Freekshow

September 3, 2008

Unit testing part 3: Design for testability

Filed under: .Net,Microsoft,Unit Testing — Freek Leemhuis @ 11:20 am

This is part 3 in a series of posts about unit testing using Visual Studio. Part 1 and part 2 focused mainly on the MS Test environment within Visual Studio.

When writing unit tests, whether one subscribes to Test Driven Development practices or not, often forces one to consider the design of the code under test. This is probably where the term ‘Design for testability’ comes from: not all code is easily testable, and you should therefore design your code so it can easily be tested. I’ve always found the term strange, as the ultimate purpose of code is not that it is tested, but that it works and is maintanable. If you can manage loose coupling through OO principles such as the Single Reponsibilty Priciple and the Open Closed Principle, then the code is testable as a result of good design.

Or, as Uncle Bob says:

“The act of writing a unit test is more an act of design than of verification”

Having said that, what is generally meant by design for testability?

Consider the following class:

public class Clown
{
   public void TellAJoke()
    {
       Joke joke = new JokeRepository().GetRandomJoke();
       Console.WriteLine(joke.question);
       Console.Read();
       Console.WriteLine(joke.punchline);
       DrumRoll.Play();
   }
}

Just a few lines of code, and we’re already in a pickle. The code is not easily testable, and the problem is tight coupling. You can not test the TellAJoke method without also invoking the JokeRepository, Joke and DrumRoll class.

The GoF : Programming too an interface, not an implementation

(It’s actually in the first chapter, so you might catch it just before you fall asleep :-))

Let’s take this advise to heart and do some refactoring:

public class Clown
{
    ISoundEffect _soundeffect;
    IDisplay _display;
    public Clown(IDisplay display, ISoundEffect soundeffect)
    {
        _soundeffect = soundeffect;
        _display = display;
    }
    public void TellAJoke(IJoke joke)
    {
        _display.Show(joke.question);
        _display.Show(joke.punchline);
        _soundeffect.Play();
    }
}

We have defined interfaces for Joke, Display and Soundeffect. We can therefore call on the might of polymorphism if we want to use alternative methods of displaying text or using sound effects (we may want to switch to trumpets in a future iteration!). In this refactoring we are using dependancy injection to remove the coupling with Drumroll(ISoundeffects) and Console (IDisplay) by adding them as an argument to the constructor.

Note that we have also defined an interface for Joke, and added the joke as an argument to be passed into the method. Because our JokeRepository retrieves jokes from a database backend, we can not really use the repository in testing: we want to keep our tests lean and mean, and that means no database trips!

We can for the purpose of testing the TellAJoke method now subsitute the IJoke parameter with our own stub:

[TestMethod]
public void Clown_TellAJoke_Test()
{
    Clown pipo = new Clown(new ConsoleDisplay(), new DrumRoll());
    //create Joke stub
    Joke jokeStub = new Joke
    {
        question = "How many developers does it take to change a lightbulb?",
        punchline = "They won't. It's a hardware problem"
    };
    //redirect console output
    var  writer = new StringWriter();
    Console.SetOut(writer);
    //call the method under test
    pipo.TellAJoke(jokeStub);
   
    string output = writer.ToString();
    Assert.IsTrue(output.Contains(jokeStub.question));
    Assert.IsTrue(output.Contains(jokeStub.punchline));
   
}

Note that we are still using the console as the method to display text, but if we decide to change that, all we would need to do is implement the IDisplay interface on an alternative implementation. The key here is that we don’t have to change our Clown class to change the implementation of display or soundeffect.

Are you mocking to me?

So what about mocking? What’s that all about? The seminal reference in this regard is Martin Fowler’s article Mocks aren’t stubs, where he explains the difference between the two. Basically, if you are using stubs you’re usually testing state, whereas mocks allow you to test the interaction between objects.

Where we have in the previous example used a stub for the Joke class, we have left out tests for the soundeffect. What we would like to test is that the soundeffect sounds when delivering the punchline.  Let’s create a stub to substitute the soundeffect – we don’t want actual sound effects when running our unit tests, fun as that might be…

 

public class SoundEffectStub:ISoundEffect
{
    public void Play()
    {
        Console.WriteLine("Sound!");
        Console.Read();
    }
}

And we could test that the ‘sound’ gets played by redirecting the console output as before. However, we are testing state, and state that is introduced by our stub. An alternative here is to use a mocking framework that will allow us to test the interaction of these objects. The only thing we want to test is that the soundeffect gets played, and it’s for this kind of behavior verification that mocking frameworks really shine.

Typemock is a popular mocking framework in the .Net space, and we can use it for our little example as so:

 [TestMethod]
public void Clown_TellAJoke_Test()
{
    MockManager.Init();
    Mock mockSound = MockManager.Mock(typeof(SoundEffectDrumRoll));
    Clown pipo = new Clown(new ConsoleDisplay(),new SoundEffectDrumRoll());

    //create Joke stub
    Joke jokeStub = new Joke
    {
        question = "How can you tell when a programmer has had sex?",
        punchline = "When he's washing the pepper spray out of his eyes."
    };

    //redirect console output
    var  writer = new StringWriter();
    Console.SetOut(writer);

    //set expectations
    mockSound.ExpectCall("Play");

    //call the method under test
    pipo.TellAJoke(jokeStub);
   
    string output = writer.ToString();
    Assert.IsTrue(output.Contains(jokeStub.question));
    Assert.IsTrue(output.IndexOf(jokeStub.punchline) > output.IndexOf(jokeStub.question));

    mockSound.Verify();

}

Typemock is different from other mocking frameworks in that it uses the .Net profiler API to monitor and intercept program execution. This makes it very powerfull, because it is not tied to the same restrictions as other Mock frameworks. Rhino Mocks or similar solutions require that you code to interfaces, and any method you want to mock must be marked as virtual. Some people have even voiced concerns that Typemock is too powerfull, in that is does not force you to design for testability. I think Roy has put that argument to bed with his reply.

In the above example we can use the original SoundEffectDrumRoll implementation, and by setting up the expectation to the Play method Typemock will make sure the method does not actually get executed, and by using the Verify method we make sure the method was in fact called.

Especially when working with legacy code you will find that Typemock offers powerfull features to allow you to test code that would otherwise not be testable without refactoring. On the other hand, if you write new code and use Typemock, you can ignore some of the design issues we’ve talked about here. But that does not mean it’s the right thing to do.


Advertisements

June 26, 2008

Unit testing with Visual Studio 2008 – Part 2

Filed under: Microsoft,Unit Testing — Freek Leemhuis @ 3:03 pm
Tags:

This is the second part of a series of blogpost on unit testing. Find the first part here.

In the first part we’ve introduced the MS Test framework, and I realise now that I’ve left out a few bits and pieces, so first off let me try to make amends. 

Initialize and Cleanup

Many times when running a test you’ll want to set up some conditions under which the test will run. After completion you’ll want to clean up these artifacts. Similarly to NUnit, where you’ll find attributes as [Setup] and [TearDown], MS Test has some attributes you can use to accomplish this.

To set up state before every test method in a test class, you can decorate a method with the [TestInitialize] attribute. Similarly, to clean up you can use the [TestCleanup] attribute.

For example, suppose you are testing some xml parsing routines. If you are testing agains some sample xml fragment, and it’s the same fragment for all your test methods in a class, you can set the fragment using the [TestInitialize] attribute:

private MyDataContext db; 

[TestInitialize()]
private void TestInit()  
{
     db = new MyDataContext(
        ConfigurationManager.ConnectionStrings["TestDatabase"].ConnectionString);
     db.Connection.Open(); 
}

 
Similarly, you can use the [TestCleanup()] attribute to dispose of any state.

[TestCleanup()]
private void WrapUp()
{
       db.Dispose();
       db = null;
}

Note: you only use a db connection setup like this when testing data access routines. For any other routines you do not really want the tests to hit the database.

So that’s pretty convenient: you don’t have to add the initialization code in the methods themselves. However, the opening and closing of the database will occur as many times as there are test methods in your test class.

You can use the [ClassInitialize] and [ClassCleanup] attributes for state changes that will execute only once for all test methods within a test class. However, for methods to use these attributes they must be static, which means that you can not use it to set properties on the instance of the test class.

Testing for exceptions

A lot of times I’ve seen developers using a try/catch block to test for exceptions. Joel for example has posted this little code snippet:

[TestMethod]
public void TestZipCodeNotNumericThrowsArgumentException()
{
       TransactionRequestInfo req = new TransactionRequestInfo();
       string testValue = "034JB";
       try
       {
           req.Zip = testValue;
           Assert.Fail("Exception not thrown.");
       }
       catch (ArgumentException aex)
       {
           Assert.IsTrue(true, "Exception Thrown Properly");
       }
}

You can see what he’s trying to do, and I’ve seen many a test use this approach. There is a better way: let’s rewrite this using the appropriate ExpectedException attribute:

[TestMethod]
[ExpectedException(typeof(ArgumentException))]
public void TestZipCodeNotNumericThrowsArgumentException()
{
    TransactionRequestInfo req = new TransactionRequestInfo();
    string testValue = "034JB";
    req.Zip = testValue;
}

You can see it pays to investigate the full capabilities of the testing framework. Note that one would probably want to use a more specific exception than ArgumentException (InvalidZipcodeException?), but that’s for another post and another day.  

On cleanup and debugging

When dealing with databases, you might on occasion use a method with the TestInitialize attribute to verify preconditions in the database, and similar you have a cleanup routine to remove any artifacts. What happend to me on more than one occasion when starting with unit tests is that I would run tests under debugging, and when an expection occurred I would stop the debugging session. This however would prevent execution of any cleanup routine, and the state would not be properly reset before the following test run. Be sure to continue debugging to allow your cleanup code to execute!  

Keyboard Shortcuts

To wrap up the specifics of the MS Test frameworks, here’s a list of keyboard shortcuts to run your tests. The running of tests through the test windows is a bit cumbersome to navigate, so these keyboard combinations will come in handy:

CTRL + R, then press T
This runs the test(s) in the current scope. That is, it runs the current test method, all the tests in the current test class, or all the tests in the namespace, respectively.

CTRL + R, then press CTRL + T
This also runs the test(s) in the current scope, but under debugging.

CTRL + R, then press C
This runs all the tests in the current test class.

CTRL + R, then press A
This runs all the tests in the solution.

Resources

MSDN documentation on Unit Testing

Write Maintainable Unit Tests That Will Save You Time And Tears (MSDN article by Roy Osherove)

If you’re looking for a good book on unit testing, I can recommend Pragmatic Unit Testing in C# with NUnit , by Andrew Hunt and David Thomas. The authors use nUnit as the test framework, but the methods and practices apply regardless of what framework you use.

Roy Osherove shares his thoughts on unit testing and his upcoming book on The Art of Unit Testing/


April 20, 2008

Unit Testing in Visual Studio 2008 – part 1

Filed under: .Net,Unit Testing — Freek Leemhuis @ 2:54 pm
Tags: , ,

Unit testing – does it need an introduction?

Unit tests can massively improve the maintainability of any application. Bugs are found right after they get introduced, and refactoring code is done with great confidence. As a consultant, I’m participating in many different development teams. In the last year or so I’ve done a number of audits and coaching sessions on client sites. One thing that has struck me is how different shops approach unit testing varies enormously. Some have been doing it for years, and are very well versed in it. Others are struggling to integrate it into their practices, and others still have not made any strides at all. So, in my experience, the practice of unit testing is not so ubiquitous as you might expect. I’ve also found that, as is writing software in general, writing good unit tests is hard to do. It requires insight and experience, and for those who start out it can be a frustrating experience. My first set of unit tests were a fragile bunch. Sometimes they would break by dozens at a time, other times they would break where the actual code would be running fine.

So the answer to the first question, ‘does it need an introduction’ would be yes, plenty of times it does! That’s why I’ve decided to dedicate a number of posts on the art of unit testing. This post is the first in the series, and here I’ll focus on the environment. I’ll introduce the build-in unit testing framework for Visual Studio 2008. In later posts I’ll talk more about test patters, design for testability, mocking and code coverage.

Introducing the Visual Studio 2008 testing framework

Microsoft has provided MS Test for unit testing in Visual Studio since version 2005. These tools were only available if you were running one of the Team system editions. Luckily, since then Microsoft has come around and for the 2008 version unit testing is also available in the Professional Edition (but not Standard Edition). Things like code coverage analysis remain limited to the Team Suite editions. Find a comparison of the features for different editions here.

First of all, you will need to add a separate Test Project to your solution. Unit test will not be stored between your source classes, but always kept in a separate test project. Select your solution and click on File, Add, New Project

Note that by default, a number of items are created in your test project.

They include a AuthoringTests.txt: a text file containing general information about testing, ManualTest1.mht: a template-type ManualTest for adding functional test descriptions to your project (I’ve never met anybody who did), and a blank unit test UnitTest1.cs. Most likely you will want to change the default settings, so you don’t have to go and manually delete these items. Luckily you can do that through the options menu by clearing the checkboxes displayed below.

We add a reference in the test project to the project we’re trying to test. We’re now ready to add a test. Let’s assume that we have a piece of code like the following:

partial void OnCompanyNameChanged()

       {

           if (CompanyName.Length > 20)

           {

               CompanyName = CompanyName.Substring(0, 20);

           }

       }

We’re using Linq to SQL and the Northwind database, where I’ve added a partial class for the Customer. In the partial class I can add validation code like the one above. The company name can contain only 20 characters, and while we’re validating things like that in the user interface, it’s good practice to validate it server-side as well. In this case, we’re not throwing an exception but simply take the first 20 characters if the name provided is longer than 20 characters.
Now we can add a test to the test project. Click on Add, New Item, Class and add the new class. For naming, there’s a number of naming conventions you can choose from. I usually name the test class as <ClassToTest>_Tests.cs, so in this case I’d name it NwindDataContext_Tests.cs.
In the test class, you can have a number of test methods, which can test one or more of the methods in the class under test. This way, you will have a 1:1 between your classes and your test classes. Of course, you can have (and often you’d want to have) more than 1 test method for each method.
Since we’ve added a plain class, we need to introduce the namespace for the testing framework:

using Microsoft.VisualStudio.TestTools.UnitTesting;

We have to define the test class and methods as public. We decorate the class with the [TestClass] attribute, and add a new testmethod. Methods are decorated with the [TestMethod] attribute.

[TestClass]

    public class Customer_Tests

    {

        [TestMethod]

        public void OnNameChanged_MoreThan20Chars_TakesFirst20Chars()

        {

            string testName = “abcdefghijklmnopqrstuvw”;

            Customer customer = new Customer();

            customer.CompanyName = testName;

            string expectedName = “abcdefghijklmnopqrs”;

            Assert.AreEqual(expectedName, customer.CompanyName,

                “OnNameChanging should take only first 20 characters”);

        }

    }

For the test methods I’m using the naming convention

<MethodUnderTest>_<StateInput>_<ExpectedResult>.

I’ve used another convention in Unit testing, in that I’ve set up an expected value and I’m comparing that with the actual value. If we then use Assert.AreEqual to compare the two, the framework infers that the first value is the expected result, and the second the actual result. The third parameter of Assert is used to specify a detailed description that is displayed in the event that the test fails.
If we run the test, (right-click, choose Run Test), we’ll see in the test result window that the test fails:

Double-clicking on the result brings up the details of the test run:

In the error message we see that the expected value differed from the actual value, and we see the description returned that we have provided in the test.
Of course, the reason why the test fails is a simple counting error: I’ve included only 19 characters in the test condition, so as the test results indicate, I’ve missed out the letter ‘t’.
Fixing this by appending the letter in the test method results in Pass:

We’ve seen a number of test windows, Clicking on the Test Menu reveal there are more testing related windows :

The Test View Window

The Test View Window is a list of all test methods in all test projects of the solution. From this window, you can make a selection of the tests you want to run, and then choose to run or to debug the selected tests.

If you have a large number of tests, as is the case here (it’s the Enterprise Library) you will sometimes want to filter the list. Here I’ve filtered on the search term ‘isolatedstorage’ to narrow down the list of test methods.

The Test List Editor

If you have a large number of tests, it is more convenient to partition the tests into separate groups. The Test List Editor is where you assign tests to particular groups.

For example, if I’m working on Caching, it can be convenient to move all tests related to caching to a separate list, so you can easily pick out a group of tests you want to run.
Personally, I don’t use the Test List Editor much. The grouping does not automatically reflect the physical grouping you’ll have in your test project, which can be confusing. Apart from that, if you’re coding it is my belief that it’s better to run all unit tests, and not a subset, since the coding might introduce an effect in the codebase that is covered by unit tests in other test lists.

The test results window.

We have already seen this window in the previous paragraphs. In this window, the results of your last run are displayed. The framework will keep by default the last 25 test runs, and you can select the results of a previous run from the dropdown box.

Back to the test

Okay, why have I not used the separate Unit test template to add a test? Well, I’ve wanted to show you the ‘bare bones’ of what makes up a unit test. If we select the template for a unit test, we get a wizard-style dialog that forces us to choose the code that we want to test. This is of course contrary to the Test Driven Development paradigm, where you want to write your tests before you write the code. Let’s run the wizard now and see where it takes us. Select New, Unit Test. You will see the following dialog displayed.

You can use the Settings button to call up the following window where you can name the new test classes and methods:


In this case, we have not changed the default names and end up with CustomerTests test class(that’s not too bad) and a OnCompanyNameChangedTest test method. This is not according to the naming convention we had in mind, so if you’re using this option beware to rename your methods so they express exactly what you’re intending to test.
In addition, there’s a fair amount of code that’s been generated in our test class. For starters, there’s this bit:

private TestContext testContextInstance;
///
///Gets or sets the test context which provides
///information about and functionality for the current test run.
///
public TestContext TestContext
{
  get
  {
  return testContextInstance;
  }
  set
  {
  testContextInstance = value;
  }
}

The TestContext has methods like TestDir (returns the path to the test folder) and, in the case of an ASP.Net test, RequestedPage(returns a reference to the aspx page). It’s nice to have these options, but in most test cases it’s a case of YAGNI and therefore clutter.
Next up: a region called Additional test attributes. This contains some very useful suggestions:

#region Additional test attributes
  //
  //You can use the following additional attributes as you write your tests:
  //
  //Use ClassInitialize to run code before running the first test in the class
  //[ClassInitialize()]
  //public static void MyClassInitialize(TestContext testContext)
  //{
  //}
  //
  //Use ClassCleanup to run code after all tests in a class have run
  //[ClassCleanup()]
  //public static void MyClassCleanup()
  //{
  //}
  //
  //Use TestInitialize to run code before running each test
  //[TestInitialize()]
  //public void MyTestInitialize()
  //{
  //}
  //
  //Use TestCleanup to run code after each test has run
  //[TestCleanup()]
  //public void MyTestCleanup()
  //{
  //}
  //
  #endregion

The content of this we’ll get to later. However, do we really want these instructions sitting in every test class? More clutter.
Finally, we get to the meat. There’s an actual test method generated:

  /// <summary>

        ///A test for OnCompanyNameChanged

        ///</summary>

        [TestMethod()]

        [DeploymentItem(“nwind.BLL.dll”)]

        public void OnCompanyNameChangedTest()

        {

            Customer_Accessor target = new Customer_Accessor();                           

            target.OnCompanyNameChanged();

            Assert.Inconclusive(“A method that does not return a value cannot be verified.”);

        }

We have a skeleton test method, and can start thinking how to rename and build this into the actual test(s) that we want to perform. Note that a [DeploymentItem] is assigned, which you really only need if you want to run your tests in a separate deployment folder.

So are my private parts exposed now?

I’m glad you noticed. This is a little trick that the framework has played in order to allow the testing of private and internal methods. It uses reflection to create a shadowed copy of the code under test, and runs the tests against that rather than against the actual code (Customer_Accessor rather than Customer). When using the New Unit Test wizard, this accessor assembly is automatically created, regardless of the existence of private members.

Now that we’ve explored the MS Test environment, we can dive into the testing itself. This I’ll save for the next post.


 

Blog at WordPress.com.