Unit Testing ASP.NET MVC Views

I have successfully jumped onto the Test Driven Development (TDD) bandwagon now and loving it. Over the last 6 months I’ve been trying to incorporate it wherever I can; my code at work is now starting to gain a number of unit tests within a codebase that initially I thought would be difficult to do automated testing on.

Recently I’ve started a personal project using ASP.NET MVC – I haven’t had much experience of MVC, only Web Forms, and wanted to expand my knowledge on .NET web development. I’ve heard many stories how ASP.NET MVC was designed from the ground up to be very easy to test so I jumped at the chance of reading up and experimenting with it. One bonus as well is that this would be all-new code; the one thing I’ve learned about TDD is that it works brilliantly with a clean-slate, with legacy code it requires quite a bit more work and slower iterations in order to not break what you’ve already got.

For most of the MVC framework I could clearly see how you could apply unit testing to your code as it focuses a lot on plain objects. One of the things I couldn’t initially get my head around though is testing controllers and verifying that views returned were correct.

Take this example of a simple controller:

public class AccountController : Controller
{
    public ActionResult Index(int id)
    {
        Account account = new Account()
        {
            Id = id,
            Name = "My Account"
        };
        return View("Index", account);
    }
}

It is actually quite easy to write a unit test for this controller and action as, thankfully, controllers are not strongly tied to anything to do with HTTP requests or responses, so testing becomes as simple as:

[Fact]
public void IndexActionReturnsView()
{
    AccountController controller = new AccountController();
    ActionResult result = controller.Index(42);
    Assert.NotNull(result);
}

But at this point I got a little stuck. How do I know that the controller action returned the view that I wanted? Initially I was thinking only in terms of controller actions returning HTML output; how can we verify HTML text output in automated tests?

The answer is you don’t, and actually the answer is a lot simpler and cleaner; we don’t verify the overall output, we simply verify that the view has the necessary information (i.e. model data) it needs to generate the HTML output (or whatever output format is required).

This can be split up into several parts:

Check the View Name

One test can be defined to determine that the correct view was returned by checking the name of the view, such as below:

[Fact]
public void IndexActionReturnsCorrectViewName()
{
    AccountController controller = new AccountController();
    
    // Cast the result of the action to a ViewResult, this will then provide
    // view information for the test to confirm
    ViewResult result = controller.Index(42) as ViewResult;
    
    Assert.Equal("Index", result.ViewName);
}

One caveat I’ve found with this though is that in the controller you must specifically request which view you want; letting the MVC framework determine it by convention – based on the action name – doesn’t seem to work for some reason.

Check the View Model Type

The next test you can run is to ensure that the view returned was supplied with the correct model type, like so:

[Fact]
public void IndexActionReturnsCorrectViewModelType()
{
    AccountController controller = new AccountController();    
    ViewResult result = controller.Index(42) as ViewResult;
    
    // Test that the model object passed to the view was the correct 
    // type
    Assert.IsType<Account>(result.Model);
}

Check the View Model Data

Finally you can then test that the view returned was supplied with the correct model data, like so:

[Fact]
public void IndexActionReturnsCorrectViewModelData()
{
    AccountController controller = new AccountController();
    ViewResult result = controller.Index(42) as ViewResult;
    
    // Cast the model in the view result to the correct type and
    // now we can test against it
    Account model = result.Model as Account;
    
    Assert.Equal(42, model.Id);
}

Conclusion

This concept, once understood, feels incredibly clean to me. If you think about it you don’t want to test how a web page looks because that could change over time thanks to re-designs, plus parsing such information would be incredibly painful. All you need to do is verify that the view was given enough details for it to carry out it’s job; can it display a title for the page, does it have the correct list of customer orders to render, etc. It has actually made me re-think some of my code designs to better match this concept.

I can see why developers love ASP.NET MVC compared to Web Forms now!

Testing Framework Review: xUnit.net

In a previous post I reviewed NUnit. For my last post in this series I will focus on xUnit.net. xUnit.net is a newer open source framework that is gaining some traction. From the xUnit.net website on CodePlex:

xUnit.net is a unit testing tool for the .NET Framework. Written by the original inventor of NUnit, xUnit.net is the latest technology for unit testing C#, F#, VB.NET and other .NET languages. Works with ReSharper, CodeRush, and TestDriven.NET.

xUnit.net is a developer testing framework, built to support Test Driven Development, with a design goal of extreme simplicity and alignment with framework features. It is compatible with .NET Framework 2.0 and later, and offers several runners: console, GUI, MSBuild, and Visual Studio integration via TestDriven.net, CodeRush Test Runner and Resharper. It also offers test project integration for ASP.NET MVC.

xUnit.net is even used internally by some high profile Microsoft projects such as:

• ASP.NET Web Stack
• Entity Framework

Integration

xUnit.net is a separate project meaning that direct Visual Studio integration support is not provided. However Visual Studio 2012 will allow different frameworks apart from MSTest to be used as the primary unit testing framework – this includes TFS builds too.

In the meantime, the following steps are required:

Download from NuGet

NuGet provides three packages for xUnit.net:

• xUnit.net – the core framework
• xUnit.net: Extensions – contains commonly used extensions for the framework (such as Theories)
• xUnit.net: Runners – contains the various test runners needed to run test assemblies

Adding these packages to a Visual Studio project is very simple as NuGet will automatically download the latest versions and insert the correct project references required.

Project Items and Snippets

Unlike MSTest which provides project items and snippets with the IDE, xUnit.net does not provide any by default. However these items are not difficult to create yourself if required and the CodePlex project page even explains how to create snippets for xUnit.net.

Standalone

Although some initial setup is required one possible benefit is that xUnit.net is a standalone framework – it can be run anywhere without requiring installation, simply by copying the correct files.

Team Build

TFS 2012 will be able to use the same Unit Test plugin model that Visual Studio 2012 uses meaning in the future it will be a lot easier to integrate xUnit.net into the Team Build process.

Until then though it is possible to use xUnit.net within Team Build but only via a custom build activity and translating the xUnit.net XML output into MSTest results. This webpage explains how it is possible to do it, though the process looks quite longwinded to me.

Writing Tests

Tests are written like this in xUnit.net:

   1: using System;

   2: using System.Collections.Generic;

   3: using System.Linq;

   4: using System.Text;

   5: using Xunit;

   6: using Xunit.Extensions;

   7:  

   8: namespace SampleCode.xUnit

   9: {

  10:     // Classes do not require attributes, xUnit.net does not care

  11:     public class CalculatorTests

  12:     {

  13:         // A "fact" is a test without any parameters    

  14:         [Fact]

  15:         public void Add_AddOneAndTwo_ReturnsThree()

  16:         {

  17:             var result = Calculator.Add(1, 2);

  18:  

  19:             // Many asserts are provided by default, API style is simple and concise

  20:             Assert.Equal(3, result);

  21:         }

  22:     }

  23: }

There are a much wider variety of assertions provided by xUnit.net by default compared to MSTest. A full list can be found here.

Data Driven Tests

Data driven tests in xUnit.net are known as theories. They are test methods that have parameters and can accept input from a number of sources. A theory looks like this:

   1: [Theory]

   2: [InlineData(1, 2, 3)]

   3: [InlineData(3, 4, 7)]

   4: [InlineData(30, 10, 40)]

   5: public void Add_AddDataValues_ReturnsExpectedResult(int first, int second, int expected)

   6: {

   7:     var actualResult = Calculator.Add(first, second);

   8:  

   9:     Assert.Equal(expected, actualResult);

  10: }

Out of the box xUnit.net can accept input from the following sources:

• Inline data
  
• Property data
  
• Excel spreadsheet
  
• OleDb connection
  
• SQL Server database

Running Tests

Until Visual Studio 2012 comes out xUnit.net tests cannot be run directly via the IDE but there are a number of other options available.

The console runner is the most basic test runner available and works from the command line.

The GUI runner is a simple standalone application with it’s own user interface and is not as full featured as the NUnit GUI runner but is capable enough. Instead of a tree view like the NUnit GUI runner this test runner presents a flat list of tests but they can be filtered down by search terms, assembly and/or trait values.

One aspect that is different from the NUnit GUI runner is that although this runner will detect and reload test assemblies when rebuilt it will not automatically run the selected tests again, unlike NUnit.

MSBuild

Unlike MSTest and NUnit, xUnit.net provides it’s own custom MSBuild task which allows direct integration into the build process. A project file can then use it similar to this:

   1: <UsingTask 

   2:     AssemblyFile="..\packages\xunit.1.9.1\lib\net20\xunit.runner.msbuild.dll" 

   3:     TaskName="Xunit.Runner.MSBuild.xunit" />

   4: <Target Name="AfterBuild">

   5:     <xunit Assembly="$(TargetPath)" />

   6: </Target>

Build output then looks similar to the following:

------ Build started: Project: SampleCode, Configuration: Debug Any CPU ------
  SampleCode -> C:\TalentQ\Experiments\UnitTestAnalysis\SampleCode\bin\Debug\SampleCode.dll
------ Build started: Project: SampleCode.xUnit, Configuration: Debug Any CPU ------
  SampleCode.xUnit -> C:\TalentQ\Experiments\UnitTestAnalysis\SampleCode.xUnit\bin\Debug\SampleCode.xUnit.dll
  xUnit.net MSBuild runner (32-bit .NET 4.0.30319.269)
  xunit.dll:     Version 1.9.1.1600
  Test assembly: C:\TalentQ\Experiments\UnitTestAnalysis\SampleCode.xUnit\bin\Debug\SampleCode.xUnit.dll
    Tests: 4, Failures: 0, Skipped: 0, Time: 0.041 seconds
========== Build: 2 succeeded or up-to-date, 0 failed, 0 skipped ==========

The MSBuild task can be configured like the console runner meaning that XML/HTML results can also be saved too. See the documentation for more details.

Another useful thing is that, because it integrates into MSBuild, any failed tests will appear as errors in the IDE error list so by definition this would make it a failed build. The only slight oddity though is that, in its current form (version 1.9.1), double-clicking the errors in the error list does not take you to the correct source code as line numbers given are referring to the project file not source files.

Additional Runners

xUnit.net also provides these test runners as standard:

• TestDriven.NET
  
• ReSharper
  
• TeamCity
  
• CruiseControl.net

Performance

Performance of running tests seems to be faster than MSTest, even with a significant number of tests to execute.

Reports

Apart from the output represented by various test runners, an XML report can be produced by either the console or MSBuild runner. Once in an XML format, this can then be transformed into another format, e.g. a HTML file to make it human readable or a *.trx (MSTest) output file so that Visual Studio can understand it.

Fortunately xUnit.net is able to do this transformation for you as long as you provide the XSLT stylesheet to use. Out of the box the following stylesheets are provided:

• HTML – transforms the XML report into a HTML, human-readable report
  
• NUnit – transforms the XML report into the same format that NUnit uses

Documentation

This in my opinion is where xUnit.net falters. Because this is a newer framework documentation is thin on the ground, especially when compared to NUnit. Usually though the features are simple enough to figure out and there is sample code provided in the CodePlex repository, but you may also have to do some searching around on the internet for an explanation of some things.

Extensibility

One of the big selling points of xUnit.net is its extensibility which is far greater than either MSTest or NUnit. Some examples are:

Report Transformations

By default the console runner can provide XML, HTML or NUnit report output, but this is actually configurable by defining further command line switches mapped to a suitable XSLT stylesheet to transform in into another format (e.g. *.trx (MSTest) format).

xUnit.net Extensions

The entire xUnit.net extensions assembly is a perfect example of its extensibility. For instance [Theory] methods are actually specialised [Fact] methods that do some additional work.

More Assertions

If there are not enough assertion functions for your liking you can implement more by extending the Assertions class rather than having to write your own wrappers for it. For example:

   1: public static class MyAssertions 

   2: {

   3:     // By using extension methods you can add more assertions

   4:     public static void Test(this Assertions assert)

   5:     { 

   6:         Assert.True(true);

   7:     }

   8: }

   9:  

  10: // By deriving from TestClass a modifiable Assert class becomes available 

  11: public class CalculatorTests : TestClass 

  12: {

  13:     [Fact]

  14:     public void CustomAssert()

  15:     {

  16:         // This is our own assertion method

  17:         Assert.Test();

  18:     }

  19: }

My Opinion

This is a tricky one. Whereas I felt that NUnit was miles ahead of MSTest, the difference between NUnit and xUnit.net is a lot smaller. To be fair you could pick either one and be extremely productive so it all comes down to nit-picking.

Both NUnit and xUnit.net have their benefits and each have a few disadvantages but in the end, after much careful thought, I’ve decided to use xUnit.net as my primary test framework for the following reasons:

1. I like the fact that XSLT stylesheets are provided with the framework so I don’t have to define my own HTML report format based on the XML output. And if I wanted to change the layout of the report I would at least have something to modify as a base.
   
2. Overall the MSBuild task is a great way of integrating xUnit.net into the build process. Whereas MSTest and NUnit could be run as a task the fact was that they were just starting a new process; the only way you would know your tests had failed was by checking the exit code of the test runner, which wouldn’t tell you anything useful.
   
3. The extensibility of framework is a real plus point. I haven’t needed to extend any features yet – consider that a testament to the basics it got right – but it’s nice to know that it is there if needed.
   
4. Finally I just feel that it has a lot of potential. It may have some niggles to iron out but I feel confident that they will be.

All of the above are very minor points; like I said I could have just as easily went with NUnit, but xUnit.net just edged ahead in my opinion.

Testing Framework Review: MSTest

Recently I wrote about what I learned from Test Driven Development (TDD) and for the most part glossed over the issue of which testing framework to use. In that post I said that I didn't like MSTest and ended up preferring xUnit.net, but was I too quick to judge? Was that a snap decision? Had I really investigated all the features of each framework?

As I am the sole TDD knowledge base at work currently I was also asked to write up an objective analysis of each the major frameworks that I was considering. I therefore thought this would be a good opportunity to share my findings to the world in case anyone else found them useful.

So I shall write three new posts giving my full review of the following .NET test frameworks I looked at and tried out: MSTest, NUnit and xUnit.net. I'm sure there are others out there but these were the ones I kept seeing through internet research on a regular basis. I shall try and present the facts as best I can but you will also see my opinions sprinkled throughout.

This first post will be focused on...

MSTest

MSTest is the testing framework from Microsoft and is built into some previous versions Visual Studio but is included as standard with Visual Studio 2010, which is the version I will be referring to from here on out. You can see the reference documentation here.

Integration

Because MSTest is built into Visual Studio itself, integration between Visual Studio and MSTest is very tight. Straight from the IDE you can:

• Create a test project
• Create a test class
• Use code snippets provided to define a test method
• Run tests using commands/shortcuts and view the results in an IDE tool window
• View report/result files.

Also, TFS 2010 has the option built-in to run MSTest assemblies as part of a Team Build process and it understands the MSTest result file format so it can be included in the build output.

A possible downside of deep integration though is that MSTest cannot run standalone without Visual Studio installed, unlike other frameworks. For instance, if a Continuous Integration (CI) build server were set up you would have to install the entire IDE just to use MSTest. It is possible to just find the necessary assemblies required for MSTest to force it to be standalone, as this blog post describes, but you would also have to fiddle around with the registry to include certain references which could be more painful than it is worth.

Writing Tests

So what do unit tests look like with MSTest? Here is a simple example:

using System;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace SampleCode.MSTest
{
    // Denotes a Test Class for MSTest to look for
    [TestClass]
    public class CalculatorTests
    {        
        // Denotes a Test Method for MSTest to run
        [TestMethod]
        public void Add_AddOneAndTwo_ReturnsThree()
        {
            int first = 1;
            int second = 2;

            var result = Calculator.Add(first, second);

            // MSTest assertion
            Assert.AreEqual(3, result);
        }        
    }
}

MSTest provides several assertion methods via the Assert class, though it does not seem as feature complete as other frameworks.

Methods provided are:

• AreEqual
• AreNotEqual
• AreSame
• IsInstanceOfType
• IsNotInstanceOfType
• IsNull
• IsNotNull
• IsTrue
• IsFalse

There are also specialised StringAssert and CollectionAssert APIs.

Exception assertions must be done using the [ExpectedException] attribute. Other frameworks have moved on from this approach, making MSTest a bit antiquated in this area. Below is an example of how to test that something has thrown an exception:

[TestMethod]
[ExpectedException(typeof(ArgumentNullException))]
public void ExceptionTest()
{
    TestMethod(null);
}

It is possible to make your own assertions but you will have to define your own Assert class to provide additional assertions. For example, this blog post explains how to define an Assert.Throws() method to replace the clunky syntax for exception checking shown above for a more functional, fluid approach.

Data Driven Tests

Originally I thought that MSTest was not able to use data driven tests - that is tests which require input from an external source - but I was wrong. MSTest does support data driven tests but not in the same way as most other frameworks.

Below is an example of such a test:

[TestMethod]
[DataSource("Microsoft.VisualStudio.TestTools.DataSource.CSV", "|DataDirectory|\\Add_DataDrivenTestCases.csv", "Add_DataDrivenTestCases#csv", DataAccessMethod.Sequential)] 
[DeploymentItem("SampleCode.MSTest\\Add_DataDrivenTestCases.csv")]
public void Add_DataDriven_ReturnsCorrectResult()
{
    var first = Convert.ToInt32(TestContext.DataRow["First"]);
    var second = Convert.ToInt32(TestContext.DataRow["Second"]);
    var expectedResult = Convert.ToInt32(TestContext.DataRow["Result"]);

    var actualResult = Calculator.Add(first, second);

    Assert.AreEqual(expectedResult, actualResult);
}

Note the following:

1. MSTest does not support parameters in test methods like other frameworks do.
2. A [DataSource] attribute is used to define where the input data comes from. This example uses a CSV file but XML and databases are supported too.
3. Because parameters are not supported, all data input has to be extracted from a TestContext class (which you have to define yourself). Getting each column of the file actually returns an object, meaning you have to do yet more work to convert values into something meaningful.

In comparison to other frameworks (which I will highlight in future posts) this feature feels incredibly clunky to me. I used Visual Studio's IDE tools to auto-generate this test method for me as I doubt I would remember the syntax for it each time. It should also be noted that you cannot have inline data - that is data defined as constants such as what NUnit supports.

Running Tests

Running tests in MSTest is incredibly easy. Simply right-click in a test code file and click "Run Tests" for the Test Window to appear, such as below:

From this window you can view detailed results of each test.

For a data-driven test, only one row appears in this window yet it is run multiple times for each row in the data source, which can appear confusing. Viewing the detailed results of a data-driven test will show the individual results though.

It is also possible to run MSTest via an MSBuild task, e.g. as an after-build step. This is done by simply executing the MSTest console application using the <Exec> task like this:

<Target Name="AfterBuild">
    <Exec Command='"C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\mstest.exe" /testcontainer:"$(TargetPath)"' />
</Target>

Build output then appears as follows:

------ Build started: Project: SampleCode.MSTest, Configuration: Debug Any CPU ------
  SampleCode.MSTest -> C:\Experiments\UnitTestAnalysis\SampleCode.MSTest\bin\Debug\SampleCode.MSTest.dll
  Microsoft (R) Test Execution Command Line Tool Version 10.0.30319.1
  Copyright (c) Microsoft Corporation. All rights reserved.

  Loading C:\Experiments\UnitTestAnalysis\SampleCode.MSTest\bin\Debug\SampleCode.MSTest.dll...
  Starting execution...

  Results               Top Level Tests
  -------               ---------------
  Passed                SampleCode.MSTest.CalculatorTests.Add_AddOneAndTwo_ReturnsThree
  Passed                SampleCode.MSTest.CalculatorTests.Add_DataDriven_ReturnsCorrectResult
  2/2 test(s) Passed

  Results               Add_DataDriven_ReturnsCorrectResult
  -------               -----------------------------------
  Passed                SampleCode.MSTest.CalculatorTests.Add_DataDriven_ReturnsCorrectResult
  Passed                SampleCode.MSTest.CalculatorTests.Add_DataDriven_ReturnsCorrectResult
  Passed                SampleCode.MSTest.CalculatorTests.Add_DataDriven_ReturnsCorrectResult
  3/3 test(s) Passed

  Summary
  -------
  Test Run Completed.
    Passed  5
    ---------
    Total   5
  Results file:  C:\Experiments\UnitTestAnalysis\SampleCode.MSTest\TestResults\petermonks_ACHILLES 2012-08-14 10_42_21.trx
  Test Settings: Default Test Settings
========== Build: 2 succeeded or up-to-date, 0 failed, 0 skipped ==========

Performance

Performance of running tests seems a little slower than other frameworks as MSTest saves all results to file meaning IO time is consumed a lot. This didn't cause any problems with my simple test suite but this might be significant if hundreds of tests are being run at once.

Extensibility

Since .NET 4 and Visual Studio 2010, MSTest has been able to be extended to some degree although doing so is hard work. For example, this blog post explains how to extend MSTest to use inline data-driven tests that have parameters, such as:

[TestMethod]
[Row(1, 2, 3)]
[Row(4, 5, 6)]
public void ParameterTest(int x, int y, int z)
{
    // Test code here...
}

However from this post it looks incredibly long-winded to implement and setup, whereas other frameworks have this ability as a built-in feature.

My Opinion

So these are the facts I researched, what do I think about MSTest now?

Overall I am still of the opinion that I was right to not use it. Having tried other frameworks, which I will give my review on in future posts, I can see that MSTest is clunky to use and not as fully featured as other frameworks. The only Unique Selling Point I can see with it is that it is built into Visual Studio so it is easy to get started with it and integrate it with other Microsoft tools, something we do require at work as we want to use TFS Build to get test results. But seeing as Visual Studio 2012 is going to include the new Unit Test Adapters to allow other test frameworks to be used instead, even this is starting to become a moot point. 

I've also noticed that some open-source Mircosoft projects don't even use MSTest and use something else. Maybe it is because they are trying to keep everything self-contained in these projects, but I read it as not even some developers in Mircosoft would consider using their own test framework.

So MSTest isn't for me. Next time I shall review NUnit.