I’m not going to go fully into what BDD is here, but my goal was to use a similar process when developing ASP.NET MVC applications as I did in Rails.  With Rails, it’s simple, Rails, Cucumber, Pickle, ActiveRecord, Factory Girl, Webrat, etc. work perfectly together.   In the .NET world, there are options, but nothing that really fit together as nicely.  I started with just wanting to test my basic application (menus, breadcrumbs, etc) and didn’t venture into model work yet, so I figured it would be as simple and efficient as it is in Rails.  Not so much.  I tweeted about this exact thing a while ago.

SpecFlow and WatiN

Did I mention how much I love Cucumber?  Well, Matt B had recommended SpecFlow to me on the .NET side, so I went and started using it.  Steve Sanderson actually has a great, detailed post on this exact topic, so I won’t go into all the things with SpecFlow.  In short, while SpecFlow is pretty cool and very much like Cucumber, using it for web testing isn’t exactly great.  There are a lot of things wrong in that space, but really the thing that turned me off was the speed of the tests.  My tests are interacting with a website, and sadly there’s not much in way of web testing under .NET.  For my tests I used WatiN.  WatiN is pretty cool, but the fundamental problem is that it uses the actual browser to run the test (I was using IE), and man, were my tests slow.  I had a suite of just 20 scenarios with 46 simple steps, for example, verifying the title of the page and such.  Nothing complex at all.  Running the scenarios with SpecFlow/WatiN, it takes around a minute and a half to run.  Yikes.  In my Rails project I’ve been working on, I have 167 scenarios with 1912 steps, and that takes about 2 minutes to run (and that’s running on a machine with only 2GB of RAM vs 8GB on the machine running the SpecFlow tests).  Quite a difference, not to mention the Rails tests are actually inserting things into a database as well.  The times to run these tests in .NET was just horrible, not very conducive to using BDD for these types of tests.  My hunch is that the instance of the browser was causing all of the delays in my test, and why my hunt was on for how to do my testing without using a full browser like Webrat can do.

** Update **
I started writing this article after just using Steve’s example as a guide, however I didn’t finish it.  I’ve been playing with other stuff, but got around to getting back to the article.  Anyway, He is using a new browser instance for each scenario (which seems like what you want so that you get a fresh instance for each scenario).  By changing the browser instance creation to be for each Feature instead, it drops the time to 30 seconds.  A bit better, but still rather slow.  Continue on for what I feel is a better option.

SpecFlow vs. Cucumber

SpecFlow has the advantage of writing your steps in C#, but with my new found love of Ruby, I don’t have a problem whipping up steps in Ruby.  I’m not one of those people who can only use one language, and really, the steps flow much nicer in Ruby than in C#.  For example:

Here’s a step defined in C#:

[When("I follow \"([^\"]*)\"")]
public void WhenIFollow(string linkText)
{
    var link = WebBrowser.Current.Link(Find.ByText(linkText));
    Assert.IsNotNull(link, string.Format("Could not find a link with the text {0}", linkText));
    link.Click();
}

And the Ruby version (with Webrat):

When /^I follow "([^\"]*)"$/ do |link|
  click_link(link)
end

Where the click_link method is provided by Webrat, but they both do the same thing: follow a link on a page.  The point is the syntax, not so much what the method body looks like.  Doesn’t the Ruby look cleaner?  For example, there is no extraneous escaping needed within the regular expression, and no need for the extra method signature markup.

One thing that SpecFlow has is integration with Visual Studio and integration with the Unit Test runner in ReSharper and such.  That’s pretty cool, but again, I’m looking for speed in these things.

Webrat and Mechanize

Webrat is really designed for testing Rack-based web applications, like those running on Rails or Sinatra.  I previously assumed you could only use it for testing Rack-based applications, but thanks to a gem called Mechanize, we can use Webrat with ANY website, oh yea!  Using Mechanize in Webrat is easy, it’s just a simple change to the env.rb file (typically found in the path /features/support/env.rb).  With that change, you can pretty much work with your ASP.NET MVC site just like you can with tests against Rails.  In fact, I just snagged a copy of web_steps.rb from my Rails application and the steps work just great.  The only difference is that you need to fully qualify your URIs.  Within a /features/support/paths.rb file (like you would have in Rails), I simply prefixed the relative paths with the absolute path to my local web server.  There is a good starting project from @cayblood on github.

In order to run the Cucumber tests, simply fire up a command prompt.  Then browse to the root folder where your features directory is located and it’s just a simple “cucumber” command to run all the tests.  Of course, you need to make sure your MVC web application is running or else your results are going to be very red.

Big Speed Improvements

So I duplicated the features in my Ruby/Cucumber/Webrat/Mechanize project (and tweak them a bit, totaling 21 scenarios and 61 steps).  Since the SpecFlow features are written in Gherkin, it’s easy to move the files over (just exclude the .feature.cs files from SpecFlow).  The tests took around one second to run.  That’s right… A minute and half with SpecFlow/WatiN down to a second.  Sweetness.  By the way, this is running on Windows as well where Ruby isn’t the speediest (note the tests in the post were using Ruby 1.8.7 and not 1.9.1 like my tweet).

Conclusion

For web testing, there is just such a difference with the Ruby stack vs. the C# stack.  I would expect much better performance if there were a headless browser option available in .NET.  I don’t have any issue with using Ruby code to test C# stuff, and the more I use Ruby, the more I dislike static-typing.  I chalk it up to just “using the right tool for the job.”  Aside from using something else that isn’t C#, the second disadvantage is VS integration.  Notice I said run your tests from a command prompt.  I know you Windows guys (well most) shudder when you hear the “command line,” but some of us aren’t scared :).  Personally, for the speed improvements, I can live without C# and VS integration.  Of course, here I go again, falling further in love with Ruby.

The fundamental thing that I think is most misunderstood about unit testing is simply not understanding what a unit test is.  In fact, Roy just recently posted an updated definition of how he would classify a unit test on his blog.  Of course a definition isn’t always enough, and in his book, Roy starts off by fully explaining all the aspects of what a unit test should be and the basic outline for how to succeed.  I would highly recommend that you visit his book’s webpage (http://manning.com/osherove/) and download the sample first chapter. 

This seriously may be the best book ever written on unit testing.  I understand that “the best” is a strong statement, but this book is really the perfect package.  Let’s look at what makes this such a winner…

Come On, The “Best”

Of course, don’t just take my (or BOO’s, or Tim’s) word for it.  Obviously, check out other’s reviews on Amazon as well.  However, there are many points that justify my opinion:

• No Fluff – It’s a slimmer book (320 pages), but packed with information.  The size of the book makes it very approachable, and it’s one of those books that you may want to read a few times to make sure everything really sinks in.  It’s a book you can recommend to just about everyone, regardless of their background.
• No Preaching – One key thing that sets this book in a class by itself is the fact that it doesn’t focus on specific methodologies (like TDD), but is simply a guide to unit testing.  The book doesn’t assume anything or force any one way of practicing the art.  Fully devoted to TDD, great.  Could care less about TDD?  This book is for you as well.
• Independent – Despite the fact that Roy works for Typemock, the majority of examples found in the book use open-source “mocking” or isolation frameworks.  Throughout the book, no one tool is pushed over another.  Roy does a good job of going over the pros and cons of many different tools, from many vendors.
• Inspiring – From the initial chapter on, this book not only educates, but it truly inspires.  This is one of those books that gets you ramped up to do something.  Whether it is starting unit testing within your organization, or enhancing existing tests, you’ll want to start right away.  It’s that feeling we geeks get when a new version of software is released.  I know I’m not the only one who goes to software conferences and such, and feels like jumping head first into the latest and greatest. 

Chapters

This is one of the few books where each chapter matters.  Even the appendices are packed with important information.  You can see overviews of each chapter anywhere, so I won’t bore you with every little detail here.  But a few of the chapters really stood out.

• Chapter 4, Interaction testing using mock objects – We’ve all heard the terms “mocks” and “stubs,” but do you know the difference?  Many books refer to these, but the terminology can be confusing.  After Chapter 4, you’ll have a good grip on what’s what.  The explanations and best practices in this chapter, set the base line for the rest of the chapters.  It would be nice if everyone could use the “common” definitions of things, but unfortunately, this practice isn’t always followed when it comes to discussions of unit testing. 
• Chapter 8, Integrating unit testing into the organization – This topic in and of itself could be described as an “art.”  You may believe in something, but conveying the belief to others isn’t your typical walk in the park.  I was very impressed that this important issue was covered in the book. 
• Chapter 9, Working with legacy code – Of course we can’t always be working on brand new development. This chapter gives a good introduction and solid approaches for dealing with legacy code.  This is another one of those chapters that’s refreshing to see within a  book of this nature.  I like how the book branched out into topics that aren’t frequently discussed.  However, this topic is really one of those that could use an entire book written about it, and Roy recommends Michael Feathers, Working Effectively with Legacy Code. Gag, legacy code, right?  It’s a reality, and remember, that new shiny thing your working on will most likely end up as someone’s headache soon enough!

More From Roy

After reading his book, I immediately started seeing what else Roy has been up to.  There are two great resources that Roy started, 5whys.com and the Team Leadership in Software group on Google.  Both will help you in working with and leading software teams.  On the unit testing/TDD front, be sure to check out Roy’s videos.  You can also check out Roy’s other sites, http://osherove.com/, http://iserializable.com/, and http://artofunittesting.com/.