42 Lines

J.B. Rainsberger has recently been railing against intgeration testing, and I find myself agreeing with that argument a great deal. Our integration tests always take too long to run, are astable with respect to changes to the UI layer, and are probably sucking time that could be better spent unit testing.

Since switching to BDD I've been enjoying writing tests specifications a lot more, but that hasn't extended to integration testing. We've been using JDave and JMock, and the results are really expressive, but attempts to extend this to the integration layer have fallen flat. I don't think it's the fault of any given technology. JDave has decent integration with wicket and webdriver, and I appreciate that this is probably the closest I'll get to writing expressive integration tests:

     public class WhenDictionaryDoesNotContainGivenWord {         
       public DictionaryPanel create() {            
           checking(new Expectations() {{                 
             one(service).lookup("hello"); will(returnValue(Collections.emptyList()));          
           }});       
           return dictionary = startComponent();   
      }                  
      public void thePanelDisplaysNoHitsMessage() {             
          query("hello");             
          FeedbackMessage message = dictionary.getFeedbackMessage();             
          specify(message.getMessage(), should.equal("No hits."));         
      }             
    }          

That's fine for a simple component, but it gets really ugly when you have a lot of nested components with content that conditionally present depending on ajax events and the underlying data. The resulting test code is still hard to read, and dependent on wicket:ids and walking the component hierarchy, even if you're careful to use IVistior.

One question I've always pondered is whether functional testing represents a belief in the gestalt, and whether that's warrented. In other words, if I unit test my models, DAOs, and Wicket Components with mocks, do I really need to drive them all in a functional test as well? Is the whole more than the sum of its parts? What am I testing at that point? Are you really ever going to put together a wicket page completely with tests and never look at the output in a container? If so, I'm impressed and also worried for your sanity. And if not, what are you testing?

When I'm writing intgeration tests I feel like I'm testing wicket, or my understanding of it, i.e. does add() really do what I think it does? Because I'm certainly not covering all the use cases for an ajax-capable web page with several components and several underlying data possibliites and several crazy users. It's a combinatorial impossibility. So I'm not really testing what integration testing claims to test.

Perhaps I just have to get used to the idea that integration tests break a lot more often that unit tests, and are harder to read, and harder to write. But J.B. has given me food for thought.