Noel Rappin Writes Here

The Testing Pyramid, from Rails 4 Test Prescriptions

It’s time for “Ask A Tester Person”, a game I haven’t played in a long time. I got a question on Twitter (several weeks ago, actually… but better late than never, right? Right?):

I am, as a Rails junior, also confused about changes in controller testing

I guess that’s technically not a question, but we’ll consider to the question to be “What the hell is up with controller testing in Rails?”

(You too can get a 750 word blog post in response to your Twitter question if you tweet me @noelrap. Be prepared to wait several weeks.)

Rails has always had very strong support for automated testing. In Rails core, this takes the form of several different customized subclasses of Minitest::TestCase, which simplify access to various parts of the framework. (RSpec provides similar functionality.) The most commonly used tests have been:

• Model tests, which have some special ActiveRecord accessors.
• Integration tests, which have a syntax for interacting with headless web pages.
• Controller tests, which have some special features for calling controller actions directly from the test and evaluating the resulting integration variables.

Over time, controller tests have become problematic. You don’t explicitly instantiate new controller objects in Rails controller tests, so setting up the test and accessing the controller object to validate its state turned out to be a little bit of a pain. This is why controller tests in Rails resort to odd workarounds, like the assigns hash to test for the values of instance variables, rather than testing instance variables all by themselves.

As Rails practices evolved to put less logic in the controller itself, the need to separately test controllers diminished. Since the controller usually just calls a bunch of ActiveRecord objects or services, the controller test often winds up duplicating the same assertions as either those ActiveRecord tests or the associated end-to-end integration tests. So, either you are aggressively using test double to stub access to the ActiveRecord objects, which is not to everybody’s taste, or you feel like you are writing the same test twice. Put another way, it’s often hard to see how to write a controller test that would fail even if the associated integration and model tests both pass.

One way of looking at controller tests is through the idea of the test pyramid, which is a term coined by Mike Cohn. The idea of the test pyramid is that the base of the pyramid is unit tests. Your test suite should have a lot of unit tests, which are fast and focused. The much smaller top of the pyramid is integration tests. Your suite should have a much smaller number of end-to-end tests, which are slower and less focused. Controller tests are in between — they aren’t quite end to end integration tests, and they typically aren’t unit tests, either, because they often test both the controller and the associated ActiveRecord. As a result controller tests, are often slower than unit tests, and brittle like end-to-end tests — the worst of both worlds. My experience is that these kinds of in-between tests are particularly hard to keep maintaining.

One of the specific proposals in DHH’s famous TDD is Dead talk and blog post was the idea that unit testing is not “a useful way of dealing with the testing of Rails applications” and that Rails should do more to encourage full-stack system tests. Leaving aside whether I agree with his diagnosis or not, one result was a welcome effort to make Rails integration tests faster so that they could replace controller tests. See the talk How To Performance by Rails core team member Eileen Uchitelle, for some idea of the effort involved.

So, in Rails 5.0, integration tests got faster, and in Rails 5.1, Capybara integration will happen by default, for more seamless integration testing.

(A quick shameless self-promotional aside, since it’s brand new: you’ll be able to read about Capybara integration in Rails 5.1 in the updated Rails 5 Test Prescriptions, which, if we’re lucky, will come out about the same time.)

Controller testing, then, has been eaten up from both sides from the pyramid. From above, full stack integration testing, which is (arguably) more meaningful but slower, got faster. From below, either you code like DHH and don’t write unit tests any more (which makes for an odd pyramid), or you code like me and you don’t have unit logic to test in controllers. In either case, you are less likely to reach for controller tests as a tool.

The takeaway, I think is:

• Controller testing is really de-emphasized in Rails 5 (and the associated RSpec release), because controller testing often overlaps with either integration testing or unit testing.
• Integration testing in Rails 5 should be much better than in the past.
• If you have enough logic in your controller that it warrants testing, you might want to think about how else to structure that code.
• DHH and I disagree on how valuable unit tests are.

Hope that helps! Please feel free to Ask A Tester Person via @noelrap.