32

I am not understanding how the TDD FIRST principle isn't being adhered to in the following code.

These are my notes about the FIRST principle:

  • Fast: run (subset of) tests quickly (since you'll be running them all the time)
  • Independent: no tests depend on others, so can run any subset in any order
  • Repeatable: run N times, get same result (to help isolate bugs and enable automation)
  • Self-checking: test can automatically detect if passed (no human checking of output)
  • Timely: written about the same time as code under test (with TDD, written first!)

The quiz question:

Sally wants her website to have a special layout on the first Tuesday of every month. She has the following controller and test code:

# HomeController
 def index
   if Time.now.tuesday?
     render 'special_index'
   else
     render 'index'
   end
 end

 # HomeControllerSpec
 it "should render special template on Tuesdays" do
   get 'index'
   if Time.now.tuesday?
     response.should render_template('special_index')
   else
     response.should render_template('index')
   end
 end

What FIRST principle is not being followed?

  1. Fast
  2. Independent
  3. Repeatable
  4. Self-checking
  5. Timely

I'm not sure which FIRST principle is not being adhered to:

  • Fast: The code seems to be fast because there is nothing complex about its tests.
  • Independent: The test doesn't depend on other tests.
  • Repeatable: The test will get the same result every time. 'special_index' if it's Tuesday and 'index' if it's not Tuesday.
  • Self-checking: The test can automatically detect if it's passed.
  • Timely: Both the code and the test code are presented here at the same time.

I chose Timely on the quiz because the test code was presented after the controller code. But I got the question wrong, and in retrospect, this wasn't a good choice. I'm not sure which FIRST principle isn't being followed here.

3 Answers 3

46

It's not Repeatable as not everyday is Tuesday :) If you run this test on Monday you will get one result, if you run it on Tuesday, a different one.

2
  • It would be adherent if there was a separate test for each day of the week then, right?
    – Geeky Guy
    Aug 2, 2013 at 19:36
  • 17
    @Renan: No, it would be adherent if one could inject a date and verify that it did the right thing when it thought that's what day it is. At that point you could comprehensively test, all at once and regardless of the current date, rather than waiting up to a week and testing each day.
    – cHao
    Aug 2, 2013 at 19:38
3

F.I.R.S.T., F.I.I.R.S.T. and FASRCS

Yes, the confusion partly has the reason, that the F.I.R.S.T. principle is not complete or concise enough concerning the "I". In courses I attended the principle was called F.I.I.R.S.T.

The second "I" stands for "Isolated". The test above is independent from other tests, but is not isolated in a separate class or project.

[Updated]:

Isolation can mean:

  • A unit tests isolates functionality out of a SUT (system under test). You can isolate functionality even out of one single function. This draws the line between unit tests or their relatives component tests and integration tests, and of course to system tests.

  • "Tests isolate failures. A developer should never have to reverse-engineer tests or the code being tested to know what went wrong. Each test class name and test method name with the text of the assertion should state exactly what is wrong and where." Ref.: History of FIRST principle

  • A unit test could be isolated from the SUT which it tests in a different developer artifact (class, package, development project) and/or delivery artifact (Dll, package, assembly).

  • Unit tests, testing the same SUT, especially their containing Asserts, should be isolated from each other in different test functions, but this is only a recommendation. Ideally each unit test contains only one assert.

  • Unit tests, testing different SUTs, should be isolated from each other or from other kind of tests of course further more in different classes or other mentioned artifacts.

Independence can mean:

  • Unit tests should not rely on each other (explicit independency), with the exception of special "setup" and "teardown" functions, but even this is subject for a discussion.

  • Especially unit tests should be order-independant (implicit independency). The result should not depend on unit tests executed before. While this sounds trivial, it isn't. There are tests which cannot avoid doing initializations and/or starting runtimes. Just one shared (e.g. class) variable and the SUT could react differently, if it was started before. You make an outside call to the operating system? Some dll will be loaded first time? You already have a potential dependency, at least on OS level- sometimes only minor, sometimes essential to not discover an error. It may be necessary to add cleanup code to reach optimal independency of tests.

  • Unit tests should be independant as much as possible from the runtime environment and not depend on a specific test environment or setting. This belongs also partly to "Repeatable". No need to fillout twenty user dialogs before. No need to start the server. No need to make the database available. No need for another component. No need for a network. To accomplish that, often test doubles are used (stubs, mocks, fakes, dummies, spies, ..).

(Gerard Meszaros' classic work on: xUnit patterns, here coining the name 'test double' and defining different kinds of)

(Test double zoo quickly explained)

(Follow Martin Fowler 2007, thinking about stubs, mocks, etc. Classic)

  • While a unit test is never totally independant from it's SUT, ideally it should be independant as much as possible from the current implementation and only rely on the public interface of the function or class tested (SUT).

Conclusion: In this interpretations the word 'isolation' stresses more the physical location which often implies logical independence to some extent (e.g. class level isolation).

No completeness concerning potentially more accentuations and meanings claimed.

See also comments here.

But there are more properties of (good) unit tests: Roy Osherove found some more attributes in his book "The art of unit testing" which I don't find exactly in the F.I.I.R.S.T. principle (link to his book site), and which are cited here with my own words (and acronym):

  • Full control of SUT: the unit test should have full control of the SUT. I see this effectively identical as being independent from the test and runtime environment (e.g. using mocks, etc.). But because of independency is so ambigous, it makes sense to spend a separate letter.

  • Automated (related to repeatable and self-checking, but not the same) identically). This one requires a test (runner) infrastructure.

  • Small, simple or in his words: "easy to implement" (again, related, but not identical). Often related to "Fast"

  • Relevant: The test should be relevant tomorrow. This one of the most difficult requirements to acchieve, and depending of the "school" there may be a need for temporary unit tests during TDD too. When a test is only testing the contracts, this is acchieved, but that may be not enough for high code coverage requirements.

  • Consistent result: Effectively a result of "enough" independency. Consistency is, what some people include in "Repeatable". There is an essential overlap, but they are not identical.

  • Self-explaining: In terms of naming, structure of the whole test, and specifically of the syntax of the assert as the key line, it should be clear what is tested, and what could be wrong if a test fails. Related to "Tests isolate failures", see above.

Given all these spedific points, it should be more clear than before, that it is all but simple to write (good) unit tests.

4
  • wouldn't Isolated include Independent? At least that's my interpretation of the term. Feb 28, 2018 at 15:25
  • Both terms seem to mean something similar, but in detail there is a list what you could mean of them. Isolated means IMHO here primarily, that 1. Unit tests isolate functionality, ideally consisting only of one single Assert, and by no way the same as an integration or system test. 2. Unit tests are often isolated in a seperate class or even project/package/assembly, but to isolate them from the SUT doesn't necessarily mean, they are really independent from the SUT, obviously.
    – Philm
    Mar 1, 2018 at 15:42
  • Independence could be understood or stressed obviously differently. Independent from what? Complete independence of the test to the SUT? Not really! Independence from the implementation of SUT (compared to the public interface)? Yes, this is one important point of test strategy (and test lifetime), but not all unit tests fulfill that. Independence of run order of other tests is one of the key points. To be very petty, isolation of tests from each other is independent from their independence (esp. order independence) from each other :-)
    – Philm
    Mar 1, 2018 at 15:50
  • Ok, sorry for that joke. I mean, you can collect them in one class or even in one test function, that is something else. More of independence: Independence of the environment is an important property of a unit test. If you need a database or a connection a web site, it's not a unit test. This is related again to "Repeatable", but again doesn't mean the same. Ok?
    – Philm
    Mar 1, 2018 at 15:55
0

Independent and Repeatable

It is not independent from date and then it would able to run repeat but technically you get the same result because you choose to

The proper way to make a test for HomeController regarding to FIRST concept is change the time before the evaluation stage

1
  • 1
    I think "Independent" stands for: dependencies on other tests. This is not the case in the example. Oct 13, 2015 at 13:33

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.