17

Is there any way to write something like a "unit test" which makes sure some code does not compile?

Why would I want such a thing? Two reasons.

1) Check the type safety of my API. I'd like a way to make sure if someone passes in a bad value, you get a compiler error, not just a runtime error. Obviously, I can just run the compiler and check for the error, but having it formalized in a unit test is good for avoiding a regression & also for documentation.

Eg., consider this test. There is some commented out code which I had used to check type-safety: https://github.com/squito/boxwood/blob/master/core/src/test/scala/com/quantifind/boxwood/EnumUnionTest.scala#L42 (lines 42 & 48 -- on line 34 I call a different API which has a runtime exception, which I can check)

It actually took me a while to get the type-safety right, so those were important checks. Now if I go and modify the underlying implementation, I can't just run my test suite -- I've got to also remember to uncomment those lines and check for a compiler error.

2) Testing error handling of macros. If a macro has some bad input, it should result in a compiler error. Same issues here, same desire to have it in a easy-to-run test-suite.

I use ScalaTest, but I'm happy to here a solution with any unit-testing framework.

3

3 Answers 3

11

As I note in a comment above, Shapeless 2.0 (not yet released but currently available as a milestone) has a very nice implementation of the functionality you're looking for, based on a solution by Stefan Zeiger. I've added a demo to your project here (note that I've had to update to Scala 2.10, since this solution uses a macro). It works like this:

import shapeless.test.illTyped

//this version won't even compile
illTyped("getIdx(C.Ooga)")

//We can have multiple enum unions exist side by side
import Union_B_C._
B.values().foreach {b => Union_B_C.getIdx(b) should be (b.ordinal())}
C.values().foreach {c => Union_B_C.getIdx(c) should be (c.ordinal() + 2)}

//Though A exists in some union type, Union_B_C still doesn't know about it,
// so this won't compile
illTyped("""
  A.values().foreach {a => Union_B_C.getIdx(a) should be (a.ordinal())}
""")

If we were to change the code in the second call to illTyped to something that will compile:

B.values().foreach {a => Union_B_C.getIdx(a) should be (a.ordinal())}

We'd get the following compilation error:

[error] .../EnumUnionTest.scala:56: Type-checking succeeded unexpectedly.
[error] Expected some error.
[error]     illTyped("""
[error]             ^
[error] one error found
[error] (core/test:compile) Compilation failed

If you'd prefer a failed test, you could pretty easily adapt the implementation in Shapeless. See Miles's answer to my previous question for some addition discussion.

1
  • Thanks Travis, sorry for the duplicate question. You went above & beyond by modifying my code for me! Also, see my answer below, about problems using it in the repl, curious if you have any thoughts on that. Nov 24, 2013 at 0:04
9

Scalatest can also do this.

Checking that a snippet of code does not compile

Often when creating libraries you may wish to ensure that certain arrangements of code that represent potential “user errors” do not compile, so that your library is more error resistant. ScalaTest Matchers trait includes the following syntax for that purpose:

"val a: String = 1" shouldNot compile

If you want to ensure that a snippet of code does not compile because of a type error (as opposed to a syntax error), use:

"val a: String = 1" shouldNot typeCheck

Note that the shouldNot typeCheck syntax will only succeed if the given snippet of code does not compile because of a type error. A syntax error will still result on a thrown TestFailedException.

If you want to state that a snippet of code does compile, you can make that more obvious with:

"val a: Int = 1" should compile

Although the previous three constructs are implemented with macros that determine at compile time whether the snippet of code represented by the string does or does not compile, errors are reported as test failures at runtime.

0

Travis Brown's answer is absolutely correct. Just in the interest of completeness, I want to add that this also works for testing macros, as demonstrated here.

One minor point: the illTyped check doesn't seem to work in the repl. It never throws an error, even if the given expression does type check. But don't let that fool you, it does work well.

> test:console
Welcome to Scala version 2.10.2 (Java HotSpot(TM) 64-Bit Server VM, Java 1.6.0_65).
Type in expressions to have them evaluated.
Type :help for more information.

scala> def foo(s: String) = s
foo: (s: String)String

scala> import shapeless.test.illTyped
import shapeless.test.illTyped

scala> foo(1)
<console>:10: error: type mismatch;
 found   : Int(1)
 required: String
              foo(1)
                  ^

scala> illTyped("""foo(1)""")

scala> illTyped("""foo("hi there")""")  // <--- works, but shouldn't!
1
  • The problem is that in the REPL the context that the code is being type-checked in isn't necessarily what you'd expect. illTyped("2 + 2") will fail as expected, as will well-typed code using any available classes defined outside of the REPL. The macro context doesn't see the definition of foo, though, so it considers foo("hi there") ill-typed and lets it past. I admit this is confusing, but it's entirely a REPL issue. Nov 24, 2013 at 0:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.