Matchers

The most frequent way to specify some expected behaviour with specs2 is to use matchers. You generally execute an action, a command or a function and then check if the actual value you get is equal to an expected one (the “arrange-act-assert” paradigm).

For example, if you create a specification for an object manipulating paths:

// describe the functionality
s2"the directoryPath method should return well-formed paths $e1"

// give an example with some code
def e1 = Paths.directoryPath("/tmp/path/to/dir") must beEqualTo("/tmp/path/to/dir/")

The must operator takes the actual value returned by directoryPath and applies it to a Matcher built with the expected value. beEqualTo is one of the many matchers defined by specs2, it just checks if 2 values are equal.

In the following sections you will learn:

Equality

The most common type of matcher is beEqualTo to test the equality of 2 values. Several syntaxes can be used, according to your own taste

Matcher Comment
1 must beEqualTo(1) the normal way
1 must be_==(1) with a symbol
1 must_== 1 my favorite!
1 mustEqual 1 if you dislike underscores
1 should_== 1 for should lovers
1 === 1 the ultimate shortcut
1 must be equalTo(1) with a literate style

There are also other notions of equality

Matcher Comment
beTypedEqualTo typed equality. a must beTypedEqualTo(b) will not work if a and b don’t have compatible types
be_=== synonym for beTypedEqualTo
a ==== b synonym for a must beTypedEqualTo(b)
a must_=== b similar to a must_== b but will not typecheck if a and b don’t have the same type
be_==~ check if (a: A) == conv(b: B) when there is an implicit conversion conv from B to A
beTheSameAs reference equality: check if a eq b (a must be(b) also works)
be a must be(b): synonym for beTheSameAs
beTrue, beFalse shortcuts for Boolean equality

Note: the beEqualTo matcher is using the regular == Scala equality. However in the case of Arrays, Scala == is just using reference equality, eq. So the beEqualTo matcher has been adapted to use the .deep method on Arrays, transforming them to IndexedSeqs (possibly nested), before checking for equality, so that Array(1, 2, 3) === Array(1, 2, 3) (despite the fact that Array(1, 2, 3) != Array(1, 2, 3)).

Now let’s check the other matchers.

Out of the box

These are the all the available matchers when you extend Specification

Specification matchers

Matching on strings is very common. Here are the matchers which can help you:

Matcher Description
beMatching or be matching check if a string matches a regular expression
=~(s) shortcut for beMatching("(.|\\s)*"+s+"(.|\\s)*")
find(exp).withGroups(a, b, c) check if some groups are found in a string
have length check the length of a string
have size check the size of a string (seen as an Iterable[Char])
be empty check if a string is empty
beEqualTo(b).ignoreCase check if 2 strings are equal regardless of casing
beEqualTo(b).ignoreSpace check if 2 strings are equal when you replaceAll("\\s", "")
beEqualTo(b).trimmed check if 2 strings are equal when trimmed
beEqualTo(b).ignoreSpace.ignoreCase you can compose them
contain(b) check if a string contains another one
startWith(b) check if a string starts with another one
endWith(b) check if a string ends with another one

Traversables can be checked with several matchers. If you want to check the size of a Traversable

  • check if it is empty
      Seq() must be empty
      Seq(1, 2, 3) must not be empty

  • check its size
      Seq(1, 2) must have size(2)
      Seq(1, 2) must have length(2) // equivalent to size
      Seq(1, 2) must have size(be_>=(1)) // with a matcher

note: you might have to annotate the haveSize matcher when using some combinators. For example: (futures: Future[Seq[Int]]) must haveSize[Seq[Int]](1).await

  • check its ordering (works with any type T which has an Ordering)
      Seq(1, 2, 3) must beSorted

Check each element individually

Then you can check the elements which are contained in the Traversable

  • if a simple value is contained
      Seq(1, 2, 3) must contain(2)

  • if a value matching a specific matcher is contained
      Seq(1, 2, 3) must contain(be_>=(2))

  • if a value passing a function returning a Result is contained (MatchResult, ScalaCheck Prop,...)
      Seq(1, 2, 3) must contain((i: Int) => i must be_>=(2))

  • note that a Seq[A] is also a function Int => A so if you want to check that a sequence is contained in another you need to use a matcher
      Seq(Seq(1)) must contain(===(Seq(1)))

  • there are also 2 specialized matchers to check the string representation of the elements
      Seq(1234, 6237) must containMatch("23") // matches with ".*23.*"
      Seq(1234, 6234) must containPattern(".*234") // matches with !.*234"

For each of the check above you can indicate how many times the check should be satisfied:

  • Seq(1, 2, 3) must contain(be_>(0)).forall // this will stop after the first failure
  • Seq(1, 2, 3) must contain(be_>(0)).foreach // this will report all failures
  • Seq(1, 2, 3) must contain(be_>(0)).atLeastOnce
  • Seq(1, 2, 3) must contain(be_>(2)).atMostOnce
  • Seq(1, 2, 3) must contain(be_>(2)).exactly(1.times)
  • Seq(1, 2, 3) must contain(be_>(2)).exactly(1)
  • Seq(1, 2, 3) must contain(be_>(1)).between(1.times, 2.times)
  • Seq(1, 2, 3) must contain(be_>(1)).between(1, 2)

Check all elements

The other types of checks involve comparing the Traversable elements to other elements (values, matchers, function returning a Result)

  • with a set of values
      Seq(1, 2, 3, 4) must contain(2, 4)
      which is the same thing as
      Seq(1, 2, 3, 4) must contain(allOf(2, 4))

  • with a set of matchers
      Seq(1, 2, 3, 4) must contain(allOf(be_>(0), be_>(1)))

  • checking that the order is satisfied
      Seq(1, 2, 3, 4) must contain(allOf(be_>(0), be_>(1)).inOrder)

Note that allOf tries to make each check be successful at least once, even if it is on the same value. On the other hand, if you want to specify that each check must succeed on a different value you should use onDistinctValues. For example this will fail:
Seq(1) must contain(allOf(1, 1)).onDistinctValues

The eachOf method does the same thing (and this example will fail as well):
Seq(1) must contain(eachOf(1, 1))

Another frequent use of Traversable matchers is to check if the Traversable have the right number of elements. For this you can use:

  • atLeast, which is actually another name for allOf, where the traversable can contain more elements than required
      Seq(1, 2, 3, 4) must contain(atLeast(2, 4))

  • atMost where the traversable can not contain more elements than required
      Seq(2, 3) must contain(atMost(2, 3, 4))

  • exactly where the traversable must contain exactly the specified number of elements
      Seq(1, 2) must contain(exactly(2, 1))

The atLeast/atMost/exactly operators work on distinct values by default (because this is easier for counting the correspondence between actual values and expected ones). However you can use onDistinctValues(false) if you don't care.

Finally, if you want to get the differences between 2 traversables:

Seq(2, 4, 1) must containTheSameElementsAs(Seq(1, 4, 2))

Numerical values can be compared with the following matchers

  • beLessThanOrEqualTo compares any Ordered type with <=
        1 must be_<=(2)
        1 must beLessThanOrEqualTo(2)

  • beLessThan compares any Ordered type with <
        1 must be_<(2)
        1 must beLessThan(2)

  • beGreaterThanOrEqualTo compares any Ordered type with >=
        2 must be_>=(1)
        2 must beGreaterThanOrEqualTo(1)

  • beGreaterThan compares any Ordered type with >
        2 must be_>(1)
        2 must beGreaterThan(1)

  • beCloseTo check if 2 Numerics are close to each other
        1.0 must beCloseTo(1, 0.5)
        4 must be ~(5 +/- 2)
        4.994 must beCloseTo(5.0 within 2.significantFigures)

  • beBetween check if a value is between 2 others
        5 must beBetween(3, 6)
        5 must beBetween(3, 6).excludingEnd
        5 must beBetween(4, 6).excludingStart
        5 must beBetween(4, 6).excludingBounds
        // with brackets notation
        5 must (be[(4, 7)])
        5 must (be[(4, 6)[)
                                                                                                                   .

There are several matchers to check Option and Either instances:

  • beSome check if an element is Some(_)
  • beSome(exp) check if an element is Some(exp)
  • beSome(matcher) check if an element is Some(a) where a satisfies the matcher
  • beSome(function: A => AsResult[B]) check if an element is Some(a) where function(a) returns a successful Result
    (note that a Seq[A] is also a function Int => A so if you want to check that a sequence is contained in Some you need to use a matcher: beSome(===(Seq(1)))
  • beSome.which(function) check if an element is Some(_) and satisfies a function returning a boolean
  • beSome.like(partial function) check if an element is Some(_) and satisfies a partial function returning a MatchResult
  • beNone check if an element is None
  • beAsNoneAs check if 2 values are equal to None at the same time
  • beRight check if an element is Right(_)

  • beRight(exp) check if an element is `Right(exp)
  • beRight(matcher) check if an element is Right(a) where a satisfies the matcher
  • beRight(function: A => AsResult[B]) check if an element is Right(a) where function(a) returns a successful Result
    (note that a Seq[A] is also a function Int => A so if you want to check that a sequence is contained in Right you need to use a matcher: beRight(===(Seq(1)))
  • beRight.like(partial function) check if an element is Right(_) and satisfies a partial function returning a MatchResult
  • beLeft check if an element is Left(_)

  • beLeft(exp) check if an element is Left(exp)
  • beLeft(matcher) check if an element is Left(a) where a satisfies the matcher
  • beLeft(function: A => AsResult[B]) check if an element is Left(a) where function(a) returns a successful Result
    (note that a Seq[A] is also a function Int => A so if you want to check that a sequence is contained in Left you need to use a matcher: beLeft(===(Seq(1)))
  • beLeft.like(partial function) check if an element is Left(_) and satisfies a partial function returning a MatchResult

There are several matchers to check Try instances:

  • beSuccessfulTry check if an element is Success(_)
  • beSuccessfulTry.withValue(exp) check if an element is Success(_)
  • beSuccessfulTry.withValue(matcher) check if an element is Success(a) where a satisfies the matcher
  • beSuccessfulTry.withValue(function: A => AsResult[B]) check if an element is Success(a) where function(a) returns a successful Result
    (note that a Seq[A] is also a function Int => A so if you want to check that a sequence is contained in Success you need to use a matcher: beSuccessfulTry.withValue(===(Seq(1)))
  • beSuccessfulTry.which(function) check if an element is Success(_) and satisfies a function returning a boolean
  • beSuccessfulTry.like(partial function) check if an element is Success(_) and satisfies a partial function returning a MatchResult
  • beFailedTry check if an element is Failure(_)
  • beFailedTry.withThrowable[T] check if an element is Failure(t: T)
  • beFailedTry.withThrowable[T](pattern) check if an element is Failure(t: T) and t.getMessage matches pattern

Testing Futures is quite easy with specs2. You can transform any Matcher[T] into a Matcher[Future[T] with the await method

Future(1) must be_>(0).await

You can also specify a timeout value and a number of retries

Future { Thread.sleep(100); 1 } must be_>(0).await(retries = 2, timeout = 100.millis)

// only retries, timeout is 1.second
Future { Thread.sleep(100); 1 } must be_>(0).retryAwait(retries = 2)

// only timeout, retries = 0
Future { Thread.sleep(100); 1 } must be_>(0).awaitFor(100.millis)

Another possibility is for you to obtain a Future[MatchResult[T]] (or any Future[R] where R has an AsResult typeclass instance). In that case you can use await directly on the Future to get a Result

Future(1 === 1).await
Future(1 === 1).await(retries = 2, timeout = 100.millis)

Scalaz Futures

All of the above is applicable to scalaz.concurrent.Future by using the method attempt instead of await.

Execution

The await/attempt methods require an implicit org.specs2.concurrent.ExecutionEnv (see here for more details). You can pass one in the body of your examples:

class MyFutureSpec extends Specification { def is = s2"""

 Let's check this scala future ${ implicit ee: EE =>
   Future(1) must be_>(0).await
 }

 Let's check this scalaz future ${ implicit ee: EE =>
   scalaz.concurrent.Future(1) must be_>(0).attempt
 }

"""
  type EE = ExecutionEnv
}

// in a mutable specification
class MyMutableFutureSpec extends mutable.Specification {

  "Let's check this scala future" >> { implicit ee: EE =>
    Future(1) must be_>(0).await
  }

  "Let's check this scalaz future" >> { implicit ee: EE =>
    scalaz.concurrent.Future(1) must be_>(0).attempt
  }

  type EE = ExecutionEnv
}

Time factor

Some actions can be a lot slower when executed on a continuous integration server rather than a developer machine and some timeouts will fail.
You can avoid this by setting the timeFactor argument which will multiply the durations used when awaiting / attempting by a constant factor.

sbt> testOnly *MyFuturesSpec* -- timeFactor 3

Now learn how to... {.ribbon .both-ribbon}

specs2 offers very compact ways of checking that some exceptions are thrown:

  • throwA[ExceptionType] check if a block of code throws an exception of the given type
  • throwA[ExceptionType](message = "boom") additionally check if the exception message is as expected
  • throwA(exception) or throwAn(exception) check if a block of code throws an exception of the same type, with the
      same message
  • throwA[ExceptionType].like { case e => e must matchSomething } or
      throwA(exception).like { case e => e must matchSomething } check that the thrown exception satisfies a property
  • throwA[ExceptionType](me.like { case e => e must matchSomething } or
      throwA(exception).like { case e => e must matchSomething } check that the thrown exception satisfies a property

For all the above matchers you can use throwAn instead of throwA if the exception name starts with a vowel for better
readability.

Maps have their own matchers as well, to check keys and values:

  • haveKey check if a Map has a given key
        Map(1 -> "1") must haveKey(1)

  • haveKeys check if a Map has several keys
        Map(1 -> "1", 2 -> "2") must haveKeys(1, 2)

  • haveValue check if a Map has a given value
        Map(1 -> "1") must haveValue("1")

  • haveValues check if a Map has several values
        Map(1 -> "1", 2 -> "2") must haveValues("1", "2")

  • havePair check if a Map has a given pair of values
        Map(1 -> "1") must havePair(1 -> "1")

  • havePairs check if a Map has some pairs of values
        Map(1->"1", 2->"2", 3->"3") must havePairs(1->"1", 2->"2")

But Maps are also Partial Functions, so:

  • beDefinedAt check if a PartialFunction is defined for a given value
        partial must beDefinedAt(1)

  • beDefinedBy check if a PartialFunction is defined for a given value
      and returns another one
        partial must beDefinedBy(1 -> true)

These matchers can be used with any object, regardless of its type:

  • beLike { case exp => result } check if an object is like a given pattern. result can be any expression using a matcher
  • beLike { case exp => exp must beXXX } check if an object is like a given pattern, and verifies a condition
  • beNull check if an object is null
  • beAsNullAs when 2 objects must be null at the same time if one of them is null
  • beOneOf(a, b, c) check if an object is one of a given list
  • haveClass check the class of an object
  • haveSuperclass check if the class of an object as another class as one of its ancestors
  • haveInterface check if an object is implementing a given interface
  • beAssignableFrom check if a class is assignable from another
  • beAnInstanceOf[T] check if an object is an instance of type T

Optional

Those matchers are optional. To use them, you need to add a new trait to your specification.

Those are additional “data” matchers

Optional data matchers

There are several matchers to check scalaz.\/ instances:

  • be_\/- checks if an element is \/-(_)
  • be_\/-(exp) checks if an element is `\/-(exp)
  • be_\/-(matcher) checks if an element is \/-(a) where a satisfies the matcher
  • be_\/-(function: A => AsResult[B]) checks if an element is \/-(a) where function(a) returns a successful Result
    (note that a Seq[A] is also a function Int => A so if you want to check that a sequence is contained in \/- you need to use a matcher: be_\/-(===(Seq(1)))
  • be_\/-.like(partial function) checks if an element is \/-(_) and satisfies a partial function returning a MatchResult
  • be_-\/ checks if an element is -\/(_)

  • be_-\/(exp) checks if an element is -\/(exp)
  • be_-\/(matcher) checks if an element is -\/(a) where a satisfies the matcher
  • be_-\/(function: A => AsResult[B]) checks if an element is -\/(a) where function(a) returns a successful Result
    (note that a Seq[A] is also a function Int => A so if you want to check that a sequence is contained in -\/ you need to use a matcher: be_-\/(===(Seq(1)))
  • be_-\/.like(partial function) checks if an element is -\/(_) and satisfies a partial function returning a MatchResult

There are several matchers to check scalaz.Validation instances:

  • beSuccess checks if an element is Success(_)
  • beSuccess(exp) checks if an element is `Success(exp)
  • beSuccess(matcher) checks if an element is Success(a) where a satisfies the matcher
  • beSuccess(function: A => AsResult[B]) checks if an element is Success(a) where function(a) returns a successful Result
    (note that a Seq[A] is also a function Int => A so if you want to check that a sequence is contained in Success you need to use a matcher: beSuccess(===(Seq(1)))
  • beSuccess.like(partial function) checks if an element is Success(_) and satisfies a partial function returning a MatchResult
  • beFailure checks if an element is Failure(_)

  • beFailure(exp) checks if an element is Failure(exp)
  • beFailure(matcher) checks if an element is Failure(a) where a satisfies the matcher
  • beFailure(function: A => AsResult[B]) checks if an element is Failure(a) where function(a) returns a successful Result
    (note that a Seq[A] is also a function Int => A so if you want to check that a sequence is contained in Failure you need to use a matcher: beFailure(===(Seq(1)))
  • beFailure.like(partial function) checks if an element is Failure(_) and satisfies a partial function returning a MatchResult

There is a special support for matching case classes, using a matcher macro. To use it you need to add the specs2-matcher-extra jar to your project and add the org.specs2.matcher.MatcherMacros trait to your specification.

Then, with the matchA matcher you can check the values of case class attributes:

// case class for a Cat
case class Cat(name: String = "", age: Int = 0, kitten: Seq[Cat] = Seq())
// a given cat
val cat = Cat(name = "Kitty", age = 6, kitten = Seq(Cat("Oreo", 1), Cat("Ella", 2)))

// this cat must be a Cat
cat must matchA[Cat]

// check the value of "name"
cat must matchA[Cat].name("Kitty")

// check the value of "age" using a matcher
def is[A](a: A) = be_==(a)
cat must matchA[Cat].age(is(6))

// check the value of "kitten" using a function returning a Result
cat must matchA[Cat].kitten((_:Seq[Cat]) must haveSize(2))

// matchers can be chained
cat must matchA[Cat]
  .name("Kitty")
  .age(is(6))
  .kitten((_:Seq[Cat]) must haveSize(2))

And if you want to know more {.ribbon .both-ribbon}

That's only if you want to check the result of other matchers!

// you need to extend the ResultMatchers trait
class MatchersSpec extends Specification with matcher.ResultMatchers { def is =
  "beMatching is using a regexp" ! {
    ("Hello" must beMatching("h.*")) must beSuccessful
  }
}

Sometimes you just want to specify that a block of code is going to terminate. The org.specs2.matcher.TerminationMatchers trait is here to help. If you mix in that trait, you can write:

Thread.sleep(100) must terminate

// the default is retries = 0, sleep = 100.millis
Thread.sleep(100) must terminate(retries = 1, sleep = 60.millis)

Note that the behaviour of this matcher is a bit different from the eventually operator. In this case, we let the current Thread sleep during the given sleep time and then we check if the computation is finished, then, we retry for the given number of retries.

In a further scenario, we might want to check that triggering another action is able to unblock the first one:

action1 must terminate.when(action2)
action1 must terminate.when("starting the second action", action2)
action1 must terminate(retries=3, sleep=100.millis).when(action2)

When a second action is specified like that, action1 will be started and action2 will be started on the first retry. Otherwise, if you want to specify that action1 can only terminate when action2 is started, you write:

action1 must terminate.onlyWhen(action2)

ExecutionEnv

The terminate matcher needs an implicit ExecutionEnv to be used. See the page to learn how to get one.

Those matchers can be used to check “content”

Optional content matchers

It is very useful to have literal Xml in Scala, it is even more useful to have matchers for it! If you want to use those matchers you need to extend the org.specs2.matcher.XmlMatchers trait:

  • beEqualToIgnoringSpace compares 2 Nodes, without considering spaces
      <a><b/></a> must ==/(<a> <b/></a>)
      <a><b/></a> must beEqualToIgnoringSpace(<a> <b/></a>)

  • beEqualToIgnoringSpace can also do an ordered comparison
      <a><c/> <b/></a> must ==/(<a> <c/><b/></a>).ordered

  • on the other hand beEqualToIgnoringSpace will not check attributes order
      <n a="1" b="2"/> must ==/(<n b="2" a="1"/>)

  • the \\ matcher is an XPath-like matcher matching if a node is a direct child of another
      <a><b/></a> must \\\\("b")

  • You can also check attribute names
        <a><b name="value"></b></a> must \\("b", "name")

  • And attribute names and values as well (values are checked using a regular expression, use the quote method if you want an exact match)
      <a><b n="v" n2="v2" n3="v3"></b></a> must \\("b", "n"->"v", "n2"->"v\\d")

  • Or the content of a Text node
      <a>hello</a> must \\("a") \\> "hello" (alias textIs)
      <a>hello</a> must \\("a") \\>~ "h.*" (alias textMatches)

  • The equivalent of \\ for a "deep" match is simply \\\\
      <a><s><c></c></s></a> must \\\\("c")

Json is a simple data format essentially modeling recursive key-values. There are 2 matchers which can be used to verify the presence of appropriate values in Strings representing Json documents:

  • /(value) checks if a value is present at the root of the document. This can only be the case if that document is an Array

  • /(regex) checks if a value matching the regex is present at the root of the document. This can only be the case if that document is an Array

  • /(key -> value) checks if a pair is present at the root of the document. This can only be the case if that document is a Map

  • */(value) checks if a value is present anywhere in the document, either as an entry in an Array, or as the value for a key in a Map

  • */(key -> value) checks if a pair is present anywhere in a Map of the document

  • /#(i) selects the ith element in a 0-based indexed Array or a Map and allow further checks on that element

Now the interesting part comes from the fact that those matchers can be chained to search specific paths in the Json document. For example, for the following document:

// taken from an example in the Lift project
val person = """{
  "person": {
    "name": "Joe",
    "age": 35,
    "spouse": {
      "person": {
        "name": "Marilyn",
        "age": 33
      }
    }
  }
}
"""

You can use these combinations:

person must /("person") */("person") /("age" -> 33.0) // by default numbers are parsed as Doubles

You can as well use regular expressions or String matchers instead of values to verify the presence of keys or elements. For example:

person must /("p.*".r) */ ".*on".r /("age" -> "\\d+\\.\\d".r)
person must /("p.*".r) */ ".*on".r /("age" -> startWith("3"))
person must /("p.*".r) */ ".*on".r /("age" -> (be_>(30) ^^ ((_:String).toInt)))

You can also access some records by their index:

person must /("person") /# 2 / "person"

Finally you can use Json matchers to match elements in an array:

val json = """{"products":[{"name":"shirt","price":10, "ids":["1", "2", "3"]},{"name":"shoe","price":5}]}"""

def aProductWith(name: Matcher[JsonType],  price: Matcher[JsonType]): Matcher[String] =
  /("name").andHave(name) and /("price").andHave(price)

def haveProducts(products: Matcher[String]*): Matcher[String] =
  /("products").andHave(allOf(products:_*))

json must haveProducts(
  aProductWith(name = "shirt", price = 10) and /("ids").andHave(exactly("1", "2", "3")),
  aProductWith(name = "shoe", price = 5)
)

The andHave method accepts any Matcher[JsonType] where JsonType is either JsonArray, JsonMap, JsonNumber, JsonString, JsonNull. In the example above we pass directly shirt and 10 as Matcher[JsonType] because there are implicit conversions from Int, Boolean, Double, String and Traversable[String] matchers (like allOf) to a Matcher[JsonType].

The Java api for files is more or less mimicked as matchers which can operate on strings denoting paths or on Files (with the org.specs2.matcher.FileMatchers trait)

  • beEqualToIgnoringSep check if 2 paths are the same regardless of their separators
      "c:\temp\hello" must beEqualToIgnoringSep("c:/temp/hello")
  • beAnExistingPath check if a path exists
  • beAReadablePath check if a path is readable
  • beAWritablePath check if a path is writable
  • beAnAbsolutePath check if a path is absolute
  • beAHiddenPath check if a path is hidden
  • beAFilePath check if a path is a file
  • beADirectoryPath check if a path is a directory
  • havePathName check if a path has a given name
  • haveAsAbsolutePath check if a path has a given absolute path
  • haveAsCanonicalPath check if a path has a given canonical path
  • haveParentPath check if a path has a given parent path
  • listPaths check if a path has a given list of children
  • exist check if a file exists
  • beReadable check if a file is readable

  • beWritable check if a file is writable
  • beAbsolute check if a file is absolute
  • beHidden check if a file is hidden
  • beAFile check if a file is a file
  • beADirectory check if a file is a directory
  • haveName check if a file has a given name
  • haveAbsolutePath check if a file has a given absolute path
  • haveCanonicalPath check if afile has a given canonical path
  • haveParent check if a file has a given parent path
  • haveList check if a file has a given list of children
File contents

The matchers from the org.specs2.matcher.ContentMatchers trait can help us check the contents of files. For example we can check that 2 text files have the same lines:

(file1, file2) must haveSameLines
file1 must haveSameLinesAs(file2)

We can check that the content of one file is contained in another one:

file1 must containLines(file2)

If the files are binary files we can also check that they have the same MD5 hash:

(file1, file2) must haveSameMD5
 file1 must haveSameMD5As(file2)

Order

It is possible to relax the constraint by requiring the equality or containment to be true regardless of the order of lines:

(file1, file2) must haveSameLines.unordered
 file1 must haveSameLinesAs(file2).unordered
 file1 must containLines(file2).unordered

Missing only

By default, (file1, file2) must haveSameLines will report misplaced lines if any, that is, lines of f1 which appear in f2 but not at the right position. However if file2 is big, this search might degrade the performances. In that case you can turn it off with missingOnly:

(file1, file2) must haveSameLines.missingOnly

Show less differences

If there are too many differences, you can specify that you only want the first 10:

(file1, file2) must haveSameLines.showOnly(10.differences).unordered

In the code above 10.differences builds a DifferenceFilter which is merely a filtering function: (lines1: Seq[String], lines2: Seq[String]) => (Seq[String], Seq[String]). The parameter lines1 is the sequence of lines not found in the second content while lines2 is the sequence of lines not found in the first content.

Directories contents

We can compare the contents of 2 directories. We can for example check if no files are missing and none has been added:

actualDir must haveSamePathsAs(expectedDir)
// with a file filter applied to both the actual and expected directories
actualDir must haveSamePathsAs(expectedDir).withFilter((file: File) => !file.isHidden)

Once we know that all files are present we can check their content:

// the default comparison expects that files are text files and that comparison must be done line by line
actualDir must haveSameFilesAs(expectedDir)

// with a file filter applied to both the actual and expected directories
actualDir must haveSameFilesAs(expectedDir).withFilter((file: File) => !file.isHidden)

// with a MD5 matcher for binary files
actualDir must haveSameFilesAs(expectedDir).withMatcher(haveSameMD5)

// it is also possible to only check the content of actual files when they exist in the expected directory
actualDir must haveSameFilesContentAs(expectedDir)
Lines contents

Files are not the only possible source of lines and it is useful to be able to check the content of a File with a Seq[String]:

file1 must haveSameLinesAs(Seq(line1, line2, line3))

This is because those 2 types implement the org.specs2.text.LinesContent trait, defining:

  • a name for the overall content
  • a method for returning the lines
  • a default method for computing the differences of 2 sequences of lines (in case you need to override this logic)

So if you have a specific type T which you can represent as a Seq[String], you can create an implicit LinesContent and then you'll be able to use the ContentMatchers:

implicit def linesforMyType[T]: LinesContent[T] = new LinesContent[T] {
  def name(t: T) = "My list of lines"
  def lines(t: T): Seq[String] = Seq()// your implementation goes here
}

And finally those matchers are Scala / Language related

Optional language matchers

Scala provides a parsing library using parser combinators.

To specify your own parsers you need to:

  • extend the org.specs2.matcher.ParserMatchers trait
  • associate the val parsers variable with your parsers definition
  • use the beASuccess, beAFailure, succeedOn, failOn, errorOn matchers to specify the results of parsing input
      strings. beAPartialSuccess, be aPartialSuccess, succeedOn.partially will allow a successful match only on part of the input
  • use haveSuccessResult and haveFailureMsg to specify what happens only on success or failure. Those matchers accept
      a String or a matcher so that
      . haveSuccessResult("r") <==> haveSuccessResult(beMatching(".*r.*") ^^ ((_:Any).toString)
      . haveFailingMsg("m") <==> haveFailingMsg(beMatching(".*r.*"))

For example, specifying a Parser for numbers could look like this:

import NumberParsers.{error, number}

import scala.util.parsing.combinator.RegexParsers

class ParserSpec extends Specification with matcher.ParserMatchers {  def is = s2"""
  Parsers for numbers

    beASuccess and succeedOn check if the parse succeeds
    ${ number("1") must beASuccess }
    ${ number("1i") must beAPartialSuccess }
    ${ number must succeedOn("12") }
    ${ number must succeedOn("12ab").partially }
    ${ number must succeedOn("12").withResult(12) }
    ${ number must succeedOn("12").withResult(equalTo(12)) }
    ${ number("1") must haveSuccessResult("1") }

    beAFailure and failOn check if the parse fails
    ${ number must failOn("abc") }
    ${ number must failOn("abc").withMsg("string matching regex.*expected") }
    ${ number must failOn("abc").withMsg(matching(".*string matching regex.*expected.*")) }
    ${ number("i") must beAFailure }
    ${ number("i") must haveFailureMsg("i' found") }

    beAnError and errorOn check if the parser errors out completely
    ${ error must errorOn("") }
    ${ error("") must beAnError }
                                                                                """

  val parsers = NumberParsers
}

Typecheck matchers

Some behaviours can be encoded with the type system and you just want to check that a given expression will typecheck:

import org.specs2.execute._, Typecheck._
import org.specs2.matcher.TypecheckMatchers._

typecheck {
    """
  // there must be a Monoid instance for Text
  Monoid[Text].zero
    """
} must succeed

You might also want to check that another expression will fail to typecheck:

typecheck {
  """
// there must be not a Monoid instance for Plane
Monoid[Plane].zero
  """
} must not succeed

typecheck {
  """
// there must be not a Monoid instance for Plane
Monoid[Plane].zero
  """
} must failWith("could not find implicit value for .* scalaz.Monoid")

Typechecked

Note that you actually don't have to use the succeed matcher because typecheck returns a Typechecked object which has an AsResult instance:

"this should typecheck ok" ! typecheck {
"""
// there must be not a Monoid instance for Plane
Monoid[Plane].zero
"""
}

This is also why you can indicate that a block of code must be marked as Pending until it typechecks:

typecheck {
  """
// there must be not a Monoid instance for Plane
Monoid[Plane].zero
  """
}.pendingUntilFixed("find a way to make this typecheck!")

Interpolated string

Another way of checking if a piece of code typechecks is to use the tc string interpolator:

tc" 1 must_== 1"

The code above uses a macros which will

  • typecheck the expression
  • replace it with the corresponding code if the expression typechecks ok
  • otherwise it will create a Typechecked object containing the reason for the failure

Note that this interpolator can not take variables at the moment (so it doesn't interpolate much :-))

Typechecking parameters

It is possible to tweak the behaviour of the typecheck method to allow errors to be reported at compile-time instead of runtime:

  • macro-expansion errors can be reported at compile-time (default behaviour is runtime)
  • implicit errors can be reported at compile-time (default behaviour is runtime)
  • parsing errors can be reported at runtime (default behaviour is compile-time)

Here is how to do it:

// to get macro errors at compile time
typecheckWith(macrosAtCompileTime)("badBadMacro")

// to get macro errors at compile time
typecheckWith(implicitsAtCompileTime)("Monoid[Plane]")

// to get parsing errors at run-time
typecheckWith(parsingAtRuntime)("#$p6")

// combine parameters
typecheckWith(macrosAtCompileTime <| implicitsAtCompileTime)("Monoid[Plane]")

Finally you can also use a string interpolator and pass parameters:

  tcw" 1 must_== 1"(parsingAtRuntime)

In the rare case where you want to use the Scala interpreter and execute a script:

class ScalaInterpreterMatchersSpec extends mutable.Specification with ScalaInterpreterMatchers {
  def interpret(s: String): String = "" // you have to provide your own Scala interpreter here

  "A script" can {
    "be interpreted" in {
      "1 + 1" >| "2"
    }
  }
}

It is highly desirable to have acyclic dependencies between the packages of a project. This often leads to describing the packages structure as "layered": each package on a layer can only depend on a package on a lower layer. specs2 helps you enforce this design property with specific matchers.

Layers definition

First you need to define the packages and their expected dependencies. Mix-in the org.specs2.specification.Analysis trait and define layers (taking specs2 as an example):

layers (
  "runner",
  "reporter",
  "specification mutable",
  "mock      form",
  "matcher",
  "execute",
  "reflect    xml  time html",
  "collection control io text main data").withPrefix("org.specs2")

The above expression defines layers as an ordered list of Strings containing space-separated package names. It is supplemented by a withPrefix declaration to factor out the common package prefix between all these packages.

By default, the packages are supposed to correspond to directories in the src/target/scala-<version>/classes directory. If your project has a different layout you can declare another target directory:

layers("...").inTargetDir("out" / "classes")

Inclusion/Exclusion

Every rule has exceptions :-). In some rare cases, it might be desirable to exclude a class from being checked on a given layer. To do this, you can use the include/exclude methods on the Layer class:

layers (
  "runner",
  "reporter",
  "specification mutable".exclude("mutable.SpecificationWithJUnit"),
  "mock      form",
  "matcher",
  "execute",
  "reflect  xml  time html",
  "collection control io text main data").withPrefix("org.specs2")

The include/exclude methods accept a list of regular expressions to:

  • exclude fully qualified class names (generally, only exclude will be necessary)
  • re-include fully qualified class names if the exclusion list is to big

Verification

Now you've defined layers, you can use the beRespected matcher to check if all the dependencies are verified:

val design = layers("...")
design must beRespected

If some dependencies are not respected:

[error] those dependencies are not satisfied:
[error] org.specs2.main x-> org.specs2.io because org.specs2.io.FileSystem -> org.specs2.main.Arguments
[error] org.specs2.main x-> org.specs2.io because org.specs2.io.FileSystem -> org.specs2.main.ArgumentsArgs

Layers as an Example

The org.specs2.specification.Analysis trait allows to directly embed the layers definition in a Specification and turn it into an Example:

class DependenciesSpec extends Specification with specification.Analysis { def is =
  "this is the application design" ^
    layers(
      "gui commandline",
      "controller",
      "backend"
    )
}

Derive matchers

The easiest way to create a new matcher is to derive it from an existing one. You can:

  • use logical operators
def beBetween(i: Int, j: Int) = be_>=(i) and be_<=(j)
  • “adapt” the actual value
// This matcher adapts the existing `be_<=` matcher to a matcher applicable to `Any`
def beShort1 = be_<=(5) ^^ { (t: Any) => t.toString.size }

// you can use aka to provide some information about the original value, before adaptation
def beShort2 = be_<=(5) ^^ { (t: Any) => t.toString.size aka "the string size" }

// !!! note: use a BeTypedEqualTo matcher when using aka and equality, otherwise you will be matching against Expectable[T] !!!
def beFive = be_===(5) ^^ { (t: Any) => t.toString.size aka "the string size" }

// The adaptation can also be done the other way around when it's more readable
def haveExtension(extension: =>String) = ((_:File).getPath) ^^ endWith(extension)
  • adapt the actual and expected values. This matcher compares 2 Human objects but set their wealth field to 0 so that the equals method will not fail on that field:
def beMostlyEqualTo = (be_==(_:Human)) ^^^ ((_:Human).copy(wealth = 0))
// then
Human(age = 20, wealth=1000) must beMostlyEqualTo(Human(age = 20, wealth=1)) // success
  • use eventually to try a match a number of times until it succeeds:
val iterator = List(1, 2, 3).iterator

// Use eventually(retries, n.millis) to use another number of tries and waiting time
iterator.next must be_==(3).eventually
  • use await to create a matcher that will match on Matcher[Future[T]] (this requires an execution environment):
Future(1) must be_>(0).await
Future { Thread.sleep(100); 1 } must be_>(0).await(retries = 2, timeout = 100.millis)
  • use attempt to create a matcher that will match on Matcher[scalaz.concurrent.Future[T]] (this requires an execution environment):
// see the Matchers-Futures reference card on how to get an ExecutionEnv
implicit val ee: ExecutionEnv = ???

  scalaz.concurrent.Future(1) must be_>(0).attempt
scalaz.concurrent.Future { Thread.sleep(100); 1 } must be_>(0).attempt(retries = 2, timeout = 100.millis)
  • use when or unless to apply a matcher only if a condition is satisfied:
1 must be_==(2).when(false)                        // will return a success
1 must be_==(2).unless(true)                       // same thing

1 must be_==(2).when(false, "don't check this")    // will return a success
1 must be_==(2).unless(true, "don't check this")   // same thing
  • use iff to say that a matcher must succeed if and only if a condition is satisfied:
1 must be_==(1).iff(true)                        // will return a success
1 must be_==(2).iff(true)                        // will return a failure
1 must be_==(2).iff(false)                       // will return a success
1 must be_==(1).iff(false)                       // will return a failure
  • use orSkip to return a Skipped result instead of a Failure if the condition is not satisfied
1 must be_==(2).orSkip
1 must be_==(2).orSkip("Precondition failed")    // prints "Precondition failed: '1' is not equal to '2'"
1 must be_==(2).orSkip((ko:String) => "BAD "+ko) // prints "BAD '1' is not equal to '2'"
  • use orPending to return a Pending result instead of a Failure if the condition is not satisfied
1 must be_==(2).orPending
1 must be_==(2).orPending("Precondition failed")    // prints "Precondition failed: '1' is not equal to '2'"
1 must be_==(2).orPending((ko:String) => "BAD "+ko) // prints "BAD '1' is not equal to '2'"
  • use zip operators to match each value of a tuple
type MyTuple = (String, String, String, Seq[(String, Double)])

val t1: MyTuple = ("a", "b", "c", Seq(("d", 1.01), ("e", 2.02)))
val t2: MyTuple = ("a", "b", "c", Seq(("d", 1.00), ("e", 2.00)))

// create a matcher by zipping matchers to the expected value
def beMatching(expected: MyTuple) = expected.zip(startWith, ===, ===, matchSequence)
// match the elements of a sequence with a zipped matcher using string equality for the first field and
// approximate Double equality for the second field
def matchSequence(expected: =>Seq[(String, Double)]) = expected.contain(_.zip(===, ==~)).inOrder

/** type inference doesn't work if this matcher, specialised to Double, is not defined */
def ==~(d: =>Double) = beCloseTo(d +/- 0.1)

t1 must beMatching(t2)

Create your own

The easiest way to create a new matcher is to create it from a function returning a tuple with a boolean and one or more messages:

// annotate the return type so that implicit conversions can transform your function into a Matcher object
// here just return a boolean and a failure message
def m1: Matcher[String] = { s: String =>
  (s.startsWith("hello"), s+" doesn't start with hello")
}

// with a success message and a failure message
def m2: Matcher[String] = { s: String =>
  (s.startsWith("hello"), s+" starts with hello", s+" doesn't start with hello")
}

// with a function taking the actual value for the failure message
def m3: Matcher[String] =
  ((_: String).startsWith("hello"), (_:String)+" doesn't start with hello")

// with 2 functions for the success and failure messages
def m4: Matcher[String] =
  ((_: String).startsWith("hello"), (s:String) => s+ " starts with hello", (s:String) => s+ " doesn't start with hello")

If you want absolute power over matching, you can define your own matcher extending Matcher:

class MyOwn extends Matcher[String] {
  def apply[S <: String](s: Expectable[S]) = {
    result(s.value.isEmpty,
      s.description + " is empty",
      s.description + " is not empty",
      s)
  }
}

In the code above you have to:

  • define the apply method (and its somewhat complex signature)

  • use the protected result method to return: a Boolean condition, a message when the match is ok, a message when the match is not ok, the “expectable” value. Note that if you change the expectable value you need to use the map method on the s expectable (s.map(other)). This way you preserve the ability of the Expectable to throw an Exception if a subsequent match fails

  • you can use the description method on the Expectable class to return the full description of the expectable including the optional description you setup using the aka method

Now learn how to…

  • use standard results (failure, success, skipped, todo…) instead of matchers
  • add descriptions to your expectations to create even better failure messages
  • use datatables to conveniently group several examples into one
  • use ScalaCheck to generate and verify data for your examples
  • use Mockito to mock the interactions with another system
  • use Forms to display actual and expected values in html tables

And if you want to know more