Scala

What's new in ScalaTest 3

Tudor Zgureanu

29 Aug 2016 • 4 min read

ScalaTest 3.0.0 was recently released. Being a major release it brings lots of features, changes (some of them potentially breaking) and new deprecations.
In this post I will focus more on the main additions and enhancements, followed by some code examples.

Installation

Add this line to your sbt build file:

libraryDependencies += "org.scalatest" %% "scalatest" % "3.0.0" % "test"

If you use maven add a dependency to your pom.xml file:

<dependency>
  <groupId>org.scalatest</groupId>
  <artifactId>scalatest_2.11</artifactId>
  <version>3.0.0</version>
  <scope>test</scope>
</dependency>

For Scala 2.10 use scalatest_2.10 artifact id.

Main improvements

Many improvements are added in this version. Here's a short list:

Full support for Scala.js
Added async styles for testing Future:
AsyncWordSpec, AsyncFunSuite, AsyncFunSpec, AsyncFlatSpec, AsyncFreeSpec and AsyncFeatureSpec. The trait versions are added as well: AsyncWordSpecLike, AsyncFunSuiteLike, AsyncFunSpecLike, AsyncFlatSpecLike, AsyncFreeSpecLike and AsyncFeatureSpecLike. The result type of tests is Future[Assertion]. A few examples can be found in the next section.
Added org.scalatest.compatible.Assertion marker trait, required for the new async testing styles. Also, org.scalatest.Assertion type alias is defined in the org.scalatest package object.
Enhanced Matchers by adding more matcher words for testing containers: oneElementOf, noElementsOf, atLeastOneElementOf, atMostOneElementOf, allElementsOf and inOrderElementsOf, which must follow contain (examples in the next section).
Changed the result type of assertions and matcher expressions from Unit to Assertion.
Changed the result type of styles to be Any instead of Unit. Therefore, compiling with -Ywarn-value-discard will not result in warnings for tests that end with an assertion or matcher expression because their result type is not Unit anymore.

Examples

Async styles

Probably one of the most expected features in Scalatest 3. Async styles allow you to test futures in an async way, as you would do in your implementation code. All async styles expect tests to have the result type of Future[Assertion].

Suppose we have a simple user service:

import scala.concurrent.Future
import scala.util.{Failure, Success}

//Don't even dare to implement this way, it's just for showcase purposes.
class UserService {

  private var users: Seq[User] = Seq(User("11111", "johndoe", 30))

  def findUser(id: String): Future[User] = Future.fromTry {
    Thread.sleep(3000)
    users.find(_.id == id) match {
      case Some(user) => Success(user)
      case _ => Failure(UserNotFoundException(s"User with id: $id not found"))
    }
  }

  def addUser(user: User): Future[User] = Future.fromTry {
    if (users.exists(_.username == user.username))
      Failure(UserAlreadyExistsException(s"User with username: ${user.username} already exists!"))
    else {
      users :+= user
      Success(user)
    }
  }
}
case class User(id: String, username: String, age: Int)
case class UserNotFoundException(message: String) extends Exception(message)
case class UserAlreadyExistsException(message: String) extends Exception(message)

And the test case:

import org.scalatest.{AsyncWordSpecLike, Matchers}
import scala.concurrent.ExecutionContext.Implicits.global

class UserServiceSpec extends AsyncWordSpecLike with Matchers {
  val userService = new UserService

  "UserService" should {
    "return a User" when {
      "searching by a valid id" in {
        val id = "11111"

        userService.findUser(id).map { user =>
          user.id shouldBe id
          user.username shouldBe "johndoe"
          user.age shouldBe 30
        }
      }
    }
  }
}

If we change user.age shouldBe 30 to user.age shouldBe 20 the test will fail:

30 was not equal to 20
ScalaTestFailureLocation: com.ted.playground.scalatest.service.UserServiceSpec$$anonfun$1$$anonfun$apply$mcV$sp$1$$anonfun$apply$mcV$sp$2$$anonfun$apply$1 at (UserServiceSpec.scala:19)
org.scalatest.exceptions.TestFailedException: 30 was not equal to 20
	at org.scalatest.MatchersHelper$.indicateFailure(MatchersHelper.scala:340)
	at org.scalatest.Matchers$AnyShouldWrapper.shouldBe(Matchers.scala:6864)
	at com.ted.playground.scalatest.service.UserServiceSpec$$anonfun$1$$anonfun$apply$mcV$sp$1$$anonfun$apply$mcV$sp$2$$anonfun$apply$1.apply(UserServiceSpec.scala:19)
	at com.ted.playground.scalatest.service.UserServiceSpec$$anonfun$1$$anonfun$apply$mcV$sp$1$$anonfun$apply$mcV$sp$2$$anonfun$apply$1.apply(UserServiceSpec.scala:16)
	at scala.util.Success$$anonfun$map$1.apply(Try.scala:236)
	at scala.util.Try$.apply(Try.scala:191)
	at scala.util.Success.map(Try.scala:236)
	at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:235)
	at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:235)
	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
	at scala.concurrent.impl.ExecutionContextImpl$AdaptedForkJoinTask.exec(ExecutionContextImpl.scala:121)
	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

Testing failed futures

Failed futures can be tested in two ways: using recoverToSucceededIf or recoverToExceptionIf:

recoverToSucceededIf is the async sister of assertThrows which is used for asserting the type of the exception the future ends in. It yields a Future[Assertion]:

  "return UserNotFoundException" when {
    "the user does not exist" in {
      val id = "1"

      recoverToSucceededIf[UserNotFoundException](userService.findUser(id))
    }
  }

recoverToExceptionIf yields a Future[T], where T is the type of the expected exception. This is useful when you want to test some of the exception's fields (e.g. ex.message):

  "return UserAlreadyExistsException" when {
    "adding a user with existing username" in {
      val username = "johndoe"
      val user = User("12345", username, 25)

      recoverToExceptionIf[UserAlreadyExistsException] {
        userService.addUser(user)
      }.map { ex =>
        ex.message shouldBe s"User with username: $username already exists!"
      }

    }
  }

recoverToExceptionIf is the async equivalent of intercept.

Worth to mention:

You can still opt for using ScalaFutures as it wasn't removed from the library.
It is possible to have synchronous tests in async Spec/Suite. However, the result type of the test body must be Assertion.

New matchers

As previously mentioned, a series of matcher words for containers were added. Here you can find some examples:

val collection = Seq(1, 2, 3, 4)

collection should contain oneElementOf List(8, 7, 6, 3)
collection should contain oneElementOf Vector(2)

collection should contain noElementsOf Seq(8, 7, 6, 5)
collection should contain noElementsOf List()

collection should contain atLeastOneElementOf List(8, 7, 6, 3)
collection should contain atLeastOneElementOf Set(1, 2, 6, 3)

collection should contain atMostOneElementOf List(8, 7, 6, 3)
collection should contain atMostOneElementOf Seq(5)
collection should contain atMostOneElementOf Vector()

collection should contain allElementsOf List(1, 3, 3, 3, 4, 3, 2)

collection should contain inOrderElementsOf List(1, 2, 3, 4)
collection should contain inOrderElementsOf Vector(1, 2, 3, 4, 1)
collection should contain inOrderElementsOf List(1, 2, 3, 4, 2)

Wrap-up

The full release notes for Scalactic/ScalaTest 3 are available here, with details about breaking changes, new deprecations and the expired deprecations. Scaladocs for ScalaTest 3 can be found here.

If you wish to play around with the code, feel free to fork my Scalatest 3 playground on github.

Enjoy the testing!