Lazy arguments in Scala

In my previous blog post we have discussed the evaluation strategies in Scala and the difference between call-by-value, call-by-name and call-by-need. Also, I have shown a small workaround for evaluating the arguments lazily (aka call-by-need). As previously presented, the workaround for call-by-need arguments consists of local lazy values which are initialised with the call-by-name arguments we want to be evaluated at most once:

def foo(cond: Boolean)(bar: => String) = {  
    lazy val lazyBar = bar
    if (cond) lazyBar + lazyBar
    else ""
}

foo(true){ // some computation ... }

This looks ok but can it be abstracted so we don't have to define these local lazy values? This would be useful especially when dealing with multiple lazy arguments.

In this blog I'm going to present a simple solution using scalameta macro annotations to remove the boilerplate required for lazy arguments.

Macro annotations to the rescue

I have defined a macro annotation @WithLazy which implements call-by-need arguments. We use @Lazy to mark the parameters we want to evaluate lazily (in our case it's elem). Let's have a look at the example bellow:

@WithLazy
def lazyFill[T](t: Int)(@Lazy elem: T): Seq[T] = Seq.fill(t)(elem)

Let's call the method with zero and a non-zero t:

lazyFill(0) {
  println("call-by-need")
  1
}

//prints nothing, yields: List()

lazyFill(4) {
  println("call-by-need")
  1
}

// prints only once: call-by-need
// yields: List(1, 1, 1, 1)

As we can see elem is evaluated at most once: when t > 0 elem is evaluated once and when t == 0 elem doesn't get evaluated.

If you want to try out this example feel free to skip to the Installation section.

Implementation details

The first and current implementation is relatively simple and consists of the following steps:

Define the @WithLazy macro annotation which implements the call-by-need functionality and @Lazy to mark the parameters to be evaluated lazily.
All the parameters annotated with @Lazy are changed to be call-by-name.
The method is wrapped with another one with the same signature.
Define local lazy values in the outer method for each @Lazy annotated parameter with its value.
The inner method name is added the "$Inner" suffix.
In the outer method body call the inner method with the corresponding local lazy values (if any).

After applying the AST transformation the method we defined above is transformed into:

def lazyFill[T](t: Int)(@Lazy elem: => T): Seq[T] = {
  def lazyFill$Inner[T](t: Int)(@Lazy elem: => T): Seq[T] = Seq.fill(t)(elem)
  lazy val elem$Lazy: T = elem
  lazyFill$Inner(t)(elem$Lazy)
}

The implementation of the @WithLazy macro annotation can be found here.

Design decisions & challenges

You may probably wonder why did I choose this approach instead of declaring the local lazy values and rewrite the function body tree to use these values instead. Currently, macro annotations support the syntactic API only which in a nutshell means that you get the AST as is, with no context, no types inferred, just the plain AST. Using the syntactic API only makes the scoping very challenging, which in my case would happen when having nested methods. This was one of the main reasons I ended up using the method wrapping approach. This approach was fairly easy to implement and it also gave me a nice starting point for further experiments.

Also, it was quite challenging to check which argument is annotated with @Lazy. Because @Lazy != @lazyargs.Lazy I had to make sure that I check the parameters for being annotated with @Lazy, @lazyargs.Lazy, @com.tudorzgureanu.lazyargs.Lazy or _root_.@com.tudorzgureanu.lazyargs.Lazy. Luckily, I was not the only one facing this problem and I was able to find a suggestion from @olafur which helped me to determine all the parameters that were annotated with @Lazy. Of course, I could just generate a list of all the sub-packages and then check if there are any parameters annotated with any "version" of @Lazy from that list. Perhaps, I should give it a try. Unfortunately, if the import was renamed there is nothing you can do about it.

The good news is that def macros are going to support the Semantic API although there is no ETA for making macro annotations to support the Semantic API.

Future plans

Probably the first improvement would be adding a naive lazy implementation that doesn't rely on lazy values, which are advertised to be resource consuming (it uses locks to provide thread safety). It will trade thread safety in exchange for performance. As we most of the time write functions whose parameters aren't involved in concurrent computation it would make sense to have this low-overhead alternative as the default implementation and the tread safe implementation enabled by another annotation (e.g. @volatile like in Dotty).
Add more test cases that cover various ways of defining functions (e.g. nested functions). This should bring more safety when doing further experiments with this library.

Installation

After experimenting enough with scalameta I decided to release the macro annotation as a small library. It's not production-ready (it might never be, although I will still spend some time improving it) so use it at your own ~~risk~~ fun.

You will need to use Scala 2.11.8+ or 2.12.x.

Add the bintray repo resolver for this project:

resolvers += Resolver.bintrayIvyRepo("tudorzgureanu", "generic")

Add the dependency to this project:

libraryDependencies += "com.tudorzgureanu" %% "scala-lazy-arguments" % "0.1.0"

Add scalameta paradise compiler plugin and scalameta dependency:

addCompilerPlugin("org.scalameta" % "paradise" % "3.0.0-M8" cross CrossVersion.full)

libraryDependencies += "org.scalameta" %% "scalameta" % "1.8.0" % Provided

Conclusions

In this blog post I have presented a relatively simple solution for adding support for call-by-need arguments without boilerplate. It is implemented using scalameta macro annotations. Hopefully, the support for lazy keyword will be extended so we can apply it on method parameters as well. The full code is available on github.

Links

Happy meta-hacking!