Learning Scala macros with Each Library
In this post, we will discuss Each Library. Each allows Scala users to write imperative syntax, which is later on translated into scalaz monadic expressions. In a way, it’s adding syntactic sugar on top of ordinary Scala. Simplicity and the way it extends ordinary code caught my attention, so I’ve decided to dig deeper.
This post will answer some basic questions about Each – how to use it, why it works and how to use the same principles in your code. Hopefully, we will demystify Scala macros along the way.
How to use Each?
Each library allows you to write operations on values enclosed in Monads in a plain, imperative syntax. In a sense it’s similar to await
from Python on C#, but without being limited to Future
s. It helps you to eliminate as much boilerplate code as possible and focus on the business logic.
Although it might sound complex in theory it’s straightforward in practice. I’m sure you will get the idea after looking at these examples.
You got the idea – import each, wrap your computation in monadic[TheMonadYouAreUsing] { ... }
and access raw values with .each
inside the code block. Neat.
You can find more examples and use cases in the project’s README. We will focus now on inspecting Each and figuring out how to use similar techniques in our code.
Why it works?
The short answer is “Scala magic”, but if you are a curious developer you probably won’t be satisfied with this answer.
The (slightly) longer answer won’t come as a surprise to a seasoned Scala Developer – combining implicits, pimp my object pattern and Scala macros.
The in-depth answer goes like this … The import that we used before import com.thoughtworks.each.Monadic._
brings us few things in scope. One of them is an implicit conversion
which is able to transform any Monad (F[A]
, where F is monad and A is the underlying type) into a ‘Pimp’ class EachOps
. The class looks simple enough
We see our familiar each
method here, but you might be surprised that it’s undefined at this point. The secret here is that each
isn’t part of the final code. It makes sure the types do match and serves as a marker. Later on the block passed into monadic
will be rewritten and each
will be substituted with real calls. We will see that in a second.
The real thing happens inside the monadic[X]
call. It takes the type information and body you passed in and expands a macro
Macro is the place where our example becomes much more convoluted. Each covers many cases that we didn’t mention here, so we will drop some details to keep it understandable. For the sake of simplicity, we will use the Future addition example from the first paragraph. The high level objective of this macro is to extract information about the code we need to transform and then build a new tree with ordinary scalaz inside.
We start of with our apply
method in MacroImplementation
.
Here c
is the whitebox context provided by scala, monad
is the monad we are using in the current block (Future
in our example) and body
is what we would like to transform. It will roughly correspond to our code
Now, that we have all the raw data in place, we can extract type information using scala.reflect.api
and build a custom MonadicTransformer class to transform the plain code into it’s final form.
Scala represents abstract syntax trees as convenient case classes, so we can use plain pattern matching to traverse and transform the code recursively. We will not analyze all the cases here, but we will get back to it in the next section.
The point of all those transformations it to simplify each expressions into plain scalaz calls like bind
invocation
Above you can see reflection API in action. To explain the code in a few words Apply
indicates that a function should be called, Select
and TermName
help us to choose what code to call exactly, Function
allows us to create a new function, List
… that’s an ordinary list to provide arguments. I encourage you to explore the documentation to learn more.
At the end of the expansion process we should be left with a code like this
It might look hard with all the $
and brackets, but you can see the pattern here – nested calls leading to a simple Int
sum at the end. The tree that we have built can now be used as a subtraction for our original block. No each
calls are needed.
That’s it. In the next section we will try to use the same pattern on our own.
Using on our own
We took a sneak peak under the hood to get some understanding what is going on, now let’s take a look how much we really did understand. We will try to implement a very simple macro on our own. We will call it with withLogging
. This macro will accept a generic function and print the return type before proceeding, so you can debug your code more easily.
Note that in order for macros to work they need to be in a separate unit of compilation – they cannot be created in the same project/module where you use them. I decided to stick to the previous example and drop the code straight into the codebase of Each, but you might want to put it into a sbt subproject.
Let’s get started with some dummy code we want to log:
As you can see we have our withLogging
macro-based function and we pass an anonymous () => Int
function into it. withLogging
takes a function and returns a new one. This new function does a simple trick: calls the argument, saves the result, prints it and returns it back. To help us with debugging we will also print the line number. Fairly straightforward stuff.
The code is as simple as the description:
We start of with defining the external interface. It’s worth pointing out that Scala macros are typesafe. You cannot (or shouldn’t) cheat the type system. If the generated code doesn’t type-check, compilation will fail. That’s a good thing.
Let’s walk through this code quickly.
It’s based around the same principles as original Each. We have a method that take the Context
of our macro transformations and a Tree
of the code that we want to transform. The following code is building and expression and an AST. To do that we use helper classes provided to us by Scala. They might look intimidating at the beginning, but if you get a better understanding of what they do it becomes simple.
What’s cool about them is that they are ordinary case classes
and all the usual tricks apply – easy instance creation (as above), destructing in pattern matching and so on.
- Function: creates a function, needs a list of arguments and a block with the body.
- Block: represents code between brackets
{...}
. - ValDef: defines a value, think
val foo = ...
. It requires a name and value. It can be given modifiers an explicit type. - Apply: calls a given function (tree) with given arguments
- Ident and TermName: are used to lookup terms.
- Literal: a “hard-coded” value.
Quite neat, isn’t it? Not that complicated either. Unfortunately, a bit verbose.
With Scala 2.11.X we have a new more natural way of building ASTs – quasiquotes. They allow you to write code as a String
with the q
interpolator to add Tree
s into the mix. It’s easier to explain with an example:
That’s even better!
Summary
In this post we have learnt a bit about Each as a tool, how it’s built and how can we use the same tricks in our code. We have played a bit with macros and shown that there’s no magic in Scala. But, most importantly, we shown that digging into an unknown code might be a excellent way to learn.
Links
- https://github.com/ThoughtWorksInc/each
- https://docs.scala-lang.org/overviews/reflection/symbols-trees-types.html
- https://docs.scala-lang.org/overviews/quasiquotes/syntax-summary.html
Do you like this post? Want to stay updated? Follow us on Twitter or subscribe to our Feed.
See also