Common Scala Gotchas

Hi. In this post, we will be exploring some unexpected gotchas we found while working with Scala. Hopefully, it will help you to better understand Scala and fix some naughty bugs :)

Evaluation Order

Consider this code

https://gist.github.com/pjazdzewski1990/cfe0e1891f8fb3e5bc09

It’s an obvious error – bar isn’t defined here, so initialization will not succeed. What about now?

https://gist.github.com/pjazdzewski1990/740b509e2fdb5ea9098b

This code will compile just fine. Unfortunately asking for the calculated value will show an issue: Test.foo produces 0. If you dig deeper the compiler will give you a hint about what’s going on

Warning:(2, 23) Reference to uninitialized value bar val foo: Int = 2 * bar ^

Essentially the same problem as above. Wrapping in an object reduced the compile error to a warning. Not good … but it gets even more confusing when you consider traits.

https://gist.github.com/pjazdzewski1990/315665c58dd58c808393

When mixing a trait we would expect our overrides to get the code under control, but it’s not the case. The result for the second case won’t be “test!”, but “null!” instead. Although the first call gives us a correct result. What’s even worse compiler won’t issue any hints or warnings here.

What can we do here? An obvious solution here is to make sayMyName a defin the base trait. This is a good practice, as we gain two things:

• flexibility – def can be overridden by vals and defs
• correctness – it will defer construction until the concrete value for the name is already given

Or you can make the computed field lazy thus changing the execution order.

Why did it happen in the first place? In both cases, the problem boils down to execution order. name value is null until we won’t override it in Implementation. Unfortunately, before it happens Scala builds the dependent field using null.

If you consider that null.asInstanceOf[Int] produces 0, the first example becomes understandable. When evaluating fields in Test object, Scala knows what fields exist, so no error is generated. But as they weren’t evaluated yet, default is used.

Race conditions in Akka

Mixing concurrency models in one unit of code is (usually) a bad idea … I understood that when debugging an Akka application.

The code went like this:

https://gist.github.com/pjazdzewski1990/bae218866daa9c25413f

In Akka we are used to stateful actors – very often it’s the simpler solution and, thanks to the actor model, we have the guarantee that only one thread will be modifying our state. It works well in bounds Akka puts on us. The problems start when we go beyond these bounds, for instance adding callback-based Futures into the mix.

Callbacks, like onSuccess, get called when the result is ready. Not when the actor is ready to safely process another message. And it can get us into trouble.

Imagine that we sent many Query objects to the actor. Since the request goes through the network we cannot say which one will get back first. As a result, we can’t say how state mutation will be interleaved. In this code, we expose our internal state to all the problems we had with mutable shared state.

This kind of bug can easily break your code, but can also easily be fixed. Just delegate handling the asynchronicity and synchronization to Akka. For instance like this:

https://gist.github.com/pjazdzewski1990/b9e550bf26448172cbcb

With this approach we still cannot say for sure what will be the order in which the responses will come in, but we will get rid of threads fighting over a shared resource as all the mutation changes will have to go via actor’s mailbox.

Map the map

Scala Maps have significant advantage over their Java counterparts. They are easier to construct and work with. Standard Scala library comes with abundance of methods to manipulate maps and their contents. You can do what you want with one or two lines of code while in Java you would need for doing the same one or two screens. There are of course gotchas and you need to be aware of them.

One of the most convenient things you can do with map is to transform it using map method. The signature of Map.map is following:

https://gist.github.com/pjazdzewski1990/0029b50b9aa8c2917eff

and actually it comes from trait TraversableOnce which means you can expect it to offer the same functionality as with all other TraversableOnces, so Lists, Seqs, Sets and whatnot. Let’s see an example:

https://gist.github.com/pjazdzewski1990/0520a103cd6d33ecd904

The gotcha here is actually valid also for Sets, but with sets it’s more vivid and actually people tend to avoid it. But recently I had to fix more than one bug related to maps where they didn’t. Let’s complicate our example a little bit:

https://gist.github.com/pjazdzewski1990/940c45e7d417aa007211

So we have a map of developer unique ids to their names and a map that maps developer id to a department code he’s working in. The goal is that we want to be able to tell developer names by the department code, maybe because we want to send customized emails depending on the department code, why not. The thing incautious developer would do was shown in the example above. But the result wouldn’t be exactly as we expected:

https://gist.github.com/pjazdzewski1990/a1dde41b2e504852f8ae

Where is poor Mr. Mason? Got fired?Problem here is that quietly map throws away those keys which happen to get overridden by subsequent result of mapping function. So if we fist mapped “frontend” -> “Mason” it soon after got replaced by “frontend” -> “McCarthy”. Here it may look simple, but if we have maps of sizes of hundreds or even bigger, and the domain hasn’t that clear rules, the threat may get blurred out.

Always map over maps, funny as it may sound, only if you are 100% certain that you will use distinct keys in resulting map. The function f: A => B we saw in the signature must be one-to-one with regard to keys. How do we fix the problem? One way to go may be to transform the map to list of tuples and then use groupBy:

https://gist.github.com/pjazdzewski1990/1708a7feaaae78fe57d9

It feels a bit more complicated, but just a bit, yet it provides the right result and it turns out Mr. Mason didn’t get fired after all.

Ask actor’s parent

Let’s depart from core scala and wander away to the world of actors. It would probably take a bit too long to provide sufficient introduction to Akka, so I will just assume you are familiar with it. To the point.

Let’s imagine we have system built of actors that interoperate with each other by exchanging messages (that’s actually a truism, I know) and that we handle some more complex tasks with dedicated actors. We may have some actors that have one instances, state and serve for a pillars of the system, but there are occasional tasks, for which we may want to create a separate actor instance for.

Or maybe that one actor sends a message to another and then expects a response – for example some route controller asks repository actor for collection of users:

https://gist.github.com/pjazdzewski1990/b5064a920bba6bd7bbda

Those who have worked with actors a bit know that these examples are quite ubiquitous in actor-based systems. We use two Akka patterns here. One is “ask”, which provides the ? operator for dispatch-and-await-response-futurething. The other is “pipe”, which allows dispatching the results of the future as a message to another actor (not necessarily sender, as it was in our examples). We will concentrate on the former.

It is important to understand what is happening under the hood. But before we shed some light on it, let’s use an example that refers to the case 1 from the snippet above.

https://gist.github.com/pjazdzewski1990/d4972834f921ad60c6e8

Ok, looks good. Problem is the parent never gets the result. We set the endpoint, make sure the message is dispatched to the parent, but the future times out nonetheless. What’s wrong? After all, didn’t we create the SpecialAgent in ServiceActor? Yes, we did! Didn’t we dispatch the message from there? Well, let’s take a look under the hood…

The ask pattern creates an internal actor that is invisible from outside. It is only this actor that sends the message to the final recipient and it is this actor that should receive the response. That response is visible from the outside as the response-future part of the pattern.

So what happens when we use context.parent ! getResult(taskDef)? We might observe it in logs if only we had used LoggingReceive instead of Receive (and by the way I encourage you to do so). With properly configured logging, we would see, that ServiceActor receives unhandled messages of type DidTheMath. So instead of landing in the future, it landed right in receive! It’s perfectly correct, if you give it a bit of ponder.

The right thing to do is to send the result back using sender ! getResult(taskDef). The lesson to learn is that when you use the ask pattern, remember that the response should always sent back to sender(). Otherwise it will end up as unhandled and you will observe Timeout-related exceptions.

Examples above are Based On True Story®

Summary

We wrote these few corner cases down, so you can learn from our mistakes and improve your code. If you seek a better understanding of Scala’s edge cases visit Scala Puzzlers

Happy hAkking!

Do you like this post? Want to stay updated? Follow us on Twitter or subscribe to our Feed.

See also

• Improving Akka dispatchers
• Require function in self-hosted ClojureScript
• Websockets Server with akka-http
• Why Developers Should Pay Attention to ZIO in 2023