07.08.2019 / By Jakub Czuchnowski

Making ZIO, Akka and Slick play together nicely

PART I – ZIO AND SLICK

All of us are eager to start writing real-world applications using ZIO (Scala library). But ZIO is still quite new, and its ecosystem is still incomplete. So while we wait for ZIO-HTTP and ZIO-JDBC (or whatever else comes in the future) to happen, we will have to end up integrating ZIO with other frameworks that can communicate with the outside world. In this article, I want to explore what this kind of integration might look like for two of the most popular libraries – Slick and Akka HTTP. All the code presented here can be found in this ZIO, Akka, and slick application repository example. It’s a “live” project so will change over time as I do more experimentation. The design and structure are loosely based on Vaughn Vernon’s “Implementing Domain-Driven Design.” While that doesn’t matter from the integration perspective, it might explain some of the naming and design choices for the project. Let’s start by adding Slick to our ZIO application.

How to integrate ZIO with Slick

Generally, when we’re adding database handling code to our applications, we want it to be isolated, testable and replaceable. We most definitely don’t want it to spill out all over the app, making it dependent on a (usually) very specialized library. We often do this simply by hiding the implementation details behind an interface and deferring any decision about which particular implementation to choose (usually using DI) until the entry point to the application. When using Slick in its pure form, you have two choices. You can either

use its `DBIOAction` as your effect type all over the application or
go to `Future` early.

Neither of these choices is something we want to go with, except for maybe in the case of the simplest of web apps. In our case, the effect of choice is ZIO, so our repository interfaces will depend purely on ZIO separating Slick from the rest of the app, while the implementation will have to deal with a transition from DBIOAction to ZIO. What about the interface and dependency injection part? For that we’ll be using Environment – a feature deeply integrated into the heart of ZIO. It’s something that after using Cats’ Kleisli a couple of times, made me fall in love with ZIO even more (when the feature was first introduced at a training I had the pleasure of attending ZIO workshop by John A De Goes ). A typical and simplified course of action in a Slick application is the following:

https://gist.github.com/jczuchnowski/cd22a0a8810f59e41ccdabd9fa5d367c

So we have our database object, we define our DBIOAction (which in particular cases is just a DBIO alias), and we run the query on our database. Finally, we get a Future in return. As mentioned before, in our application, we don’t want to deal with a DBIOAction. We want to program with ZIO. But at the same time, we also want to delay committing to a particular database until the “end of the world.” First of all, let’s create our Slick-agnostic repository interface representing our database operations. We want to end up with something along the lines of

https://gist.github.com/jczuchnowski/e97d8b808a0ca0e790109141b593e839

The return type of IO[E, A] is just an alias for ZIO[R, E, A] where R is Any. Before we get to the implementation, I’d like to give a quick introduction to Environment – something that (just as with the Database object) will allow us to postpone our decision about the choice of any particular implementation until it’s inevitable.

Intro to ZIO Environment

If you have ever tried to write an application the FP way and you wondered how to do DI the “functional way,” then you have probably stumbled upon the Reader monad. And if you actually then tried to use it in practice, you probably needed some effect type, and so you ended up with Kleisli. Now, some argue that Reader is not for dependency injection, but in this article I’m not going to dive into semantics (although it’s a fascinating topic!) and let this slide… In the end, you might have eventually ended up using Kleisli[F, A, B] throughout your app: Kleisli[F, A, B] Using ZIO will be similar in some ways. ZIO[R, E, A] From our (simplified) point of view, what Cats calls Configuration [A], ZIO calls Environment [R]. An abstract effect type of [F] in Kleisli is, in turn, fixed and integrated into ZIO (and it’s an IO as you might have guessed). The additional parameter that ZIO introduces is the Failure channel [E]. When writing apps with Slick and Cats, I usually ended up with Kleisli[DBIO, Config, T] – where DBIO was the effect type, Config was my injected configuration, and T was some result type. With ZIO it’s similar in some ways, but on the other hand, it’s more natural. IO is a part of the structure, with a lot of operators and with an additional failure channel that you’d otherwise need to emulate with Either[E, A]. You can find more information on Environment and its usage on the official documentation page and J. De Goes’ blog. It’s worth reading:

https://degoes.net/articles/testable-zio

Introducing DatabaseProvider component

As I mentioned earlier, when working with Slick we need to provide a Database object that represents the database we’re going to run the queries on. For this, we’ll introduce a DatabaseProvider component whose sole purpose will be to provide the Database object:

https://gist.github.com/jczuchnowski/5b2b0fb120c7f55e22b5282277c367d3

Its sole member is a `db` function, which will be the source of the database object for the application. This will be universal enough to provide a different implementation for both production and test environments. Here’s an example production implementation, assuming we’re using the H2 database:

https://gist.github.com/jczuchnowski/7693041e6af1e4e0375bcf12d81b5379

The UIO[A] is an alias of ZIO which doesn’t need any Environment and cannot fail. In this case, we know very well it might fail, but a failure at this level is hardly recoverable, and we might as well crash the app at startup.

Introducing Repository components

We’ll deal with our repositories in the same way. Our interface will now become a repository component according to the same pattern as the DatabaseProvider:

https://gist.github.com/jczuchnowski/3ba488ff14eb0215a1a89991b0ffe2fe

We could now implement the interface and provide a production-ready repository, but we’re missing one part. If we’re using Slick, we’ll be operating using DBIOAction as a result. This isn’t what we want. We want to work with ZIO. In this specific case with IO – which is an alias for ZIO with R = Any. At the time of writing, there’s no interop module for ZIO and Slick, so at the next step, we’ll have to implement simplified interop functions for our case.

From DBIO to ZIO

Now we need to convert DBIOAction (or DBIO) into ZIO representation. As I mentioned at the beginning about Slick, we need a Database object to be able to run the queries. And we don’t want to decide right away what database it is. For that purpose, we’ve introduced the DatabaseProvider component. So how to access it without providing the actual implementation? ZIO companion object has a unique method for that:

https://gist.github.com/jczuchnowski/321fd47c3ade14263c87f915d3083414

What’s great about this, is that with this piece of code the compiler will automatically infer that the type of the resulting ZIO will require a DatabaseProvider at the Environment position. Now we have the database, and we can run the action:

https://gist.github.com/jczuchnowski/46ee38c7fa754174f8afaf072fef89cc

But by doing this, we get a scala.concurrent.Future as a result. We don’t want eager Future at this point (right?), we want ZIO. Fortunately, ZIO integrates with Future easily and so will provide us with a smooth transition:

https://gist.github.com/jczuchnowski/7e91947a883504ef9c3c8d38e8b7b47f

In the end, we end up with this:

https://gist.github.com/jczuchnowski/3bded5747019bcdd7281c918ac765b1c

From StreamingDBIO to ZStream

Slick has another mode for providing results from the database – streaming. If we run our action using the stream() method, we’ll get a Publisher instead of a Future. The publisher is part of Reactive Streams specification, and luckily enough the ZIO community has thought about providing us with an interop module that will convert the reactive stream into ZIO’s ZStream.

https://gist.github.com/jczuchnowski/5eccbe1d03f7838aa0422c7ae2fe12c4

Gluing it all together

Now that everything is in place we’re left with one last task. Bringing all the pieces together – probably somewhere at the application’s entry point. Let’s imagine we have a service that will perform some data access and calculations based on a portfolioId and will return the current portfolio status.

https://gist.github.com/jczuchnowski/d202252b5e7335643f4e0d4bfad257c3

At some point, we will call this method, and eventually, we will want to get the PortfolioStatus out of ZIO. To do that, we need to use ZIO’s DefaultRuntime and its unsafeRun* family of methods. The most straightforward of these is unsafeRun. But before we do that, we have to provide Environment for our ZIO effect which is of the type `AssetRepository with PortfolioRepository`. To do this, let’s define our Live environment:

https://gist.github.com/jczuchnowski/51219274f40901020fcd5acd2ad4d1e8

Now having the environment, we can eventually get the result of our computation.

https://gist.github.com/jczuchnowski/07e8304f32db3e2a2c508a3985da3e60

In simpler circumstances, that would be it. We get the result; the program ends (or not). But here we’re creating a web application, so the final step has to integrate with Akka HTTP. Our “end of the world” for ZIO is actually when a response to an HTTP request is created. In the second chapter of this article, I will try to show you how it can be done.

What else to keep in mind?

Transactions

With this approach, you’ll be able to do database transactions only at the level of repositories. Usually, it’s enough, but I can imagine it could be too constraining in some cases. Unfortunately, I haven’t looked into this subject yet, so I can’t help, but if by any chance you have any ideas or solutions, please let me know.

Thread pools

Slick uses its thread pool and ExecutionContext to handle db (blocking) operations. At the same time, ZIO also has a thread pool specifically cut out for blocking operations. It might be tempting to use one OR the other instead of both. You could achieve it either:

by passing `ioExecutionContext` from Slick to ZIO, or
by implementing your AsyncExecutor that uses ZIO’s IO thread pool, and creating the Database object with it.

Before you go with any of these solutions, make sure you know what you’re doing. ZIO’s approach is different from Slick’s. ZIO’s blocking thread pool is unbounded, while Slick takes a more nuanced approach to its database thread pool. Ultimately it might be best to leave it as is – Slick will take care of its operations, and ZIO uses unbounded blocking pool either way.

Tests

Last but not least, – the tests. I haven’t covered them in this article, as it seems overly long as it is. But they are there. Take a look at my ZIO, Akka and Slick repo – it just works. That’s all I have for now. In the second part, I will show you how you can plug Akka HTTP on top of this application. In the meantime, you’re welcome to take a look into the repository to see the work in progress. Also, if you’re interested in integrating ZIO with another respected stack of http4s and doobie, you might want to take a look at these:

Great article by Wiem Zine https://medium.com/@wiemzin/zio-with-http4s-and-doobie-952fba51d089
Sample project by Maxim Schuwalow https://github.com/mschuwalow/zio-todo-backend

Do you like this post? Want to stay updated? Follow us on Twitter and let us know in the comments!