When old-school developer goes freestyle
Full source code is available on GitHub.
This post is about creating relatively simple project with Freestyle, my adventures during this process and some afterthoughts.
Self-description from the project website: Freestyle is a library that enables the building of large-scale modular Scala applications and libraries on top of Free monads/applicatives.
I’m limited to single blog post, so I’ll make non-trivial-scale application instead of large-scale one. Precisely it is a module to perform registration and log in with credentials or GitHub’s OAuth 2.0 and http4s server that uses this module in
HttpServices (endpoints) for issuing JWT.
This article describes my journey and feelings during it, so it is written in the first person and past tense.
I guess most blog posts are written by someone either excited or angry about something. This post is not such a case. I got resistant to hypes and I won’t write a post that is a gospel of [put any buzzword here] saving our codebases. On the other hand my previous unsuccessful experiences with free algebrashaven’t discouraged me.
Free Monads is a topic which I’ve tried to get into by writing non-trivial applications at least few times but all attempts ended when I thought: All this boilerplate and additional types in codebase are not worth the benefits. Accidental complexity grows too big.”.
I’ve judged that I prefer to stick to “traditional” style, which perhaps is not so easily testable, and does not allow to run a program against many interpreters (I’d love to hear from someone who has more than one for tests and one for production).
Here, please don’t get me wrong, when I began I was thinking that Interpreter pattern is a good thing, although too expensive, a trade off that doesn’t pay off. Have Freestyle changed my opinion about it? You can read the whole story or jump to conclusion.
How it unfolded
Before I dived deeper in solving somehow real problem that I face in my daily programmers life, the registration and log in, I had started with something really simple, namely arithmetic, just to get used to Freestyle, and ran over against first line of hurdles.
The target was to write free programs for calculating mathematical formulas. Following quick start I’ve created such algebras:
HighMath traits to two separate free algebras. I’ve combined these algebras together in
AllTheMath with help of Freestyles
@module to obtain new free algebra with operations of both components.
The next step was to write a program that uses given algebras or to write interpreters. I chose to write a simple program to evaluate math formulae first, here it is:
At glance: Freestyle uses macros to generate boilerplate code for you, it expands macros indicated by
There were some negative consequences of using macros I faced. The first one was that my Intellij Idea indicated errors:
cannot find symbol FS in
@free trait HighMath is expanded to
trait HighMath[FF$15532[_]] extends _root_.freestyle.internal.EffectLike[FF$15532]and
EffectLike contains type alias
FS, thus sbt build was OK. I just had to live with “red margin” in the IDE.
The second problem seemed more severe, because it caused sbt build failure!
Fortunately, that were missing imports of
freestyle.implicits._. Imports had been repeatedly removed by my beloved Idea, until I disabled organize imports in the IDE.
The last problem I encountered can be painful for ones who like to use sbtREPL. It turns out that
console task failed because of
org.scalameta % paradise % 3.0.0-M9, which is required by current version of Freestyle. Unfortunately this made working with examples and learning a bit harder.
A very, very positive thing I’ve learned when facing problems with Freestyle was helpfulness of developers on the 47deg/freestyle Gitter channel. They are so open and kind! Thank you guys again!
After fixing compile-time errors I wanted to get the ultimate reward of a programmer: feeling when the whole code finally works. So I’ve rushed with interpreter implementation. I haven’t been fancy about it, selection between
Id has been won by the simpler one.
The finale of my example in the arithmetic domain is running the computation:
In the last line, I run the program
Formulas.`(a+b)^2` in the
AllTheMathalgebra, interpreted to
Id monad. Imports, reported as unused by Idea, by the way, bring in scope my
Id interpreters for
HighMath and Freestyle machinery (including one that wires my handlers into
One thing worth noting at this point is that I still can write programs that use only
BasicMath and I don’t need anything related to
HighMath to run them. Only lines of code that bound them together were in
Type annotations in wrong places
I’ll come back to my formulae definition:
def `(a+b)^2`[F[_]](a: Int, b: Int)(implicit A: AllTheMath[F]): FreeS[F, Int]Opinions may be different but I don’t really like, and find it redundant, to add type parameter and implicit parameter. The good news is that there are two ways of dealing with this.
The first one is placing methods inside module definition. I had to rename method because some non alphanumeric characters in names cause macro expansion failure.
Ironically, it causes a dual problem: I cannot define a return type, because macro adds type parameter with a semi-random name to the trait it operates on.
The second one requires some boilerplate code, but it can pay off in the case of a big number of methods. The trick is to define a class to capture type, like here:
Irrelevant type annotations are gone, the relevant one is present.
That’s it for the simple case of arithmetic. I’ve run into some problems with not found symbols and had to find out how to place types annotations, but that just happens in the learning period. I’ve become resistant to IntelliJ Idea false positives and imports management. In the end amount of boilerplate was acceptable for me. After this prelude, I was ready to solve the original problem.
Log in the domain at glance:
- allows to sign up – register an account
- allows to log in – obtain JWT
- sign up and log in are possible with email and password or GitHub
This tiny slice of real log in domain was already demanding enough for my goals. It turned out that I needed
GitHubClient at least. I wanted to have logging effects under control to be able to test if the most important events were not left without a trace, that brought
Log algebra in. The fourth and the last component of the free algebras set was
You don’t really need to contemplate my algebras deeply, I write them down mostly to show the volume of what is required.
Further, I’ve joined (somehow following Freestyle getting started guide)
Log to obtain
GitHub and put
Log to get
Final programs required
Deps, a module with all algebras.
At this point I could see the huge benefit of Freestyle: its macros derived a type for each
@free trait method and required types coproducts for each
@module. Thus, it really did reduce the amount of boilerplate to great extent!
You can see that I’ve defined a method inside module
GitHub leaving it without the return type annotation. For more advanced
LoginPersistenceprograms I’ve really wanted to see return types. I’ve defined
PersistencePrograms boilerplate class, but I find it a fair trade off.
Log in programs
Let me share the implementation of two out of four programs: log in with a password and register with a password. I’ve used a helper (boilerplate) class to capture types again:
For registering user I’ve chosen to use inner methods:
I nested for-comprehensions in
I’ve found out that when using Freestyle sometimes I need to put a lot of type annotations, you can observe this in the case of both methods. To be honest I don’t know why, because in PersistencePrograms it works without them.
Besides that (and that I use a monad for logging in Scala the first time) these programs don’t look very different to ones I would write if I have committed to some particular monad or used tagless final style. In some time I’ve gained the skill to quite quickly deal with errors caused by too few type annotations.
Writing handlers is nothing more than implementing the interface in a particular monad. Because I’ve used http4s in this project I’ve chosen to interpret in
fs2.Task. There no are significant differences to imperative style. Nice thing was, that I could very easily obtain handlers for
fs2.Task when I had
cats.Id handlers. That holds true every time when there is a natural transformation from one monad to another.
Links to implementations of interpreters for interested readers:
- PrintlnLogger – not really educative
- InHouseGHClient – simple, home-made client that retrieves data I needed
- LoginDoobieHandler – Database access based on Doobie
- IdJwtHandler – pdi.jwt based
Idinterpreter for issuing and validating JWTs
To run programs for compound-free algebra in target monad, an interpreter for it is required. User has to provide interpreters for each component algebra in implicit scope only because one of the super-powers of Freestyle is that it constructs compound interpreter from implicitly available interpreters for you. I’m really glad that I didn’t have to make it manually.
Such code reminds me of my preferred style of (static) dependency injection in Play Framework.
The next example presents using implicitly available interpreters, creating my
Programs instance and interpreting one of the programs to
The code above concludes the effort of using Free Monads with Freestyle. In the end, there is some boilerplate in production – factitious classes (
PersistencePrograms) for sake of having return type annotations in public methods and extra type annotations in functions bodies.
Personally, I didn’t find that I needed a huge amount of extra code to have free algebras based application. Of course, I didn’t reach this satisfying outcome easily, but I was still in the early learning phase.
Free algebras way of writing programs yields two kinds of objects to test: interpreters and programs. I haven’t implemented tests for interpreters because I’ve assumed that it won’t differ from testing regular service implementations.
When it came to testing programs, I had a real problem in finding what is endorsed way of doing it. Ok, I appreciate you may now think: “this guy is really not-so-bright”, but I’ll dare to be honest: this lack of guidance, and problems with finding it, made me lose some time in unproductive doubts.
Finally, I’ve used advice from Gitter channel and have used mocked interpreters. This way I could easily inject failures when testing error paths. I’ve written these mock interpreters manually because Freestyle handlers methods are
protected[this], thus one can’t mock them with scalamock.
Not finished specs for register/login programs can be found in ProgramsSpecs.scala.
There is another thing that worries me about testing when using Free Algebras: a problem with lack of hierarchy and scalability.
For example when testing
Programs I cannot mock
PersistencePrograms(class I’ve created to have return type annotations, not for dependency injection). It contains pure functions, so there is no reason to mock them, you can tell, but then I have to mock all their dependencies! I have to fit all the problem in my head at once, what turns out to be hard sometimes.
On the other hand, again, it’s a trade-off with a benefit: I couldn’t make a false assumption about how
PersistencePrograms works and “mis-mock” it.
- testing interpreters is not different
- testing programs require to mock all (leaf) interpreters
- one won’t make mistake in the middle levels
- using scalamock is hard because Freestyle protects handlers methods
def transact(c: ConnectionIO)method, for support of Doobie database access. Helper for transforming
DoobieM.Handler is in place.
Wiring Doobie in is easy. The trap is in testability. I’ve found
ConnectionIO really untestable, I wasn’t able to match on queries or updates in my test handler and I don’t believe that H2 is a solution for tests when in production another database is used. I’ve decided to throw
DoobieM code away and implement my own, higher-level, algebra for interacting with DB. If I had written tests for its interpreter, then I would provision some DB and place these tests in the integration suite.
I don’t want to speculate about reasons but that is just a fact: I have been restarting Idea, few times per day at least, when playing with Freestyle. I turned type-aware highlighting off but it didn’t help a lot. IDE still was very sluggish and I was really irritated sometimes. Whole system was affected. This really was a huge blow for my productivity, the biggest impediment I’d say.
I started the evaluation of Freestyle with mixed feelings about Free Algebrasprogramming, perhaps more skeptical than enthusiastic. Some people even doubt that it is feasible with current programming languages but macros-based approach that Freestyle brings to the table looks promising. There are some areas to improve, two most important for me I’ve pointed already: problems with types annotations and suffering IDE.
I’m happy with the code that I’ve written but the process was quite painful a few times. Although Nice Guys at Gitter eased the pain! I think the ultimate question is: would I use Freestyle in my next commercial project (let’s assume I make it alone)? Today the honest answer is: no, I would not. But I truly hope that it will change soon.
Cost-benefit-analysis significantly changed widely for me, Freestyle reduced boilerplate and accidental code tremendously! For sure I’ll keep an eye on Freestyle.
Please take a look at the type signature for
Deps free algebra after stripping all packages names:
Do you like this post? Want to stay updated? Follow us on Twitter and let us know in the comments!