How to set up Bazel build tool for your Scala project

You have probably encountered this problem while working with SBT and bigger projects. I’m talking about compilation times and test execution times, in other words, having to wait instead of working. Imagine working with a build tool that rebuilds only what is necessary, using a distributed cache, so if module A is built by one of your team members you won’t have to do it again. Or imagine being able to run builds of different parts of your project in parallel, or run tests only for the affected code that has been changed. Sounds promising right? That’s why, in this tutorial, I will be showing you what Bazel build is and how to set your project up in Scala.

Introduction to Bazel build


Bazel is a build tool from Google, which allows you to easily manage builds and tests in huge projects. This tool gives huge flexibility when it comes to the configuration and granularity of the basic build unit. It can be a set of packages, one package or even just one file. The basic build unit is called a target, the target is an instance of rules. A rule is a function that has a set of inputs and outputs; if the inputs do not change then the outputs stay the same. By having more targets (the disadvantage of this solution is having more build files to maintain) where not all of them depend on each other, more builds can run in parallel because Bazel build uses incremental builds, so it rebuilds only the part of the dependency graph that has been changed, as well as only running tests for the affected parts.

It can distribute, build and test actions across multiple machines, and then build and reuse previously done cached work, which makes your builds even more scalable.

Bazel can also print out a dependency graph, the results of which can be visualized on this page webgraphviz.com

So if your project takes a lot of time to build, and you don’t want to waste any more time, this tool is what you need. Speed up your compile times, speed up your tests, speed up your whole team’s work!

In this tutorial, we will be using Bazel version 1.0.0.

Project structure

We will be working on a project with this structure:
├── BUILD
├── WORKSPACE
├── bazeltest
│   ├── BUILD
│   └── src
│       ├── main
│       │ └── scala
│       │ └── bazeltest
│       │     └── Main.scala
│       └── test
│           └── scala
│               └── bazeltest
│                   └── MainSpec.scala
├── dependencies.yaml
└── othermodule
    ├── BUILD
    └── src
        ├── main
        │   └── scala
        │       └── othermodule
        │           └── Worker.scala
        └── test
            └── scala
                └── othermodule
                    └── WorkerSpec.scala

So we have two modules called: bazeltest and othermodule.
Bazeltest will depend on othermodule.

Workspace file setup

Each project has one WORKSPACE file, where we will define things like Scala version and dependencies. If in the project directory there is a  subdirectory with a WORKSPACE file, then while doing our builds this subdirectory will be omitted.
To make it work with Scala, then let’s take an already prepared boilerplate WORKSPACE file from:
https://github.com/bazelbuild/rules_scala#getting-started

Be aware of the change in rules_scala_version. Rules_scala_version is commit’s sha. So if you want to use the newest version of the rules, check GitHub repository and copy-paste commit’s sha.
We also have to add at the end of the file:
load(“//3rdparty:workspace.bzl”, “maven_dependencies”)
maven_dependencies()

This will be used by a third-party tool called bazel-deps, but we will come back to this at the next step.

So after the changes:
rules_scala_version=“0f89c210ade8f4320017daf718a61de3c1ac4773” # update this as needed

load(“@bazel_tools//tools/build_defs/repo:http.bzl”, “http_archive”)
http_archive(
    name = “io_bazel_rules_scala”,
    strip_prefix = “rules_scala-%s” % rules_scala_version,
   type = “zip”,
    url = “https://github.com/bazelbuild/rules_scala/archive/%s.zip” % rules_scala_version,
)

load(“@io_bazel_rules_scala//scala:toolchains.bzl”, “scala_register_toolchains”)
scala_register_toolchains()

load(“@io_bazel_rules_scala//scala:scala.bzl”, “scala_repositories”)
scala_repositories()

# bazel-skylib 0.8.0 released 2019.03.20 (https://github.com/bazelbuild/bazel-skylib/releases/tag/0.8.0)
skylib_version = “0.8.0”
http_archive(
    name = “bazel_skylib”,
   type = “tar.gz”,
    url = “https://github.com/bazelbuild/bazel-skylib/releases/download/{}/bazel-skylib.{}.tar.gz”.format (skylib_version, skylib_version),
    sha256 = “2ef429f5d7ce7111263289644d233707dba35e39696377ebab8b0bc701f7818e”,
)

load(“//3rdparty:workspace.bzl”, “maven_dependencies”)
maven_dependencies()

scala_repositories((
    “2.12.8”,
    {
      “scala_compiler”: “f34e9119f45abd41e85b9e121ba19dd9288b3b4af7f7047e86dc70236708d170”,
      “scala_library”: “321fb55685635c931eba4bc0d7668349da3f2c09aee2de93a70566066ff25c28”,
      “scala_reflect”: “4d6405395c4599ce04cea08ba082339e3e42135de9aae2923c9f5367e957315a”
    }
))
 


If you wish to set a specific Scala version, add code from: https://github.com/bazelbuild/rules_scala#getting-started
scala_repositories((

    "2.12.8",

    {

       "scala_compiler": "f34e9119f45abd41e85b9e121ba19dd9288b3b4af7f7047e86dc70236708d170",

       "scala_library": "321fb55685635c931eba4bc0d7668349da3f2c09aee2de93a70566066ff25c28",

       "scala_reflect": "4d6405395c4599ce04cea08ba082339e3e42135de9aae2923c9f5367e957315a"

    }

))

In this file, we will setup the Scala rules and everything else that is needed to compile the Scala project.

BUILD files setup


To write BUILD files we will use the following methods:
  1. load – which loads the Bazel Scala rules, and extensions
  2. scala_binary – generates a Scala executable
  3. scala_library –  generates a .jar file from Scala source files.
  4. scala_test – generates a Scala executable that runs unit test suites written using the scalatest library.

Start from the BUILD file in a project folder.
load("@io_bazel_rules_scala//scala:scala.bzl", "scala_binary")
scala_binary(
    name = "App",
    deps = [
        "//bazeltest"
    ],
    main_class = "bazeltest.Main"
)
  We have named it App, just one dependency to the bazeltest package. In deps, we list our dependencies, where our own modules or third party can be. Main_class is our entry point.

In the bazeltest package BUILD file:
load("@io_bazel_rules_scala//scala:scala.bzl", "scala_library", "scala_test")
 
scala_library(
   name = "bazeltest",
   srcs = ["src/main/scala/bazeltest/Main.scala"],
   deps = [
       "//othermodule",
       "//3rdparty/jvm/joda_time:joda_time"
   ],
   visibility = ["//:__pkg__"]
)
 
scala_test(
    name = "test-main",
    srcs = ["src/test/scala/bazeltest/MainSpec.scala"],
    deps = [":bazeltest"]
)

Our Main.scala file will use some external third party dependency such as joda date time, and Worker from the subpack package. In srcs we set our Main.scala file, but it could be a list of files, listed one by one or a  matching path pattern for example:
glob(["src/main/scala/bazeltest/*.scala"]) 
( then we use glob ), could even be a package with all the subpackages, such as:
glob(["src/main/scala/bazeltest/**/*..scala"]) 
and in deps all the necessary dependencies, so for this example our own sub pack package plus third part joda date time. For now, it points to the 3rdparty folder which does not exist yet, this will be done at one of the next steps so don’t worry. Visibility is used to define which other targets can use this target as a dependency, in this example, we define a project folder containing the main BUILD file.
Now the BUILD file for othermodule:
load("@io_bazel_rules_scala//scala:scala.bzl", "scala_library", "scala_test")
 
scala_library(
     name = "othermodule",
     srcs = glob(["src/main/scala/othermodule/*.scala"]),
     deps = [],
     visibility = ["//bazeltest:__pkg__"]
)
 
scala_test(
    name = "test-othermodule",
    srcs = ["src/test/scala/othermodule/WorkerSpec.scala"],
    deps = [":othermodule"]
)
Here we have set up a visibility param to the bazeltest package. So only this package can read from this one. If other packages try to reach this, we will see an error.  

Dependencies

We will use a third-party tool for this: https://github.com/johnynek/bazel-deps
Open the dependencies.yaml file and put this there:
options:
 buildHeader: [
   "load(\"@io_bazel_rules_scala//scala:scala_import.bzl\", \"scala_import\")",
   "load(\"@io_bazel_rules_scala//scala:scala.bzl\", \"scala_library\", \"scala_binary\", \"scala_test\")"
 ]
 languages: [ "java", "scala:2.12.8" ]
 resolverType: "coursier"
 resolvers:
   - id: "mavencentral"
     type: "default"
     url: https://repo.maven.apache.org/maven2/
   - id: "hmrc"
     type: "default"
     url: https://hmrc.bintray.com/releases
 strictVisibility: true
 transitivity: runtime_deps
 versionConflictPolicy: highest
 
dependencies:
 joda-time:
   joda-time:
     lang: java
     version: "2.10.4"
 
 com.typesafe.scala-logging:
   scala-logging:
     lang: scala
     version: "3.9.0"
 
 com.typesafe.akka:
   akka-http:
     lang: scala
     version: "10.1.7"
 
 org.scalatest:
   scalatest:
     lang: scala
     version: "3.0.8"
 
replacements:
 org.scala-lang:
   scala-library:
     lang: scala/unmangled
     target: "@io_bazel_rules_scala_scala_library//:io_bazel_rules_scala_scala_library"
   scala-reflect:
     lang: scala/unmangled
     target: "@io_bazel_rules_scala_scala_reflect//:io_bazel_rules_scala_scala_reflect"
   scala-compiler:
     lang: scala/unmangled
     target: "@io_bazel_rules_scala_scala_compiler//:io_bazel_rules_scala_scala_compiler"
 
 org.scala-lang.modules:
   scala-parser-combinators:
     lang: scala
     target:
       "@io_bazel_rules_scala_scala_parser_combinators//:io_bazel_rules_scala_scala_parser_combinators"
   scala-xml:
     lang: scala
     target:
       “@io_bazel_rules_scala_scala_xml//:io_bazel_rules_scala_scala_xml"
(Language is always required and may be one of java, Scala, Scala/unmangled. This is important, if you define an invalid language then errors will occur. Replacements are used for internal targets instead of Maven ones.)  

Save the system variable of this project path, for example (working on a Mac): export MY_PROJ_DIR=`pwd`
We will need this in a minute.

  Clone https://github.com/johnynek/bazel-deps and enter the bazel-deps folder. Ensure that this tool uses the same rules_scala commit sha.
Open the WORKSPACE file inside the bazel-deps and look for this:
git_repository(

    name = "io_bazel_rules_scala",

    remote = "https://github.com/bazelbuild/rules_scala",

    commit = "0f89c210ade8f4320017daf718a61de3c1ac4773" # HEAD as of 2019-10-17, update this as needed

)
  Commit is of course what we need to change ( if it is different than in our WORKSPACE file in rules_scala_version ).

  bazel run //:parse generate -- --repo-root "$MY_PROJ_DIR" --sha-file 3rdparty/workspace.bzl --deps dependencies.yaml

  This will download dependencies into a 3rdparty folder into your project directory.
INFO: Analyzed target //:parse (0 packages loaded, 0 targets configured).

INFO: Found 1 target...

Target //src/scala/com/github/johnynek/bazel_deps:parseproject up-to-date:

  bazel-bin/src/scala/com/github/johnynek/bazel_deps/parseproject

  bazel-bin/src/scala/com/github/johnynek/bazel_deps/parseproject.jar

INFO: Elapsed time: 0.168s, Critical Path: 0.01s

INFO: 0 processes.

INFO: Build completed successfully, 1 total action

INFO: Build completed successfully, 1 total action

wrote 26 targets in 8 BUILD files

The first run

Before doing the first run, let’s implement our Main and Worker classes.
package bazeltest
 
import othermodule.Worker
import org.joda.time.DateTime
 
object Main extends App {
  println("IN MAIN now: "+DateTime.now().plusYears(11))
  val worker = new Worker
  worker.doSomething()
 
 
  def status(): String = "OKi"
}
package othermodule
 
class Worker {
 
  def doSomething() : Int = {
    println("Doing something")
    12345
  }
 
  def pureFunc(): String = "ABC"
 
}
bazel run //:App
INFO: Analyzed target //:App (1 packages loaded, 2 targets configured).

INFO: Found 1 target...

INFO: From Linking external/com_google_protobuf/libprotobuf_lite.a [for host]:

/Library/Developer/CommandLineTools/usr/bin/libtool: file: bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf_lite/io_win32.o has no symbols

INFO: From Linking external/com_google_protobuf/libprotobuf.a [for host]:

/Library/Developer/CommandLineTools/usr/bin/libtool: file: bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/error_listener.o has no symbols

INFO: From Building external/com_google_protobuf/libprotobuf_java.jar (122 source files, 1 source jar):

warning: -parameters is not supported for target value 1.7. Use 1.8 or later.

Target //:App up-to-date:

  bazel-bin/App

  bazel-bin/App.jar

INFO: Elapsed time: 52.246s, Critical Path: 23.22s

INFO: 194 processes: 189 darwin-sandbox, 5 worker.

INFO: Build completed successfully, 198 total actions

INFO: Build completed successfully, 198 total actions

IN MAIN now: 2030-10-11T11:26:07.533+01:00

Doing something
The first run takes some time because it has to download the dependencies, so don’t worry.

Unit tests

Now let’s write some simple unit tests:
package bazeltest
import org.scalatest._
 
class MainSpec extends FlatSpec with Matchers {
 
  "status" should "return OK" in {
    Main.status() shouldBe "OKi"
  }
 
}
package othermodule
import org.scalatest._
 
class WorkerSpec extends FlatSpec with Matchers {
 
    val worker = new Worker()
    
      "do something" should "return 12345" in {
        worker.doSomething() shouldBe 12345
      }
    
      "pureFunc" should "return ABC" in {
        worker.pureFunc() shouldBe "ABC"
 

And run them: bazel test //bazeltest:test-main
INFO: Analyzed target //bazeltest:test-main (0 packages loaded, 0 targets configured).

INFO: Found 1 test target...

Target //bazeltest:test-main up-to-date:

  bazel-bin/bazeltest/test-main

  bazel-bin/bazeltest/test-main.jar

INFO: Elapsed time: 1.047s, Critical Path: 0.89s

INFO: 3 processes: 2 darwin-sandbox, 1 worker.

INFO: Build completed successfully, 4 total actions

//bazeltest:test-main                                                    PASSED in 0.5s

 

Executed 1 out of 1 test: 1 test passes.

INFO: Build completed successfully, 4 total actions
bazel test //othermodule:test-othermodule

INFO: Analyzed target //othermodule:test-othermodule (0 packages loaded, 0 targets configured).

INFO: Found 1 test target...

Target //othermodule:test-othermodule up-to-date:

  bazel-bin/othermodule/test-othermodule

  bazel-bin/othermodule/test-othermodule.jar

INFO: Elapsed time: 1.438s, Critical Path: 1.29s

INFO: 2 processes: 1 darwin-sandbox, 1 worker.

INFO: Build completed successfully, 3 total actions

//othermodule:test-othermodule                                           PASSED in 0.6s

 

Executed 1 out of 1 test: 1 test passes.

INFO: Build completed successfully, 3 total actions
Try now to change the status method from Main, to return “OK” instead of “OKi”. Run the tests again: bazel test //bazeltest:test-main
INFO: Analyzed target //bazeltest:test-main (0 packages loaded, 0 targets configured).

INFO: Found 1 test target...

FAIL: //bazeltest:test-main (see /private/var/tmp/_bazel_maciejbak/16727409c9f0575889b09993f53ce424/execroot/__main__/bazel-out/darwin-fastbuild/testlogs/bazeltest/test-main/test.log)

Target //bazeltest:test-main up-to-date:

  bazel-bin/bazeltest/test-main

  bazel-bin/bazeltest/test-main.jar

INFO: Elapsed time: 1.114s, Critical Path: 0.96s

INFO: 3 processes: 2 darwin-sandbox, 1 worker.

INFO: Build completed, 1 test FAILED, 4 total actions

//bazeltest:test-main                                                    FAILED in 0.6s

  /private/var/tmp/_bazel_maciejbak/16727409c9f0575889b09993f53ce424/execroot/__main__/bazel-out/darwin-fastbuild/testlogs/bazeltest/test-main/test.log

 

INFO: Build completed, 1 test FAILED, 4 total actions
bazel test //othermodule:test-othermodule
INFO: Analyzed target //othermodule:test-othermodule (0 packages loaded, 0 targets configured).

INFO: Found 1 test target...

Target //othermodule:test-othermodule up-to-date:

  bazel-bin/othermodule/test-othermodule

  bazel-bin/othermodule/test-othermodule.jar

INFO: Elapsed time: 0.150s, Critical Path: 0.00s

INFO: 0 processes.

INFO: Build completed successfully, 1 total action

//othermodule:test-othermodule                                  (cached) PASSED in 0.6s

 

Executed 0 out of 1 test: 1 test passes.

INFO: Build completed successfully, 1 total action
Bazel build sees what has been changed, and runs tests only for the affected classes. So test results for othermodule are taken from the cache, and only the main tests run. The test failed because we didn’t change the results in the Spec file, so the change expected the result in the test to the Main.status() shouldBe “OK”. Run tests again: bazel test //bazeltest:test-main
INFO: Analyzed target //bazeltest:test-main (0 packages loaded, 0 targets configured).

INFO: Found 1 test target...

Target //bazeltest:test-main up-to-date:

  bazel-bin/bazeltest/test-main

  bazel-bin/bazeltest/test-main.jar

INFO: Elapsed time: 1.389s, Critical Path: 1.22s

INFO: 2 processes: 1 darwin-sandbox, 1 worker.

INFO: Build completed successfully, 3 total actions

//bazeltest:test-main                                                    PASSED in 0.6s

 

Executed 1 out of 1 test: 1 test passes.

INFO: Build completed successfully, 3 total actions

Dependency graph

We can easily visualize our dependency graph: In the command line run: bazel query --noimplicit_deps "deps(//:App)" --output graph
digraph mygraph {

  node [shape=box];

  "//:App"

  "//:App" -> "//bazeltest:bazeltest"

  "//bazeltest:bazeltest"

  "//bazeltest:bazeltest" -> "//bazeltest:src/main/scala/bazeltest/Main.scala"

  "//bazeltest:bazeltest" -> "//3rdparty/jvm/joda_time:joda_time"

  "//bazeltest:bazeltest" -> "//othermodule:othermodule"

  "//othermodule:othermodule"

  "//othermodule:othermodule" -> "//othermodule:src/main/scala/othermodule/Worker.scala"

  "//othermodule:src/main/scala/othermodule/Worker.scala"

  "//3rdparty/jvm/joda_time:joda_time"

  "//3rdparty/jvm/joda_time:joda_time" -> "//external:jar/joda_time/joda_time"

  "//external:jar/joda_time/joda_time"

  "//external:jar/joda_time/joda_time" -> "@joda_time_joda_time//jar:jar"

  "//bazeltest:src/main/scala/bazeltest/Main.scala"

  "@joda_time_joda_time//jar:jar"

  "@joda_time_joda_time//jar:jar" -> "@joda_time_joda_time//jar:joda_time_joda_time.jar\n@joda_time_joda_time//jar:joda_time_joda_time-sources.jar"

  "@joda_time_joda_time//jar:joda_time_joda_time.jar\n@joda_time_joda_time//jar:joda_time_joda_time-sources.jar"

}

Loading: 12 packages loaded

Paste results to webgraphviz.com Bazel build Scala graph

Generate jar

bazel build //:App
INFO: Analyzed target //:App (0 packages loaded, 0 targets configured).

INFO: Found 1 target...

Target //:App up-to-date:

  bazel-bin/App

  bazel-bin/App.jar

INFO: Elapsed time: 0.085s, Critical Path: 0.00s

INFO: 0 processes.

INFO: Build completed successfully, 1 total action

Bazel build: Summary

In this post, we showed what is bazel, when to use it, and how to make basic configuration. It can take some time to properly set up complex projects using bazel build, but I guarantee you, in the end, it will speed up the whole team’s work.


Useful links

  1. Official Bazel build documentation https://docs.bazel.build/versions/1.0.0/bazel-overview.html
  2. Building Scala with Bazel build- Natan Silnitsky https://www.youtube.com/watch?v=K2Ytk0S4PF0
  3. Building Java Applications with Bazel https://www.baeldung.com/bazel-build-tool

See also

Authors

Maciej Bąk

I am a backend developer since 2013 February. During my career, I worked in many different projects, requiring high traffic, NoSQL and SQL databases, microservices connecting each other via akka, Kafka, RabbitMQ. I am continually improving my skills, currently in functional programming with the ZIO library, Kafka, and Spark. Besides work, I love reading books, learning Spanish (as I live in Canarias islands since 2016 November ), and enjoy the sun!

Latest Blogposts

24.11.2022 / By  Daria Karasek

ZIO Test: What, Why, and How?

Functional World meetup is a global event aimed at improving learning and knowledge sharing among developers with a passion for functional programming. This year’s edition was focused on the ZIO Scala library. In that vein, we invited developer and ZIO enthusiast Marcin Krykowski to speak about the functional power of ZIO Tests. Marcin previously worked […]

24.11.2022 / By  Daria Karasek

Find the Perfect Specialists for Your Project: Test Team-Matching Startup Teamy.ai

Teamy.ai is a startup that integrates industry-leading team-matching algorithms with various workforce management capabilities in a single platform. It’s a new technology from experienced software house Scalac and offers a range of services, from AI-powered candidate matching to extensive skills mapping.  Teamy.ai is almost ready to launch. And Scalac is looking for businesses to test […]

17.11.2022 / By  Daria Karasek

Scalac’s Latest Development: Team-Matching Software, Teamy.ai 

Teamy.ai, a startup from Scalac, is a team-matching software application that pairs best-fit tech specialists with specific business requirements.  Finding the right employees for your business is challenging. Not only do you need to identify candidates with the right skills, but they must also want to work with your stack, have enough experience, be compatible […]

Need a successful project?

Estimate project