blog/content/posts/creating-a-blog-with-bazel/02-compiling-a-kotlin-application-with-bazel/index.md

+++
template = "article.html"
title = "Compiling a Kotlin application with Bazel"
date = 2019-12-08T11:30:00+11:00
description = "A comprehensive guide to building Kotlin applications with Bazel, including dependency management, testing, and static analysis with Detekt and Ktlint."

[taxonomies]
tags = ["bazel", "kotlin"]
+++

This post will describe how to compile a small application written in Kotlin
using [Bazel](https://bazel.build), tests, as well as how to use static
analyzers.

## Phosphorus

Phosphorus is the application that this post will cover. It's a small utility
that I wrote to check if an image matches a reference. If it doesn't, Phosphorus
generates an image highlighting the differences. The goal is to be able to check
that something generates an image in a given way, and doesn't change - at least
if it's not expected. The actual usage will be covered later in this series.
While it's not open-source yet, it's something I intend to do at some point.

It's written in Kotlin, as a couple external dependencies (
[Clikt](https://ajalt.github.io/clikt/) and [Dagger](https://dagger.dev/)), as
well as a few tests. This is the structure:

{% mermaid(caption="Phosphorus's class diagram") %}
classDiagram
    namespace loader {
        class ImageLoader {
            <<interface>>
        }
        class ImageIoLoader {
        }
    }

    namespace differ {
        class ImageDiffer {
            <<interface>>
        }
        class ImageDifferImpl {
        }
    }

    namespace data {
        class Image
        class DiffResult
    }

    class Phosphorus

    ImageIoLoader ..|> ImageLoader
    ImageDifferImpl ..|> ImageDiffer
    Phosphorus --> ImageLoader
    Phosphorus --> ImageDiffer
{% end %}

The `differ` module contains the core logic - comparing two images, and
generating a `DiffResult`. This `DiffResult` contains both the straightforward
result of the comparison (are the two images identical?) and an image
highlighting the differences, if any. The `loader` package is responsible for
loading and writing images. Finally, the `Phosphorus` class orchestrates all
that, in addition to processing command line arguments with Clikt.

## Dependencies

Phosphorus has two dependencies: Clikt, and Dagger. Both of them are available
as Maven artifacts. In order to pull Maven artifacts, the Bazel team provides a
set of rules called
[rules_jvm_external](https://github.com/bazelbuild/rules_jvm_external/). The
idea is the following: you list a bunch of Maven coordinates and repositories,
the rule will fetch all of them (and their transitive dependencies) during the
loading phase, and generate Bazel targets corresponding to those Maven
artifacts, on which you can depend. Let's see how we can use them. The first
step is to load the rules, in the `WORKSPACE`:

```python
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "rules_jvm_external",
    sha256 = "62133c125bf4109dfd9d2af64830208356ce4ef8b165a6ef15bbff7460b35c3a",
    strip_prefix = "rules_jvm_external-3.0",
    url = "https://github.com/bazelbuild/rules_jvm_external/archive/3.0.zip",
)
```

Then, we can load and call `maven_install` with the list of Maven coordinates we
want, in the `WORKSPACE` too:

```python
load("@rules_jvm_external//:defs.bzl", "maven_install")

maven_install(
    artifacts = [
        "com.github.ajalt:clikt:2.2.0",
        "com.google.dagger:dagger:2.25.2",
        "com.google.dagger:dagger-compiler:2.25.2",
        "com.google.truth:truth:1.0",
        "javax.inject:javax.inject:1",
        "junit:junit:4.12",
    ],
    fetch_sources = True,
    repositories = [
        "https://maven.google.com",
        "https://repo1.maven.org/maven2",
        "https://jcenter.bintray.com/",
    ],
    strict_visibility = True,
)
```

A couple of things to note:

- We're also downloading [JUnit](https://junit.org/junit4/) and
[Truth](https://truth.dev/), that we're going to use in tests
- `maven_install` can try to download the sources, if they're available on
Maven, to be able to see them directly from the IDE

At this point, Clikt, JUnit and Truth are ready to be used. They are exposed
respectively as `@maven//:com_github_ajalt_clikt`, `@maven//:junit_junit` and
`@maven//:com_google_truth_truth`.

Dagger, on the other hand, comes with an annotation processor and, as such,
needs some more work: it needs to be exposed as a Java Plugin. Because it's a
third party dependency, this will be defined in `//third_party/dagger/BUILD`:

```python
java_plugin(
    name = "dagger_plugin",
    processor_class = "dagger.internal.codegen.ComponentProcessor",
    deps = [
        "@maven//:com_google_dagger_dagger_compiler",
    ],
)

java_library(
    name = "dagger",
    exported_plugins = [":dagger_plugin"],
    visibility = ["//visibility:public"],
    exports = [
        "@maven//:com_google_dagger_dagger",
        "@maven//:com_google_dagger_dagger_compiler",
        "@maven//:javax_inject_javax_inject",
    ],
)
```

It can now be used as `//third_party/dagger`.

## Compilation

Bazel doesn't support Kotlin out of the box (the few languages natively
supported, Java and C++, are currently getting extracted from Bazel's core, so
all languages will soon share a similar integration). In order to compile some
Kotlin code, we'll have to use some Starlark rules describing how to use
`kotlinc`. A set of rules is available
[here](https://github.com/bazelbuild/rules_kotlin/). While they don't support
Kotlin/Native, they do support targeting both the JVM (including Android) and
JavaScript.

In order to use those rules, we need to declare them in the `WORKSPACE`:

```python
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "io_bazel_rules_kotlin",
    sha256 = "54678552125753d9fc0a37736d140f1d2e69778d3e52cf454df41a913b964ede",
    strip_prefix = "rules_kotlin-legacy-1.3.0-rc3",
    url = "https://github.com/bazelbuild/rules_kotlin/archive/legacy-1.3.0-rc3.zip",
)

load("@io_bazel_rules_kotlin//kotlin:kotlin.bzl", "kotlin_repositories", "kt_register_toolchains")

kotlin_repositories()

kt_register_toolchains()
```

Once that's done, we have access to a few rules:

- `kt_js_library`
- `kt_js_import`
- `kt_jvm_binary`
- `kt_jvm_import`
- `kt_jvm_library`
- `kt_jvm_test`
- `kt_android_library`

We're going to use `kt_jvm_binary`, `kt_jvm_library` as well as `kt_jvm_test`.

As JVM-based languages have a strong correlation between packages and folder
structure, we need to be careful about where we store our source code. Bazel
handles a few names as potential Java "roots": `java`, `javatests` and `src`.
Anything inside a directory named like this needs to follow the package/folder
correlation. For example, a class
`fr.enoent.phosphorus.client.matcher.Phosphorus` can be stored at those
locations:

- `//java/fr/enoent/phosphorus/Phosphorus.kt`
- `//tools/images/java/fr/enoent/phosphorus/Phosphorus.kt`
- `//java/tools/images/src/fr/enoent/phosphorus/Phosphorus.kt`

In my repo, everything Java-related is stored under `//java`, and the
corresponding tests are in `//javatests` (following the same structure).
Phosphorus will hence be in `//java/fr/enoent/phosphorus`.

Let's see how we can define a simple Kotlin library, with the `data` module. In
`//java/fr/enoent/phosphorus/data/BUILD`:

```python
load("@io_bazel_rules_kotlin//kotlin:kotlin.bzl", "kt_jvm_library")

kt_jvm_library(
    name = "data",
    srcs = [
        "DiffResult.kt",
        "Image.kt",
    ],
    visibility = [
        "//java/fr/enoent/phosphorus:__subpackages__",
        "//javatests/fr/enoent/phosphorus:__subpackages__",
    ],
)
```

And that's it, we have our first library ready to be compiled! I won't describe
all the modules as it's pretty repetitive and there's not a lot of value into
doing that, but let's see what the main binary looks like. Defined in
`//java/fr/enoent/phosphorus/BUILD`, we have:

```python
load("@io_bazel_rules_kotlin//kotlin:kotlin.bzl", "kt_jvm_binary")

kt_jvm_binary(
    name = "phosphorus",
    srcs = [
        "Phosphorus.kt",
    ],
    main_class = "fr.enoent.phosphorus.PhosphorusKt",
    visibility = ["//visibility:public"],
    deps = [
        "//java/fr/enoent/phosphorus/differ",
        "//java/fr/enoent/phosphorus/differ/impl:module",
        "//java/fr/enoent/phosphorus/loader",
        "//java/fr/enoent/phosphorus/loader/io_impl:module",
        "//third_party/dagger",
        "@maven//:com_github_ajalt_clikt",
    ],
)
```

Note the name of the `main_class`: because it's a Kotlin class, the compiler
will append `Kt` at the end of its name. Once this is defined, we can run
Phosphorus with this command:

```
bazel run //java/fr/enoent/phosphorus -- arguments passed to Phosphorus directly
```

## Tests

As mentioned previously, the test root will be `//javatests`. Because we need to
follow the packages structure, the tests themselves will be under
`//javatests/fr/enoent/phosphorus`. They are regular JUnit 4 tests, using Truth
for the assertions.

Defining unit tests is really straightforward, and follows really closely the
pattern we saw with libraries and binaries. For example, the `ImageTest` test is
defined like this, in `//javatests/fr/enoent/phosphorus/data/BUILD`:

```python
load("@io_bazel_rules_kotlin//kotlin:kotlin.bzl", "kt_jvm_test")

kt_jvm_test(
    name = "ImageTest",
    srcs = ["ImageTest.kt"],
    deps = [
        "//java/fr/enoent/phosphorus/data",
        "@maven//:com_google_truth_truth",
        "@maven//:junit_junit",
    ],
)

```

This will define a Bazel target that we can invoke like this:

```
bazel test //javatests/fr/enoent/phosphorus/data:ImageTest
```

Hopefully, the output should look like this:

```
//javatests/fr/enoent/phosphorus/data:ImageTest                          PASSED in 0.3s
```

Once this is done, it's possible to run
`ibazel test //javatests/fr/enoent/phosphorus/...` - Bazel will then monitor all
the test targets defined under that path, as well as their dependencies, and
re-run all the affected tests as soon as something is edited. Because Bazel
encourages small build targets, has some great caching, and the Kotlin compiler
uses a persistent worker, the feedback loop is really quick.

## Static analysis

For Kotlin, two tools are quite useful:
[Detekt](https://arturbosch.github.io/detekt/), and
[Ktlint](https://ktlint.github.io/). The idea to run them will be really
similar: having two supporting test targets for each actual Kotlin target,
running Detekt and Ktlint on its sources. In order to do that easily, we'll
define some wrappers around the `kt_jvm_*` set of rules. Those wrappers will be
responsible for generating the two supporting test targets, as well as calling
the original `kt_jvm_*` rule. The resulting macro will be entirely transparent
to use, the only difference being the `load` call.

Let's see what those macros could look like. In `//java/rules/defs.bzl`:

```python
load(
    "@io_bazel_rules_kotlin//kotlin:kotlin.bzl",
    upstream_kt_jvm_binary = "kt_jvm_binary",
    upstream_kt_jvm_library = "kt_jvm_library",
    upstream_kt_jvm_test = "kt_jvm_test",
)
def kt_jvm_binary(name, srcs, **kwargs):
    upstream_kt_jvm_binary(
        name = name,
        srcs = srcs,
        **kwargs
    )

    _common_tests(name = name, srcs = srcs)

def kt_jvm_library(name, srcs, **kwargs):
    upstream_kt_jvm_library(
        name = name,
        srcs = srcs,
        **kwargs
    )

    _common_tests(name = name, srcs = srcs)

def kt_jvm_test(name, srcs, size = "small", **kwargs):
    upstream_kt_jvm_test(
        name = name,
        srcs = srcs,
        size = size,
        **kwargs
    )

    _common_tests(name = name, srcs = srcs)

def _common_tests(name, srcs):
    # This will come soon, no-op for now
```

With those wrappers defined, we need to actually call them. Because we're
following the same signature and name as the upstream rules, we just need to
update our `load` calls in the different `BUILD` files.
`load("@io_bazel_rules_kotlin//kotlin:kotlin.bzl", "kt_jvm_test")` will become
`load("//java/rules:defs.bzl", "kt_jvm_test")`, and so on. `_common_tests` will
be responsible for calling Detekt and Ktlint, let's see how.

### Detekt

[Artem Zinnatullin](https://twitter.com/artem_zin) published a
[set of rules](https://github.com/buildfoundation/bazel_rules_detekt/) to run
Detekt a week before I started writing this, making things way easier. As usual,
let's start by loading this in the `WORKSPACE`:

```python
http_file(
    name = "detekt_cli_jar",
    sha256 = "e9710fb9260c0824b3a9ae7d8326294ab7a01af68cfa510cab66de964da80862",
    urls = ["https://jcenter.bintray.com/io/gitlab/arturbosch/detekt/detekt-cli/1.2.0/detekt-cli-1.2.0-all.jar"],
)

http_archive(
    name = "rules_detekt",
    sha256 = "f1632c2492291f5144a5e0f5e360a094005e20987518d228709516cc935ad1a1",
    strip_prefix = "bazel_rules_detekt-0.2.0",
    url = "https://github.com/buildfoundation/bazel_rules_detekt/archive/v0.2.0.zip",
)
```

This exposes a rule named `detekt`, which defines a build target, generating the
Detekt report. While there are a few options, we'll keep things simple. This is
what a basic invocation looks like, in any `BUILD` file:

```python
detekt(
    name = "detekt_report",
    srcs = glob(["**/*.kt"]),
)
```

We can integrate that in our `_common_tests` macro, to generate a Detekt target
automatically for every Kotlin target:

```python
def _common_tests(name, srcs):
    detekt(
        name = "%s_detekt_report" % name,
        srcs = srcs,
        config = "//java/rules/internal:detekt-config.yml",
    )
```

All our Kotlin targets now have a `$name_detekt_report` target generated
automatically, using a common Detekt configuration.

The way this `detekt` rule work is by creating a build target, that generates
the report. Which means that it's not actually a test - which is what we were
trying to achieve. In order to do this, we can use
[Bazel Skylib](https://github.com/bazelbuild/bazel-skylib)'s `build_test`. This
test rule generates a test target that just has a dependency on other targets -
if any of those dependencies fails to build, then the test fails. Otherwise, it
passes. Our macro becomes:

```python
def _common_tests(name, srcs):
    detekt(
        name = "%s_detekt_report" % name,
        srcs = srcs,
        config = "//java/rules/internal:detekt-config.yml",
    )

    build_test(
        name = "%s_detekt_test" % name,
        targets = [":%s_detekt_report" % name],
    )
```

And there we have it - a `$name_detekt_test` that is actually a test, and will
fail if Detekt raises errors.

### Ktlint

Ktlint doesn't have any existing open-source rules. Let's see how we can write
our own minimal one. It will take as inputs the list of files to check, as well
as an optional [editorconfig](https://editorconfig.org/) configuration, that
Ktlint supports natively.

The definition of the rules will be split in three files: two internal files
defining respectively the _action_ (how to invoke Ktlint) and the _rule
interface_ (what's its name, its arguments...), as well as a third, public file,
meant to be consumed by users.

Let's start by downloading Ktlint itself. In the `WORKSPACE`, as usual:

```python
http_file(
    name = "com_github_pinterest_ktlint",
    executable = True,
    sha256 = "a656342cfce5c1fa14f13353b84b1505581af246638eb970c919fb053e695d5e",
    urls = ["https://github.com/pinterest/ktlint/releases/download/0.36.0/ktlint"],
)
```

Let's move onto the action definition. It's a simple macro returning a string,
which defines how to invoke Ktlint, given some arguments. In
`//tools/ktlint/internal/actions.bzl`:

```python
def ktlint(ctx, srcs, editorconfig):
    """Generates a test action linting the input files.

    Args:
      ctx: analysis context.
      srcs: list of source files to be checked.
      editorconfig: editorconfig file to use (optional)

    Returns:
      A script running ktlint on the input files.
    """

    args = []

    if editorconfig:
        args.append("--editorconfig={file}".format(file = editorconfig.short_path))


    for f in srcs:
        args.append(f.path)

    return "{linter} {args}".format(
        linter = ctx.executable._ktlint_tool.short_path,
        args = " ".join(args),
    )
```

Pretty straightforward - we combine both Ktlint's executable path, the
editorconfig file if it's provided, and the list of source files.

Now for the rule interface, we will define a rule named `ktlint_test`. Building
a `ktlint_test` target will mean generating a shell script to invoke Ktlint with
the given set of argument, and running it will invoke that script - hence
running Ktlint as well. In `//tools/ktlint/internal/rules.bzl`:

```python
load(":actions.bzl", "ktlint")

def _ktlint_test_impl(ctx):
    script = ktlint(
        ctx,
        srcs = ctx.files.srcs,
        editorconfig = ctx.file.editorconfig,
    )

    ctx.actions.write(
        output = ctx.outputs.executable,
        content = script,
    )

    files = [ctx.executable._ktlint_tool] + ctx.files.srcs

    if ctx.file.editorconfig:
        files.append(ctx.file.editorconfig)

    return [
        DefaultInfo(
            runfiles = ctx.runfiles(
                files = files,
            ).merge(ctx.attr._ktlint_tool[DefaultInfo].default_runfiles),
            executable = ctx.outputs.executable,
        ),
    ]

ktlint_test = rule(
    _ktlint_test_impl,
    attrs = {
        "srcs": attr.label_list(
            allow_files = [".kt", ".kts"],
            doc = "Source files to lint",
            mandatory = True,
            allow_empty = False,
        ),
        "editorconfig": attr.label(
            doc = "Editor config file to use",
            mandatory = False,
            allow_single_file = True,
        ),
        "_ktlint_tool": attr.label(
            default = "@com_github_pinterest_ktlint//file",
            executable = True,
            cfg = "target",
        ),
    },
    doc = "Lint Kotlin files, and fail if the linter raises errors.",
    test = True,
)

```

We have two different parts here - the definition of the interface, with the
call to `rule`, and the implementation of that rule, defined as
`_ktlint_test_impl`.

The call to `rule` define how this rule can be invoked. We define that it
requires a list of `.kt` and/or `.kts` files named `srcs`, an optional file
named `editorconfig`, as well as a hidden argument named `_ktlint_tool`, which
is just a helper for us to reference the Ktlint binary - to which we pass the
file we defined in the `WORKSPACE` earlier.

The actual implementation is working in multiple steps:

1. It invokes the `ktlint` action we defined earlier, to generate the script
that will be invoked.
2. It generates an action to write that script, in a file referred as
`ctx.outputs.executable` (which Bazel knows how to handle and what to do with
it, we don't need to worry about where it is or anything, it won't be in the
source tree anyway).
3. It computes a list of files that are needed to run this target. This is what
allows Bazel to ensure hermeticity - it will know that this rule needs to be
re-run if any of those files are changed. If the target runs in a sandboxed
environment (which is the default on most platforms, as far as I'm aware), only
those files will be available.
4. It returns a `Provider`, responsible for holding a description of what this
target needs.

Finally, we write a file that only exposes the bits users should care about.
It's not mandatory, but makes a clear delimitation between what is an
implementation detail and what users can actually rely on. In
`//tools/ktlint/defs.bzl`:

```python
load(
    "//tools/ktlint/internal:rules.bzl",
    _ktlint_test = "ktlint_test",
)

ktlint_test = _ktlint_test
```

We just expose the rule we wrote in `rules.bzl` as `ktlint_test`.

Once this is done, we can use this `ktlint_test` rule where we needed it, in our
`_common_tests` macro for Kotlin targets:

```python
def _common_tests(name, srcs):
    ktlint_test(
        name = "%s_ktlint_test" % name,
        srcs = srcs,
        editorconfig = "//:.editorconfig",
    )

    detekt(
        name = "%s_detekt_report" % name,
        srcs = srcs,
        config = "//java/rules/internal:detekt-config.yml",
    )

    build_test(
        name = "%s_detekt_test" % name,
        targets = [":%s_detekt_report" % name],
    )
```

And there we have it - all our Kotlin targets have both Detekt and Ktlint test
targets. Because we're exposing those as Bazel targets, we automatically benefit
from its caching and remote execution capabilities - those linters won't re-run
if the inputs didn't change, and can run remotely, with Bazel being aware of
which files are needed on the remote machine.

## Closing thoughts

But what's the link between generating a blog with Bazel and compiling a Kotlin
application? Well, almost none, but there is one. The class diagram included
earlier in this article is generated with a tool called
[PlantUML](http://plantuml.com/), which generates images from a text
representation of a graph. The next article in this series will talk about
integrating this tool into Bazel (in a similar way as we did with Ktlint), but
also how to test the Bazel rule. And to have some integration tests, Phosphorus
will come in handy!