Lumen - Statically compiled Erlang for x86

Underjord is a tiny, wholesome team doing Elixir consulting and contract work. If you like the writing you should really try the code. See our services for more information.

The Lumen Project is an ambitious compiler development effort to create a complimentary set of compilers and tools that allow developers to get the power of the Erlang VM, The BEAM, in places it does not traditionally fit. Such as the browser. Currently the project is at an early released stage as covered in this talk. It does not yet implement all of Erlang OTP and as such won’t handle most Erlang/Elixir you could throw at it. I want to show something that it does do. And while the project is a complicated compiler project you do not need to know that stuff to try it out. This should be achievable for most developers and to ensure that I wasn’t talking out of my rear I had my assistant developer, Elin Olsson, follow the instructions without my input and it worked out well.

So while the big goal is the WebAssembly (WASM) target the compiler is built on top of LLVM and could also be great for making static binaries for x86 and friends. There are still some missing facilities in WASM for handling the kind of light-weight processes and scheduling we enjoy in Erlang land. The team is working with the WASM crowd to make those things happen. While waiting we get an x86 target to play with.

So, to get started we follow the instructions at the Lumen repo. This post will get outdated. The project README is more likely to stay up to date. We’ll cover the steps here anyway.

The basic requirements are the following:

• CMake
• Ninja (recommended)
• CCache (recommended)

Installing these on MacOS is done like so (assuming homebrew is installed):

brew install cmake ninja ccache

Now, let’s get the Lumen repo from Github and cd into it.

git clone git@github.com:lumen/lumen.git
cd lumen

First, we need a modified version of LLVM. Thankfully a precompiled version is provided for Linux and x86 Mac. Below we provide the instructions for MacOS, for Linux, use the previous link. The instructions reference $XDG_DATA_HOME as an environment variable, it is recommended to export XDG variables in general, but if you have not, just replace the usages of $XDG_DATA_HOME below with $HOME/.local/share, which is the usual default for this XDG variable.

mkdir -p $XDG_DATA_HOME/llvm/
cd $XDG_DATA_HOME/llvm/
wget https://github.com/lumen/llvm-project/releases/download/lumen-12.0.0-dev_2020-10-22/clang+llvm-12.0.0-x86_64-apple-darwin19.5.0.tar.gz
tar -xzf clang+llvm-12.0.0-x86_64-apple-darwin19.5.0.tar.gz
rm clang+llvm-12.0.0-x86_64-apple-darwin19.5.0.tar.gz
mv clang+llvm-12.0.0-x86_64-apple-darwin19.5.0 lumen
cd -

Lumen and related tooling such as Eir is built with Rust. So you need rustup which is a Rust installer. Lumen builds off of the Rust nightly due to some features, such as WebAssembly, requiring nightly features. Then we use the Rust package manager cargo to install some additional requirements.

# Install rustup by running this script and follow the onscreen instructions:
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

# Set default Rust version to match Lumen's CI:
rustup default nightly-2021-01-29

# Install the wasm32 targets for the toolchain
rustup target add wasm32-unknown-unknown --toolchain nightly-2021-01-29

# Install cargo-make to run build tasks in project
cargo install cargo-make

Then we need to build our compiler. Replace $XDG_DATA_HOME with $HOME/.local/share as above if needed.

LLVM_PREFIX=$XDG_DATA_HOME/llvm/lumen cargo make

At this stage we encountered a problem. The build failed with a rustup error. This was solved by hard coding the toolchain version in Makefile.toml, line 76.

CARGO_MAKE_TOOLCHAIN = "nightly-2021-01-29"

With this fix the build passed, and we have a compiler. Let’s make sure it runs at all by calling its help (from project root).

bin/lumen --help

Let’s create some of our own code, to start we just set up separate directory:

cd ..
mkdir our_code
cd our_code

To actually try things out we need to write some code. The process for Elixir code is less convenient at the moment so we’ll go with an erlang example. This is our hello world in Erlang:

-module(init).
-export([start/0]).
-import(erlang, [display/1]).
-spec start() -> ok | error.
start() ->
   display("Hello World!").

So now we can compile that:

../lumen/bin/lumen compile --output-dir . hello.erl

That generates a binary for us called hello.out. We can run it. Usually you have to set the executable bit the first time:

chmod +x hello.out
./hello.out

To make things a bit more interesting we can make it do multiple things. Lets spawn a couple of processes that will output things as well as output from the initial process.

-module(init).

-export([start/0]).

-import(erlang, [display/1]).

-spec start() -> ok | error.
start() ->
  spawn(fun() -> server("Foo") end),
  spawn(fun() -> server("Bar") end).

server(Message) ->
  display(Message).

A single static binary running from Erlang code, compiled via Rust. Bakeware achieves something similar, as do certain options in Distillery I believe, but that ships the entire Erlang runtime and has a decompression step on the first run due to this. With Lumen the runtime is compiled in. At later points, due to removing hot code updates, there will be optimizations such as dead code elimination and simply not shipping the entirety of Erlang and OTP with each binary. I think that’s quite a compelling concept. update: this is actually already a thing for NIFs, they aren’t linked in if they aren’t used, I really should run these things by the Lumen team :)

The tradeoffs with the BEAM being a VM are generally size and startup time. And it gives us immense flexibility with absurd capabilities like hot code updates. The downside is only for some very particular use-cases where size and startup time matter more. The web is one of these but far from the only one. I would expect Lumen binaries, even when the project has matured a bit, to still be larger than something like Rust and Go binaries but I honestly don’t know. I would also expect them to start a lot faster than your average BEAM-based release. So I’m keen on using Lumen for two things in the future. Elixir-based CLI and web frontend code. With WASM the world does open up quite wide with WASM poised to being a general target for anything that needs to be sandboxable and fast. It is being considered as a container replacement, edge compute, serverless functions, embedded, so many things.

So that’s a preview of the Lumen compiler. I tried to do some timer:sleep/1 stuff but found out that wasn’t implemented yet. If you are curious to see what you do have I think this gave a decent view if you know your Erlang module names. So going into the timer folder there I can see what functions do exist and there isn’t presently a sleep_1.rs.

So this example outputs x86 binaries. The next step is WASM and while I’ve been out of the loop for a bit a bird just told me that there’s a big PR rolling in that should get us to WASM output or thereabouts. If you are keen on helping Lumen move forward, check the issues on the lumen repo and you can help implement more of Erlang/OTP. Maybe get me that sleep function? You can find the runtime BIFs label in the issues.

Big props to Elin for helping me work through this. She deserves a 75% credit on this blog post. All the annoying work was done by her and I’m very happy to know that you can be unfamiliar with the project and get from start to finish. Also thanks to Luke and Brian for correcting me on some important details.

If you have thoughts or question, don’t hesitate to let me know. I tend to both read and respond via email at lars@underjord.io and on Twitter where I’m @lawik.

Underjord is a 4 people team doing Elixir consulting and contract work. If you like the writing you should really try the code. See our services for more information.

Note: Or try the videos on the YouTube channel.