We made an Erlang Cluster on the last stream, with Raspberry Pis. That was great fun. And not as hard as it could have been which opens the door for some wild things :)

Next up is to add more polish to my calendar gadget which will be today at 13.30 CEST and you can find the livestream session here and probably activate a notification for it. Picture of the gadget in the repo README. I expect us to do some eInk visual work and some LiveView UI at the very least. Maybe even extend the functionality a bit.

Regular Programming episode 12 is up and we talk about vacations. Very much in the spirit of Swedish vacation mentality I imagine. I’d be curious to hear international reflections on that conversation.

Not Invented Here - Reinventing a Better Wheel

Something weird and wonderful about Elixir as a community and ecosystem is that it has a wild tendency to reinvent common libraries and tools. Sort of. And I like it, because I don’t think it is done in arrogance or naïveté (the marks on that word, yees) but rather from a mix of ambition and pragmatism.

Elixir is built on top of Erlang. Erlang runs on the BEAM virtual machine which is a very powerful runtime environment for Actor-based systems. Systems built in Erlang and Elixir can integrate and interact directly via common patterns like supervision and message passing. They can offer each other re-usable pieces of infrastructure and run-time parts in the form of supervision trees or applications.

For example, when you bring in Ecto to manage your database you get to choose when to initialize your Repo module in your application supervision tree. And if you need more fine-grained control you can spin up separate repository instances with their own pooling settings in whatever structure you like.

So re-implementing something for Elixir is usually done because you want to be able to leverage the entirety of the actor model.

I’m excited about the Membrane Framework which wraps up video tooling primitives in high-level Elixir abstractions because it lets me deal with this media inside Elixir. I can get messages saying what the current audio level is at a tight interval, if a new thumbnail has been made mid-stream and I can choose how those Pipelines fit into my Supervision structure. Is one always up? Do we start them dynamically as new streams of video come in or new devices are detected?

Rethinking things in Elixir is what produced Phoenix LiveView. The runtime is audacious and whenever we try to use something that is just a binary blob outside of it we are keenly aware that we are leaving things on the table. We make that trade-off all the time. Sometimes we just need a Go binary to do some work for us and it can essentially be a crude actor over a Port. Some libraries wrap NIFs with good reason (Membrane, for example).

But a lot of things get ported into Elixir or are built entirely because of what this environment offers. Nerves does IoT on Linux-devices differently, building on Buildroot for the Linux part and then quickly handing things off to the BEAM runtime because it is a good environment for resilient applications. Scenic is doing UI straight over to OpenGL with Processes for UI state with some very cool consequences.

There are a ton of libraries that use Elixir and the BEAM to do the thing they do differently than what it would look like in other platforms. I feel like the community and platform show them that they can be more ambitious, audacious and that doing so isn’t just re-inventing the wheel but with less features. It actually provides new, novel and different value.

Typical examples:

• Phoenix Presence - Part of Phoenix, keeping eventually consistent state about who is online
• Oban - Work queues library, incredibly full-featured, especially at Pro
• PromEx - Alex went hard on this Prometheus + Grafana library
• SiteEncrypt - Removing the need for a separate Certbot install for SSL encryption
• Finch - A very capable HTTP client with some cool features
• Wallaby - Browser automation testing directly in Elixir, driving browsers
• Telemetry - A shared foundation for getting data out of BEAM applications, an Erlang library
• Livebook - Collaborative code notebooks that can be run standalone, against local projects, against remote project, this one is just wild
• Nx - Bringing numerical computing to Elixir without breaking the abstractions and making wild mutability happen. The foundation for Machine Learning with Elixir.

I’m forgetting a bunch as well. I’m writing unusually early and will blame that.

I like that there is this sense of not necessarily “we have to port everything to Elixir because all other things suck” but rather “I wonder what this would do on the BEAM” or “I bet we could make this cooler”.

The distinctive qualities of the runtime is a bit of a forcing function there. It would be way more convenient if the outside world could be just plugged right in. But it is also incredibly clear once you are familiar with the way that it works how quickly that would break down. So we port things and adapt them and see if this wild and wonderful platform can let them be something even greater.

Outside of the BEAM I try not to be too precious about language. Going full polyglot enables a different kind of power and flexibility. But if you buy into the BEAM for a project you really benefit from things that play well there. It isn’t harder to integrate with the BEAM than with other runtimes. But the outside code tends to be noticeably separate while you can build a very cool whole from the parts on the inside the BEAM.

What are the most interesting projects you’ve seen in BEAM land? Am I incredibly wrong?  I'm curious what you think and you can let me know at lars@underjord.io or on Twitter where I'm @lawik.

Thanks for reading, I appreciate your time.

- Lars Wikman