Leverage the difference

I mentioned in the previous newsletter that I made good use of the ability of the BEAM to do runtime tracing and connecting and disconnecting telemetry handlers at runtime. This is essentially how operating the BEAM is very different from operating a Python app. Unless you build it in a very unusual way you will not have this in most ecosystems. There are more differences that we can and should leverage.

Overall I think you should stay close to "the basics". Meaning database (Postgres) for source of truth. Write regular-ass code that does the expected thing. Most of what you do requires zero custom GenServers. We do this with NervesHub. Lots of data just goes into the db. Keeps things simple.

The first stage of leveling up is to introduce more signals that you can use to do live things. If you let Phoenix PubSub publish something to a relevant topic on DB changes you can pick those up and act on them in very exciting ways. The most obvious and easy thing is to shove relevant information over a Channel or LiveView to update a UI somewhere in near real-time. Cheap, easy and scales surprisingly well.

At some point you may want a more holistic approach to that. Not just reacting to your app changing the data store but rather learn when data changes and let your app distribute that knowledge. Peter Ullrich recently wrote up a big honkin' tour of tracking Postgres changes. The answer is rarely LISTEN/NOTIFY and the cool stuff happens if you can do the WAL thing. Consuming the WAL is a super-power.

Of course any language could do that. But for most languages and runtimes that'd be a separate service, a message queue or event bus, and then somehow integrating the worker consuming those events with the rest of your application. Maybe through a pubsub broker. We don't even think about it. GenServe consumes the stream. Erlang Distribution. Phoenix PubSub. Done. We were already there. Now we could react live to someone doing a bulk import or an admin fixing a borked record manually.

The concurrent nature of the runtime and the process model makes it much less concerning to be holding state. It makes it obvious over time what you need to be thinking about when operating GenServers. This is why I think any production app should have two nodes clustered. It will immediately teach you to respect the really hard distributed systems problems and avoid naive singletons.

Elixir thrives on events. If we can be told when a thing happens instead of needing to poll we can do all the cool stuff with nice and low latency. So look for ways to  be told things. Webhooks, WebSockets, TCP connections. These are your friends because Elixir makes it much easier than it typically is. Unfortunately there isn't always something on offer. We also have a phenomenal interface for turning less-than-live upstream data sources into as-live-as-possible things. A GenServer that polls a resource at a reasonable interval and emits the events is generally straightforward to build and then you are back to consuming messages in your happy ol' way.

A great example of all this is Oban. Oban is the Sidekiq or Celery of Elixir. But really it isn't. That'd be selling it incredibly short. I've heard that Sidekiq is a significant contributor to the outsized cost of hosting a Mastodon instance. Mastodon is built in Rails and since it requires a lot of background processing it uses Sidekiq. Akkoma is a Mastodon-compatible Fediverse server built in Elixir and last I checked it uses Oban a fair bit. It will do more work on way less hardware.

Oban uses Postgres heavily, also supports MySQL and SQLite but the Postgres is the big one, and it relies on it. Oban uses BEAM fundamentals to optimize things away from the more performance sensitive parts of Postgres. And it all gets to be fast, event-friendly and you don't add a service.

All this requires some thought and consideration to do in serious distributed production apps of course. And when done well unlocks awesome possibilities. Speed being an unbeatable feature in many systems.

In Nerves devices you generally are dealing with a single device and then you don't have nearly so much concern about solving distributed problems. Single GenServers actually become a very appropriate solution for many things. And if you want PubSub you might be using PropertyTable since that is already an underpinning of VintageNet's networking. And most hardware is either interrupt-driven, meaning it produces events. Or you poll it at an interval which then lets you decide what to do based on the values read. It all shakes out to events and messaging in the end.

And for hobby projects like my desktop control dealio called auto you don't have many very serious concerns and that thing keeps no data store and just spins up a bunch of GenServers to do things. It is all Phoenix.PubSub, all the time. Very quick way to get rolling. In the longer term you may run into the need to centralize some of the logic but this works well currently.

We have a runtime that behaves differently. Definitely use that.

Thank you for reading. I appreciate it.