Level flooring

How much can we raise the floor?

Without incurring deeply painful trade-offs?

This is a fundamental tension in software engineering but I think it is a daily and very concrete concern of the Elixir ecosystem. We can look at LiveView, Ash Framework and Nerves all carry examples of this and different choices.

One popular architectural choice that raises the floor of capability is Event Sourcing. And it is fundamentally a way to keep a higher level of resolution for your data than what you get if you mutate the existing data. An append log modelling the full history rather than the state at any one time. Some people definitely want to Event Source All The Things but most proponents that are somewhat serious about it do consider it a selective tool suitable for parts of a domain. Opting everything into always being event-sourced would be raising the floor but also locking you into some very hard problems around retention and performance.

LiveView raised the floor in terms of providing a high-level abstraction for writing server-driven web UI. It could have attempted the more audacious effort of supporting local first, fully offline websites and more. The moment the state splits off from the server and can live both locally and server-side you have a hilarious amount of new stuff to consider. They chose to raise the floor for a constrained set of use-cases but in a way that was sustainable and maintainable.

The local-first audacious UI-paradigm still has a torch-bearer in Hologram which is tackling a lot of hard things. Btw, if your company wants local-first Elixir, you should reach out to Bart about supporting his effort. I've been shepherding his EEF stipend and he is one focused and productive individual. I think Hologram has the potential to be a next disruptor from the ecosystem. Another different choice and trade-off made.

Phoenix chose to be less stuff than Ruby on Rails. Solving fewer problems for you. To a large extent because the way the problems are solved in Rails are not necessarily satisfying solutions. So instead of locking the ecosystem down that path they left Phoenix way simpler and more open-ended as a web framework. The floor they've kept high is around performance, useful building blocks for real-time applications and a great not-quite-ORM data layer with Ecto.

Ash tackled the Rails use-case. But instead of rebuilding Rails on top of Phoenix with the known set of good and bad from Django, Rails, et. al. he chose to try something more ambitious. Something way harder. Ash brings up the floor for access policy and safety. It raises the abstraction level a ton to make you model your system more as data to allow more automation, more derived structures, interfaces (APIs, UIs, etc). The trade-off being that you accept a decent chunk of Ash ideas. You put a decent bit of trust in Ash itself. But in return you get a considerably more fool-resilient system and you get a lot of tedious stuff for free. Code you don't need to write or maintain.

Erlang itself makes interesting choices in this regard. It enforces some constraints around immutability and message passing and in return provides trivial concurrent computing with a lot of safe defaults in terms of pre-emption and resilience to errors.

Nerves is interesting in this regard. Any Nerves device would be expected to support or implement a baseline of the Nerves functionality. They are all recommended industry practices. A/B partitions for safe hands off recovery on failed updates. Watchdog reboot and recovery. Minimal OS, for security, simplicity and minimizing the number of things that can fail. Read-only root filesystem, for many reasons. Partial/delta firmware updates to save on data usage.

A while back Frank implemented the RPi tryboot mechanism meaning the Raspberry Pi devices running Nerves can now fall back properly between A and B partitions even when devices fail to boot. This means the RPi devices are now closer to the level of the more industrial boards that Nerves runs on. By default. We want this for all users.

I am eyeing Secure Boot. Which is not one mechanism. It is one mechanism per device. But there are steps we can take toward it. I want to provide a Raspberry Pi system that is utilizing all the RPi security features available. By default. This is a hard floor to raise for everyone though. Working with Secure Boot tends to mean locking down bootloaders, setting permanent keys. It will probably always be an option, not fully default. We can add some integrity checks by default, that's cool. Full Secure Boot, disk encryption and such would mean making very limiting choices for the developer. Probably we ship dm-verity by default, meaning we get extra integrity checks and the foundations are there for extending the Chain of Trust. It also makes a variety of disk encryption schemes relatively easy.

The Elixir ecosystem is recognized, over and over again, for having this nice higher quality baseline. Higher default effort and aspiration. Tempered by pragmatism.

What are the best examples of raising the floor you know of. In Elixir or beyond?

Thanks for reading.