Physical knobs & userspace drivers in Elixir

Underjord is a tiny, wholesome team doing Elixir consulting and contract work. If you like the writing you should really try the code. See our services for more information.

I like production gear. Audio, video and .. miscellaneous. I like things that seem like they’ll make me oh-so-very productive. I’m a sucker prosumer as a hobby. Let’s talk about Elgato, Elixir and why I never actually get anything done. This is unfortunately not sponsored by a prosumer brand, or anyone, that’d have been something.

Ad-style pitch for a business-thing: I help companies find competent, capable or promising Elixir developers. If you build in Elixir and want to spend less time finding great candidates, reach out.

I was first introduced to Elgato for their lighting panels, the arms that hold them up and the Camlink 4K. I was stepping up my streaming/meeting camera situation by putting my then only good camera, a Sony ZV-1, on a stand and improving the lighting.

At the time the camera didn’t do USB streaming, recent firmware allows it to do 720p.. I got the Elgato Camlink 4K instead. It takes the HDMI out from the camera and lets me treat it as a webcam via USB. It has been essentially problem-free for me under Linux continuously. It can hang but I’d blame my USB controllers more than the hardware, other hardware has also complained.

The Elgato Keylights have been nice enough as well. Hook them up to wifi via the app, a Mac or Windows. Then they are accessible to be controlled and they provide nice diffused light that does the job. As I moved from Mac to Linux I started controlling them with a Python script using leglight and also some experiments with Nerves and my own keylight library.

I’ve been eyeing the Stream Deck because all the Mac productivity podcasts I have a bad habit of following has been hyping it up. Of course. No Linux support. So I held off. Then I saw the Stream Deck Plus which has physical knobs. And then the Stream Deck Pedal which is a pedal. I am not hard to impress. Unfortunately.

No Linux support from Elgato but I’ve been feeling up for a challenge. I checked and saw existing implementations in Python, Node, etcetera and therefore I considered it doable. I’ve ported from Pyton before with good results. And if I got hard stuck it seemed like streamdeck-ui is a thing. (note: no Stream Deck Plus support at time of writing.).

The Stream Decks are hidraw (not hi draw, hid raw) devices in Linux terminology which essentially means they can be driven with a very simplistic interface and all the knowledge about the specific device is enshrined in the userspace code that talks to it. Pretty much the same thing in other OS:es. If you are using the Stream Deck software to set things up, there is literally nothing saved on the device. The software is polling the device for activity, generating and pushing new images whenever the screens need to change and such. Even the Stream Deck Plus with a neat variety of controls and interfaces does very little beside retain an image and provide input signals.

I’d like my code to work on both Linux and the occasional Mac (I still run an Apple laptop) and I don’t mind accidental Windows support. Rather than work with Elixir libraries for straight hidraw I looked for something using the library hidapi which provides cross-platform abstractions with a simple API. I found hid on Hex.pm, unfortunately the source repo is gone.

This is already much more elaborate than the Elgato Keylight. That’s a very straight-forward JSON API. This was more challenging but also a lot more interesting.

The hidapi library provides:

• hid_enumerate - list devices
• hid_open - open a device
• hid_write - write an “output report” to a device
• hid_read - read an “input report” from a device
• hid_send_feature_report - this is another type of write
• hid_get_feature_report - this is another type of read
• hid_close

There are some other I don’t care about as well providing more info about the device and such.

The NIF library I had didn’t have all of those hooked up, I believe I needed to add sending feature reports. I imagine the original author just happened to not need that. I also fixed the makefile to work on recent MacOS and actually fixed a bug or two in the C code. As someone who doesn’t write C, essentially ever, that felt like an achievement. You can see my changes here.

This stuff happened during the process of porting. I worked off of python-elgato-streamdeck as a reference implementation. Particular props to this PR for Plus support. I also checked my work against node-elgato-stream-deck a few times as I wasn’t sure the Python one had it right. Some weird build issue with dynamic shared libraries and ctypes meant I never actually got the Python one running.

The device offers the following functionality:

• 8 buttons aka. “keys” with full color LCD screens under them. Down, Up. That’s all they do.
• An LCD touchscreen strip. 800x100, full color. Can register short tap, long press and a weird kind of drag.
• 4 knobs, turns in steps, left, right, can also send multiple steps if turned quickly. Can also be pressed as a button, up/down.
• Controllable LCD brightness (it is really just one panel).

All of this is done over that USB HID interface.

I had very little trouble with the Elixir port overall. My most long-standing wtfs where around sending images to the StreamDeck and making sure they worked. I have a stream of me struggling to update the LCD where in the end I had just flipped two fields in the messages I was sending. Buttons and knobs took very little time. Compared to when I was making Inky over 3 years ago I have written Elixir pretty much daily since. It is nice to feel that the language fades away when solving problems.

Building and matching binaries could have been more challenging if it wasn’t for all the work I did with The Changelog for chapter support where I implemented an ID3v2 library for Elixir. That made me very familiar with all of the binary pattern matching and constructing binaries.

Let’s actually have some code in here:

def set_brightness(d, percent) when is_integer(percent) do
	percent = min(max(percent, 0), 100)
	payload = rightpad_bytes(<<0x03, 0x08, percent>>, 32)

	write_feature(d, payload)
end

Not too complicated. How about reading keys?

def read_key_states(d) do
	{:ok, binary} = read(d, 14)
	binary
end

Also simple. Writing a new chunk of image to the LCD is a bit more involved..

def set_lcd_image(d, x, y, width, height, binary) do
    send_lcd_image_chunk(d, binary, x, y, width, height, 0)
end

defp send_lcd_image_chunk(_, <<>>, _, _, _, _, _), do: :ok

defp send_lcd_image_chunk(d, binary, x, y, width, height, page_number) do
    if width + x > 800 do
      raise "too wide"
    end

    if height + y > 100 do
      raise "too high"
    end

    bytes_remaining = byte_size(binary)
    payload_length = @config.image.report.touchlcd_payload_length
    length = min(bytes_remaining, payload_length)

    {bytes, remainder, is_last} =
      case binary do
        <<bytes::binary-size(payload_length), remainder::binary>> ->
          {bytes, remainder, 0}

        bytes ->
          {bytes, <<>>, 1}
      end

    header =
      [
        0x02,
        0x0C,
        <<x::size(16)-unsigned-integer-little>>,
        <<y::size(16)-unsigned-integer-little>>,
        <<width::size(16)-unsigned-integer-little>>,
        <<height::size(16)-unsigned-integer-little>>,
        is_last,
        <<page_number::size(16)-unsigned-integer-little>>,
        <<length::size(16)-unsigned-integer-little>>,
        0x00
      ]
      |> IO.iodata_to_binary()

    16 = byte_size(header)

    payload = header <> bytes

    payload = rightpad_bytes(payload, @config.image.report.touchlcd_length)

    1024 = byte_size(payload)

    case write(d, payload, "set lcd image chunk") do
      {:ok, _} ->
        send_lcd_image_chunk(d, remainder, x, y, width, height, page_number + 1)

      err ->
        err
    end
  end

Still not unreasonable. Just longer.

I shared some in-progress work both in this Short and on the livestream.

The end result is the streamdex library which supports the Stream Deck Plus right now as well as the Stream Deck Pedal (literally three keys, very simple). It is pretty rough, alpha-level, as I only took it to the point where I could use it. If you have a Stream Deck and interest in Elixir, feel free to contribute your device.

As a self-taught web dev it is wild to me that I’m writing something that can be fairly called device drivers right now. I’m not claiming they are advanced or impressive. Userspace drivers like this one live on top of many layers of more complex implementation. I just know that teenager me is nodding his head right now. He is pretty proud.

I’m also very glad that someone else has done the reverse engineering. I did fire up Wireshark a couple of times, actually how I found the final swapped byte for the LCD, but I don’t want any credit for figuring out how these work.

Ever since I got it fully operational I’ve been hacking at a crude but effective project for running a bunch of my computer .. stuff. That’s also public.

So far I’ve tried:

• Lights
  • keys for on/off
  • a knob for brightness that can be pressed for on/off
  • a knob for light temperature and pressing the knob resets that to a good default
• Calendar
  • Show current and upcoming calendar events on the LCD which I mostly ripped from my calendar gadget eInk project. Simplistic display still, much more fun to be had.
• Play/Pause
  • A key that toggles and uses dotool to send the media play/pause key event to the OS. Made a quick and nasty dotool_elixir library that pulls that together and compiles the Go code.
• Mic mute
  • Both a key for toggle and a pedal for press. Sends micmute key event to the OS. does not work currently on my KDE :(
• Key icons
  • Made a small library called bs_icons which pulls the Bootstrap Icons repository and provides those for me in my Elixir project. I then mangle into keys.

Building on the backs of the community and ecosystem like this I have gotten amazing mileage out of Kip Cole’s great Image library. I’ve also used resvg for turning SVG into PNG, which I apparently could have done with Image. On the plus side that also brought in Rust as well which meant achieving basic hipster compliance. Now I just need to change my HID library to use Zig and I’ll have aced it. That might happen actually.

Things yet to try:

• Touchscreen
  • Want to render much more interesting UI on it. Image can do so much fun stuff and I bet I could make it play nice with Brian Cardarella’s easing library. I’ve already tested a bit and Image can update the screen at just shy of 60hz. No idea if that looks reasonable though. We can expect to push 30hz with no issue I expect. Image provides operations like blur and ripple which I’d love to animate. I made a Short showing the silliness I’m after.
  • Using the touch at all. I’m most curious about the drag. I couldn’t actually see that Elgato uses that so it might be too crap. But I want to see what I can make it do. Build a whole little touch UI in there.
• Update keys while being pressed, trigger events on key up after down instead of down. Those kinds of niceties.

This is a really fun thing to poke around with when I have some moments for myself. Just making it do more and more. Exploring how I can combine more and more sources of data, inputs and outputs. I have a very simplistic GenServer-driven approach right now that I’d like to think about abstracting a bit. If I want to do better UI I need to keep the state of the UI differently and I’ll need to manage timing for animations.

I have a bunch of ideas around how to wrap libraries that provide new capabilities in a way where they plug into this in a more flexible way. Early thoughts. Might go nowhere.

When they market macro-pads like this for productivity I don’t think they usually mean that you’ll write tons of lines of code to make them do things. For me, this device has made me very productive. In a very specific way.

Underjord is a 4 people team doing Elixir consulting and contract work. If you like the writing you should really try the code. See our services for more information.

Note: Or try the videos on the YouTube channel.