jssfr.de

Darktable styles mimicing Fujifilm film simulations, version 1.1

2026-01-04T17:30:00+00:00

This post announces a patch release for the darktable styles I published two days ago.

This patch fixes the following two issues:

Module version conflict warnings when applying or hovering over the styles
Lingering modules when applying any other style after Classic Neg., Acros or Monochrome.

The first issue was caused by me trying to be too clever when disabling conflicting modules (filmic rgb, sigmoid and base curve).

The second issue was caused by Classic Neg., Acros and Monochrome using additional modules which were not in use by the other styles. Since styles only ever append to the history, this meant that for example the colour tint of Classic Neg. would remain if you applied Velvia on a picture which previously had Classic Neg. applied.

The above changes have no influence on the visuals, so the new version is called 1.1 and is available on the dtsolve page.

Fujifilm Filmsimulationen als Darktable-Stile, Version 1.1

2026-01-04T17:30:00+00:00

Mit diesem Artikel möchte ich ein Patch-Release für die darktable-Stile, die ich vorgestern veröffentlicht habe, bekanntgeben.

Dieser Patch behebt folgende Probleme:

Darktable zeigte Versionskonfliktwarnungen an, wenn die Stile angewandt wurden oder mit der Maus darüber gefahren wurde.
Die Stile für Classic Neg., Acros und Monochrome haben beim Stilwechsel Module zurückgelassen.

Beim ersten Problem habe ich schlicht versucht zu clever zu sein als es darum ging, über die Stile Module auszuschalten, welche mit den Stilen im Konflikt stehen (Filmic RGB, Sigmoid und Basiskurve).

Das zweite Problem entstand dadurch, dass Classic Neg., Acros und Monochrome Module verwenden, die von den anderen Stilen nicht verwendet werden. Dadurch wurden diese Module beim Stilwechsel nicht ausgeschaltet oder zurückgesetzt, sodass z.B. die Farbtönung von Classic Neg. in den Spitzlichtern und den Schatten zurückbleiben konnte.

Die obigen Änderungen haben keinerlei Einfluss auf das visuelle Ergebnis, daher trägt die neue Version die Nummer 1.1 und ist ab sofort auf der dtsolve-Seite zum Herunterladen verfügbar.

Darktable Styles mimicing Fujifilm's Film Simulations

2026-01-02T08:30:00+00:00

TL;DR: I made Darktable styles which look quite close to what Fujifilm does with its Film Simulations on X-series interchangable (and non-interchangable, I suppose) lens cameras. Get the styles on the dtsolve overview page.

Table of Contents

Motivation
- More flexibility
- Better user experience
The styles
- Acros
- Astia
- Classic Chrome
- Classic Neg
- Eterna
- Eterna Bleach Bypass
- Monochrome
- Pro Neg. Hi
- Pro Neg. Std
- Provia
- Reala Ace
- Velvia
Comparison to similar projects
What's next

Motivation

Before I got my Fujifilm camera in 2023, I had been shooting with a 2000s-era Canon camera. The JPEGs produced by the Canon were usually okay, but not stunning, which made me do postprocessing on them using darktable fairly often. With the Fujifilm X-T4, I faced a much different challenge: I was unable to get close to the visual appeal of the out-of-camera (OOC) JPEGs using darktable, to the point that I mostly stopped using the RAW files by early 2024.

The reason for that is what Fujifilm calls "Film Simulations". Apparently, they sampled film stock and made magic which made their JPEGs look as if shot on that specific analogue film. The styles range from highly saturated styles aimed at landscape photography ("Velvia") via solid standard profiles ("Pro Neg Hi") to completely monochrome ("Acros"). Those film sims can be selected at will and the live view in the viewfinder and on the LCD shows a realistic preview of what the photo's JPEG will look like.

This is actually a good thing first and foremost: it saves a lot of time! I can do a lot of development right in the camera, especially since it not only supports the film sims themselves, but allows tweaking them further, with options like tone curve adjustments, saturation changes and even a basic "local contrast" filter ("Clarity").

But, there are two downsides: lack of flexibility in the cases where postprocessing is actually needed and, in particular for newcomers, a bad user experience.

More flexibility

Sometimes there are situations where the JPEG just does not cut it. There can be two main reasons for this:

A snapshot taken quickly, leading to suboptimal exposure, wrong film sim, bad white balance etc.
The want to do extra or more complex processing, such as masked edits in order to remove light pollution or sharpening/blurring specific areas.

Some of this can be addressed by sending the files back to the camera and re-exporting them using the Q button. That way, one can either just tweak the initial recording parameters (such as white balance, film sim, tone curve etc.) or even do a high-resolution (16-bit TIFF) export of the image, which in theory lends itself to quite a bit of additional processing.

The 16-bit TIFF files are, however, huge (158 MiB on the 26 MP sensor of the X-T4, while a compressed RAW is around 20-30 MiB). Also, this process is cumbersome. First you have to copy the files back to the camera, then you have to fiddle with the buttons to reexport it. Afterwards you need to copy the files back and check if the result pleases and otherwise repeat the entire process¹.

Better user experience

The other reason for doing this was that I did not like the first user experience people have with darktable. When you import a RAW file in darktable, it shows you the embedded thumbnail by default, which looks (almost, or exactly) like the OOC JPEG.

Once you open the picture in the darkroom though, it looks very different. Typically it is much blander, both in terms of colour and in terms of contrast.

I envision a future where darktable can apply a style which matches the look of the OOC JPEG almost exactly by default, giving the user a solid starting point for further edits.²

Enter: the styles.

The styles

To download the styles, go to the dtsolve overview page.

These styles were created by letting a numeric solver tune the darktable parameters until the result got as close as possible to the corresponding OOC image. No Fujifilm code has been reverse engineered for this.

The styles do not reproduce the original look completely. Due to the nature of how they are made (using already-existing darktable modules), a 100% reproduction of the original style would be quite unlikely to be possible for any given style. However, most of them come close enough to pass for the real thing, unless in a side-by-side comparison.

Nonetheless, let's do the side-by-side comparison below.

Tip: This section works best with JavaScript enabled, because it will allow you to more interactively compare the images.

The left side is always the OOC JPEG and the right side is the RAW processed with the styles (plus lens correction, setting the white balance to "as shot in camera", and setting the exposure module to +0.7 EV without any other compensations).

Acros

Astia

Classic Chrome

Classic Neg

Eterna

Eterna Bleach Bypass

Monochrome

Pro Neg. Hi

Pro Neg. Std

Provia

Reala Ace

Velvia

Comparison to similar projects

There have been others who did similar things in the past. To my knowledge, all of these attempts have been based on creating look-up tables (LUTs). These look-up tables map an input colour to an output colour and thus allow to apply all kinds of visual effects to an image.

LUTs have, however, two important downsides:

Effects which are baked into a LUT cannot easily be tweaked, since applying a LUT is an all-or-nothing type of thing. This is most problematic in the case of film simulations, because they include the crucial tone mapping step of the editing workflow.

During tone mapping, the brightness curve of the input RAW image is adjusted to produce a pleasant, often contrasty image. In darktable, tone mapping is typically handled by the filmic rgb, sigmoid or AgX (new in 5.4) modules.

A LUT which includes the tone mapping step cannot be mixed with a darktable module which does tone mapping. That means that parameters which configure the tone mapping are baked into the LUT and cannot be (easily) tweaked when the LUT is applied, for example for a slight reduction of contrast in the highlights.

(Colours, on the other hand, can typically be tweaked nicely even when a LUT is in use, e.g. using the rgb primaries or color balance rgb modules.)
LUTs cannot be (easily) packed into redistributable dtstyle files. While some LUT types are actually embeddable, normally darktable just references the path to the LUT file. That complicates redistribution of the styles significantly, in particular when LUT effects are supposed to be mixed with non-LUT effects.

The styles created here do not use any LUTs and are thus very malleable.

What's next

There are three key areas where I want to build on this work:

Some styles are not yet perfect and need more refinement to be released. In particular, Nostalgic Neg is currently unsolved and it does not look like it can be done with darktable as it is today.
So far, the in-camera tone curve adjustments and dynamic range modes are not supported yet. This is currently WIP.
In the future, I want darktable to apply the styles automatically to any Fujifilm RAW image based on its EXIF metadata. This will require some extensions to darktable itself and/or its Lua API to make the necessary information available to tools like apply_camera_style.lua.

Once the infrastructure for that is there, I want to have the styles packaged with darktable, so that any darktable user can profit from them.

And of course, anyone³ can go ahead and use dtsolve to produce their own dtstyle files for their favourite look. And the basis does not even have to be an OOC JPEG, it can be anything for which you can obtain a RAW reference image (for example, you could solve against some LUT you downloaded to make it malleable).

Finally, I'll publish a "making of" of the styles which I have shared here. That post will be very technical and aimed more at hackers than at photographers. Stay tuned if you are part of that intersection.

There is also the FUJIFILM X RAW STUDIO software which automates the process, but it is not available on Linux (not even with Wine). ↩
In fact, darktable 5.0.0 has started shipping styles and a Lua module which supposedly help with that.

Unfortunately, the styles shipped with darktable are sometimes far off the actual OOC image. This project is part of an effort to change that. ↩
Okay, not anyone, I suppose. dtsolve requires a Linux system and is not the most user-friendly tool there is (nor do I intend to make it that---but others are free to). ↩

Fujifilm Filmsimulationen als Darktable-Stile

2026-01-02T08:30:00+00:00

TL;DR: Ich habe Stile für Darktable gemacht, welche sehr nah an den Fujifilm Filmsimulationen dran sind, die man von den Kameras der X-Serie kennt. Die Stile kann man auf der Übersichtsseite für dtsolve herunterladen.

Table of Contents

Warum?
- Mehr Flexibilität
- Einsteigerfreundlichkeit
Die Stile
- Acros
- Astia
- Classic Chrome
- Classic Neg
- Eterna
- Eterna Bleach Bypass
- Monochrome
- Pro Neg. Hi
- Pro Neg. Std
- Provia
- Reala Ace
- Velvia
Ähnliche Projekte
Wie geht es weiter?

Warum?

Bevor ich im Jahr 2023 zu Fujifilm gewechselt bin, hatte ich eine Canon-Kamera aus den späten 2000ern. Die "out-of-camera" JPEGs (OOC-JPEGs), die die Canon produziert hat, waren okay, aber nicht gerade umwerfend, weshalb ich die RAW-Dateien oft mit darktable nachbearbeitet bzw. entwickelt habe. Mit der Fujifilm X-T4 hatte ich plötzlich ein ganz anderes Problem: Die JPEGs sahen so gut aus, dass ich echte Schwierigkeiten hatte, mit darktable auch nur in die Nähe dieses Looks zu kommen. Das führte letztendlich dazu, dass ich seit Anfang 2024 die RAW-Dateien quasi nicht mehr genutzt habe.

Der Grund dafür ist das Feature, das Fujifilm "Filmsimulationen" nennt. Anscheinend ist Fujifilm hingegangen und hat klassische Analogfilme hergenommen und dann Magie praktiziert die dafür sorgt, dass ihre JPEGs so aussehen als ob sie mit dem jeweiligen charakteristischen Analogfilm aufgenommen worden wären. Die dabei verfügbaren Stile gehen von einem recht gesättigten Film für Landschaftsfotografie ("Velvia") über einen soliden Standardlook ("Pro Neg Hi") bis hin zu komplett schwarz/weiß ("Acros"). Diese Filmsimulationen können beliebig gewählt werden und man sieht im Live View direkt, wie das Foto am Ende aussehen wird.

Insgesamt ist das erstmal etwas Gutes: Dieser Prozess spart unmengen an Zeit! Einen Großteil des Entwickelns das ich früher in Darktable gemacht habe, mache ich heute direkt in der Kamera. Die kann nämlich zusätzlich zu den Filmsimulationen auch noch andere übliche Bearbeitungsschritte wie leichte Anpassungen der Helligkeitskurve oder Sättigungsänderungen und es gibt sogar einen einfachen "Lokaler Kontrast"-Effekt.

Trotzdem gibt es zwei Nachteile: mangelnde Flexibilität, wenn doch mal etwas im Nachgang angepasst werden muss, und für Einsteiger insbesondere eine recht frustrierende User Experience.

Mehr Flexibilität

In manchen Situationen reicht das JPEG einfach nicht aus, zum Beispiel weil:

man nur schnell einen Schnappschuss gemacht und dabei die Belichtungszeit versemmelt, die falsche Filmsimulation drin oder den Weißabgleich verstellt hatte oder
man gerne weitere Entwicklungsarbeiten durchführen möchte, z.B. gezielte, maskierte Bearbeitungsschritte um Lichtverschmutzung wegzuretuschieren oder Einzelteile des Bildes nachzuschärfen

Den ersten Stichpunkt insbesondere kann man lösen, indem man die RAW-Dateien zurück auf die Kamera lädt und mithilfe der Q-Taste neu entwickelt. Dabei kann man entweder die ursprünglichen Aufnahmeparameter verändern (also Weißabgleich, Filmsimulation, Helligkeitskurve etc.) oder man kann sich gleich eine hochauflösende Bilddatei (16-bit/Kanal TIFF) exportieren lassen. Letzters ist zumindest in der Theorie eine solide Grundlage für weitere Bearbeitungen.

Die 16-bit TIFF-Dateien sind allerdings riesig (wir reden von 158 MiB beim 26 Megapixel-Sensor der X-T4, im Gegensatz zu 20-30 MiB (verlustfrei) komprimiertem RAW). Außerdem ist der Prozess sehr umständlich: Erst muss man die Datei wieder auf die Kamera kopieren, dann mit den Tasten hantieren um den Export so einzustellen wie man ihn gerne hätte. Schließlich müssen die Dateien auch wieder zurück und dann erst kann man auf dem großen Bildschirm gucken, ob das Ergebnis passt¹.

Einsteigerfreundlichkeit

Der andere Grund warum ich mich mit diesem Thema befasst habe, ist, dass mir die Einsteigererfahrung mit darktable nicht gefallen hat. Wenn man eine RAW-Datei in Darktable importiert, sieht man erstmal das eingebettete Vorschaubild, welches (fast) genauso wie das JPEG aussieht.

Sobald man das Bild aber per Doppelklick in der Dunkelkammer öffnet, sieht es ganz anders aus: üblicherweise viel zu sanft im Kontrast und in den Farben.

Ich wünsche mir eine Welt, in der darktable den Stil des JPEGs beim Laden automatisch auf die RAW-Datei anwendet. So sollten Einsteiger:innen einen guten Ausgangspunkt für ihre eigene Bearbeitung haben².

Auftritt: die Stile.

Die Stile

Die Stile gibt es auf der dtsolve Übersichtsseite zum Herunterladen.

Um diese Stile zu erzeugen, habe ich einen numerischen Löser die darktable-Parameter so lange verstellen lassen, bis das Ergebnis so nah wie möglich am zugehörigen OOC-JPEG dran war. Insbesondere wurde kein Fujifilm-Code hierfür reverse engineert.

Da diese Stile nur bestehende Darktable-Module so konfigurieren, dass das Ergebnis möglichst korrekt aussieht, können sie in der Regel nicht zu 100% an das Original herakommen. Allerdings sind die meisten so dicht dran, dass man sie schon direkt neben das Original halten muss, um den Unterschied zu sehen.

Eine solche Gegenüberstellung gibt es im Folgeden.

Tipp: Dieser Abschnitt funktioniert am Besten, wenn JavaScript aktiv ist. Dann gibt es nämlich einen sehr interaktven Vergleich der Bilder.

Auf der linken Seite ist immer das OOC JPEG zu sehen und die rechte Seite zeigt das RAW-Bild mit dem dtsolve-Stil. Zusätzlich zum Stil wurde die Objektivkorrektur eingeschaltet, der Weißabgleich auf "wie mit der Kamera aufgenommen" und das Belichtungs-Modul auf die standardmäßigen +0.7 EV eingestellt.

Acros

Astia

Classic Chrome

Classic Neg

Eterna

Eterna Bleach Bypass

Monochrome

Pro Neg. Hi

Pro Neg. Std

Provia

Reala Ace

Velvia

Wie geht es weiter?

Es gibt drei Gebiete wo ich weitermachen will:

Ein paar Stile sind noch nicht perfekt und brauchen noch Feinschliff, damit ich sie veröffentliche kann. Das betrifft derzeit vor allem Nostalgic Neg. und Sepia. Bei Nostalgic Neg. scheint es mir aktuell so, als ob die verfügbaren Darktable-Module für diesen konkreten Look schlicht nicht ausreichen.
Bisher können die Stile lediglich die Filmsimulation selbst abbilden. Ich möchte auch noch die anderen Kameraeinstellungen wie die Anpassungen der Helligkeitskurve oder die DR100/DR200/DR400-Modi unterstützen.
Zukünftig möchte ich wie gesagt, dass Darktable diese Stile automatisch anhand der EXIF-Informationen im RAW-Bild anwenden kann. Dafür werden einige Anpassungen in Darktable selbst erforderlich sein, um die nötigen Informationen in der Lua-API zur Verfügung zu stellen und dann z.B. das bestehende apply_camera_style.lua-Skript zu ergänzen.

Sobald das umgesetzt ist, möchte ich die Stile direkt mit Darktable ausliefern lassen, sodass alle Nutzer:innen was davon haben.

Natürlich können auch alle³ anderen hergehen und dtsolve verwenden, um ihre eigenen dtstyle-Dateien für ihre Lieblingslooks zu erzeugen. Die Grundlage muss nichtmal ein OOC JPEG sein, es reicht irgendetwas wofür man eine zugrundeliegende RAW-Datei beschaffen kann. Man könnte z.B. eine LUT anwenden und dann dtsolve drauf loslassen, um sie in normale Darktable-Module umzuwandeln und so leichter anpassbar zu machen.

Zu guter Letzt werde ich demnächst ein "Making Of" für die Stile, die ich hier vorgestellt habe, veröffentlichen. Das wird dann sehr technisch (noch technischer als dieser Post) und zielt eher auf die Schnittmenge von Hacker:innen und Fotograf:innen ab. Falls ihr zu dieser Schnittmenge gehört... bleibt dran.

Die Existenz der FUJIFILM X RAW STUDIO-Software, die Teile von diesem Prozess automatisiert, ist mir bekannt. Allerdings läuft das Programm nicht auf Linux (auch nicht mit Wine). ↩
Tatsächlich liefert darktable seit 5.0.0 Stile und ein Lua-Modul mit, die das Problem lösen sollen.

Leider sind die mit darktable ausgelieferten Stile stand heute teilweise sehr weit vom tatsächlichen out-of-camera JPEG entfernt. Dieses Projekt hier hat auch zum Ziel, das zu ändern. ↩
Naja, vielleicht nicht alle. dtsolve erfordert ein Linuxsystem und ist nicht wirklich das Benutzerfreundlichste Tool der Welt. Derzeit habe ich auch keine Pläne das zu ändern, aber Andere können das gerne tun. ↩

ROCKPro64 as Multimedia System and NAS: A long journey

2022-09-26T19:00:00+00:00

Update (2026-01-01): Upon migrating this post from the old site, I noticed that I should probably leave some remarks about the current usage of the RockPro64.

In particular, it is not used for music playback or DRM-protected video content anymore.

The S/PDIF hack turned out to be too unreliable. The signal got interrupted quite often for unclear reasons, even when reducing the powersave modes. Another downside was that I wanted to have surround sound even outside movies, which meant that S/PDIF was quite the limitation.

Due to the HDMI limitations of the RockPro64, I switched back to a Pi for music playback (which now uses an NFS share from the RockPro64 as data source at least).

Playback of DRM-protected content has also moved back to x86 laptops, because while libwidevinecdm can be made to work on the RockPro64, the kodi plugins for the services are so unreliable that I eventually gave up on that.

Abstract

The ROCKPro64 is a single-board computer based on the RK3399 hexacore ARM system-on-chip. It supports PCIex4, hardware decoding of many modern video codecs and gigabit ethernet. In this article, I describe my journey to convert the device into a NAS-multimedia-system hybrid, solving challenges such as DRM incompatibilities and performance issues.

TL;DR:

Use a kernel from debian-backports
If you do any kind of realtime/media playback/high throughput: Disable the deepest sleep state and potentially frequency scaling.
If you want to use mpd: patch it against the unaligned access or get a more recent release.
If you want to use kodi with libwidevinecdm: install an arm64 kernel with armhf userland and patch your glibc.

Introduction

When I set up our home theater, I initially wanted it to be able to work without any "real" computers. I didn't want to have a plug a laptop in some HDMI cable in order to watch a movie or a series.

Unfortunately, it didn't work out that way. When I installed kodi on the Raspberry Pi 3B, I quickly realized that the digital rights management (DRM) logic required by some streaming services is too heavy for the poor little Pi. 1080p content would not play fluently or overheat the device within minutes (ever heard how short the audio buffer of the Pi is? that's how you'll find out).

So in the past months, full-blown x86 laptops have served as input to the HDMI channel. That is neither efficient nor convenient, given that many modern SoCs come with hardware-enabled decoding of common video codecs (even though I doubt that this can be usefully used for DRM content) and that ARM platforms may be much less power-consuming than x86 systems. And while it's possible to run Kodi on the laptop to gain its remote-controllability and integration with infrared remotes for instance (and thanks to the IR forwarder I built earlier, I can actually send IR commands via the network to the laptop), even the bi-weekly fights with xscreensaver or DPMS over when it's appropriate to turn off an output are enough.

Then someone in some chat mentioned the ROCKPro64. The ROCKPro64 is a single-board computer based on the Rockchip RK3399 system-on-chip. It features an ARM hexacore, a Mali GPU and a Hantro video processing unit. It can decode VP8, VP9 and H.264 in hardware, has gigabit ethernet and a PCIe x4 slot.

Previously, my network attached storage needs are fulfilled by a 2015 intel box which I bought in a haste when my desktop PC died (I needed somewhere to plug the disks to get to the data). It consumes 35 W in idle (with disks in standby!), so it goes without saying that it's not always online. Needless to say, that is bad for backups.

Looking at the hardware specs, the ROCKPro64 looked like it could fullfil both needs: With its hardware decoding and the hexacore (which can be clocked up to 1.8 GHz on the more powerful cores) it should easily be able to render even full HD DRM content. The PCIe x4 slot should be sufficient to connect a SATA controller which can serve the RAID 1 from the NAS.

This way, I could get rid of the Intel box, have an always-on NAS, and a more convenient home theater. Splendid, right?

Well, there are some pitfalls.

Contents

Abstract
Introduction
Shipping and other meta
Installing Debian
- Installing Debian with arm64 kernel and armhf userland
Setting up the Multimedia System
Setting up the NAS system
Fixing the Throughput
Bottom line

Note

Before we get into the nitty-gritty details, one word of warning: I will generally not repeat things you can find elsewhere here. In particular, I'll just write "apply this patch and rebuild" instead of giving you a step-by-step guide on how to actually do that. The reason is that this article contains enough information which will (hopefully!) be out of date in a year or two, even without me providing step-by-step guides for processes better documented elsewhere.

Shipping and other meta

The first thing which annoyed me was that I only found out after the fact that PINE64 ships from Asia. I don't like the thought of having that piece of hardware express-shipped via airplane. If I had known earlier, I would've gone with a local retailer, even though that would've meant a 20%+ markup on the price.

Other than that, the transaction and shipping went just fine. They don't seem to handle VAT correctly, so you will have fun with customs, though.

Installing Debian

As I mentioned, I want to do more with this than just media. Otherwise, I would probably have gone for LibreELEC or a similar Just-enough OS (JeOS) for this purpose. Hence, I will be using Debian, my favourite OS and the same system I use on all my other computers (except smartphones...).

There are ready-made installer images for the ROCKPro64 available right from Debian, which is nice.

At first, I was confused about how to boot this; I am used to images which contain an OS then self-inflate to the entire medium (like for the Raspberry Pi). I did not buy the eMMC module, so the only thing the board would boot off is the micro SD slot. So I had to boot the installer off the medium I was going to install on (which is actually possible, because the installer loads itself into memory on boot).

It took me a few attempts to get a bootable arm64 Debian system, however. To get a bootable system, I:

Did not delete the installer partition (that turned out to be irrelevant, but also useful because it offers a way to boot into a shell when things don't work)
Created a 2 GiB ext2 /boot with bootable flag
Created a 25 GiB ext4 / without any special flags.

I did not enable swap (also probably irrelevant) for the sake of the SD card.

There are a few peculiarities about this:

This is not a UEFI system, even though some pages seem to claim it is. There were no efivars in sysfs or procfs, so even if U-Boot (which is the bootloader which they ship in the SPI flash) is capable of providing an UEFI environment, it somehow doesn't provide it to the Debian installer.
U-Boot is massively underdocumented. We'll get to the challenges related to U-Boot later, when we have to re-install the system while bypassing some of the debian-installer steps.
U-Boot looks for boot.scr script files (or, as we'll see later, extlinux/extlinux.conf). The first of which it finds, it'll execute and boot. In the standard deployment of U-Boot on the ROCKPro64, it only searches on the eMMC and on the μSD.

Note

In between, I actually re-flashed the SPI because I thought some intermittent issues I had were related to an outdated U-Boot version. An image which flashes U-Boot into the SPI flash when booted can be found on GitHub.

The first thing I tested was obviously Kodi. Kodi and DRM-protected content. Which did not work, because libwidevinecdm, the library handling the DRM-protection, is not available for arm64.

Hence, I needed an armhf (32-bit) userland, just like on the Raspberry Pi. Now Debian supports something called MultiArch, where you can have things from different CPU architectures installed simultaneously.

Unfortunately, that didn't work out for Kodi, because Kodi depends on python3 (not just libpython3-*) and that binary and executable package is not multiarch capable.

Installing Debian with arm64 kernel and armhf userland

Now for the fun question: how do you install Debian with arm64 kernel but armhf userland?

Note

What this article describes is how I did it, not how I would do it again. The easiest way is probably using debootstrap and qemu on another system in order to generate a root filesystem you can then just dd onto the SD card. I was not in the mood of wrangling with qemu-user, even though the internet™ claims it's easy enough. I preferred to wrangle with a system I (thought I) understood, which would be the Debian installer (henceforth abbreviated as d-i, because I can't be bothered to type out "Debian installer" all the time).

What I did was this:

Boot the installer and go through the steps until it starts with "Installing the base system". This is where d-i calls the debootstrap utility.
Enter a shell using Ctrl+Alt+F2 and hitting enter.
Use ps and cat /proc/../cmdline to find the complete command line of the debootstrap call.
Kill debootstrap. Switch back to the main TTY (Ctrl+Alt+F1) to confirm that d-i noticed that debootstrap is gone. If it has not, also kill the parent of debootstrap. Iterate until an error message pops up.
Clean out the /target directory (but be careful not to delete mountpoints).
Call debootstrap again, but with --arch=armhf
Wait for debootstrap to finish

Now d-i is obviously a bit unhappy, but you can skip past that unhappiness. You'll now have to go through the steps manually (it will throw you back into the list of steps all the time). It will sometimes try to do the base system installation again, but notice that the chroot isn't empty. You must then refuse its question to continue, at which point it'll continue with the step you actually asked for. This may not work for the bootloader installation.

Once the installation is over, you enter your shell again. You need to chroot into the /target (make sure that /proc, /sys and /dev are bind-mounted!) and then run dpkg --add-architecture=arm64, followed by apt-get update and apt-get install linux-image-arm64 linux-base, to get the kernel installed.

Edit /etc/kernel-img.conf and add image_dest = /boot. Find the kernel version you just installed by looking into /boot and run linux-update-symlinks install $version $image_path, where you replace $version with that version and $image_path with the absolute filename of the vmlinuz file associated with that kernel.

You should now have /boot/vmlinuz as a symlink pointing at the kernel image.

Create the /boot/extlinux directory and inside that an extlinux.conf file with the following contents:

default l0
menu title Debian
prompt 0
timeout 5

label l0
menu label Debian (armhf)
linux /vmlinuz
initrd /initrd.img
fdt /dtb
append root=UUID="9d348833-9ec0-4bc9-b2b4-520fc4076740" apparmor=0

Make sure to substitute the UUID in there with the UUID of your SD card partition. You can find it by running e.g. blkid /dev/mmcblk1*.

Note

Normally, there is the flash-kernel package which is supposed to create appropriate configuration in /boot for U-Boot to understand. In my mixed arm64/armhf setup, I was unable to get it to work, though. It would generate boot.scr files which U-Boot then refused to run (SCRIPT FAILED).

Note

Nontheless, you should definitely install flash-kernel if d-i did not install it for you. It is required in order to update the devicetrees in /boot.

You can now reboot the system and should hopefully end up in your freshly installed Debian.

Setting up the Multimedia System

The kernel version in the stable archive is currently 5.10.x, and many features are not enabled during build time. That has been fixed in the more recent kernel versions, so you'll likely have to install a kernel from Debian backports if you want to use any of the more advanced features (such as video acceleration). I hear that 5.17.x is the least you want.

Installing the Music Player Daemon (mpd)

You can install and configure mpd as usual. However, because of a bug which has been fixed in mpd upstream, it is possible that you may not be able to index your library. The symptom is that mpd gets killed with the SIGBUS signal, triggered by an unaligned memory access.

Your options are:

Build your own patched mpd package
Check if by now 0.23.7 has reached the Debian repository or complain to the maintainer if not

As I was already deep in debugging this, I chose the first path (built my own package). The patch is simple:

diff --git a/src/tag/ApeLoader.cxx b/src/tag/ApeLoader.cxx
index 596ac820..3ede6a1d 100644
--- a/src/tag/ApeLoader.cxx
+++ b/src/tag/ApeLoader.cxx
@@ -72,11 +72,15 @@ try {
    /* read tags */
    unsigned n = FromLE32(footer.count);
    const char *p = buffer.get();
+   uint32_t u32buffer;
    while (n-- && remaining > 10) {
-           size_t size = FromLE32(*(const uint32_t *)p);
+           // fix alignment before dereferencing as uint32_t
+           memcpy(&u32buffer, p, sizeof(uint32_t));
+           size_t size = FromLE32(u32buffer);
            p += 4;
            remaining -= 4;
-           unsigned long flags = FromLE32(*(const uint32_t *)p);
+           memcpy(&u32buffer, p, sizeof(uint32_t));
+           unsigned long flags = FromLE32(u32buffer);
            p += 4;
            remaining -= 4;

You can just add that to the patch series of the mpd package and rebuild it locally.

Because it is already fixed upstream, I anticipate the fix becoming available soon-ish in Debian. If I get around to reverting the mpd version to one from the archives, I may even file a bug to make the maintainers aware.

HDMI issues and Enabling S/PDIF output

Now this is a nasty one. The board features multiple audio outputs. There are at least:

I2S pins exposed on the Pi-compatible header
HDMI (also via I2S internally)
Analog (via I2S)
S/PDIF

Unfortunately, there are only 3 direct memory access (DMA) blocks available for these. So you will have to disable one of them in order to make S/PDIF work.

If you have an Audio/Video Receiver (AVR) device, you may want to use HDMI directly even for audio. Unfortunately, it seems impossible to output audio with the ROCKPro64 if no display is connected and turned on in the HDMI chain. I mean this quite literally; there was simply no audio output (without any error indication) if I connected the ROCKPro64 to my Denon AVR until I also connected a display on the other end of the Denon AVR and turned it on (this took me a while to figure out…).

Note

I am really annoyed by that, so if you know a fix, let me know. I am tempted to pull HDMI pin 19 high to see if that helps, but haven't gathered the motivation yet to hack a cable. See the footer below the post for contact info.

So if you want headless music playback of considerable quality with just the things available on the board, you'll have to enable S/PDIF.

The enablement of S/PDIF is controlled via the so-called devicetree. You can read more about devicetrees on the Linux and the Devicetree page of the kernel documentation or on devicetree.org.

The current devicetree used by your board is available in compiled form at /boot/dtb (TODO: verify). To edit it, we first need to decompile it using dtc -I dtb -O dts < /boot/dtb > ~/full.dts. That will emit a few warnings, but that's ok (I hope).

Devicetree source files are organized in blocks (much like C code), delimited by curly braces.

We now have to find the blocks which match spdif. Use text search for that. You should find a status = "disabled" in those blocks. Replace that with status = "okay". Do not replace any other status = "disabled" entries!

Now because of the DMA issue mentioned above, you'll have to pick one of the i2s blocks (search for i2s@ to find them) and change the status = "okay" in it to status = "disabled". I used the first one, which corresponds (I think) to the I2S pin headers I don't plan to use anyway.

If there is no status = "okay" in the block, you may add a status = "disabled".

The best way to install your new device tree is to compile it and put it in /etc/flash-kernel/dtbs/rk3399-rockpro64.dtb. This can be done in a single step:

dtc -I dts -O dtb < full.dts > /etc/flash-kernel/dtbs/rk3399-rockpro64.dtb

Followed by:

cp --backup /etc/flash-kernel/dtbs/rk3399-rockpro64.dtb /boot/dtbs/$(uname -r)/rockchip/rk3399-rockpro64.dtb

(By using cp instead of flash-kernel, you have a .bak file to recover the original devicetree in case you made a mistake which renders your system unbootable.)

By putting the file in /etc/flash-kernel/dtbs, you ensure that you will get your modified device tree on subsequent kernel updates. The flipside of this method is that you will not benefit from a kernel update which ships with an improved device tree. As things like the cooling fan levels are also configured inside the device tree, you may want to diff the upstream device tree against your local device tree regularly. You can find the original dtb file in /usr/lib/linux-image-$(uname -r)/rockchip/rk3399-rockpro64.dtb.

After a reboot, you should see the SPDIF card in aplay -l.

Note

I eventually also disabled the other (analog) I2S output.

Installing kodi

You should probably install kodi from backports straight away. If you then attempt to play DRM-protected content, you'll find that kodi crashes.

The reason is that libwidevinecdm uses a new feature called RELR, which needs support by the runtime dynamic linker used. The linker is provided by the glibc source package and unfortunately, officially support for RELR is not even in upstream yet.

In order to fix that I took the patch against libc from LibreELEC, because it seemed smaller and easier to backport. Unfortunately, it did not apply cleanly to the libc from debian. This is the patch backported to the current glibc 2.31-13+deb11u4.

Building libc took several hours on the ROCKPro64 and it is a good opportunity to test your cooling rig :-).

Once you installed the patched libc, and possibly rebooted, it is possible to play DRM-protected content with Kodi. At least with Netflix, however and in contrast to the Raspberry Pi, 1080p content is not delivered to Kodi. I'll still have to figure that one out.

Setting up the NAS system

This is pretty straightforward. I just got a PCIe x4 SATA controller (no raid functionality) plugged it in and it worked out of the box. The one I have is based around the JMicron JMB585 chipset, if that matters to anyone.

There was one catch, though (fixed below): When using rsync to copy data off the disks attached to the ROCKPro64, it would only reach 65 MiB/s at most. Given that the disks are spinning disks with more than 170 MiB/s sequential read performance and in a RAID1 configuration, this was certainly unexpected.

Fixing the Throughput

There were two issues with the build at this point:

Music played back with mpd had gaps.
rsync did not deliver full gigabit line rate, even through rsyncd (i.e. without ssh).

I debugged both issues at length. In fact, I spent several days, including the use of an oscilloscope, just to understand what was going on with the music playback. I may make a detailed post debugging this later on, because it was a pretty weird journey.

Either way, both of these issues share a common root cause: The cluster-sleep cpu idle state.

I don't know for sure what that is about, but I think it boils down to this: When a CPU has nothing to do, in addition to reducing the clock frequency, it may also be put in a sleep state, which reduces power consumption at the cost of latency. A CPU may have multiple of those for different levels of "idleness" (however that is measured).

As it turns out, the deepest sleep state on the ROCKPro64 (cluster-sleep) is too deep.

Apparently, waking up from that state takes long enough that it both:

Prevents mpd (and other audio things, actually) from providing data to the sound driver in time, or the sound driver from sending that data to the hardware in time.
Introduces sufficient latency inside rsync itself to limit the bandwidth.

To disable the sleep state (until the next reboot, anyway) as well as cpufreq scaling, run:

echo 1 | tee /sys/devices/system/cpu/cpu*/cpuidle/state2/disable
echo performance | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

(Disabling only one of those did not help reliably.)

Now you may fear that this will make your board burn immediately, but in fact it did not have any influence on the CPU temperature I measured (though I still have to check the power consumption).

FWIW, the above change is also required to get reliable fluent media playback with kodi.

Bottom line

The ROCKPro64 is an affordable base for a power-efficient NAS system, if combined with a PCIe-to-SATA converter.

It is also suitable for music playback and playback of local media, but you'll probably have issues with streaming services due to resolution restrictions because of insufficient widevine certification levels.

There are issues, as detailed above, but they can be circumvented and will eventually be fixed upstream (except the power management thing I suppose).

In contrast to the Raspberry Pi and the surrounding ecosystem, the ROCKPro64 definitely has a worse "out of the box" experience at this point. However, if you put the work in, you are rewarded with a reliable and performant system.

I got the ROCKPro64 in July of this year and since then, it has served as NAS and music player mostly. Due to the lack of Full HD on Netflix, I haven't used it for kodi much and that use case is still served using x86 laptops.

It was still worth it, even if only because having reliable backups is really nice.

Happy hacking!

Designing an Infrared Forwarder/Sniffer/Injector Microcontroller Firmware

2022-02-26T12:00:00+00:00

In the previous post, I explained the broad setup and gave an early out for those only interested in remote controlling a Denon AVR (or similar device) from a Raspberry Pi (or similar device).

In this post, on the other hand, I'll go all in.

As mentioned at the end of the previous post, it is possible to do all of the things I did here with a Raspberry Pi, without more dedicated hardware, with various drawbacks. It is also much less fun. Here is the fun way.

I decided to make a device which can forward infrared signals in real-time, while at the same time providing a "transcript" of those signals via a serial interface. During times where no signal is being forwarded, it can also take such a transcript via a serial interface and turn it into a real IR signal. I call it a "Infrared Forwarder/Sniffer/Injector Device", or IRFSID.

Because I like a challenge, I picked the smallest microcontroller I had available to me, the Atmel ATtiny2313a 8-bit MCU. It has 2048 bytes of flash program memory, 128 bytes of RAM and 128 bytes of EEPROM. It can safely be driven at 8 MHz of clock speed, with most of the instructions completing in a single clock cycle (with the notable exception of addressed loads/stores, which we'll get back to later). Most instructions take 2 bytes (some take 4 bytes), leaving us with about 1000 instructions.

It comes with a USART peripherial, two timers (one 8 bit, one 16 bit, both PWM-capable), the ability to trigger interrupts on pin changes and some irrelevant stuff we're not going to use.

Bird's Eye Perspective

         │RX pin                         │IR in pin
         │                               │
         │               ┌───────────────┤
         │               │               │
         ▼               │               ▼
┌──────────────────┐     │        ┌────────────┐
│                  │     │        │            │
│  UART Receiver   │     │        │  Analyzer  │
│                  │     │        │            │
└────────┬─────────┘     │        └──────┬─────┘
         │               │               │
         ▼               │               ▼
┌────────────────────┐   │         ┌───────────┐
│                    │   │         │           │
│  Signal generator  │   │         │  UART TX  │
│                    │   │         │           │
└────────┬───────────┘   │         └─────┬─────┘
         │               │               │
         └──────────┐    │               │
                    ▼    ▼               │
              ┌────────────────┐         │
              │                │         │
              │  Multiplexer   │         │
              │                │         │
              └───────┬────────┘         │
                      │                  │
                      ▼                  │
              ┌────────────────┐         │
              │                │         │
              │    Modulator   │         │
              │                │         │
              └───────┬────────┘         │
                      │                  │
                      │                  │
                      ▼IR out pin        ▼ TX pin

The above is a simple block diagram of the architecture we're looking at. The IR in pin is connected to an external IR demodulator IC (such as the TSOP1738) and from there, receives the demodulated signal. The RX pin is connected to the UART TX of the Pi and receives instructions for generating an IR signal.

The multiplexer is responsible for selecting which of the two signals is fed to the modulator for forwarding. The modulator generates the 38 kHz carrier according to its input.

The signal from the IR in pin is also fed to an analyzer component which emits the timing values of the signal (duration of spaces and pulses) on the UART TX pin, which is connected to the UART RX of the Pi.

First implementation: Forwarding unit without multiplexing

It is clear that the modulation should be done in hardware. If we were to bit-bang (eww) the output, we would have to adhere to a strict timing where every ~211 instructions (or less, if 4 byte instructions are involved), we get around to toggle a pin. With that many moving parts in the design, such a strict schedule would be challenging to achieve. Also, there is no need.

The 8-bit timer can easily be used to generate the signal, even when running on an 8 MHz clock. On the ATtiny2313a, when operating a timer in phase-correct PWM mode, the input clock effectively gets divided by four. If we then let the timer count to 52, we get a 38.46 kHz carrier. As the bandpass of one of the IR chips inside the Denon is from 37 kHz to 40 kHz, it seemed safer to err on the high side (otherwise, 53 would've been the value of choice).

We can toggle the PWM output using a single bit in the timer register (COM0A0), which means that we only need three clock cycles to set, clear or toggle it.

Using a pin-change interrupt on the IR input pin, we can now easily implement a forwarder: Whenever the pin input changes, we set the COM0A0 bit to the value at the input pin.

The code for this simple thing, plus some scaffolding for the multiplexing we'll need in the future, can be viewed online.

In addition to the things described here, it has two more additional features:

It already uses the 16 bit timer to keep track of how long the individual pulses/gaps are.
It prolongs the signal by a few 38 kHz cycles; the idea here is that we know that the TSOP1738 is bound to introduce a "delay" in recognizing a pulse, while it will turn off its output almost immediately once a pulse is over. By extending each pulse by a few cycles, we recover this lost part of the signal which may help a downstream decoder properly recognizing the command.

It compiles to 378 bytes of flash and needs one byte of RAM to hold the multiplexer state (which is mostly unused). It also has a hold timer to keep the "lock" on the multiplexer for some time after the last pulse. We'll get to that in more detail later.

Global Variables Are Expensive

As hinted at earlier, arbitrary memory access is slow on the ATtiny2313a; it takes twice as long as most other instructions and it also takes twice as many bytes. The previous version of the code uses a global variable in order to multiplex the access to the modulator.

However, accessing global registers is fast: just a single-cycle instruction. If we look carefully at the program, we'll find that the global variable is really just mirroring information we already hold implicitly in the global registers. In particular:

In the initial state, timer 1 is disabled and the PWM output is also disabled.
If we are forwarding a pulse, the PWM output is enabled (though timer 1 is disabled)
After forwarding a pulse, we use timer 1 to prolong it, so timer 1 is enabled and the PWM output is also enabled.
After the prolonged output, we are forwarding a space for up to a certain time (after which we release the lock on the non-existent multiplexer), so we still need timer 1, but the PWM output is disabled.

Concisely:

Idle: Timer 1 disabled, PWM disabled.
Forwarding pulse: Timer 1 disabled, PWM enabled.
Prolonged pulse: Timer 1 enabled, PWM enabled.
Space: Timer 1 enabled, PWM disabled.

So we can just use those register values as indication in which state we are! This allows us to get rid of the slow global variable access and it reduces the code size, too.

The result can be seen in commit 38c2ae1. 296 bytes of flash, that's a 21% reduction of space. And of course it reduces the latency in the forwarding chain, because we do not need the global loads/stores.

Sending a transcript via UART

To send the transcript of the signal (duration of pauses/spaces including information about whether it is a pause or space), we need to keep track of how long they are. That means we cannot use the enablement of timer 1 anymore. However, we can use another bit: In the case of forwarding a pulse, we do not care about setting an upper limit; hence, we can use the information about whether a comparision with the compare register is enabled or not to know in which of the four states we are.

With that sorted out, forwarding the signal to the UART becomes only a matter of writing the durations into the UART data register. For this, we need to talk about baud rates.

Turns out, 8 MHz is a terrible clock if you want to achieve one of the standard baud rates. The fastest baud rate which can be hit with reasonable accuracy is 38400 (see how similar it is to the 38 kHz PWM we do?). Anything above that will have way too high of a difference.

Luckily, many UART things for Linux nowadays support "arbitrary" non-standard baud rates. In normal mode, the UART of the ATtiny2313a uses a clock division by sixteen for the baud rate generator (mostly because of how the receiver works, oversampling the signal significantly). That leaves us with 500 kHz. We divide that further by 5 to achieve the non-standard baudrate of 100000 Baud.

Transmitting an 8 bit frame using 100000 Baud takes 100us. That is about four strobes of the 38 kHz carrier and just half of the assumed lower bound of pulse durations of valid IR signals. That means we can not assume that we'll be able to transmit two 8 bit frames to relay a duration; that would risk that we overrun some buffer.

(Of course, we could put a buffer into the 128 bytes of RAM, however, given the slowness of loads and stores, I'd like to avoid accessing global memory in the fast path of the IR forwarding.)

Instead, we do the following:

Transmit durations in units of 16us
Use a single bit to indicate pauses vs. pulses
When the duration counter overruns (at 2032us), we emit a frame showing just that duration.

As we explicitly encode a pause vs. pulse, the receiver on the other end can accumulate multiple such overflow frames to reconstruct the actual duration of the pulse/pause. This makes it our encoding O(t), but this is okay: The longest pulse/pause we expect to transmit is just a few dozen milliseconds.

To detect overrun of the counter, we use the compare/match register A of the timer 1. The compare/match register B is used for timing purposes. The reason for doing it this way (even though we previously used A for timing purposes) is that the interrupt priority of the compare/match with register A is higher than the priority for the pin change, meaning that we're less likely to lose overrun information here.

In transmitting, we use the least significant bit to indicate a pause. By running timer 1 on units of 8 us, this allows us to just mask out the lowest bit of the timer value and copy it over into the uart register otherwise.

Example

Assuming an IR signal which consists of a 9000us pulse, 4500us pause, 310us pulse, 740us pause, 310us pulse (making something up here), the sniffer will emit the following bytes (in hex, linebreaks on each level change for readability):

fe fe fe fe 6c
ff ff 37
26
5d
26

Receiving instructions via UART

Now for the real multiplexing!

Receiving data from the UART is easy. After enabling the receiver and the interrupt, the interrupt handler is called whenever a byte has been received from the UART line. Before we get into the processing, we'll have to talk about the data format.

Data format

There are two options:

Use the UART as command interface ("enable output", "disable output") and let the host device handle the timing.
Use the same format as used the transcript, i.e. "enable/disable output for X microseconds".

The advantage of the former format is that it is dead-simple and could be directly mapped to the COM0A0 output. A key downside is that this relies on strict timing of UART frames; that is generally not possible with Linux host devices, as there could be arbitrary buffers involved. In addition, it would be impossible to emit signals shorter than 100us (the frame duration on the 8-bit UART) and timing frames which are just barely above 100us would be extra challenging.

The second option has the advantage that we can script the ATtiny2313a to emit a signal by pushing the description of the signal to it via the UART and have it then asynchronously generate the described signal. The downside here is that we'll need a buffer to keep the parts of the script which have not been executed yet in memory. In addition, we'll have to make sure that we do not overrun this buffer while receiving from the UART. That would, in theory, again require intricate timing on the host device, but there's a neat way around that.

For the multiplexing use case, we'll have to "lock" the modulator at any time to either the UART source or the demodulated source; it cannot be driven by both. In case of the first mode, we'd have to introduce further commands in order to manage that locking. If the modulator is currently locked, this would have to be signalled to the sender or we'd have to steal the output. We cannot signal it to the host device, because the code space for UART frames on the transmitter lane is already completely full. We could use some out-of-band signalling, but that would likely break the timing. Stealing the output would obviously cause corruption in whatever was being forwarded there and potentially also corruption of the signal which was about to be generated from the UART input, unless that defensively includes a long pause at the beginning.

The second option seems more compatible with multiplexing: We can simply buffer the script until the modulator is available, and we can lock the transmitter for exactly as long as the described signal is going.

That said, we can just reuse the same format we use for the transcript on the UART input.

Buffering and Flow Control

As the UART will generally receive faster than we execute the script, we'll have a small buffer in RAM which buffers the incoming script until it is processed. If the host system was transmitting too fast, however, we'd overrun the buffer.

To avoid this, we'll employ a thing commonly supported by UARTs, which is called Hardware Flow Control. Hardware Flow Control allows the embedded device to signal to the host whether it is currently able to receive data. This is done using an active-low Clear-To-Send (CTS) signal in parallel to the RX line. That means that whenever the buffer is nearing its high water mark or whenever we are currently forwarding an IR signal from the demodulated pin, we can, using the CTS signal, tell the host device that it needs to wait for a moment.

Thanks to the data format which is effectively a transcript of the to-be-sent signal, this does not cause timing issues: The signal will start later, but it will still be fully intact.

In practice, it turns out that we need to have at least four bytes of space before de-asserting the CTS signal; the host UARTs I was testing with continued sending a few frames before they reacted to the CTS signal.

Signal generation

Whenever the buffer becomes non-empty (and we can safely forward), we enable signal generation mode. In that mode, timer 1 is programmed to the duration as indicated in the next byte in the buffer. The output is set to correspond to the pause bit.

Once the timer elapses, it checks whether there are more bytes in the buffer. If there are not, signal generation mode is exited. If there are, the timer is reprogrammed and the output is configured according to the bytes value.

It's that simple.

During signal generation mode, the pin-change interrupt for forwarding mode is disabled to avoid it from interfering with the signal generation. Vice versa, the receive interrupt is disabled while forwarding is going on (though this may actually cause loss of data due to the fact that the host may react to the de-assertion of CTS only slowly; that's an edge case which could see improvement).

Wrapping things up

The final code can be observed in the current main.c. There are more detailed explanations about the state machine in there and helper functions which hopefully make the code more comprehensible.

We now have a piece of firmware, but no hardware to go with it. The hardware around it is pretty simple, just level shifters and the capacitors you typically need to provide a stable power supply for a microcontroller. The schematics and a layout suitable for soldering in DHT on a prototyping board are available in the GitHub repository (as Kicad files).

Remotely controlling a Denon AVR-1910 from Linux

2022-02-26T11:00:00+00:00

The Denon AVR-1910 is, by current standards, an ancient device. It does not have ethernet. It does not have wifi. There are no apps to control it. Only an old-school infrared remote control. As ours lives in a closed cabinet, the infrared remote control unit isn't even usable. The signals do not reach the receiver.

This is compensated somewhat by being able to control volume and number of audio channels via the network through PulseAudio on the Raspberry Pi, which is the main source of signals for the Denon. However, generally it is preferable to control the volume through the last segment of an audio processing chain, if possible, to get the best quality possible. In addition, when it comes to how stereo signals are upmixed to the 5.1 output, there are various options available in the Denon (and no sensible option in PulseAudio) and they do not apply to any situation equally.

Hence, it would be nice to be able to control the device remotely.

Available remote control interfaces

There are three options:

HDMI Consumer Electronics Control (HDMI-CEC) allows devices in an HDMI "tree" (with a TV or similar device usually being the root of the tree) to exchange commands. This can be a TV forwarding remote control codes to upstream devices (such as DVD players) or a playback device which wakes up the TV or asks it to switch source.
Hack the port on the Denon which is curiously labelled "Room-to-room remote control in".
Use an IR blaster to inject infrared remote control signals right into the photodiode the Denon uses to receive signals from the physical IR remote control unit.

Option 1: HDMI-CEC

Option one would obviously be the most modern. I ran a few tests using the Pi and they were mostly successful: I was able to control volume and power status via HDMI-CEC. However, this comes with a nasty downside: The receiver, when the HDMI-CEC endpoint is enabled via the menu, will not turn off the auxiliary power output. That means that as long as the AVR has power, even if it is just in standby, the subwoofer connected to the aux out is fully powered. That is a huge waste of power. The subwoofer does have an auto standby mode, but I found that this is unreliable during quiet pieces of music or tracks with sections of low bass, effectively removing the lows.

So with HDMI-CEC not being a viable option. I briefly considered getting a switched power outlet and control that via the Pi to then control the Denon via HDMI-CEC, but a research did not reveal any device which doesn't depend on wifi working and being available. That I consider annoying, because the A/V cabinet is otherwise mostly autonomous from the access point (being connected via ethernet to other devices and thus able to survive even while the access point is down).

The other approach for switching power to the Denon would be something home brew, but I'd really not like to mess with mains voltage.

Option 2: Room-to-Room Remote Control In

This is a peculiar feature of the Denon, and massively underdocumented. In the manual, it ominously refers to "room-to-room functionality", without giving any concrete example on how you would connect it to anything. It is not described what kind of protocol is spoken there.

Luckily, it is possible to find a service manual for the Denon AVR-1910 in the internet, with complete parts lists, schematics and PCB layouts. I'll not bore you with the details of other people's schematics. The gist is, after some AC coupling, the RC in is fed into a Sony CXA1511L IR demodulator IC. This takes a remote control signal modulated with 38 kHz (just like the hardware remote) and outputs it on some pin which is then, presumably, fed into some IC deeper inside the AVR.

This is actually pretty neat. It would allow connecting, for instance, the Raspberry Pi and somehow generating that signal to inject remote control commands. The downside is that even after studying the schematics in detail, I'm not convinced that connecting GND and a signal from the Pi would not potentially cause havoc to either device or the audio quality. Pis are not exactly known for being low-noise and audio equipment tends to be finnicky about that.

That led me to wonder whether there would be a way to somehow electrically insulate this and couple the signal into the Denon optically...

Option 3: Infrared Blaster

... when I realized that there is a very standard way to do so: Just tape an IR emitter diode in front of the IR photodiode of the Denon and make it emit whatever I need it to.

This is clearly the safest option. It would also be very reliable if the diode is placed in close proximity and fed with sufficient power. It does not require electrically coupling two pieces of hardware which may never have been meant to be connected.

So this is, finally, the way I'll go.

How to generate the signals? And what signals even?

Now that we know how to feed control signals, we need to figure out what signals to feed. And how to generate them.

Infrared remotes work by modulating a "slow" signal on top of a carrier square wave, typically between 35 kHz and 42 kHz. In case of the Denon, it is 38 kHz (or compatible to that, anyway). That means a signal looks roughly like this:

Demodulated signal:

     ______________________________________           _____________
____|                                      |_________|             |_______

Modulated signal:

     _   _   _   _   _   _   _   _   _   _            _   _   _   _
____| |_| |_| |_| |_| |_| |_| |_| |_| |_| |__________| |_| |_| |_| |_______

With a 38 kHz carrier, each low-high cycle of the modulated signal is 26.316us long.

The modulated signal is then used to drive an infared light-emitting diode. Those are easily fast enough to work with a 38 kHz signal, so that's not a problem. The light travels through space toward the photodiode in the receiving device, where there's typically a bandpass selecting for the carrier (to exclude any non-signal noise, for instance heat or sunlight). Then the carrier is demodulated back into the original signal. The bandpass and other filtering will typically include a delay, which is why most IR protocols would avoid too short bursts, because those may be missed.

Now that we know how the signal would be modulated, we'd have to learn what to modulate. This is more tricky, because it is also often underdocumented. Luckily, there is the IRMP (Infrared-Multiprotocol-Decoder) project with excellent documentation on many different infrared remote protocols, including the one used by Denon.

The Denon protocol encodes a binary zero with a 310us burst, followed by 745us of pause. A binary one is encoded by a 310us burst and a 1780us pause. The signal is terminated by a 310us pulse without defined pause afterward (i.e. a pause much longer than 1780us; the stop bit). That way, 15 bits are encoded which make up the complete IR command.

Now we know how bits are encoded, but we don't know which bits to encode.

There are two ways to find that out: One option is to search the internet for documentation on the specific IR commands used by Denon. The other option is to teach lircd (Linux Infrared Remote Control Daemon) to understand the encoding, point a remote at a photodiode at a lircd-compatible device and let it learn the commands.

In fact, surprisingly, in this case both options actually work. It is possible to find the IR command set for the AVR-3803, which is not model I have, but which (un-?)surprisingly uses much of the same IR commands.

Getting Infrared Signals In and Out of a Raspberry Pi

The Raspberry Pi comes with pre-made configuration which can be used to read demodulated IR signals from any arbitrary GPIO. Likewise, it can generate modulated IR signals on arbitrary GPIOs. This is achieved by bit-banging, which is CPU intensive and in general ugly. Nontheless, let us briefly talk about this option just in case you're not as interested in hardware and microcontrollers as I am.

The gpio-ir device tree overlay can be used to turn any GPIO pin into an IR receiver (provided you feed it demodulated IR signals)
The gpio-ir-tx device tree overlay can be used turn any GPIO pin into an IR transmitter (emitting modulated IR signals)

You can read more about using the ready-made device tree overlays with lircd. But this is not the road I'll go down.

Along all the way of research, I came up with the idea of not ditching the remote control unit altogether, but instead making it usable by placing an IR receiver diode somewhere and forwarding that signal to the Denon, too. I would like to be able to control the Pi (or rather, software running on it) using the many spare buttons on the remote control, while simultaneously using it to control the Denon, all from a single IR receiver.

To implement this on the Pi requires one of two things:

A realtime thing listening on the events from /dev/lirc0, replaying them immediately. This has no latency, but I am fairly certain that a /dev/lirc0 only supports a single reader at a time, which would mean that this requires messing with lircd to do what we want.
Let lircd interpret the signals, listen for key events from lircd and then make it send them again through the photodiode. This is totally possible; lircd does, in fact, support sending. However, here the problem becomes latency: lircd has to wait for the full key sequence to be received, which can take between 10 ms and 100 ms depending on the remote signal coding. That is a noticeable latency which can get annoying when doing things like volume control.

I don't like either of those options.

I do happen to have an ATtiny2313a 8-bit microcontroller in stock though. Those happen to have timers. And pin change interrupts. And a USART. And a PWM output which can be used to generate an 38 kHz carrier signal.

In the next post (Designing an Infrared Forwarder/Sniffer/Injector Microcontroller Firmware), we'll go into the nitty gritty details of what a microcontroller needs to do to achieve this goal. If you're not into making your own hardware things, I'd highly recommend to go back to that post about using the ready-made device tree overlays with lircd; it should get you a long way.

Making the Isolation bearable: Self-hosted Audio/Video Conferencing with Jitsi Meet and Mumble

2020-03-21T13:00:00+00:00

Update (2026-01-01): Things have changed a lot in the past nearly six years. The jitsi-meet package in Debian has evolved significantly. Firefox and Jitsi Meet are now best friends. Stability has improved overall. If you have a host to spare, you can probably just install jitsi-meet, follow the debconf questions, set up DNS and it works. Or use their docker-compose.

I'm going to leave this up nontheless, I like to not break links.

Update (2020-03-25): I published the Ansible role I wrote to set up the server. Feel free to use and improve it; it is still buggy though and does not include the manual fixes to the nginx config I mentioned below. Grep for FIXME and TODO.

Introduction and Story Hook

My wife and I have a pen&paper group (let’s get this out of the way). That means we meet every week for an evening to basically tell a story together, with some rules. It can be a very fun activity, but the isolation rules which have been put in place (rightfully!) rule out physically meeting up with each other.

In order to still be able to continue the story and get some social interaction, a solution had to be found.

I am active in the instant messaging standards community, however, I have never really delved into A/V conferencing. While it has been long on my to do list to implement one of the various standards which exist around the real-time transmission of audio and video data, I never got around to do it.

So, here goes. In the first part, this post will describe how I set up jitsi-meet on my own server. In addition, I’ll say a few words about Mumble. A few, because for one it is super easy to set up, but also because it is audio-only. I only deployed mumble as a fail-safe fallback option, which we luckily did not need.

In the second part of the post I will present some measurements I took during our pen&paper session last night. Six people, seven Jitsi Meet clients, five hours.

Let’s dive right in.

Setting up Jitsi Meet

Jitsi Meet is based on the XMPP messaging standard. However, Jitsi instances do not like to federate much and from my experience, it does not make sense to re-use any existing XMPP deployment and instead keep it completely separated.

My successful install of Jitsi Meet (I stubbornly tried it in a few different ways before that) was mostly based on the Quick Install guide from the Jitsi Meet team themselves. I suggest you go there, do that and then only look here if anything does not work.

Prerequisites

My system is a Debian buster virtual server with two CPUs and four gigabytes of RAM. To preempt a bit of the later measurements, the entire system did not exceed a usage of 1.2 GiB, so RAM-wise it is slightly overpowered.

Since Jitsi Meet is, among others, a web application, port 80 and 443 need to be free. In addition, you’ll have to be able to send and receive traffic on ports 4443/tcp and 10000/udp, via both IPv4 and IPv6 (if you have that; neither IPv4 nor IPv6 is a strict requirement, but you’ll need it if you have IPv4/IPv6-only users). If you are behind NAT, you can configure Jitsi Meet to understand that.

If you already have a webserver installed, you’ll have to configure the web part yourself. There is a Manual Install document which goes into some of the aspects you need to handle in such a case.

Installing Jitsi Meet

To install Jitsi Meet on Debian, you first have to enable the apt repository by the Jitsi developers. If you, like me, do not like enabling third-party repositories on productive servers, I suggest you put your Jitsi instance in an LXC-based container. Or maybe inside Docker. Though if you go with Docker, you may be interested in docker-jitsi-meet instead.

You need to follow the linked Quick Install document up to the point where apt install jitsi-meet is called. During the installation you’ll be asked for two things: A domain name under which you want to serve your instance (which you cannot easily change later) and whether you want to use existing certificates.

We’ll refer to the domain name you gave as $HOSTNAME in the following.

They then suggest to install Let’s Encrypt certificate using the script they provide. I found that this script downloads certbot-auto from the EFF website without further signature checking and executes it as root. This isn’t ideal.

You can instead use certbot, but you have to patch the nginx configuration created by the package slightly by adding the following lines to /etc/nginx/sites-enabled/$HOSTNAME.conf, where $HOSTNAME is the domain name you gave Jitsi during the installation with apt.

location ~ ^/\.well-known/.+$ {
    try_files $uri @root_path;
}

Then you can install certbot (apt install certbot) and request a certificate:

# certbot certonly -d $HOSTNAME --webroot --webroot-path /usr/share/jitsi-meet

Again, replace $HOSTNAME with the domain name of your Jitsi Meet installation. Note that certbot automatically puts itself into crontab so renewal will happen automatically. It will, however, not auto-reload nginx. To achieve that, you can add your own crontab entry which calls systemctl reload nginx, for example once per day or week.

Jitsi Meet configured nginx to load the certificates from /etc/jitsi/meet/, so we have to create symlinks there so that nginx finds the correct certificates:

# rm /etc/jitsi/meet/$HOSTNAME.{crt,key}
# ln -s /etc/letsencrypt/live/$HOSTNAME/fullchain.pem /etc/jitsi/meet/$HOSTNAME.crt
# ln -s /etc/letsencrypt/live/$HOSTNAME/privkey.pem /etc/jitsi/meet/$HOSTNAME.key
# systemctl reload nginx

If you want to use IPv6, you also need to fix the nginx configuration which will currently only offer the Jitsi service on IPv4. To do that, you need to duplicate the listen directives in /etc/nginx/sites-enabled/$HOSTNAME.conf to also listen on IPv6. So instead of listen 80;, you need the following:

listen 80;
listen [::]:80;

And analogously for listen 443 ssl;.

At this point, you should already be able to reach the Jitsi Meet instance from your browser.

Authentication

As you noticed, your Jitsi Meet instance is currently available to anyone who knows the domain name. This is not ideal. You can secure the instance by enforcing authentication. This is documented in the Secure Domains section of the Jicofo service.

Setting up Mumble

Mumble is a low-latency audio team chat which has its origins in the gaming scene. It has been an early challenger of the proprietary TeamSpeak software. In contrast to Jitsi, it cannot be used from the browser but requires a client to be installed on each user’s machine. The client software is available for Linux, MacOS and Windows and there exist inofficial apps for Android and iOS.

It is very easy to set up. You need to be able to open a UDP and TCP port (by default this is 64738) and that’s it. Install mumble with apt:

# apt install mumble-server

... that’s it.

To allow your users to securely identify your mumble server, you can re-use the certificates you generated for Jitsi and tell mumble to use them via the config file in /etc/mumble-server.ini:

sslCert=/etc/letsencrypt/live/$HOSTNAME/fullchain.pem
sslKey=/etc/letsencrypt/live/$HOSTNAME/privkey.pem

Remember to then also create a cron job to reload the certificates.

To log into your Mumble server as privileged user (for example, to add channels or distribute privileges to other users), you first need to set a super user password:

# cd /var/lib/mumble-server
# murmurd -readsupw

This will ask for a password which will subsequently be the SuperUser password of the server. You may have to restart mumble after this and then you should be able to log into it as SuperUser.

Why even mumble when you have Jitsi? Videoconferencing and WebRTC are tricky technologies which are easily disturbed by machines which are not powerful enough or by network issues. Mumble is very resilient and low on resource consumption, so it may be an alternative if the circumstances do not allow the use of Jitsi Meet.

Using Jitsi Meet

Using Jitsi Meet is dead simple. You open the website (https://$HOSTNAME) of your installation in the browser and hit go. Then you can copy the link of the page you ended up on and send it to your friends and family.

If they want to join from mobile, there is a quirk. This requires an app to be installed. And rumor has it that it will only work if you set the Jitsi server in the settings (even though it is technically included in the URL already, so this doesn’t really make sense).

Typing the name in the current release of the App is kind of annoying because there is a weird bug with the text input. Copy&Pasting works though.

Once that is sorted out, users should be able to join your Jitsi Meet conferences even from their phones. In my short test, I was able to join a video conference with four users via mobile data while on the go outside.

Resource Consumption

Jitsi Meet is a video conferencing software, and as such it needs a bit of server resources (both CPU and network) to operate.

In contrast to the hardware solution, the Jitsi Meet videobridge software will not re-encode or mix video. This means that in theory the CPU use should not be too high. During our pen&paper session last night, I let Prometheus monitor the resource usage on the box where I installed Jitsi.

Experiment Setup

Debian Buster VM at Hetzner (CX21, but with only 20 GB disk because I opted out of the disk resize when scaling up from the CX11 I had before)
Jitsi Meet installed in an LXC container (managed via libvirt)
IPv6 directly routed to the LXC container, IPv4 via DNAT (stateful, more on that later)
Initially five people, later six people peak, with seven clients (more on that later)
Monitoring with Prometheus:
- CPU time metrics of the VM via node-exporter
- Memory use metrics of the VM
- Network traffic on eth0 of the VM
- Network traffic on the various Jitsi ports (80, 443, 4443/tcp and 10000/udp) using nftables counters and the nftables_exporter
The VM also functions as a shell server, so there were two poezio instances running which caused a baseline of CPU and memory use.

Timeline

First off, you can look at the data yourself. I took an interactive snapshot on my Grafana instance for you to look at.

For everyone else, there’s also a screenshot:

In the second graph on the left side, I marked a few points of interest. I will now go through them one by one.

A around 16:45, we set up the conference. My wife and I joined first, then shortly after 17:00, the first player came along. We chatted a bit and optimised the sound setup. Then the second player tried to join, but was unable to get audio or video to work.

I looked into it and found error messages indicating that no WebRTC session came to be between the Videobridge and the player’s computer. After a while of messing around, I noted that I forgot to configure Videobridge to make it aware of the IPv4 NAT, and this was the first person to not have IPv6. After applying the NAT configuration from the documentation ...
... I restarted the videobridge which kicked everyone out of the conference. The two players from before were able to join (this time with audio!) and a third player came along shortly thereafter. This worked for quite a while stably.
We had issues with one of the two Firefox users getting frozen video. This happened immediately as soon as a second Firefox user was present, which was odd. I recommended one of them to switch to Chromium. That fixed the issue for them, but the other Firefox user still had frozen video. The switch to Chromium is what caused the drop in traffic.

There are two reasons for this: First, the video quality actually dropped a bit when the player switched. Second, there is a problem with Jitsi Meet and Firefox. Normally, Jitsi Meet will continuously send three video streams to the Videobridge: A low definition one, a standard definition one and a high definition one. Then the other clients indicate which version of the streams they want and the videobridge will only send the requested one to the other clients.

Most of the time, you don’t have a participant in full screen, so getting a HD stream makes no sense; your Jitsi Meet in the browser will request SD (if you use the tiled view) or LD (if the participants are folded to the side) video streams.

Since Firefox isn’t able to send multiple stream versions, Jitsi Meet only sends the HD one. That means that everyone will always get the HD stream of all firefox participants, even if they’re only a 64x64 square on the side. We’ll see more data to support this in a bit.
The last player joined. They also joined with Firefox, and immediately we see the traffic jump up by a significant bit. We still had the issue that one member was getting frozen video streams.
At this time, we tried to fix the frozen video streams again (they didn’t go away by switching the other user to chromium).

I set up a victim firefox on my laptop to catch the "disease of the frozen streams" while the new player switched to chroimum. That worked. So I turned of video and audio for my firefox client and let it sit there for the rest of the session.

The drop in traffic was again caused by a player switching to Chrome, though this time without loss in video quality.

After the initial hiccups, Jitsi worked mostly smoothly. One player was having a bad connection and dropped out for a few seconds a few times, but other than that it was pretty good and reliable.

Resource Consumption summary

In the last section of the graph, the setup was continuously as follows:

Seven clients
Five clients which received video
Five clients which sent audio
Two firefox clients, only one of which sent video
Five chrome/chromium clients

The CPU load in the graphs is normalised to full load on all cores. That means if you see a CPU load of 100% in the graph, it implies that all cores were fully utilised. Note that the blue line in the CPU graph is in fact the load5 metric of the system and uses the right axis, while the other data uses the left axis and is CPU time spent per second.

The Jitsi Meet conference thus used approximately 80% of a CPU core continuously. Due to the high amount of threads, the load5 was wiggling around 2.0. Memory use started out at ~950 MiB without Jitsi Meet running and then gradually rose to approximately 1.15 GiB. This means approximately 200 MiB of memory used by the server-side components of Jitsi Meet for seven clients.

The memory has not been fully released afterwards though, which makes me think that some of this is overhead caused by memory fragmentation.

The traffic totalled at about 75 GiB (50 GiB tx, 25 GiB rx) on the server side. With only one Firefox client sending video, we had about 2 MiB/s tx and 1 MiB/s rx on the server, which is quite low all over all. I suspect that eliminating the last Firefox client will improve the situation even more.

Summary

I hope this was interesting and/or helpful to someone. Here’s a quick summary:

Jitsi Meet is easy to set up, if you have a box where you don’t have an HTTP or XMPP server running.
Mumble is even easier, but we didn’t need it luckily.
Jitsi Meet is a decent free video conferencing tool.
However, it is no fun with Firefox. There is a reason they don’t officially support Firefox. Use Google Chrome or Chromium if you can.

On Centralisation of Code Hosting Infrastructure—An Argument

2019-07-08T18:35:00+00:00

Update (2026-01-01): This post was written in 2019. A lot of water and at least one bridge has since gone down the Elbe. We've seen facism rise more clearly than before, we've seen the neglience of Microsoft in regards to caring for the ecosystems they control and a lot more. I've since come to learn about the Fediverse and seen what decentralized moderation can do.

I think some arguments are still sound. The centralization does come with benefits regarding defense against abuse. But that only works as long as the definition of abuse is a sensible one.

These days, I think the future indeed lies in federated forges, based on already-existing federated standards (be it XMPP or ActivityPub or whatever). I'm glad to see that this is a direction we're moving toward.

Nontheless, I'll keep the post up here for the sake of archival.

Many, many Free/Libre Open Source Software (FLOSS) projects have their code and/or related content (mostly issues, wikis and pull requests, but also documentation and websites) pooled on large cloud providers. The most notable one is GitHub, probably followed with a large margin by BitBucket and GitLab (I don’t have the actual numbers on this, but I don’t think anyone will disagree on this).

Earlier this year, GitHub was bought by Microsoft. Microsoft, historically The Devil in FLOSS circles. This has created a huge outcry and an exodus from the GitHub platform to others, most notably GitLab (who played the event very cleverly, PR-wise). They saw an increase of GitHub imports by at least 30 dB.

Of course, the question is, what are you even fearing Microsoft could do? As a FLOSS project, there is typically not much private information; the code data and issues are public (but see below). And what makes Microsoft, in this context, worse than the old GitHub management? Or the current GitLab management? BitBucket?

In this vein, I’ll discuss the pros and cons I see for the current centralisation of code hosting to major providers (or, in reality, mostly GitHub actually).

What Services Are We Talking About?

In case it isn’t clear, here’s the services which those platforms provide:

Authentication and Authorization: They allow you to log in, and allow you to control who has write access to the projects data, possibly in rather fine-grained steps.
Code hosting: They provide access to the code to the public.
Issue/Pull request management: They provide tools to create and manage issues and pull requests. Often, rather advanced code review tools are included.

What Are The Threats?

To figure this out, let us look at what is at stake. Treating the code hosting platform as an attacker and figure out what they can do. Going into classic security theory, let us look at how the platform can danger the three security goals Confidentiality, Integrity and Availability (CIA; funny, right?).

Confidentiality

This one is easy and hard at the same time. Easy, because the vital data of FLOSS projects is public. Hard, because the part which isn’t varies from platform to platform. Here’s a common denominator:

User passwords
User emails (if different from what is in the commit logs anyways)
IP addresses used to access the service during pushes, pulls or when editing data in the webinterface (e.g. issues)
Timing of interactions with the website (not necessarily correlating with commit times)

The users have to trust the platform with this data in order to use it. The platform could abuse the data for profiling or leak it to the public (probably inadvertently).

Integrity

This is a big one. The platform has control over the code and the metadata (commits, issues etc.). This could lead to major issues, for example, if the platform started to include obfuscated crypto currency mining code in all Node.js projects (contrived example!).

To prevent this, maintainers need to take a look at commit logs and audit new commits. Thanks to how modern distributed version control systems (DVCS) work, there is no way such a platform could inject code into old commits without it being noticed by everybody. So the only way to threaten the integrity of the code would be by appending commits to the tip of e.g. the master branch.

Depending on the project size and organisation, such an attack is probably feasible: many people working on the master branch concurrently will lead to folks not being wary when they can’t push because a new commit has been added in the meantime.

Availability

Of course, the service can shut down at any instant. It can cease to provide free services to FLOSS projects. It could restrict its platform to projects which are not under the GPL. All of this would be threats to the availability of the service to FLOSS projects.

This is a bummer, but thanks to DVCS, at least the code is easily replicated on another platform, or even a self-hosted thing, if one platform chose to do this.

Other data can typically be exported using specialised tools; paranoid (or rather, cautious) projects could set up a periodic task which downloads all the non-code data from the service, to be able to restore it somewhere else in case it turns against them.

What Are The Benefits?

To figure out the benefits of a large centralised platform, let us look at the drawbacks or challenges you face when providing code hosting for your (or a bunch of people’s) projects.

You need to grant those people write access to your server. They will be able to upload arbitrary data, which will then be publicly available (I’m going to let aside the issues of code injection attacks on your domain, because I assume that whatever code hosting software you use is secure against that). This means that you either have to employ monitoring and filtering of the content which goes up, be prepared to deal with someone uploading stuff you really do not want to have under your domain (and somebody else noticing), or trust your users.

For now, I’d say that "trust your users" is going to work in this scenario. Let this be a friends&family code hosting service; nobody will intent to harm you or the service. So that’s not an issue.

However, your users will want to be able to receive contributions from wider community. This means that "everyone" must be able to create issues and pull requests for the projects. This means that they need accounts, or that you open up the instance for anonymous comments/issues (really not a good idea).

Having to create an account is a bad user experience. Not having to create a new account for each project I want to contribute to is one of the great benefits of GitHub and similar (older) platforms. Some systems have solved this with "shim" accounts, such as the Prosody issue tracker: You "sign up" using just your email address in the same step when you send your first comment. A confirmation email is sent; once confirmed, your comment is posted and you can post further comments, until your browser forgets about the secure cookie which has been set for you.

This type of stuff would be nice to have in software such as Gitea. However, from my experience with co-hosting a large forum community, I learnt that once you get big enough, spammers will implement the few lines of code to match your specific flavour of confirmation email (a concept they’ve been dealing with since the early days of phpBB). So this is not going to fly in the long run.

Aside from that, there’s another big issue. If you want people to post pull requests, they (with the current model of how things work) need to fork the repository on your instance. And then they need to be allowed to add code to that fork. And that’s the point where you not only granted your friends&family write access on your server, but the entire internet.

You will be faced with abuse. It is just a matter of time.

I think those two (not having to create an account for each project, and not having to deal with abuse as a project maintainer) are major points which speak for the use of a few, centralised code hosting platforms. And they are big ones, compared to the threats from the previous section.

Where To Go From Here?

Of course, as I’ve written in an earlier post, I value my privacy. I also value decentralisation in general; having too much power in a few hands is rarely a good thing, because it poses the temptation to abuse it.

So what are the steps we, as the FLOSS community, have to take forward?

A federated authentication service, which provides all the typical account stuff (user name, email, possibly avatar and short bio, real name, etc.) and allows to control precisely with which platform which amount of data is shared. The important part is that, for example when changing your avatar or email address, you don’t have to go to all the various code hosting instances and change it there, but have a single, centralised (for you; it is still federated so that other folks could be using other services) point where you can update this info.

The service would then be responsible for pushing the changes to the consumers of your user data. It would also push deletions of (parts of) the user data when you revoke access to (parts of) it.

One way of implementing this would be using the extensible messaging and presence protocol (XMPP), standardised by the IETF. It offers a way to share this information, and to push updates to consumers. In addition, it supports federation so that each (group of) users can host their own server to prevent lots of user data accumulating in a single place.

Full disclosure: I have been working on XMPP and XMPP clients for a few years now and am member of the XMPP council for the term 2018/19.
A way to create pull requests cross-instance. This requires that the code hosting instances can talk to each other during user interaction (such as posting (review) comments, adding commits, etc.). You would want to have notifications about new comments in your "home" instance, and likewise you’d want your new commits to be visible on the "target" instance right away without further action.

This, again, could be solved with XMPP. Huh, I didn’t intend this post to be an advertisment piece for XMPP, but here goes. Essentially, this is the kind of ground feature set required for federated social networking. And there has been prior art for doing this type of stuff in XMPP. goffi, who is well known in the XMPP community for their work on social network use-cases and Salut-á-Toi, has laid the groundwork for a decentralised code forge based on XMPP.

However, even with such decentralised tools, we would still have to have people who provide the service. Project maintainers are not always interested in also hosting services. So, just like with email and instant messaging, the ideal would be if groups would flock together and host their own services together, spreading the load among those who also happen to have an interest in maintaining services.

Conclusio

So, uh. This went in a different direction than I originally intended to. To wrap the argument up:

For the current state of the art, I think that we have to live with centralised code hosting platforms such as GitHub. It does its job fairly well. It doesn’t hurt (too much). It takes a massive load off the FLOSS community by handling abuse and providing a rather stable and reliable service.

In the future, we might come up with technologies which allow us to avoid this and have secure, reliable and usable decentralised code hosting. I’m hoping that we get there eventually.

Stable vs. Rolling Release—An Argument

2018-12-03T17:13:00+01:00

Since I have integrated myself into the XMPP/Jabber community more, I came into contact with more people whose opinion I respect a lot, but whose views also differ a lot from my own.

One of the recurring topics in a tech chat room I usually hang around in is the frustration caused by so-called "stable" Linux distributions.

With that, folks typically mean a distribution like Debian, Ubuntu, Red Hat Enterprise Linux (RHEL), SuSE Linux where releases happen every few years and then the software does not get major upgrades until the next release. In contrast to that, there exist distributions such as ArchLinux where there are no defined releases; very recent software is always available for each ArchLinux installation, as soon as the package repositories get the update (which is typically very soon after the upstream release).

Definitions: What Does "Stable" Mean (to Me)?

Stable Linux Distributions

When I think of a "stable" (I’ll be leaving out the quotation marks from now on) Linux distribution, there are a few key properties for me:

It has well defined releases. A release, for me, consists of a set of specific package versions which are shipped (distributed) together. When installing for example the initial "Stretch" Debian release (Debian 9.0), you will always get the very same packages on each install. It is reproducible.

This is not 100% true, because there are point-releases in Debian; For example, there exists the Debian 9.1 release. So if you made an install of Debian 9 before and after a point release, you’d get different setups. But as long as no additional point release happens, you will get identical stuff.
There has been some type of quality assurance (QA) before a release is made. The QA does not necessarily need to be explicit or paid or whatever; the QA may simply consist of distributing a release candidate for a certain amount of time and waiting for bug reports (and then triaging and possibly fixing those before the release).
There exists a non-trivial (two or more years) time window where support is more or less guaranteed for (at least a well-defined non-empty and non-trivial subset of) the packages included in the release. With support, I mean that grave issues (leading to the package being unusable) are fixed with updates. This most notably includes security issues.
There exists security support. This means, to me, that security issues are addressed in a very timely fashion (for example, within less than 48 hours) after a fix becomes available, if at all possible. If it is not possible to fix the issue, I expect a workaround to be announced (even if the "workaround" is "Turn off all your machines running X and hide in the woods").

Note that none of the above says "it must not contain any bugs". Also, it says nowhere that I expect all bugs to be fixed during the lifetime of a release.

It also doesn’t say that software should be kept up-to-date (in fact, I consider up-to-date software an anti-feature sometimes; see below). In fact, it is rare to have a non-rolling-release distribution which has releases more often than once per year.

Ubuntu and Fedora do have release cycles more often than that, but they also don’t offer long-term support (longer than one year) on every of those releases (or any, in case of Fedora), making them a weird hybrid of rolling release and stable.

To summarise it in one paragraph: Stability, in this context, means to me that I can predict the behaviour of the operating system and that some effort is put into being able to being able to continue to operate it just like I did yesterday without having to expect breakage by security issues or unexpected updates.

Examples of stable distributions, in my mind, are Ubuntu (Long-Term-Support only), Debian, and RHEL (and derivatives).

Rolling Release

What I wrote above about stable distributions means a lot of management overhead for those. In contrast to this, there exist rolling release distributions such as ArchLinux (I know that others exist, but unfortunately, this is the only one I ever used, so I can only speak for it; likewise, the folks I argue with also only use Arch or a derivative of it as far as I know).

A rolling release (RR) distribution is essentially a single package repository where updates are pushed. When you install an RR distribution from scratch, you always get a different installation (installing it without internet access is also typically rather futile or at least frowned upon) because it is likely that an update to the package repositories has happened since the last time.

The Argument: Rolling Release vs. Stable

Now for the main argument.

As I mentioned in the introduction, this post came into being because I see people frustrated with stable software distributions. And I also see people "forcing" (no wrenches involved!) me to justify my personal choice of using a stable distribution on all of my servers.

I try to make this argument as balanced as possible. In the end, it all comes down to personal choice.

I will grant rolling release distributions a massive advantage: they provide recent/up-to-date software. This is a major win for many use-cases. I, for example, like to work with modern Python features. I can fully understand folks who love to work with Rust or Golang or other rather fast-moving languages ("fast" is here to be taken with respect to the typical two-year-or-more release cycle of stable distributions).

In those environments, you really and absolutely do not want to be slowed down by a distribution which releases once every other year. And this is in fact the reason why I’m using Debian unstable on my development machines.

However, on my servers, I prefer stable a lot. And here’s why.

Why I Prefer Stable Distributions

Quality Assurance

A stable distribution provides me with a combination of packages which are very likely known to work in that specific combination. Most packages are provided by somebody who actually uses them. This combined with the typically significant user base of well-known stable distributions means that it is likely that all of the typical configuration combinations are actually in use somewhere.

When a new release is in the making for a stable distribution, folks will try the various things in the testing channel and report issues when they don’t work as expected. The issues are then fixed. This is a somewhat crowdsourced non-automated integration test for the packages in the distribution. From software development, I know that unit tests (in this analogy and from perspective of the packaging architecture, the tests run after/during the package build are the unit tests) are good and all, but without integration tests, they are very, very useless.

However, this is the world of Free Software. I fully know that I cannot rely on somebody having tested a specific combination; if I care about it, I’ll have to do it myself (which I do). When something goes wrong, I can downgrade my systems to the previous stable release (which I did when dovecot-ldap broke our use-cases in the 2.4 release).

Mix & Match

As I mentioned, I myself like to use modern features of the languages I work with. This means that sometimes, I’ll have to go off the beaten stable path. For those cases, there exist (with Debian) the backports repository. You can (using apt pinning) select specific packages (or even versions) which shall be sourced from the backports repository. Package metadata makes sure that you don’t do something blatantly stupid (like upgrading the perl version to something which isn’t compatible with the apt release you’re using), so there’s that.

But aside from that, you are now kind of on your own. There is no official security support for those packages (although the maintainer might still make an effort on their own to provide that), and they have not been "integration tested" during the release candidate phase.

However, a backport package doesn’t come out of nowhere. There has been a need for it, and somebody will likely be running it already in production somewhere. So a few backport packages might not hurt (you still have to be careful, obviously, especially with security upgrades).

I generally try to avoid using backports (or, shudder testing) on production systems.

Now, in response to this, I often hear something along the lines of "if you’re using backports, why aren’t you using rolling release altogether?", which is a fair question.

My reply to that is: The stable distribution provides me with a base system I can rely on. It will behave predictably.

Update 2018-12-08: I reworded the part starting from there until the end of this subsection after some useful feedback.

Now, with backports in the mix, I have to take care that they don’t mess up the system. I will have to re-do this whenever the backport gets an update. In contrast to the rest of the system, I lose the security support; this means, most notably, when a security fix comes out, I am faced with the choice of upgrading the package ASAP (the fix will be based on the next upstream release, so it will most likely also carry new features and bugs), delay the upgrade further by doing additional tests (e.g. on a dedicated test setup) and/or (temporarily) remove/disable the package altogether.

Of course, this is not ideal (and this is the reason why I’m not using backports in general); but having to do this for one (or a few) packages is still much better than having to do it for all the packages.

Time Is A Factor

To be clear: I do most of my admin work in my free time. I run communication services for friends & family (think instant messaging, email, voice chat); I run various static websites. And all of that with the full infrastructure stack: virtualisation, OSPF-based routing, own DNS servers, etc. etc.

I do this because I value my privacy and believe that self-hosting is in most cases preferable to using a hosted service. And also because building and maintaining infrastructure can be a hell lot of fun.

However, I also have a life besides this. You know, somewhere I need to get those friends & family I can provide the services for!

As much as I like the feeling of being important when doing sysadmin work and being appreciated, the appreciation typically ends when you have to cancel plans when something breaks and you need to fix it to ensure that nobody loses their email for that day.

And this is where all of this ties together. A stable distribution, and some self-restrain from always having to use the most recent fancy stuff, makes such events extremely rare. New features most of the time also mean new bugs. Stable releases do not get new features. They only get very specific security updates (which may of course also introduce new (potentially grave) bugs, like in the famous Debian SSL disaster of 2008), and they get those updates only when absolutely necessary.

I can thus, with good conscience, allocate rather slim time slots to run the daily upgrades on the servers. I’ll have an eye on the monitoring (and I of course use apt-listbugs and apt-listchanges and read through the notes while upgrading), but I’ve never had the issue that an upgrade broke something in Debian.

To Wrap It All Up

A stable distribution is no magic bullet. It merely provides me with a reliable and predictable basis. It can’t provide me with the most recent software all the time. It provides me with security upgrades. The packages are known-ish to work in the combinations. And it is unlikely that an upgrade will break all the things.

It means I have to live with fewer/less modern features. It means I have to work around bugs which might have been fixed three years ago in the upstream source. And you know what? That’s fine with me. What’s not fine with me is having to cancel plans because a required security fix also pulled in a bug which stops users from being able to log into their email accounts.

jssfr.de

Darktable styles mimicing Fujifilm film simulations, version 1.1

Fujifilm Filmsimulationen als Darktable-Stile, Version 1.1

Darktable Styles mimicing Fujifilm's Film Simulations

Motivation

More flexibility

Better user experience

The styles

Acros

Astia

Classic Chrome

Classic Neg

Eterna

Eterna Bleach Bypass

Monochrome

Pro Neg. Hi

Pro Neg. Std

Provia

Reala Ace

Velvia

Comparison to similar projects

What's next

Fujifilm Filmsimulationen als Darktable-Stile

Warum?

Mehr Flexibilität

Einsteigerfreundlichkeit

Die Stile

Acros

Astia

Classic Chrome

Classic Neg

Eterna

Eterna Bleach Bypass

Monochrome

Pro Neg. Hi

Pro Neg. Std

Provia

Reala Ace

Velvia

Ähnliche Projekte

Wie geht es weiter?

ROCKPro64 as Multimedia System and NAS: A long journey

Designing an Infrared Forwarder/Sniffer/Injector Microcontroller Firmware

Bird's Eye Perspective

First implementation: Forwarding unit without multiplexing

Global Variables Are Expensive

Sending a transcript via UART

Example

Receiving instructions via UART

Data format

Buffering and Flow Control

Signal generation

Wrapping things up

Remotely controlling a Denon AVR-1910 from Linux

Available remote control interfaces

Option 1: HDMI-CEC

Option 2: Room-to-Room Remote Control In

Option 3: Infrared Blaster

How to generate the signals? And what signals even?

Getting Infrared Signals In and Out of a Raspberry Pi

Making the Isolation bearable: Self-hosted Audio/Video Conferencing with Jitsi Meet and Mumble

Introduction and Story Hook

Setting up Jitsi Meet

Prerequisites

Installing Jitsi Meet

Authentication

Setting up Mumble

Using Jitsi Meet

Resource Consumption

Experiment Setup

Timeline

Resource Consumption summary

Summary

On Centralisation of Code Hosting Infrastructure—An Argument

What Services Are We Talking About?

What Are The Threats?

Confidentiality

Integrity

Availability

What Are The Benefits?