- Shell 100%
| scripts | ||
| LICENSE.txt | ||
| README.md | ||
These scripts will set up your own verification server, based on the numbered scripts in the scripts/ directory. They have only been tested on a few machines so far, so please read the below carefully – and check the scripts before running them. Use them at your own risk (see „16. Limitation of Liability“ in the AGPL license file included with this repository) 😉
Requirements
These scripts must be run on either
- Debian based machines (so Debian, Ubuntu, Linux Mint etc) utilizing the
aptpackaging system - RPM based systems (Fedora, RHEL, CentOS etc) using
dnf - Arch based systems (Arch, Manjaro, EndeavourOS etc) using
pacman
They have so far only be tested on Linux Mint and Debian (bookworm), Fedora (41), and Arch Linux. They should be run under the OS user that will operate the builder later. For the first step, this user must have permission to use sudo to run apt-get/dnf/pacman commands . Those elevated permissions (sudo) are only needed for the initial setup (to install the software), and thus can be withdrawn once the setup is complete; they are not needed to operate a builder).
While RPM and Arch based systems are supported, too, not all packages are available there. Missing will be e.g. apksigner, dexdump and trydiffoscope. None of these are needed to run a builder; they're used by some rbhelper functions. If you e.g. want to analyze differences in classes*.dex files, you will need dexdump. dexdump and apksigner are part of the is part of the Android build-tools and can be obtained from there (e.g. Android SDK Build-Tools 34; a list of available downloads can be found e.g. at Android SDK Offline). On Arch based systems, you could also install the android-sdk-build-tools packages via AUR for apksigner and dexdump to become available, though that might be quite an overhead. Diffoscope is available via the full diffoscope package on Arch as well as RPM, and alternatively can be used via their Podman image.
Usage
- login as the user who shall operate the builder, and checkout this repo (or download and unpack the
*.zip/*.tar.gzfile) - change into the
rbuilder_setup/scripts/directory - Make sure to first copy
.config.sampleto.config, and adjust the latter to your intended installation wishes. Each setting is explained there. - Run
setup.sh– either after making it executable (chmod +x setup.sh) and running./setup.sh, or by simply runningbash setup.shfrom within this directory. Append| tee install.logto have a protocol of your installation stored toinstall.log, e.g.bash setup.sh | tee install.log
The setup.sh script will invoke each of the numbered scripts in their intended order. The settings from your .config will be used for this:
01_sudo_requirements_*.shwill be run usingsudoto install the required packages viaapt/dnf/pacman.02_venv.shwill set up a Python virtual environment (venv) with Python tools not available via their corrsponding system packages.03_rbtlog.shwill set uprbtlogitself.- with
RBTLOG_INSTALL_RBHELPER=1in your.config,04_rbhelper.shwill set uprbhelperand its required tools.
Notes on installation size
On Debian based systems, a full install will use about 2.7 GB of disk storage (at the time of this writing, i.e. 3/2025).
The hugest part of that is Diffoscope, which with its full install (RBTLOG_INSTALL_RBHELPER_DIFFOSCOPE=3) will be close to 1.7 GB. As this is optional (only needed to debug failed RBs) you might wish to skip it (you can always use try.diffoscope.org via your web browser) or go with RBTLOG_INSTALL_RBHELPER_DIFFOSCOPE=1 to use the command-line wrapper of that website. A third option is to use their podman image, as described in the config file – after all, you have podman installed anyway.
Second largest part is apksigner + bat + dexdump with ~460 MB, if RBTLOG_INSTALL_RBHELPER_APKDIFF=1 is set in your config. Again, these are not needed to run a builder, but to debug failed RBs.
So what remains is about 500 MB for the „mandatory parts“ (including Podman and Python).
Afterwork needed?
While these setup scripts should install everything needed, they cannot complete your „personal configuration“ – which you will have to do yourselves:
- rbhelper: check the
rbhelper.configand adjust it to your needs. E.g. set your NTFY channel if you want notifications, and configure your proxy to speed up things (and save bandwidth). Also set yourGITHUB_TOKENto not get limited on update checks. - rbtlog: this repo was checked out via
https. You will need to add your own remote(s) (usinggit remote add) if you plan to publish your results. You will also have to update itscodebranch (and your working branches from that) from time to time. Further make sure to always sign each commit (on your „public repo“) with your GPG key:
this setup only creates an example repo to play with. Your „real one“ will need to have all of its commits GPG signed (see e.g. Sign your commits in our wiki on Developer Best Practices) – which should ideally not happen on the builder itself, but on your own local machine. For that you will need GPG (install it withapt install gpg), as described on the linked wiki page. - reproducible-apk-tools: nothing to configure here – but again, you'll have to keep them updated. In the VEnv (for rbtlog), but also the copy rbhelper uses for manual checks of failed RBs.
A special note on local proxies
If you plan to use a proxy, and want to run it on the same host rbtlog will run on, you might run into problems with Podman 5+ – where networking switched from slirp4netns to pasta, no longer exposing the host. You know you're affected by this, if you see errors like
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/jammy/InRelease Could not connect to 192.168.83.10:3128 (192.186.83.10). - connect (111: Connection refused)
As the example squid.conf here allows access for localhost, and sticks with 3128 as port, the best way is to go with HTTP_PROXY="http://127.0.0.1:3128" in your environment (e.g. set via your rbhelper.config), and placing the following in your $XDG_CONFIG_HOME/containers/containers.conf, to map port 3128 from localhost in the container to localhost on the host:
[network]
pasta_options=["--tcp-ns", "3128:3128",
"--udp-ns", "3128:3128"]
Should that for some reason not work for you, please consult the podman man page for a proper solution matching your needs. To quickly get things working (again), you can e.g. revert back to the behavior before Podman 5, by simply adding the following to your $XDG_CONFIG_HOME/containers/containers.conf instead:
[network]
default_rootless_network_cmd = "slirp4netns"
or enable host network via
[containers]
netdns = "host"
Note that „the easiest“ does not imply „the best“ – so security-wise, the latter two approaches are not what we'd recommend. They are just quick ways to get it working, e.g. for testing. Describing a proper full networking setup, is not in the scope of this document.
Updates
From time to time, you will need to update (parts of) your installation:
rbtlog
From within the rbtlog directory. The following is based on the assumption that origin points to the upstream repo (which it does with the „demo repo“), and runner is the public repo you've set up for your builder (not done automatically with these setup scripts). Further, it assumes a „simple install“ as per the .config.example (with RBTLOG_USE_MASTER=0), so you need to adjust the values accordingly for your real setup (also include -S where needed if you're signing your commits, unless you've set up git to do that automatically):
signme='-S' # set to an empty string if you're not signing your commits (you should sign with a „production builder“)
git checkout code
git pull $signme origin code
git push runner code
git checkout fraggle/log
git merge $signme code
git push origin fraggle/log
rbhelper
Similarly from within the rbhelper directory. As long as rbhelper has no tagged releases, simply update from HEAD:
git pull origin master
For its reproducible-apk-tools, from within that directory, do the same (the setup checked out from HEAD – of course you're free to switch to a tagged release there).
Virtual Env
Assuming you've installed this to $HOME/venvs (see the RBTLOG_VENV_BASE variable in the setup script's config):
cd $HOME/venvs/repro-apk
. bin/activate
for pkg in apksigcopier repro-apk; do [[ -n "$(pip list --local | grep $pkg)" ]] && pip install $pkg -U; done
deactivate
The for loop here ensures that apksigcopier is skipped when it was installed from its Debian package.
Usage examples
Here are some (hopefully) useful examples. This is by far no extensive or even complete list, but just to get you started. As very first step, source the rbhelper/.env file, to have rbhelper commands available. If you've set up a dedicated user for your builds, you might wish to include that call with your .bashrc, .profile or .bash_aliases.
In the following examples, com.example.app is used for the package name of the app you want to operate on, v1.2.3 is used for an example tag to process, ab123cd (even though too short for this) as example commit hash.
- update the recipe for a given app:
rblog update-recipes com.example.app(to update all recipes, simply skip the package name) - test-run a build recipe:
testbuild com.example.app v1.2.3 - build an app based on a given recipe (tag), but from a different commit, and compare it against a local APK (useful for checking if a failed RB has been fixed):
rblog build com.example.app:v1.2.3:ab123cd:/home/builder/example.apk local
(note that/home/builder/example.apkcan also be an URL to pull the APK from, likehttps://example.com/example.apk) - build all tags for a given app that have no entries in their corresponding
log/*.jsonand have those entries added:
rblog update-log com.example.app
again, to do that for all the apps you've set up recipes for, simply ommit the package name. - write the index for all existing logs:
rblog writeIndex - just build the APK, but do not check for RB (useful for developers who want to ensure a clean release build that passes RB with other builders):
rblog build com.example.app:v1.2.3::none local - list commands/tools available with rbhelper:
rbhelp
More details can be found in the documentation of the corresponding tools – and in our Verification Builder wiki page.
License
rbuilder_setup is licensed as AGPL-3.0-or-later (see the LICENSE.txt):
This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License version 3 as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.
Funding
This project is funded through NGI Mobifree Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet program. Learn more at the NLnet project page.
