Thanks to visit codestin.com
Credit goes to github.com

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
c5b8347
First working draft
frzb Jan 26, 2022
5bb2c11
Remove gcc-9 and secure boot related packages
frzb Jan 26, 2022
a9f7e8f
Remove grub-common
frzb Jan 26, 2022
3e4bb12
Remove autoconf and dialog package, cleanup
frzb Jan 26, 2022
b22e2d7
Draft for Navi with ROCr OpenCL
frzb Jan 27, 2022
bcc32d9
Remove grub-common
frzb Jan 27, 2022
539a80a
Remove dialog package, cleanup
frzb Jan 27, 2022
1b63c7a
Rename to amdgpu, bump to 21.40.2
frzb Jan 27, 2022
ccd5905
Cleanup
frzb Jan 27, 2022
876ecf9
Just copy modules (*.ko)
frzb Jan 27, 2022
127859b
Use navi script for polaris plugin
frzb Jan 28, 2022
0d07687
Introduce directory for enabled plugins
frzb Jan 28, 2022
c38d904
Add libnuma1
frzb Jan 29, 2022
d6e944e
amd_iommu_v2.ko changed to iommu_v2.ko
frzb Jan 29, 2022
c86849b
Add upstream amdgpu firmware
frzb Jan 29, 2022
b0ac8ce
Update teamredminer to v0.9.1
frzb Feb 1, 2022
cf77ae9
Add .keep files to keep empty directories in VCS
frzb Feb 7, 2022
a6b0b2b
Change of IOMMU module for AMD for Polaris GPU
frzb Feb 13, 2022
1e8e81e
Move TRM arguments to the environment
frzb Feb 13, 2022
73810f7
TRM v0.9.2.1 and working arguments via environment
frzb Feb 18, 2022
5956fee
Bump Team Red Miner so v0.9.4
frzb Mar 6, 2022
c70b910
Add ability to select fimrware to include
frzb Mar 6, 2022
5bf1eec
Bump Team Red Miner tp 0.9.4.2
frzb Mar 12, 2022
8c028b4
Enhanced coinbootmaker -l and -h output
frzb Mar 12, 2022
32cdb3f
Made cleanup of cbm-host interface conditional
frzb Mar 12, 2022
ae308af
Made existence check for merged mountpoint quiet
frzb Mar 12, 2022
83e34f2
Improved message for used release tag
frzb Mar 12, 2022
4f0ecf5
Add filter for pre-releases
frzb Mar 12, 2022
1eeaeca
Cleanup leftover namespace, reduce noisy output
frzb Mar 14, 2022
ba98448
Legacy and ROCr OpenCL with the same plugin script
frzb Mar 14, 2022
f16a7cf
Add AMDGPU driver 21.50.2
frzb Mar 15, 2022
1266dcf
Fix apt-ge command
frzb Mar 15, 2022
06daccb
Enable usage of upstream firmware
frzb Mar 18, 2022
370687d
Separate tunable variables
frzb Mar 18, 2022
0bd0769
Updated metadata for Navi and Polaris AMDGPU
frzb Mar 18, 2022
e004f70
Add AMDGPU Vega plugin build script
frzb Mar 18, 2022
87fc1c0
Updated info block about GPU series
frzb Mar 19, 2022
e9725ea
Added WORKERNAME
frzb Mar 19, 2022
c2f622b
Reworked start of teamredminer by calling exec
frzb Mar 20, 2022
608e6e8
Rework versioning
frzb Mar 21, 2022
5ee0bab
Fix indentation
frzb Mar 23, 2022
ae0a1b4
Rearranged logic order to make ROCr OpenCL default
frzb Mar 23, 2022
17fa550
Got rid of redundant deletion of mocked update-initramfs
frzb Mar 23, 2022
87b830f
Remove plugin dir after build
frzb Mar 28, 2022
21bde5f
Use docker-compose from develop
frzb Apr 9, 2022
d4e566f
Added package cacheing for apt to Coinbootmaker
frzb Apr 10, 2022
d43f22b
Reworked OpenCL package logic, dual stack is valid
frzb Apr 10, 2022
2187d2f
Excluded /boot directory from plugins
frzb Apr 10, 2022
fc2432c
Added purging to apt autoremove
frzb Apr 10, 2022
df5f512
Reworked exec call variables expansion for TRM
frzb Apr 11, 2022
da0a848
Added well-known yet undocumented OpenCL variables
frzb Apr 11, 2022
1b79427
Added plugins for ROCr and Legacy OpenCL
frzb Apr 11, 2022
57e26f0
Remove extra space
frzb Apr 12, 2022
c4cc7da
Cleanup of renamed enable directory
frzb Apr 12, 2022
094d356
Updated README_template to reflect current changes
frzb Apr 12, 2022
e7b2ead
Apply automatic changes of Coinboot plugin table [skip ci]
frzb Apr 12, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions VERSIONING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Versioning of Coinboot

## Scheme

Versioning uses [CalVer](https://calver.org/) with the format:

**0Y.0M.0D-modifier**

**0Y**        - Zero-padded year - 06, 16, 106
**0M**        - Zero-padded month - 01, 02 ... 11, 12
**0D**        - Zero-padded day - 01, 02 ... 30, 31
**-modifier**  - An optional text tag, such as "dev", "alpha", "beta", "rc1"


## Examples

**22.05.01-rc0**

**22.01.30**
13 changes: 12 additions & 1 deletion coinbootmaker/coinbootmaker
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ RELEASE=${RELEASE:-latest}
if [ $RELEASE = latest ]; then
RESPONSE=$($CURL --silent "https://api.github.com/repos/${GITHUB_REPO}/tags")
sleep 5
while ! TAG=$(echo $RESPONSE | jq -r '[ .[].name | select(test("^pre.*") | not) ] | sort | last'); do
while ! TAG=$(echo $RESPONSE | jq -r '[ .[].name | select(test("^pre.*") | not) ] | sort | last' ); do
echo "Calling the Github API has failed, repeat ..."
RESPONSE=$($CURL --silent "https://api.github.com/repos/${GITHUB_REPO}/tags")
sleep 5
Expand Down Expand Up @@ -169,6 +169,8 @@ sudo tee etc/hosts << EOF 1> /dev/null
127.0.1.1 coinbootmaker
EOF

[ -d $BASEDIR/cache/apt ] || mkdir $BASEDIR/cache/apt

cd $LOWER
# Generate container spec file config.json
# We have to manipulate the config.json created by 'runc spec'.
Expand Down Expand Up @@ -384,6 +386,15 @@ sudo tee ./config.json << EOF 1> /dev/null
"bind",
"ro"
]
},
{
"destination": "/var/cache/apt",
"type": "none",
"source": "$BASEDIR/cache/apt",
"options": [
"bind",
"rw"
]
}
],
"linux": {
Expand Down
9 changes: 4 additions & 5 deletions debirf/scripts/create_plugin.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,21 +41,22 @@
PLUGIN_DIR = "/mnt/plugin/rootfs"

EXCLUDE = (
".*__pycache__.*",
".wget-hsts",
"/boot",
"/dev/",
"/proc/",
"/run/",
"/sys/",
"/tmp/",
"/usr/src",
"/usr/include",
"/usr/share/dbus-1/system-services",
"/usr/src",
"/vagrant",
"/var/cache",
"/var/lib/apt/lists",
"/var/lib/dpkg/[^info]",
"/var/log",
".*__pycache__.*",
".wget-hsts",
r".*\.cache",
)

Expand Down Expand Up @@ -135,8 +136,6 @@ def main(arguments):
archive_name = arguments["<plugin_name>"] + ".tar.gz"

create_tar_archive(archive_name, files_for_plugin_archive)

print("------------------------------------")

print("------------------------------------")

Expand Down
19 changes: 11 additions & 8 deletions plugins/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ For more information how to boot your machines with Coinboot visit: https://coin

## Requirements

* Coinboot with kernel `5.4.0-58-generic`
* Coinboot with kernel `5.11.0-46-generic`

### If you want to build plugins on your own:

Expand All @@ -42,7 +42,7 @@ into the `plugins` directory of your Coinboot setup.
To build Coinboot plugins on your own use `coinbootmaker`.

```
Usage: coinbootmaker [-i] [-l] [-h] [-p <plugin name> <path to initramfs>]
Usage: coinbootmaker [-i] [-l] [-h] [-p <plugin name>]

-i Interactive mode - opens a shell in the build environment
-p <file name> Plugin to build
Expand All @@ -53,20 +53,23 @@ Usage: coinbootmaker [-i] [-l] [-h] [-p <plugin name> <path to initramfs>]
For example:

```
$ ./coinbootmaker -p ethminer /tmp/coinboot-initramfs-4.15.0-43-generic
$ ./coinbootmaker -p ethminer
```

`coinbootmaker` takes a path to a Coinboot-Initramfs to create an environment for building the plugins
`coinbootmaker` uses a Coinboot RootFS release to create an environment for building the plugins
by converting the given Initramfs into a Container image and run it.
The plugin creation script located at `src` is executed in that `coinbootmaker` container and the resulting
plugin archive is written to the `build` directory.
plugin archive is written to the `build` directory.
#### Setting a Release
You can specify the used Coinboot RootFS be setting the `RELEASE` enviroment variable.
The default value is `RELEASE=latetst` and will pulll the latest release of the Coinboot RootFS.

### Run `coinbootmaker` interactivly (`-i`)

For developing and debugging

```
$ ./coinbootmaker -i /tmp/coinboot-initramfs-4.15.0-43-generic
$ ./coinbootmaker -i
```

* You are entering the build environment
Expand All @@ -77,7 +80,7 @@ $ ./coinbootmaker -i /tmp/coinboot-initramfs-4.15.0-43-generic

* When your are done: Execute `$ create_plugin.py finish <name-of-your-plugin>`

* Place the created plugin archive into `./plugins` on the host where you run the Coinboot Docker container
* Place the created plugin archive into the directory `./plugins/enabled` on the host where you run the Coinboot Docker container

Up on the next boot the changes your made in your plugin are ready to be used on your Coinboot machines!

Expand All @@ -98,5 +101,5 @@ [email protected]

## Contribution

Fork this repo.
To contribute your own plugins fork this repo.
Make a pull request to this repo.
19 changes: 11 additions & 8 deletions plugins/README_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ For more information how to boot your machines with Coinboot visit: https://coin

## Requirements

* Coinboot with kernel `5.4.0-58-generic`
* Coinboot with kernel `5.11.0-46-generic`

### If you want to build plugins on your own:

Expand All @@ -34,7 +34,7 @@ into the `plugins` directory of your Coinboot setup.
To build Coinboot plugins on your own use `coinbootmaker`.

```
Usage: coinbootmaker [-i] [-l] [-h] [-p <plugin name> <path to initramfs>]
Usage: coinbootmaker [-i] [-l] [-h] [-p <plugin name>]

-i Interactive mode - opens a shell in the build environment
-p <file name> Plugin to build
Expand All @@ -45,20 +45,23 @@ Usage: coinbootmaker [-i] [-l] [-h] [-p <plugin name> <path to initramfs>]
For example:

```
$ ./coinbootmaker -p ethminer /tmp/coinboot-initramfs-4.15.0-43-generic
$ ./coinbootmaker -p ethminer
```

`coinbootmaker` takes a path to a Coinboot-Initramfs to create an environment for building the plugins
`coinbootmaker` uses a Coinboot RootFS release to create an environment for building the plugins
by converting the given Initramfs into a Container image and run it.
The plugin creation script located at `src` is executed in that `coinbootmaker` container and the resulting
plugin archive is written to the `build` directory.
plugin archive is written to the `build` directory.
#### Setting a Release
You can specify the used Coinboot RootFS be setting the `RELEASE` enviroment variable.
The default value is `RELEASE=latetst` and will pulll the latest release of the Coinboot RootFS.

### Run `coinbootmaker` interactivly (`-i`)

For developing and debugging

```
$ ./coinbootmaker -i /tmp/coinboot-initramfs-4.15.0-43-generic
$ ./coinbootmaker -i
```

* You are entering the build environment
Expand All @@ -69,7 +72,7 @@ $ ./coinbootmaker -i /tmp/coinboot-initramfs-4.15.0-43-generic

* When your are done: Execute `$ create_plugin.py finish <name-of-your-plugin>`

* Place the created plugin archive into `./plugins` on the host where you run the Coinboot Docker container
* Place the created plugin archive into the directory `./plugins/enabled` on the host where you run the Coinboot Docker container

Up on the next boot the changes your made in your plugin are ready to be used on your Coinboot machines!

Expand All @@ -90,5 +93,5 @@ [email protected]

## Contribution

Fork this repo.
To contribute your own plugins fork this repo.
Make a pull request to this repo.
152 changes: 152 additions & 0 deletions plugins/src/amdgpu/21.40.2.40502/amdgpu_navi.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
---
# Copyright (C) 2018, 2020 - 2022 Gunter Miegel coinboot.io
#
# This file is part of Coinboot.
# This software may be modified and distributed under the terms
# of the MIT license. See the LICENSE file for details.
#
# Please notice even while this script is licensed with the
# MIT license the software packaged by this script may be licensed by
# an other license with different terms and conditions.
# You have to agree to the license of the packaged software to use it.

plugin: AMDGPU Polaris
archive_name: amdgpu_navi_rocr_opencl
version: 21.40.2.40502
description: AMD Navi GPU (RX 5000/RX 6000 family) firmware and driver with support for ROCr OpenCL 2.1
maintainer: Gunter Miegel <[email protected]>
source: https://www.amd.com/en/support/kb/release-notes/rn-amdgpu-unified-linux-21-40-2
run: |
set -e

sudo apt update

# Specfiy AMD Radeon GPU series codename.
# RX 400/500 series: polaris
# RX Vega series: vega
# RX 5000/6000 series: navi
GPU_CODENAME=navi
AMDGPU_INSTALL_PACKAGE=amdgpu-install_21.40.2.40502-1_all.deb
if [ ! -z $DEBIRF_KERNEL ]; then
KERNEL_RELEASE=$DEBIRF_KERNEL
fi

PACKAGES="wget linux-headers-$KERNEL_RELEASE"
# We don't install any xserver stuff, similar to amdgpu-pro installer
# '--compute'
#AMDGPU_OPENCL_PACKAGES='clinfo-amdgpu-pro opencl-orca-amdgpu-pro-icd'rocm-opencl-runtime
AMDGPU_OPENCL_PACKAGES='rocm-opencl-runtime libnuma1'
# The DKMS package is installed explictily to purge it without wiping the dependcies.
AMDGPU_DKMS_PACKAGE='amdgpu-dkms'
AMDGPU_CACHE_PATH=/mnt/cache/amdgpu
EXTRA_MODULES_PACKAGE="linux-modules-extra-$KERNEL_RELEASE"
EXTRA_MODULES_PACKAGE_FILE=$(apt-cache show $EXTRA_MODULES_PACKAGE | grep -oP $EXTRA_MODULES_PACKAGE.*deb)
UPSTREAM_FIRMWARE_ARCHIVE=linux-firmware-20211216.tar.gz

sudo apt-get install $PACKAGES --yes

if [ -d $AMDGPU_CACHE_PATH ]; then
mkdir -p $AMDGPU_CACHE_PATH
fi

# Let's cache the downloaded archive and the decompressed
# archive on the build host - time saving once more.
# Download is only allowed with this referer!
if ! [ -f "$AMDGPU_CACHE_PATH"/"$AMDGPU_INSTALL_PACKAGE" ]; then
wget --progress=bar:force \
https://repo.radeon.com/amdgpu-install/21.40.2/ubuntu/focal/"$AMDGPU_INSTALL_PACKAGE" -P $AMDGPU_CACHE_PATH
else
echo "$AMDGPU_INSTALL_PACKAGE is already in build cache."
fi

if ! [ -f "$DEBIRF_ROOT/tmp/$AMDGPU_INSTALL_PACKAGE" ]; then
cp -r "$AMDGPU_CACHE_PATH/$AMDGPU_INSTALL_PACKAGE" /tmp/
else
echo "$AMDGPU_INSTALL_PACKAGE is already copied to the chroot."
fi

if ! [ -f "$AMDGPU_CACHE_PATH"/"$UPSTREAM_FIRMWARE_ARCHIVE" ]; then
wget --progress=bar:force \
https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/snapshot/$UPSTREAM_FIRMWARE_ARCHIVE -P $AMDGPU_CACHE_PATH
else
echo "$UPSTREAM_FIRMWARE_ARCHIVE is already in build cache."
fi

if ! [ -f "$DEBIRF_ROOT/tmp/$UPSTREAM_FIRMWARE_ARCHIVE" ]; then
cp -r "$AMDGPU_CACHE_PATH/$UPSTREAM_FIRMWARE_ARCHIVE" /tmp/
else
echo "$UPSTREAM_FIRMWARE_ARCHIVE is already copied to the chroot."
fi

# The amdgpu-pro-install script is not running properly non-interactive
# it uses 'apt-get install' and asks for (Y/n).
# So we install the packages on our own.
# 1. Create the local deb repository
# 2. Update the package list
# 3. Install the packages

sudo apt-get install --yes /tmp/$AMDGPU_INSTALL_PACKAGE

sudo sed -i 's/^#deb /deb /' /etc/apt/sources.list.d/amdgpu-proprietary.list

sudo apt-get update


# TODO mock update-initramfs called by /var/lib/dpkg/info/amdgpu-dkms.postinst

sudo tee /usr/bin/update-initramfs << EOF
#!/usr/bin/bash

true
EOF

sudo chmod 755 /usr/bin/update-initramfs

# FIXME: /opt/amdgpu-pro/lib/x86_64-linux-gnu/libOpenCL.so.1 missing when installing
# $AMDGPU_DKMS_PACKAGE and $AMDGPU_OPENCL_PACKAGES at once.
# sudo apt-get install --yes $AMDGPU_OPENCL_PACKAGES $AMDGPU_DKMS_PACKAGE
sudo apt-get install --yes $AMDGPU_OPENCL_PACKAGES

#sudo apt-mark hold $AMDGPU_OPENCL_PACKAGES

sudo apt-get install --yes $AMDGPU_DKMS_PACKAGE

# Purge installed packages of this modules to free up the space.
# But keep the OpenCL related packages.

# The module amdgpu.ko requires iommu_v2.ko which comes with the linux-modules-extra-
sudo apt-get download "$EXTRA_MODULES_PACKAGE"

# dpkg-deb -x needs a full path!
sudo dpkg -x $PWD/$EXTRA_MODULES_PACKAGE_FILE /tmp/
sudo rm -rf $PWD/$EXTRA_MODULES_PACKAGE_FILE

sudo find /tmp/lib -name 'iommu_v2.ko' -exec install -vD {} /lib/modules/"$KERNEL_RELEASE"/kernel/drivers/iommu/amd/iommu_v2.ko \;

sudo rm -rf /tmp/lib

sudo cp -v /lib/modules/"$KERNEL_RELEASE"/updates/dkms/amd* /tmp/

sudo cp -Rv /lib/firmware /tmp/

sudo apt-get purge $PACKAGES --yes

# Additional packages which can be removed to save space.
sudo apt-get purge --yes --allow-remove-essential ^gcc-9 grub-common dialog
sudo apt autoremove --yes

dpkg -l|grep opencl

sudo cp -v /tmp/amd*.ko /lib/modules/"$KERNEL_RELEASE"/updates/dkms/

sudo mkdir -p /lib/firmware/updates/amdgpu
sudo find /tmp/firmware -name "$GPU_CODENAME*" -exec cp -v {} /lib/firmware/updates/amdgpu/ \;

UPSTREAM_FIRMWARE_DIRECTORY=$(basename -s '.tar.gz' $UPSTREAM_FIRMWARE_ARCHIVE)/amdgpu
sudo tar -xzvf /tmp/$UPSTREAM_FIRMWARE_ARCHIVE -C /tmp -P $UPSTREAM_FIRMWARE_DIRECTORY
sudo cp -rv /tmp/$UPSTREAM_FIRMWARE_DIRECTORY /lib/firmware/updates/

sudo depmod $KERNEL_RELEASE

# Remove mocked update-initramfs
sudo rm -v /usr/bin/update-initramfs
Loading