This release moves domain randomization of Bullet simulations to the new PyBullet interface. This change makes it easier to customize environments for reinforcement learning, and simplifies the previous dictionary-based runtime configuration of the Bullet interface.

Added

• envs: Add external force application to PyBullet backend
• envs: Add inertia randomization to PyBullet backend
• envs: Add joint friction modeling to PyBullet backend
• envs: Add torque control noise to PyBullet backend
• envs: Add torque measurement noise to PyBullet backend
• envs: Get contact points via the PyBullet backend
• examples: Add PyBullet inertia randomization example
• examples: Annotate PyBullet external force example
• utils: Add ExternalForce class
• utils: Add PointContact class

Changed

• envs: Expose backend in all UpkieEnv environments
• envs: Move external forces from C++ BulletInterface to Python PyBulletBackend
• envs: Standardize Gymnasium dtypes to float32 across environments
• examples: Reorganize examples into backend-specific sub-directories
• examples: Update PyBullet external-force to new backend

Fixed

• envs: Add back UpkieEnv.log function to log data in Python
• utils: Compatibility with old SciPy API with Python 3.9

Removed

• Breaking: cpp: Remove contact monitoring from Bullet interface
• Breaking: cpp: Remove external force processing from Bullet interface
• Breaking: cpp: Remove inertia randomization from Bullet interface
• Breaking: cpp: Remove joint friction from Bullet interface
• Breaking: cpp: Remove joint properties from Bullet interface
• Breaking: cpp: Remove torque control noise from Bullet interface
• Breaking: cpp: Remove torque measurement noise from Bullet interface
• Breaking: envs: Remove get_bullet_action function
• Breaking: envs: Remove set_bullet_action function
• Breaking: spines: Remove inertia randomization from Bullet spine

This patch release fixes odometry configuration in PyBullet simulations, as well as a compilation error for the keyboard sensor on ARM64 macOS.

Added

• examples: Add PEP-723 inline dependencies to example scripts

Fixed

• envs: Use wheel radius from robot configuration in PyBullet odometry
• sensors: Fix keyboard build error on ARM64 macOS

This release refactors the robot software in major ways:

• Simulators: Upkie now runs out of the box in PyBullet and Genesis.
• Reinforcement learning: policy training using RL Baselines3 Zoo.
• Parallel simulation: new simulation environments run directly in Python and can be parallelized.
• Agents: standard behaviors, like the MPC balancer and the trajectory player, are now directly packaged in the repository.

Here is an example of the same agent running in two different simulators:

The new simulation environments are easier to interface with reinforcement-learning frameworks, while the spine workflow (with the simulation-real bridge) is still active for testing and deploying to real Upkies.

Gymnasium environments

The upkie library can register standard Gymnasium environments:

Detailed changelog

Added

• Major: Add Gymnasium environments with the Genesis simulator
• Major: Add Gymnasium environments with the PyBullet simulator
• Major: Add rlb3_trainer agent based on RL Baselines3 Zoo
• Major: Add trajectory-player agent
• agent: Add start_mpc_balancer.sh script to the repository
• agent: Import MPC balancer from previous Pink balancer
• config: Load user configuration from ~/.config/upkie
• control: Create upkie.controllers submodule
• control: Import MPC balancer class from the Pink balancer
• cpp: Add documentation to the moteus actuation interface
• cpp: Controller pipeline run by the spine after observers
• deps: Add dependency on PyYAML
• envs: Abstract base class for environments named UpkieEnv
• envs: Add Genesis backend
• envs: Add PyBullet backend
• envs: Add internal upkie.envs.backends submodule
• envs: New Upkie-Mock-Pendulum environment
• envs: New Upkie-Mock-Servos environment
• envs: New Upkie-PyBullet-Pendulum environment
• envs: New Upkie-PyBullet-Servos environment
• examples: Genesis lying genuflection example
• examples: Trying a Genesis simulation environment
• examples: Trying a PyBullet simulation environment
• exceptions: Raise UpkieTimeoutError upon spine timeout
• logging: Add disable_warnings helper function
• model: List joint names in Model class
• spines: Log buffer size of the logger itself
• utils: Joystick utility class
• utils: Wrap quaternion conversion function from SciPy

Changed

• Breaking: Move upkie.spine submodule to upkie.envs.backends.spine
• Breaking: Rename UpkieGroundVelocity Gymnasium environment to Upkie-Spine-Pendulum
• Breaking: Rename UpkieGroundVelocity wrapper to UpkiePendulum
• Breaking: Rename UpkieServos Gymnasium environment to Upkie-Spine-Servos
• Breaking: Rename interfaces C++ namespace from actuation
• Breaking: Rename upkie.logging submodule from upkie.utils.spdlog
• CICD: Move coverage and lint rules from the Makefile to Pixi tasks
• CICD: Separate jobs for C++ and Python library testing
• CICD: Switch to Pixi in CI workflow
• CICD: Switch to pytest for Python unit tests
• Clean up Python cache directories upon make clean
• Restrict Bazel to C++ tasks and use Pixi for Python tasks in development workflow
• cpp: Fix Bazel imports of C++ packages going up the dependency tree
• cpp: Limit rate of SynchronousClock skip warnings and report frequency drop
• cpp: Update mpacklog.cpp to version 3.2.0
• deps: Bump loop-rate-limiters to 1.2.0
• envs: Allow gain scales larger than one in servo environments
• envs: Allow parallel simulations, clearing up the CPU bottleneck
• envs: Class UpkieServos is now a subclass of UpkieEnv
• envs: Drop Gymnasium environment versioning
• examples: Allow users to select the Gymnasium environment in MPC example
• examples: Rename PyBullet lying genuflection example
• logging: Use an "upkie" logger rather than the global logger
• model: Load the URDF description from upkie_description by default
• nit: Remove brackets around SI units in documentation and comments
• spines: Factor common observers and sensors between spines
• spines: Throttle clock warning frequency based on time and performance drop
• tools: Update script to dump moteus servo configurations
• utils: Generalize abs_bounded_derivative_filter to vector inputs
• utils: Generalize bounded_derivative_filter to vector inputs
• utils: Use pathlib rather than os.path in Raspberry Pi detection

Fixed

• exceptions: Fix error message reporting of UpkieError base class
• exceptions: Make sure we stop the spine even upon interruption
• spines: Add exception handling to all three spines
• spines: Handle empty user replies when no joystick is found
• utils: Fix incorrect check on inability to set affinity from an interpreter
• utils: Remove unnecessary requirement to configure the agent process as root

Removed

• Breaking: Remove output bound from abs_bounded_derivative_filter
• Breaking: Remove output bound from bounded_derivative_filter
• cpp: Remove palimpsest Bazel package that is already upstream from mpacklog

This patch release triggers a correction to the release packaging workflow in order to distribute pre-compiled Bullet and pi3hat spines.

This release introduces a C++ controller pipeline API. It also removes servo-position and servo-torque sub-environments that have become unused by downstream agents.

These two changes are preparatory to v9.0.0 where controller pipelines will be put to practice, and downstream agents will be moved to the main Upkie repository (so that Gymnasium environments and the agents use them and are maintained are all in one repository).

Added

• Development workflow based on Pixi
• cpp: Controller pipeline run by the spine after observers

Changed

• cpp: Spines now take a controller pipeline as constructor argument
• envs: Wait one second when resetting environments on the Raspberry Pi (thanks to @Tordjx)

Fixed

• CICD: Update release workflow images to ubuntu-latest
• Spine: Fix full-logger issue when substepping a simulation spine

Removed

• envs: Remove UpkieServoPositions and UpkieServoTorques environments

This new version makes UpkieGroundVelocity a wrapper around UpkieServos, in preparation for non-servo simulation spines. Alongside comes a new example on domain randomization and a regular batch of fixes.

Added

• docs: Install libtinfo5 when building from source on Debian-based distros
• examples: Domain randomization by environment wrapping
• sensors: Sensor pipeline that runs at every spine cycle

Changed

• Breaking: envs: Make UpkieGroundVelocity a wrapper around UpkieServos
• Breaking: envs: Merge base and servos environments
• CICD: Setup Micromamba in coverage job
• Report error messages in simulation script
• Update minimum Gymnasium version to 1.0
• Update palimpsest to version 2.3.2
• cpp: Spine constructor now takes an additional sensor-pipeline argument
• envs: Hard-code spine retries to ten attempts

Fixed

• examples: Fix warning in model predictive control example
• Spine: Fix logging when substepping a simulation spine

Removed

• envs: Remove unused WheeledInvertedPendulum environment

This release adds collision observations to simulation environments, pixi-pack integration and additional unit tests. It is a major release as get_reward functions have been removed from all Gymnasium environments. Otherwise comes a regular batch of fixes and additions to the documentation.

Thanks to @Tordjx for contributing to this release 👍

Added

• Add pixi-pack integration
• actuation: Add collision-with-environment observation (thanks to @Tordjx)
• docs: Start Kinematics page
• utils: Add clear_shared_memory utility function

Changed

• Bazel: Treat warnings as errors (except the one we can't avoid)
• Minimum Python version is now 3.9
• envs: Observation-based reward wrapper
• envs: Unit test for the base step function
• examples: Group simulation-related examples in a sub-directory

Fixed

• Bazel: Ignore .pixi directory as it can contain unrelated Bazel files
• Fix unused variable warning in Bullet interface

Removed

• Breaking: Remove get_reward functions from all environments
• Makefile: Remove conda packing rules, now deprecated in favor of pixi

This release adds link inertia randomization to the Bullet interface contributed by @Tordjx 👍 as well as non-major API updates and a regular batch of fixes.

Added

• Configure pixi in pyproject.toml
• Handle GLIBC version incompatibility in start_simulation.sh
• Spine: Throw an exception when a servo reply has invalid torque
• Update downloaded simulation spine in cache if outdated
• actuation: Added link inertia randomization to the Bullet interface (thanks to @Tordjx)
• actuation: Check maximum torques before sending commands
• cpp: Add ServoError exception
• model: Static model for joints with position, velocity and torque limits
• pixi: Task to generate the documentation by pixi -e docs make

Changed

• CICD: Build release spines with Ubuntu 20.04 rather than 22.04
• CICD: Switch to pixi for documentation workflow
• CICD: Update upload-artifact action to v4
• Remove layout argument to actuation interfaces
• actuation: Rename Interface::initialize_action to Interface::reset_action
• actuation: Rename Interface::servo_joint_map to Interface::servo_name_map
• deps: Upate to palimpsest 2.2.1

Fixed

• examples: Lying genuflection example (thanks to @one-for-all)
• examples: Unwrap environments to get their neutral actions
• Update start_simulation.sh for systems where -v is not defined

Removed

• env: Remove unused leg_return_period and hard-code it to one second
• env: Replace contingent parse_first_observation by a reset override
• model: Remove C++ upkie::model namespace
• observers: Remove upper-leg and wheel joints from configurable parameters

This release gives one more careful 👀 to spine cycles, with the main two updates:

• Combine action and observation steps in the spine-agent IPC protocol, halving the number of waiting intervals between the two.
• Fix observation consistency between the run and simulate spine functions: thanks to @Tordjx for pointing this out 👍

This iteration also produced some more design documentation in the following discussions:

• Starting CAN cycles after sleep or after action
• Shift in actions sequence between agent and simulator

Added

• Makefile: Conda environment packing and unpacking rules
• Support Python 3.12
• tools: Add ps_thread command

Changed

• Breaking: Spine and spine interface revisions:
  • Observations are now returned upon reset and step
  • Spine: Remove separate observation state and observation request
  • SpineInterface: Setting an action now returns an observation
  • SpineInterface: Starting the spine now returns an observation
  • StateMachine: Rename spine FSM state from "act" to "step"
  • docs: Update spine FSM specification in the documentation
• Breaking: actuation: Move resolution to static configuration
• Breaking: actuation: Move servo layout to static configuration
• Breaking: envs: Rename LowPassFilterAction to AddLagToAction
• CICD: Pin Ubuntu workflows to 22.04
• Default kd gain for wheel servos is now 0.3
• Move Python spine exceptions to upkie.exceptions
• docs: Turn environment page into an index
• tools: Make output directory an argument in dump_servo_configs
• tools: Simplify servo configuration script

Fixed

• BulletInterface: Fix typo in a comment
• CICD: Install Doxygen with specific version from conda-forge
• Spine: Fix observation consistency between run and simulate
• envs: Clamp ground velocity action in UpkieGroundVelocity

Removed

• Bazel: Remove legacy rules for PyPI dependencies
• Move PID balancer to its own repository
• deps: Remove dependency on PyYAML
• examples: Remove Bazel BUILD file
• model: Remove duplicate maximum torque constant

This release adds several features to the Bullet simulation spine (custom initial configuration, uncertainty on measurements) as well as a random-push wrapper contributed by @Tordjx, thanks 👍

Added

• BulletInterface: Apply custom initial joint configuration, if provided
• BulletInterface: Report velocity of the base link in groundtruth
• BulletInterface: Torque control noise to joint-property configuration
• BulletInterface: Torque measurement noise to joint-property configuration
• BulletInterface: Uncertainty on IMU accelerometer and gyroscope measurements
• Forward command-line arguments to the spine in start_simulation.sh
• ImuData: Add linear velocity field
• envs: Add a random-push wrapper (thanks to @Tordjx)
• examples: Apply an external force to lift an Upkie in sim
• examples: Custom initial state with a non-zero joint configuration
• examples: Simulation with joint friction
• examples: Simulation with sensor noise
• spines: Add variant argument to BulletSpine

Changed

• BulletInterface: Move simulation body poses to sim.bodies
• BulletInterface: Rename internal unit-test getters
• actuation: Log simulation groundtruth to sim

Fixed

• BulletInterface: Fix application of external forces
• CICD: Enable stable-baselines3 environment-check unit tests
• docs: Add missing documentation pages
• docs: Document Gym environment wrappers
• envs: Handle spine errors raised in base env constructor
• envs: Register UpkieServoPositions and UpkieServoTorques environments

Removed

• CICD: Remove pycodestyle as we now use ruff for Python linting
• BulletInterface: Remove unused orientation/position unit-test getters