Low Level (RL Policy) — Python Example

Source: rbq_sdk/python/example/sport/rbq_low_level.py

Python port of rbq_low_level.cpp.
Subscribes to IMU, joystick, and leg-joint feedback topics, publishes joint position references, and runs a learned walking policy (ONNX) at 10 Hz.

Behavior overview

A single 500 Hz control loop (2 ms period) runs in a background thread. A keyboard-driven state machine controls which mode is active:

State	Key	What the control loop does
Idle	—	Tracks rt/rbq/ref/leg_joint/final and republishes it as a pass-through
Motion	x / z	Cosine-interpolates joints to ready or ground postures
Control	c	Loads the ONNX policy, runs inference at 10 Hz (every 5th tick), publishes MotionRef_

Prerequisites

• rbq_sdk_py installed — see Python SDK Overview.
• onnxruntime: pip install onnxruntime
• A policy directory containing info.json and policy.onnx.
• Robot stack running (simulation or hardware).

Install & Run

cd RBQ/rbq_sdk/python
pip install -e .
pip install onnxruntime

cd RBQ/rbq_sdk/python/example/low_level
# networkInterface is optional (defaults to loopback for simulation)
python3 rbq_low_level.py -p <policyDir> [networkInterface]
# e.g.
python3 rbq_low_level.py -p ../../../../rbq_simulator/rbq_gym/policy/rbq10/
python3 rbq_low_level.py -p ../../../../rbq_simulator/rbq_gym/policy/rbq10/ eth0

Unlike the C++ version, sudo is not required — Python does not use a real-time thread.

Run with simulation

1. From RBQ/, start the simulator:

   bash scripts/sim.bash

2. In RBQGUI: Connect → Auto Start Robot → Stance (key 2).
3. Run the example (loopback — no interface needed for sim):

   python3 rbq_low_level.py -p ../../../../rbq_simulator/rbq_gym/policy/rbq10/

Keyboard controls

Key	Action
q	Quit
z	Go to ground posture (folding → ground, ~4.8 s)
x	Go to ready posture (ground → folding → ready, ~2.8–5.6 s)
c	Start Walk / Control mode (loads policy from -p)

Keyboard is read in non-canonical, no-echo mode — no Enter required.

Policy directory

The -p directory must contain:

File	Purpose
info.json	Policy hyper-parameters (run_name, policy_dt, action_scale, stiffness/damping, obs/action sizes, normalization scales, default joint angles, command ranges)
policy.onnx	ONNX model exported from the training framework

Supported policy names (from info.json run_name):

Name	Observation vector	Notes
rbq10	gyro, proj_grav, command, dof_pos, dof_vel, action	Single-frame
rbq10_trot	gyro, proj_grav, command, 4×dof_pos history (0,2,4,6), 4×dof_vel history, action×2, added_mass	7-frame history
rbq10_trot_run	Same as rbq10_trot	Faster run variant

DDS topics

Direction	Topic	Type	Description
Subscribe	rt/rbq/cmd/joy/final	Joy_	Joystick axes for velocity commands
Subscribe	rt/rbq/info/imu	Imu_	Angular velocity + orientation quaternion
Subscribe	rt/rbq/info/leg_joint	LegJointInfo_	Joint position and velocity (12 joints)
Subscribe	rt/rbq/ref/leg_joint/final	MotionRef_	Final blended reference (used in Idle)
Publish	rt/rbq/ref/leg_joint/owner_20	MotionRef_	Joint position references with PD gains
Publish	rt/rbq/cmd/motion/joint_owner/_20	JointOwnershipCmd_	Claim ownership of all 12 joints

Joint ordering

Joints indexed 0–11:

HR: HRR(0), HRP(1), HRK(2) → HL: HLR(3), HLP(4), HLK(5) → FR: FRR(6), FRP(7), FRK(8) → FL: FLR(9), FLP(10), FLK(11)

Key implementation notes

• No RT thread: Python's time.sleep() is used for 2 ms pacing. Timing jitter is higher than the C++ RT thread — acceptable for simulation and testing, but less suitable for highly dynamic real-robot use.
• Observation pre-initialization: msg_imu, msg_leg, and msg_joy are pre-initialized with zero defaults in SharedState. The policy runs immediately on launch even before the first DDS sample arrives; a throttled warning is printed if any subscriber is still cold.
• Decimation: Policy inference runs every 5th loop tick (10 Hz from the 500 Hz loop), matching policy_dt in info.json.
• Ownership: JointOwnershipCmd_ is published once on entering Motion or Control state, claiming all 12 joints at priority 20.
• Posture detection: go_to_motion_ready() checks the four hip-pitch angles against 60° to determine if the robot is grounded, inserting a folding step if needed.

Troubleshooting

Problem	Likely cause
No motion / topics silent	Stack not running or wrong network interface
Policy, PolicyParams not loaded	-p path wrong, or info.json / policy.onnx missing
NaN or Inf in output data	Policy received invalid observation; check IMU and joint topics
Timing warnings in log	System too loaded for 2 ms; close other processes

See also

• Low Level (RL Policy) (C++)
• High Level Command (Python)
• Python SDK Overview
• Isaac Gym
• Isaac Lab