Built on Burn

Reinforcement
learning,
built from parts
that mesh.

ModuRL is a deep reinforcement learning framework for Rust, with training algorithms, environments, models, policy distributions, storage, schedules, and logging.

Get started

Backends

  • CUDA
  • Metal
  • ndarray

Environments

  • Classic control
  • Box2D
  • MuJoCo
  • Atari

Build an agent

Select an algorithm, and for PPO or A2C a network topology, to see an assembly example.

Algorithm
Network
your_agent.rs RUST
// CartPole · separate models · PPO
use burn::tensor::Device;
use modurl::prelude::*;
use modurl_gym::classic_control::cartpole::CartPoleV1;

let device = Device::flex().autodiff();
let env = CartPoleV1::builder().seed(7).build()?;
let env = VectorizedGymWrapper::from(env);
let mut env = DeviceMultiGymWrapper::new(env, Device::flex(), device.clone());
let observation_space = env.observation_space();
let action_space = env.action_space();
// actor_model, critic and optimizers: setup omitted; see the guide.
let policy = ProbabilisticPolicyModel::new(
    actor_model,
    CategoricalDistribution,
);

let networks = SeparatePPONetwork::builder()
    .actor_network(policy)
    .critic_network(critic)
    .actor_optimizer(actor_optimizer)
    .critic_optimizer(critic_optimizer)
    .actor_learning_rate(3e-4)
    .critic_learning_rate(3e-4)
    .build()?;

let mut agent = PPOAgent::builder()
    .observation_space(observation_space)
    .action_space(action_space)
    .network_info(networks)
    .batch_size(2_048)
    .mini_batch_size(64)
    .clip_range(ConstantSchedule::new(0.2))
    .training_horizon(100_000)
    .build()?;

agent.learn(&mut env, 100_000)?;

Algorithms

Implementations for on-policy and off-policy training.

Benchmarks

End-to-end Breakout PPO throughput.
40,960 transitions with matched preprocessing and training workloads.

Benchmark methodology
BREAKOUT / PPO TRANSITIONS / SEC ↑
ModuRL 6,000 /s
CleanRL 4,178 /s
Stable-Baselines3 3,737 /s

CUDA · Throughput only, not final reward or sample efficiency. Emulator versions differ. See the benches README for methodology and fairness controls.

Quick start

Train PPO on CartPole.

git clone https://github.com/ModuRL/ModuRL.git
cd ModuRL
cargo run --release -p examples --example ppo_cartpole

Requires a Rust toolchain with edition 2024 support. Read the getting started guide

ModuRL gear mascots

Start training in Rust.