Matthew BatacanA reinforcement-learning agent trained to hold a heading in cross-wind and cross-current while steering an outrigger canoe. Race it, or watch it against a hand-coded baseline. The physics and training are in ama-flow.
blue is the selected trained model ·red is the hand-coded baseline steersman.
blue is you ·red is the selected trained model.
Race over
| canoe | heading error | distance | avg speed |
|---|---|---|---|
- ← paddle port
- → paddle starboard
- 1 forward · 2 draw · 3 poke · 4 rudder
Your selection fires automatically every stroke, so there's no paddle key. It's highlighted at the bottom of the view.
The green compass arrow points where to steer; wind and current show the same way. The small map in the corner shows the whole course.
Seed 1, fully trained (750k)
Best of three seeds: beats the baseline on all 9 wind x current cells.
run 8aadacc9750k of 750k stepsseed 1registry v3
ent_coef 0.01lr 0.0003gamma 0.998 envswind arm 0.5 m
Settings
training
- batch_size
- 256
- checkpoint_freq
- 50000
- ent_coef
- 0.01
- eval_freq
- 20000
- gamma
- 0.99
- learning_rate
- 0.0003
- n_envs
- 8
- n_eval_episodes
- 10
- n_steps
- 512
- policy
- ActorCriticPolicy
- run_name
- ppo-canoe-wind-recal-seed1
- seed
- 1
- total_timesteps
- 750000
hull
- added_inertia_yaw_kg_m2
- 4713.45
- added_mass_surge_kg
- 30.0
- added_mass_sway_kg
- 400.0
- air_density_kg_m3
- 1.225
- damping_surge_linear
- 10.0
- damping_surge_quadratic
- 21.0
- damping_sway_linear
- 300.0
- damping_sway_quadratic
- 2500.0
- damping_yaw_linear
- 2000.0
- damping_yaw_quadratic
- 28000.0
- mass_kg
- 630.0
- wind_cp_offset_m
- 0.5
- wind_drag_coeff
- 1.1
- windage_area_lateral_m2
- 8.0
- yaw_inertia_kg_m2
- 9426.9
ama & crew
- ama_sway_drag_extra_quadratic
- 500.0
- ama_yaw_bias_coeff
- 10.0
- crew_paddle_offset_m
- 0.6
- crew_seat_force_ranges_n
- ((30.0, 50.0), (32.0, 48.0), (34.0, 46.0), (32.0, 48.0), (30.0, 50.0))
- crew_seat_sides
- (PORT, STARBOARD, PORT, STARBOARD, PORT)
- crew_switch_period_strokes
- 12
Results vs baseline · won 9/9 cells
Cross-wind and cross-current, 10 episodes per cell. Lower heading error and higher speed are better; the baseline is the hand-coded steersman.
| wind | current | heading error (model / baseline) | speed m/s (model / baseline) | model wins |
|---|---|---|---|---|
| 10 kt | 0.5 kt | 2.1 / 4.1 deg | 3.05 / 3.03 | yes |
| 10 kt | 1 kt | 2.0 / 4.1 deg | 3.05 / 3.03 | yes |
| 10 kt | 1.5 kt | 2.0 / 4.0 deg | 3.05 / 3.04 | yes |
| 15 kt | 0.5 kt | 4.4 / 4.6 deg | 3.00 / 2.96 | yes |
| 15 kt | 1 kt | 3.8 / 4.6 deg | 3.00 / 2.96 | yes |
| 15 kt | 1.5 kt | 3.3 / 4.5 deg | 3.00 / 2.97 | yes |
| 20 kt | 0.5 kt | 6.0 / 8.6 deg | 2.91 / 2.82 | yes |
| 20 kt | 1 kt | 5.9 / 7.1 deg | 2.92 / 2.85 | yes |
| 20 kt | 1.5 kt | 5.8 / 6.3 deg | 2.93 / 2.86 | yes |
Training curves
Logged every 20k steps during training; the line stops where this checkpoint was saved.
Seed 2, fully trained (750k)
Same recipe, different seed: wins the light-wind cells but loses RMS heading error at 15-20 kt.
run 2d87ed75750k of 750k stepsseed 2registry v4
ent_coef 0.01lr 0.0003gamma 0.998 envswind arm 0.5 m
Settings
training
- batch_size
- 256
- checkpoint_freq
- 50000
- ent_coef
- 0.01
- eval_freq
- 20000
- gamma
- 0.99
- learning_rate
- 0.0003
- n_envs
- 8
- n_eval_episodes
- 10
- n_steps
- 512
- policy
- ActorCriticPolicy
- run_name
- ppo-canoe-wind-recal-seed2
- seed
- 2
- total_timesteps
- 750000
hull
- added_inertia_yaw_kg_m2
- 4713.45
- added_mass_surge_kg
- 30.0
- added_mass_sway_kg
- 400.0
- air_density_kg_m3
- 1.225
- damping_surge_linear
- 10.0
- damping_surge_quadratic
- 21.0
- damping_sway_linear
- 300.0
- damping_sway_quadratic
- 2500.0
- damping_yaw_linear
- 2000.0
- damping_yaw_quadratic
- 28000.0
- mass_kg
- 630.0
- wind_cp_offset_m
- 0.5
- wind_drag_coeff
- 1.1
- windage_area_lateral_m2
- 8.0
- yaw_inertia_kg_m2
- 9426.9
ama & crew
- ama_sway_drag_extra_quadratic
- 500.0
- ama_yaw_bias_coeff
- 10.0
- crew_paddle_offset_m
- 0.6
- crew_seat_force_ranges_n
- ((30.0, 50.0), (32.0, 48.0), (34.0, 46.0), (32.0, 48.0), (30.0, 50.0))
- crew_seat_sides
- (PORT, STARBOARD, PORT, STARBOARD, PORT)
- crew_switch_period_strokes
- 12
Results vs baseline · won 3/9 cells
Cross-wind and cross-current, 10 episodes per cell. Lower heading error and higher speed are better; the baseline is the hand-coded steersman.
| wind | current | heading error (model / baseline) | speed m/s (model / baseline) | model wins |
|---|---|---|---|---|
| 10 kt | 0.5 kt | 3.0 / 4.1 deg | 3.05 / 3.03 | yes |
| 10 kt | 1 kt | 2.4 / 4.1 deg | 3.05 / 3.03 | yes |
| 10 kt | 1.5 kt | 2.4 / 4.0 deg | 3.05 / 3.04 | yes |
| 15 kt | 0.5 kt | 7.5 / 4.6 deg | 3.00 / 2.96 | no |
| 15 kt | 1 kt | 6.3 / 4.6 deg | 3.00 / 2.96 | no |
| 15 kt | 1.5 kt | 5.0 / 4.5 deg | 3.01 / 2.97 | no |
| 20 kt | 0.5 kt | 10.1 / 8.6 deg | 2.91 / 2.82 | no |
| 20 kt | 1 kt | 10.1 / 7.1 deg | 2.92 / 2.85 | no |
| 20 kt | 1.5 kt | 8.9 / 6.3 deg | 2.93 / 2.86 | no |
Training curves
Logged every 20k steps during training; the line stops where this checkpoint was saved.
Seed 0, fully trained (750k)
Wins 7 of 9 cells; narrow losses at 15 kt / 0.5 kt and 20 kt / 1.5 kt.
run 8c3ea0be750k of 750k stepsseed 0registry v2
ent_coef 0.01lr 0.0003gamma 0.998 envswind arm 0.5 m
Settings
training
- batch_size
- 256
- checkpoint_freq
- 50000
- ent_coef
- 0.01
- eval_freq
- 20000
- gamma
- 0.99
- learning_rate
- 0.0003
- n_envs
- 8
- n_eval_episodes
- 10
- n_steps
- 512
- policy
- ActorCriticPolicy
- run_name
- ppo-canoe-wind-recal-seed0
- seed
- 0
- total_timesteps
- 750000
hull
- added_inertia_yaw_kg_m2
- 4713.45
- added_mass_surge_kg
- 30.0
- added_mass_sway_kg
- 400.0
- air_density_kg_m3
- 1.225
- damping_surge_linear
- 10.0
- damping_surge_quadratic
- 21.0
- damping_sway_linear
- 300.0
- damping_sway_quadratic
- 2500.0
- damping_yaw_linear
- 2000.0
- damping_yaw_quadratic
- 28000.0
- mass_kg
- 630.0
- wind_cp_offset_m
- 0.5
- wind_drag_coeff
- 1.1
- windage_area_lateral_m2
- 8.0
- yaw_inertia_kg_m2
- 9426.9
ama & crew
- ama_sway_drag_extra_quadratic
- 500.0
- ama_yaw_bias_coeff
- 10.0
- crew_paddle_offset_m
- 0.6
- crew_seat_force_ranges_n
- ((30.0, 50.0), (32.0, 48.0), (34.0, 46.0), (32.0, 48.0), (30.0, 50.0))
- crew_seat_sides
- (PORT, STARBOARD, PORT, STARBOARD, PORT)
- crew_switch_period_strokes
- 12
Results vs baseline · won 7/9 cells
Cross-wind and cross-current, 10 episodes per cell. Lower heading error and higher speed are better; the baseline is the hand-coded steersman.
| wind | current | heading error (model / baseline) | speed m/s (model / baseline) | model wins |
|---|---|---|---|---|
| 10 kt | 0.5 kt | 2.8 / 4.1 deg | 3.05 / 3.03 | yes |
| 10 kt | 1 kt | 2.4 / 4.1 deg | 3.05 / 3.03 | yes |
| 10 kt | 1.5 kt | 2.3 / 4.0 deg | 3.05 / 3.04 | yes |
| 15 kt | 0.5 kt | 5.4 / 4.6 deg | 3.00 / 2.96 | no |
| 15 kt | 1 kt | 4.5 / 4.6 deg | 3.00 / 2.96 | yes |
| 15 kt | 1.5 kt | 3.9 / 4.5 deg | 3.01 / 2.97 | yes |
| 20 kt | 0.5 kt | 6.9 / 8.6 deg | 2.91 / 2.82 | yes |
| 20 kt | 1 kt | 7.1 / 7.1 deg | 2.92 / 2.85 | yes |
| 20 kt | 1.5 kt | 6.8 / 6.3 deg | 2.93 / 2.86 | no |
Training curves
Logged every 20k steps during training; the line stops where this checkpoint was saved.
Seed 1, early (150k)
The seed 1 run a fifth of the way through training, before it learns to hold a heading.
run 8aadacc9150k of 750k stepsseed 1
ent_coef 0.01lr 0.0003gamma 0.998 envswind arm 0.5 m
Settings
training
- batch_size
- 256
- checkpoint_freq
- 50000
- ent_coef
- 0.01
- eval_freq
- 20000
- gamma
- 0.99
- learning_rate
- 0.0003
- n_envs
- 8
- n_eval_episodes
- 10
- n_steps
- 512
- policy
- ActorCriticPolicy
- run_name
- ppo-canoe-wind-recal-seed1
- seed
- 1
- total_timesteps
- 750000
hull
- added_inertia_yaw_kg_m2
- 4713.45
- added_mass_surge_kg
- 30.0
- added_mass_sway_kg
- 400.0
- air_density_kg_m3
- 1.225
- damping_surge_linear
- 10.0
- damping_surge_quadratic
- 21.0
- damping_sway_linear
- 300.0
- damping_sway_quadratic
- 2500.0
- damping_yaw_linear
- 2000.0
- damping_yaw_quadratic
- 28000.0
- mass_kg
- 630.0
- wind_cp_offset_m
- 0.5
- wind_drag_coeff
- 1.1
- windage_area_lateral_m2
- 8.0
- yaw_inertia_kg_m2
- 9426.9
ama & crew
- ama_sway_drag_extra_quadratic
- 500.0
- ama_yaw_bias_coeff
- 10.0
- crew_paddle_offset_m
- 0.6
- crew_seat_force_ranges_n
- ((30.0, 50.0), (32.0, 48.0), (34.0, 46.0), (32.0, 48.0), (30.0, 50.0))
- crew_seat_sides
- (PORT, STARBOARD, PORT, STARBOARD, PORT)
- crew_switch_period_strokes
- 12
Results vs baseline · won 0/9 cells
Cross-wind and cross-current, 10 episodes per cell. Lower heading error and higher speed are better; the baseline is the hand-coded steersman.
| wind | current | heading error (model / baseline) | speed m/s (model / baseline) | model wins |
|---|---|---|---|---|
| 10 kt | 0.5 kt | 18.6 / 4.1 deg | 2.77 / 3.03 | no |
| 10 kt | 1 kt | 16.8 / 4.1 deg | 2.79 / 3.03 | no |
| 10 kt | 1.5 kt | 14.5 / 4.0 deg | 2.81 / 3.04 | no |
| 15 kt | 0.5 kt | 37.9 / 4.6 deg | 2.20 / 2.96 | no |
| 15 kt | 1 kt | 37.4 / 4.6 deg | 2.22 / 2.96 | no |
| 15 kt | 1.5 kt | 36.3 / 4.5 deg | 2.24 / 2.97 | no |
| 20 kt | 0.5 kt | 11.1 / 8.6 deg | 2.68 / 2.82 | no |
| 20 kt | 1 kt | 47.3 / 7.1 deg | 1.93 / 2.85 | no |
| 20 kt | 1.5 kt | 51.5 / 6.3 deg | 1.79 / 2.86 | no |
Training curves
Logged every 20k steps during training; the line stops where this checkpoint was saved.