Robotics
Drone Hunter
A Webots simulation where a Mavic 2 Pro hunts a wandering ground rover. A reinforcement learning policy decides where to steer, classic control keeps the aircraft flying, and the rover is found by colour in the camera image.
Source codeOne sign flip between training and deployment took time on target from almost nothing to 87%.
01 / 06The drone closes in on the rover it spotted with its camera
Overview
A drone that follows a moving target has two jobs: stay in the air, and decide where to go. This project keeps the first job classical, because attitude and altitude hold are a solved problem, and hands the second to a reinforcement learning policy trained with PPO. About 15 times a second, the policy outputs just two numbers: how hard to turn and how hard to push forward.
Training uses a shortcut. The trainer reads the rover's position straight from the simulator, so it never renders a camera frame, which is the slowest part of the simulation. When the drone flies for real, the same inputs are rebuilt from pixels: a magenta blob gives the bearing, and the blob's height in the image plus the altimeter gives the range.
After 120,000 training steps the drone keeps the rover in its sights about 87% of the time and centres it smoothly. Keeping the right distance is still weak: range drifts out past the 9 m target until the rover slips out of view and the search sweep has to find it again.
Features
- Watch a replay of the chase in 3D in the browser, orbit around it and scrub through the timeline
- Train the pursuit policy with PPO inside Webots in about 800 episodes
- Fly the trained policy from the camera alone, with coast and search modes when the rover is lost
- Compare against the earlier hand-tuned controller in a sibling world on the same task
- Reuse the Gymnasium environment, reward and observation for other guidance experiments
Challenges
- Webots returns garbage sensor readings until the simulation has stepped once. Feeding that into the motor mix made the physics explode, with altitudes of 9,320 m and speeds of 1,464 m/s, so the controllers now step before reading anything.
- A policy that reverses the stick every 64 ms drove the pitch to 66 degrees, where lift collapses. Smoothing the command with a 100 ms filter and quadrupling the rate damping kept the aircraft flyable under any command sequence.
- Two well meant changes to the observations, clamping the bearing at the frame edge and holding the last range, made learning flatline near 7% time on target. Honest geometry plus a separate visibility flag reached 54% in training.
- The trained policy turned the wrong way when flying from the camera, because image columns grow to the right while the trainer measured angles with left as positive. Every logged bearing came out negative, which gave it away, and one minus sign fixed it.