From zero hardware experience to a fault-tolerant RL octocopter in 2.5 weeks

A first-time hardware builder documents going from an idea to a flying eight-rotor drone in under three weeks, designing the frame in Fusion 360, CNC-milling it from G10 fiberglass and carbon fiber, and hand-assembling the motors, propellers, and electronics. The initial aircraft flies as a conventional Betaflight-controlled octocopter, which is inherently tolerant of a single motor loss thanks to large thrust overcapacity and a fast PID loop — but its static mixer collapses the moment a second motor dies, which is precisely the gap the project aims to close with reinforcement learning.

The engineering interest is in the RL work. A lightweight 43.4k-parameter MLP policy, trained in simulation, learns to keep the drone aloft through single, dual, and even some untrained triple-motor failures, and it debunks the author’s assumption that certain ‘uncompensatable-yaw’ failure geometries are unrecoverable. Getting there hinged on diagnosing two subtle bugs rather than tuning hyperparameters: motor commands saturating at the action clip boundary with no corrective gradient (fixed by squashing outputs through a tanh residual around hover throttle), and a reward structure where surviving paid nothing because a survival bonus exactly cancelled an altitude penalty (fixed by raising the survival reward).

The remaining challenge is sim-to-real. The builder collected system-identification data using a physical pendulum-swing method to estimate the drone’s moments of inertia, and made a key architecture decision: rather than bolting on a separate companion computer (a Raspberry Pi or Teensy) to issue eight per-motor commands over MSP — which fights Betaflight’s disarm safety model — the policy will be compiled directly into the Betaflight firmware on the existing STM32H743 flight controller, cutting loop latency. The approach draws on the ‘Learning to Fly in Seconds’ and Neuroflight papers, with domain randomization intended to bridge remaining model inaccuracies.