Drone navigation with deep RL
A quadrotor learns to fly through dense, GPS-denied forest from depth images alone.
BS thesis, team of three, grade A · COMSATS University Islamabad · 2025 – 2026
- SAC
- policy, vs a TD3 baseline
- UE5
- AirSim simulation
How it is built, step by step
- Step 1
The world
A dense forest in AirSim on Unreal Engine 5; the drone sees only noisy depth images.
- Step 2
The policy
A maximum-entropy Soft Actor-Critic acts on those images.
- Step 3
A critic that knows more
The asymmetric privileged critic sees clean depth and the true state while training, so the policy learns faster than it could alone.
- Step 4
Training at a distance
Experience streams over gRPC to a remote GPU trainer; a ground-control app shows telemetry and versions the models.
- Step 5
Measured
Benchmarked against a TD3 baseline. Grade A.
In detail
A maximum-entropy Soft Actor-Critic policy on noisy depth images, trained against a privileged critic that sees the clean state, and benchmarked against TD3.
The thesis trains vision-based quadrotor navigation through dense forest in AirSim on Unreal Engine 5. The policy is a maximum-entropy Soft Actor-Critic acting on noisy depth images; it is trained against an asymmetric privileged critic that sees clean depth and state, and benchmarked against TD3 in PyTorch. Everything is in simulation.
My contributions: reward-shaping design (potential-based progress, obstacle proximity, heading and altitude terms), checkpointing and evaluation tooling, a gRPC pipeline streaming experience to a remote GPU trainer, and the ground-control desktop app (Electron, React/TypeScript, FastAPI) for live AirSim telemetry, mission control and model versioning.