Skip to content

Drone navigation with deep RL

A quadrotor learns to fly through dense, GPS-denied forest from depth images alone.

BS thesis, team of three, grade A · COMSATS University Islamabad · 2025 – 2026

SAC
policy, vs a TD3 baseline
UE5
AirSim simulation

How it is built, step by step

  1. Step 1

    The world

    A dense forest in AirSim on Unreal Engine 5; the drone sees only noisy depth images.

  2. Step 2

    The policy

    A maximum-entropy Soft Actor-Critic acts on those images.

  3. Step 3

    A critic that knows more

    The asymmetric privileged critic sees clean depth and the true state while training, so the policy learns faster than it could alone.

  4. Step 4

    Training at a distance

    Experience streams over gRPC to a remote GPU trainer; a ground-control app shows telemetry and versions the models.

  5. Step 5

    Measured

    Benchmarked against a TD3 baseline. Grade A.

Drone navigation with deep RL: system sketchactionstrainstelemetryAirSim · UE5dense forestDepth imagesnoisySAC policymax-entropyPrivileged criticclean depth + stateReward shapingprogress · obstacles · headingRemote GPU trainergRPC experience streamGround-control appElectron · FastAPITD3 baseline

In detail

A maximum-entropy Soft Actor-Critic policy on noisy depth images, trained against a privileged critic that sees the clean state, and benchmarked against TD3.

The thesis trains vision-based quadrotor navigation through dense forest in AirSim on Unreal Engine 5. The policy is a maximum-entropy Soft Actor-Critic acting on noisy depth images; it is trained against an asymmetric privileged critic that sees clean depth and state, and benchmarked against TD3 in PyTorch. Everything is in simulation.

My contributions: reward-shaping design (potential-based progress, obstacle proximity, heading and altitude terms), checkpointing and evaluation tooling, a gRPC pipeline streaming experience to a remote GPU trainer, and the ground-control desktop app (Electron, React/TypeScript, FastAPI) for live AirSim telemetry, mission control and model versioning.

Next: claude-nonstop

Write to me about a role

umer.jahangier@gmail.com

Subject: Role: let's talk

A line about the team and the role is enough; I reply within a day.

Open your mail app