10.71.0.0/16) as an identity boundary. Two independent mechanisms hang off that trust, and this page covers both:
Inbound: check_vpn_ip
A route decorator that authenticates drone-originated callbacks by source IP. No JWT, no per-device token — if the request arrives from a
10.71.* address (or a whitelisted X-Drone-IP), it is trusted as that drone.Outbound: jumphost routing
The rosbridge
Connection multiplexes many drones through a single nginx jumphost using x-drone-ip / x-drone-port headers, selected by JUMPHOST_IP.Inbound: the check_vpn_ip middleware
check_vpn_ip is a plain decorator (not Flask-JWT) defined in src/middleware/drone_vpn.py. It resolves the caller’s drone IP into request.vpn_ip and lets the handler through, or rejects with 400. The wrapped handler then identifies the drone with generic_drone_service.get_drone_by_ip(request.vpn_ip) (src/service/drone_service.py:116 — a simple Drone.ip == vpn_ip lookup).
IP resolution order
The decorator checks four sources in order and takes the first that matches (src/middleware/drone_vpn.py:32):
| # | Source | Accepted when | Notes |
|---|---|---|---|
| 1 | request.remote_addr | starts with 10.71. | The real socket peer |
| 2 | X-Real-IP header | starts with 10.71. | Set by the jumphost nginx |
| 3 | X-Forwarded-For header | starts with 10.71. | startswith matches the leftmost (client) entry |
| 4 | X-Drone-IP header | SITL container name and ENABLE_SITL, or a safe local prefix and IS_LOCAL_ENVIRONMENT | Non-VPN fallback for SITL/dev |
The X-Drone-IP whitelist
When no 10.71.* source is present, the header X-Drone-IP is accepted only for the prefixes in _ALLOWED_LOCAL_IP_PREFIXES (src/middleware/drone_vpn.py:12):
src/middleware/drone_vpn.py
SKYHUB_SITL_*container names are trusted wheneverENABLE_SITLis true — including on non-local / production deployments (drone_vpn.py:56). This is how remote SITL drones, which have no10.71.*VPN address, authenticate their callbacks. For SITL the “IP” column of thedronesrow literally stores the container name, soget_drone_by_ipstill resolves.- All other local prefixes (
127.,172.1x.,192.168.,10.223.) are trusted only whenIS_LOCAL_ENVIRONMENT(i.e.DEPLOYMENT_ENVIRONMENT=local).
Routes protected by check_vpn_ip
These are the only endpoints with no JWT — the VPN source IP is their entire authentication. All are drone/gamepad → Gateway callbacks:
| Route | Method | Source | Purpose |
|---|---|---|---|
/api/v1/drone/pull | GET | drone_routes.py:1638 | Drone self-update: returns temporary STS assume-role AWS creds (1 h) to pull core images from ECR |
/api/v1/authenticate_upload | POST | asset_routes.py:54 | Presigned S3 upload URL + create the Asset row |
/api/v1/complete_upload | POST | asset_routes.py:172 | Mark an upload complete; may dispatch a deferred execution report |
/api/v1/executions/start | POST | execution_routes.py:131 | Begin execution tracking on ARM |
/api/v1/executions/{id}/complete | POST | execution_routes.py:190 | Finish execution on land/disarm |
/api/v1/executions/{id}/log | POST | execution_routes.py:272 | Attach an uploaded .bin log asset (triggers async analysis) |
/api/v1/executions/current | GET | execution_routes.py:338 | The in-progress execution for the calling drone |
GET /api/v1/drone/activate is not in this list. Activation uses a separate 10-digit token header (bootstrap, before the drone is on the VPN); only the later self-update /drone/pull uses check_vpn_ip. See Drone Management & Control Actions and Executions, Assets & Reports for the full request/response shapes.The spoofing caveat
Preserve these invariants when refactoring:- The Gateway must sit behind the jumphost / inside the VPC; in production the API is reached via nginx on the single WireGuard EC2 (TLS-terminated, port 5000 -> gateway API), never over a public ALB (no ALB/ELB exists in prod).
- Only the jumphost nginx should be allowed to set
X-Real-IP/X-Forwarded-For; do not let clients inject them. - The drone network isolation itself (per-user iptables from
user_drone_access) is enforced by the User VPN service, not by this middleware.
Minor footgun to leave in place: the middleware package file is misnamed
___init__.py (three leading underscores) in src/middleware/, so it is not a real package __init__. Imports work only because drone_vpn is imported by explicit module path (from src.middleware.drone_vpn import check_vpn_ip).Outbound: jumphost routing for rosbridge
Command dispatch and telemetry travel the other direction — Gateway → drone rosbridge (WebSocket, port 9090). A single nginx jumphost fronts every drone’s rosbridge and demultiplexes based on two request headers the Gateway injects. This lets one public/edge host serve an entire fleet without a per-drone port map. The connection object is built per drone insrc/rosbridge/connection.py; see Rosbridge Connection & Reconnect for the socket internals and Connection Pool & Startup Wiring for pooling.
Direct vs jumphost selection
The mode is chosen by one line — an emptyJUMPHOST_IP means direct (src/rosbridge/connection.py:27):
src/rosbridge/connection.py
connection.py:159):
src/rosbridge/connection.py
The HTTP reachability probe
Before opening the WebSocket,start_connection() runs _test_connection() (connection.py:202): an HTTP GET against the same target with a DRONE_REACHABILITY_TIMEOUT-second connect/read timeout. Rosbridge answers non-WebSocket requests with HTTP 400, which the probe treats as “alive and reachable.” Any other status re-raises.
SITL IP translation
Thedrone_ip passed into Connection is not always drone.ip. DroneControlService._get_client rewrites it for SITL drones (src/service/drone_control_service.py:551), because their ip column holds a container name:
| Drone type | drone_ip used | Condition |
|---|---|---|
| Physical | drone.ip (its 10.71.* VPN address) | always |
| SITL, remote | settings.SITL_HOST (= DOCKER_HOST_IP, e.g. <office-docker-host>) | REMOTE_DOCKER_ENABLED=true |
| SITL, local | host.docker.internal | REMOTE_DOCKER_ENABLED=false |
<office-docker-host>) — but from the jumphost/nginx perspective both are just an x-drone-ip:x-drone-port pair. That interchangeability is what lets SITL and physical drones share the exact same rosbridge/topic contract. See SITL Drone Lifecycle.
Configuration
Only the VPN/jumphost-relevant vars are shown here; the full catalogue lives in Gateway Environment Variables.| Variable | Default | Production value | Purpose |
|---|---|---|---|
JUMPHOST_IP | "" (empty → direct) | jumphost-private.skyhub-prod.internal | Host nginx fronts rosbridge; empty disables jumphost routing |
JUMPHOST_PORT | 9090 | 9090 | Jumphost port for rosbridge |
DRONE_REACHABILITY_TIMEOUT | 2 | 2 | Connect/read timeout (s) for the pre-socket HTTP probe |
DRONE_NETWORK_CIDR | 10.71.0.0/16 | 10.71.0.0/16 | WireGuard drone plane used by IPService (not read by the middleware — see caveat above) |
DOCKER_HOST_IP / SITL_HOST | <office-docker-host> | <office-docker-host> | On-prem office server IP that remote SITL rosbridge connects to |
VPN_SERVICE_IP | None | jumphost-private.skyhub-prod.internal | External WireGuard status service (VPN_Service status probes) |
VPN_SERVICE_PORT | 5050 | 5050 | Port of the VPN status service |
In production,
jumphost-private.skyhub-prod.internal resolves to <jumphost-private-ip> inside the AWS VPC. The public side of the same box (the single t4g.nano WireGuard EC2 instance, public <prod-ingress-ip>) is also the WireGuard server and TLS ingress — making it the platform’s architectural keystone and primary single point of failure. Network layout is documented in Network & VPN Topology and VPC, WireGuard Jumphost & nginx Routing.Setting up a jumphost deployment
Point rosbridge through the jumphost
Set
JUMPHOST_IP to the private jumphost DNS/IP and JUMPHOST_PORT to the port nginx listens on for rosbridge (9090). Leaving JUMPHOST_IP empty makes every Connection dial the drone IP directly — correct only when the Gateway shares the drone’s L3 network.Configure nginx to route by header
The jumphost nginx must read
x-drone-ip and x-drone-port from the upgrade request and proxy_pass the WebSocket to http://$http_x_drone_ip:$http_x_drone_port. These are the exact header names emitted by Connection (lower-case).Wire the drone-callback path back
Ensure drone callbacks arrive with a
10.71.* source, or the jumphost sets X-Real-IP to the drone’s WireGuard address. Confirm clients cannot forge X-Real-IP / X-Forwarded-For upstream of nginx.Debugging 400 IP not authenticated
A physical drone's callback is rejected
A physical drone's callback is rejected
The request did not arrive with a
10.71.* source. Check that the drone is actually on the WireGuard plane, and that the jumphost preserves the drone IP in remote_addr / X-Real-IP / X-Forwarded-For. If nginx terminates the connection, its X-Real-IP must be the drone’s 10.71.* address, not the jumphost’s own IP. The middleware logs which source it picked (Picked X-Real-IP: ...) or No valid VPN IP detected.A SITL callback is rejected
A SITL callback is rejected
SITL uses
X-Drone-IP: SKYHUB_SITL_<n>, which is accepted only when ENABLE_SITL=true. Verify the header value starts exactly with SKYHUB_SITL_ and matches the ip stored on the drones row (that is what get_drone_by_ip looks up).A local-dev tool is rejected
A local-dev tool is rejected
Non-SITL
X-Drone-IP prefixes (127., 172.17-20., 192.168., 10.223.) are trusted only when DEPLOYMENT_ENVIRONMENT=local. On a server deployment these are refused by design — do not “fix” this by widening the whitelist in production.The route is reached but the drone is None
The route is reached but the drone is None
check_vpn_ip passed (an IP was resolved) but get_drone_by_ip(request.vpn_ip) returned no row — the resolved IP/container name does not match any drones.ip. This surfaces as a downstream 404/500, not a 400.Related pages
Rosbridge Connection & Reconnect
The
Connection / SmartSocket internals, keepalive and bounded reconnect.HTTP API Overview & Auth Models
Where VPN-IP trust sits among JWT and token/signature auth.
User VPN & Network Isolation
The WireGuard planes and per-user iptables that make source-IP trust safe.
VPC, WireGuard Jumphost & nginx
How the single jumphost is provisioned and routes traffic in AWS.

