The User VPN service (skyhub_user_vpn) is the platform’s network-security plane. It is a small Python/Flask daemon that runs WireGuard + iptables on a single host and continuously reconciles the live WireGuard peer set against the shared PostgreSQL database. Its job is threefold:
  1. Give every Dashboard operator and every physical drone its own WireGuard tunnel with a stable private IP.
  2. Enforce per-user isolation — an operator’s traffic can only reach the drones they own.
  3. Publish generated client configs to S3 so the Gateway can hand them out on demand.
This is what turns a physical drone’s 10.71.x address into something the Gateway can actually route a rosbridge connection to. SITL drones do not use this service — they are Docker containers reached over DOCKER_HOST_IP (see SITL Simulator).
The service runs on the single WireGuard EC2 instance (skyhub-prod-skyhub-wireguard-server, t4g.nano, private <jumphost-private-ip>, public <prod-ingress-ip>) — the same box that is the platform’s public TLS ingress and jumphost. It cannot run on Fargate because it needs network_mode: host and NET_ADMIN/NET_RAW to manage kernel WireGuard interfaces (skyhub_user_vpn/docker-compose.yml). The prod image is skyhub-prod-vpn-image. See VPC, WireGuard Jumphost & nginx Routing.

​
Three WireGuard planes on one host

The service models three independent WireGuard networks (“planes”), each with its own interface, UDP listen port, and /16 range. Defaults come from libs/config/settings.py and are set explicitly in docker-compose.yml.
PlaneInterfaceUDP portRangeWho connectsManager class
corewg05182210.69.0.0/16On-prem hosts: Docker host (<office-docker-host>), SITL host (<office-host>), SigNozCoreVPN
userusers05182310.70.0.0/16Dashboard operator laptopsUserVPN
dronedrones05182410.71.0.0/16Physical drones (rosbridge on 10.71.x:9090+)DroneVPN
Each plane is a subclass of EntityVpnManager (libs/service/wireguard.py) that overrides only its data_dir, config_dir, interface, port, range, and the SQL that lists its entities (libs/service/core_vpn.py, customer_vpn.py, drone_vpn.py).
The core plane is not actually managed by this service yet. In libs/wrapper.py, CoreVPN is commented out — only UserVPN, DroneVPN, and the IptablesManager start. The wg0 interface is still managed out-of-band (there is a TODO: change after all VPNs are managed by this service in settings.py:29). So at runtime this service owns exactly two tunnels (users0, drones0) plus the firewall.
The drone plane is created with use_real_interface=True and MTU = 1350 (cellular-friendly); the user plane uses the default MTU = 1500. Every generated client config routes all four ranges through the tunnel — AllowedIPs = 10.69.0.0/16, 10.70.0.0/16, 10.71.0.0/16, 172.31.0.0/16 (bridged_ranges in settings.py:44) — which is why a drone on drones0 can reach the Gateway and DB in the AWS VPC (172.31.0.0/16), and a user on users0 can reach drones on 10.71.x. Peers also receive DNS = 172.31.0.2 so they can resolve *.skyhub-prod.internal names.

​
The reconciliation loop and S3 config distribution

Each plane manager runs a background tick() loop (wireguard.py:383) every DB_REFRESH_SECONDS (default 10s). It never mutates the database — it treats the DB as the source of truth and makes the WireGuard peer set match it.
1

Read desired state from the DB

UserVPN selects every user with a non-null ip; DroneVPN selects every non-SITL drone, mapping drone.ip (a 10.71.x address stored on the row). SITL rows are excluded (WHERE d."type" != 'sitl').
2

Diff against current peers

It computes create_this = their_ips - our_ips and delete_this = our_ips - their_ips. On the very first run it seeds vpn_entities from {data_dir}/mappings.json in S3 so a restart doesn’t churn every peer.
3

Generate keys + client config per new entity

For each new IP, _generate_vpn_config_for_entity mints a fresh keypair and pre-shared key and produces two files: a server-side [Peer] block and a client-side access.conf (its own [Interface] + the server [Peer] with Endpoint = EXTERNAL_IP:port and PersistentKeepalive = 25).
4

Write configs to S3 + re-render + hot-reload

Both files are uploaded to the VPN_BUCKET. The full server config is re-rendered to /etc/wireguard/{iface}.conf and applied live with wg syncconf {iface} <(wg-quick strip {iface}) — a hot reload that does not drop existing tunnels.
A separate monitor() thread polls wg show {iface} dump once per second to track per-peer rx/tx bytes, latest handshake, and an is_active flag; this feeds the /drones_status endpoint.

​
S3 bucket layout (VPN_BUCKET)

The bucket is skyhub-{env}-user-vpn (prod: skyhub-prod-user-vpn). Per plane, data_dir/config_dir are user_data/user, drone_data/drone, core_data/core.
S3 keyWritten byPurpose
{data_dir}/server_private_key, server_public_keyplane managerServer WG keypair, persisted so keys survive restarts
{data_dir}/{iface}.confplane managerRendered server interface config
{data_dir}/mappings.jsonplane managerip→entity snapshot used to seed the first run
{config_dir}/{entity_id}/access.confplane managerClient config the peer imports (the Gateway presigns this)
{config_dir}/{entity_id}/peer.confplane managerThe [Peer] block merged into the server config
The Gateway never talks to WireGuard directly — it just presigns user/{user_id}/access.conf or drone/{drone_id}/access.conf from this same bucket (skyhub_gateway_service/src/service/vpn_service.py:21-43, 30s expiry) and returns the URL to the Dashboard or drone. The layout above is exactly what those keys resolve to.

​
Per-user iptables isolation

The IptablesManager (libs/service/iptables.py) is the security core. On startup _prepare() installs MASQUERADE (NAT) rules so traffic can flow user↔drone and core↔drone across interfaces. Then, on every change cycle, it enforces a default-deny posture on the user plane by appending, as the last rule: 
iptables -A FORWARD -i users0 -j REJECT
Everything an operator is allowed to reach is expressed as ACCEPT rules inserted before that REJECT. Those rules are derived from the database: for each (user_ip → drone_ip) pair the manager adds an ACCEPT on the drone’s rosbridge port plus a fixed set of service ports.
PortProtocolUse
drone.port (e.g. 9090+n)tcprosbridge (Gateway control + telemetry)
8554udp, tcpRTSP video
14900udpMAVLink (mavp2p udps:0.0.0.0:14900)
22tcpSSH
A FORWARD ACCEPT rule is precise — source {user_ip}/32, destination {drone_ip}/32, in-interface users0, matched protocol/port (libs/models/rule.py:41). The loop diffs rules_in_db against rules_in_runtime, applies the delta, deduplicates any stray duplicates, and re-asserts the trailing REJECT. Net effect: a user’s WireGuard traffic reaches only the drones they own, and only on those ports.
Ownership source is narrower than the reference SQL. The design intent — and the standalone sql_4_iptables.sql file in the repo — derives access from user_drone_access (explicit grants) unioned with direct ownership. But the live query in IptablesManager.read_database_state (iptables.py:242) currently joins "user" to drone on drone.user_id only (direct ownership) and excludes SITL. So today, shared-access grants recorded in user_drone_access are not reflected in the firewall by the running code. If you are auditing “which users can reach which drones,” read the live rules via the API below rather than assuming user_drone_access is applied. See Database Schema Overview.

​
The :5050 status & firewall API

main.py starts a Flask app (dev server, or gunicorn/gevent in Dockerfile.ecr) bound to 0.0.0.0:5050 (HTTP_PORT, default 5050). Routes live in libs/routes/vpn_routes.py; most responses are cached ~1s (the /version liveness route is cached 15 min). This is a read-only introspection surface — it never changes tunnels.
MethodPathReturns
GET/status/user/<ip>{ip, status} — 200 if the user IP is a known peer, else 404
GET/status/drone/<ip>{ip, status} — same for the drone plane
GET/firewallFull per-user allow state (IptablesManager.current_state)
GET/firewall/user/<ip>One user’s current allow state
GET/drones_statusLive wg stats (rx/tx, handshake, is_active) keyed by drone id — send {"ips": [...]} as a JSON body
GET/version{"version": "canned flesh"} liveness ping
The Gateway consumes this over http://{VPN_SERVICE_IP}:{VPN_SERVICE_PORT} (vpn_service.py:45-77). For example, the fleet page’s connectivity indicators come from POST-style /drones_status: 
curl -s -X GET "http://<jumphost-private-ip>:5050/drones_status" \
  -H "Content-Type: application/json" \
  -d '{"ips": ["10.71.0.5", "10.71.0.6"]}'

​
How 10.71.x drone addressing reaches the Gateway

A drone is only reachable once its tunnel is up. The Gateway gates activation on the VPN status API and then routes rosbridge to the drone’s 10.71.x address through the jumphost. Two Gateway-side pieces close the loop:
  • Activation wait (src/routes/drone_routes.py:1585): the activation handler loops while not get_vpn_service().status_drone_vpn(drone.ip) (≤15s) so it only hands back the drone’s VPN config + ECR credentials once the WireGuard handshake is confirmed.
  • VPN-source-IP trust (src/middleware/drone_vpn.py): once on the plane, all drone→Gateway callbacks (/drone/pull, executions, asset upload) authenticate purely by source IP — check_vpn_ip accepts the request only if remote_addr / X-Real-IP / X-Forwarded-For starts with 10.71., then resolves the drone via get_drone_by_ip. This is one of the platform’s three coexisting auth models (JWT, VPN-IP trust, token/signature). See VPN IP Authentication & Jumphost Routing and Authentication & Security Model.
The Gateway (an ECS Fargate task at <vpc-host-ip>) is not itself on the drone plane, so it reaches 10.71.x:9090 through the jumphost using x-drone-ip/x-drone-port headers (JUMPHOST_IP / JUMPHOST_PORT default 9090). That routing and the pooled rosbridge connection are documented in DroneControlService & Rosbridge Dispatch and Network & VPN Topology.

​
Configuration reference

Environment variables (libs/config/settings.py; values below are the defaults / docker-compose.yml settings).
VariableDefaultPurpose
CORE_VPN_RANGE / CORE_VPN_PORT / CORE_IFACE10.69.0.0/16 / 51822 / wg0Core plane (currently unmanaged)
USER_VPN_RANGE / USER_VPN_PORT / USER_IFACE10.70.0.0/16 / 51823 / users0Operator plane
DRONE_VPN_RANGE / DRONE_VPN_PORT / DRONE_IFACE10.71.0.0/16 / 51824 / drones0Drone plane
EXTERNAL_IP<dev-vpn-endpoint> (dev)Public WireGuard endpoint written into client access.conf
AWS_RANGE172.31.0.0/16Added to peer AllowedIPs so tunnels reach the VPC
DNS_SERVER172.31.0.2DNS pushed to peers (resolves *.skyhub-prod.internal)
KEEPALIVE_DURATION25PersistentKeepalive seconds
DB_REFRESH_SECONDS10Reconcile loop interval
FORCE_UPDATEFalseRegenerate every peer config each cycle
SITL_HOST<office-host>Substituted for SITL drone IPs in the reference queries
REAL_INTERFACE / SECOND_INTERFACEens5 / ens6Physical NICs used for NAT/masquerade
HTTP_PORT5050Status API bind port
VPN_BUCKETskyhub-dev-user-vpnS3 bucket for keys + peer configs
DB_HOST / DB_NAME / DB_USERNAME / DB_PASSWORDdatabase.skyhub-*.internal / skyhub / <redacted>Shared PostgreSQL (read-only use)
Gateway-side knobs that pair with this service: VPN_SERVICE_IP / VPN_SERVICE_PORT (5050), VPN_BUCKET (shared), DRONE_NETWORK_CIDR (10.71.0.0/16), JUMPHOST_IP / JUMPHOST_PORT.
Troubleshooting a drone that is unreachable from the Gateway. Work outward:
  1. Is the tunnel up? GET /status/drone/<ip> on :5050 — 404 means the peer isn’t in the WireGuard config, i.e. the row’s ip is null or the reconcile loop hasn’t run.
  2. Is there recent traffic? GET /drones_status with the drone’s IP — check latest_handshake_ts / is_active.
  3. Is the firewall allowing it? GET /firewall/user/<user_ip> — confirm an ACCEPT exists for user_ip → drone_ip on the rosbridge port. Remember the live rules come from direct ownership (drone.user_id), not user_drone_access (see the warning above).
  4. Only then look at the jumphost/rosbridge hop (DroneControlService & Rosbridge Dispatch).

Ecosystem Overview

Where the VPN sits among Janus, WHIP, SITL, and the WS Proxy.

VPN IP Auth & Jumphost

The Gateway’s check_vpn_ip middleware and jumphost rosbridge routing.

Network & VPN Topology

The full three-plane picture across the platform.

Billing, Calendar, VPN & Video API

The Gateway HTTP endpoints that presign VPN configs and report status.