SITL Drone Lifecycle (Docker Orchestration)

SITLDroneService (src/service/sitl_drone_service.py, ~1500 lines) is the heaviest service in the gateway. It turns a single POST /api/v1/drone {type: sitl} request into a running 3-container Docker stack — an ArduPilot SITL simulator, a CORE container (MAVROS + rosbridge + video), and a gamepad container running the agent — plus a shared Redis container. Once the stack is up, the gateway connects rosbridge to it and treats it identically to a physical drone: the same DroneControlService command dispatch, the same telemetry emission, the same Janus video room. That interchangeability is the whole point of SITL. This page documents the container topology, the arithmetic that derives every port from a single container number, the local-vs-remote Docker split (including ECR auth), the readiness waits that make creation a long synchronous operation, the default ArduPilot params applied on boot, and the reverse-order cleanup that runs on any failure.

​

Two gates before anything starts

SITL creation is guarded twice. Understanding both saves a lot of “why is nothing happening” debugging. To actually run SITL you need one of:

• DEPLOYMENT_ENVIRONMENT=local (local Docker via docker.from_env()), or
• REMOTE_DOCKER_ENABLED=true (remote Docker over the office server, with ECR auth).

​

The container stack

SITL is not one container. The SITL image runs no ROS — CORE does. Four moving parts: {n} is the container number (1..101); {name} is the user-supplied drone name with spaces replaced by _. Companion names are derived by string substitution in _get_companion_container_names() (sitl_drone_service.py:234) — SKYHUB_SITL_ → SKYHUB_CORE_ / SKYHUB_GAMEPAD_ — so the number and name must stay identical across the three containers or delete/cleanup will miss containers.

​

SKYHUB_ROLE: two containers, one image

The core and agent halves ship as a single image (APP_IMAGE_NAME, default skyhub-app:latest). SKYHUB_ROLE selects which supervisord programs a container runs: vehicle (both halves — the image default, and what an aircraft runs), core (ROS only), agent (agent only). SITL keeps them as two containers on purpose — separate CPU/memory caps per half, and on Fargate the agent container is essential=false so it can die without killing the task. That makes the role variable mandatory rather than cosmetic:

• start_core_container sets "SKYHUB_ROLE": "core" (sitl_drone_service.py:712)
• start_gamepad_container sets "SKYHUB_ROLE": "agent" (sitl_drone_service.py:549)

Omit either and the container falls back to vehicle: the gamepad container would also start MAVROS, rosbridge and the drone node, competing with the CORE container for the same MAVLink endpoint, and the CORE container would run a second copy of the agent talking to the same Redis and the same API. CORE_IMAGE_NAME and GAMEPAD_IMAGE_NAME still exist, but both default to APP_IMAGE_NAME. They are per-half overrides so a deployment can pin one half to an older image without a code change.

​

Container numbering and port math

find_available_container_number() (sitl_drone_service.py:262) lists all containers (including stopped) whose name starts with SKYHUB_SITL_, parses the number from parts[2], and returns the first free integer starting at 1. It hard-caps at 101 (raises ResourceNotFoundException beyond that). A Docker listing failure raises DroneCreationFailed rather than falling back to 1. The whole method runs under a lock and records the number in _reserved_container_numbers, because the Docker listing alone races: gunicorn runs a single gevent worker in which every Docker/HTTP call yields, so two concurrent creates would otherwise see the same gap and collide on rosbridge port, MAVLink ports, ROS_DOMAIN_ID and the skysim slot. Callers must call release_container_number() once the container exists or creation has failed. Every other port is pure arithmetic off that number n. There is no port registry — collisions are prevented only by n being unique. The sshd port is derived rather than left at the vehicle default of 322 because every SITL container group shares a network namespace with its neighbours or the host — the second container to boot found 322 taken and its sshd sat in FATAL for the life of the container. ROSBRIDGE_PORT, GAMEPAD_WS_PORT and SKYHUB_SSH_PORT are all per-container for the same reason.

​

Local vs remote Docker networking

The single biggest branch in this service is use_remote_docker (REMOTE_DOCKER_ENABLED). It changes the Docker client, the network mode of every container, and how the gateway later reaches rosbridge.

​

Creation sequence

save() (sitl_drone_service.py:986) is async, long, and synchronous internally (it blocks on healthchecks and readiness probes). The order matters and every failure branch triggers reverse-order cleanup.

​

Per-container environment

The env dicts are the real contract with each image. Highlights (see start_container, start_gamepad_container, start_core_container):

​

Default ArduPilot params

After boot, _set_default_params() (sitl_drone_service.py:1352) sleeps a fixed 5s (MAVROS param services register after rosbridge is up) and pushes a param batch through DroneControlService.set_params. Everything not marked hardcoded comes from a SITL_* env var in src/application/settings.py: The method also sets simulation speed (SIM_SPEEDUP), navigation and airframe params (WPNAV_SPEED, RTL_ALT, MOT_THST_HOVER, FRAME_CLASS/FRAME_TYPE), the coulomb-count failsafes (BATT_LOW_MAH / BATT_CRT_MAH, derived from SITL_BATT_LOW_PCT / SITL_BATT_CRT_PCT) and the camera backend. set_params routes through the pooled rosbridge connection like any other command — see DroneControlService & Rosbridge Dispatch and Rosbridge Connection & Reconnect.

​

Deletion and cleanup ordering

Both intentional deletion and failure cleanup stop containers in reverse dependency order: core → gamepad → sitl. Preserve this — CORE holds the rosbridge/DDS session and should go first.

• delete(drone) (sitl_drone_service.py:1235): guards is_server_environment, deletes all S3 assets (non-fatal), reads container_name from drone.mac, derives companion names, then stop_core_container → stop_gamepad_container → stop/remove the SITL container (30s timeout), and finally super().delete() removes the DB row. The shared Redis is left running.
• _cleanup_containers(sitl, gamepad, core) (sitl_drone_service.py:863): the failure path used inside save(). Each stage of save() passes progressively more containers to clean up, so a gamepad failure removes only SITL, a CORE failure removes SITL + gamepad, and a post-start DB failure removes all three.

​

Gotchas for future editors