The video_stream module is the on-drone half of SkyHub’s live video path. It runs inside the skyhub container’s ROS half as a drone_node module, turns whatever camera the vehicle carries into a single H264 GStreamer pipeline, and pushes that stream to the WHIP ingest server — which republishes it into a Janus VideoRoom that the Dashboard subscribes to over WebRTC. Nothing about the video travels through the Gateway or rosbridge; the only thing the control plane does is create the room and hand the drone its credentials. This page covers the drone side end to end: how the stream is gated on/off, the CameraFactory driver set, hardware vs software encode selection, the gst-launch subprocess model, and the WHIP endpoint client. It is written against docker/core/src/modules/video_stream/.
Out-of-band by design. The Gateway creates the Janus room and pushes {room_number, room_password, room_mgmt_token} to the drone over the rosbridge ROS topic video_room_details; from there the drone talks directly to the WHIP server. See Janus Video Rooms & On-Drone Video Control for the Gateway side of this handshake.

​
End-to-end pipeline

The drone-side entrypoint is the ROS2 DroneNode in docker/core/main.py. It subscribes to three std_msgs/String topics — video_room_source, video_room_details, video_room_state — plus /mavros/state, and forwards each into the module via ModuleLoader.dispatch_command(...) / dispatch_drone_state(...) (docker/core/main.py:154-187). The module itself never touches rosbridge; it only reacts to those dispatched events and to Redis.

​
Stream state machine (arm / disarm / manual / force-start)

Whether the pipeline is running is decided entirely inside Module._transition_state() (docker/core/src/modules/video_stream/module.py:399). Three inputs combine:
  • status — START or STOP, set by video_room_details/video_room_state.
  • drone — derived from /mavros/state against required_state (VIDEO_STREAM_DRONE_STATE, either ARMED or CONNECTED).
  • gates — force_start, _manual_start, or the drone reaching required_state.
The decisive line is module.py:415:
1

Auto-start on state

When status == START and the drone reaches required_state (e.g. ARMED), can_stream becomes true and, if room prerequisites are present, the stream starts.
2

Manual start persists

A video_room_details/video_room_state message carrying status: START sets _manual_start = True (module.py:359-382). A manual start streams regardless of drone state and keeps streaming even while disarmed — until an explicit STOP or a disarm clears it.
3

Disarm always stops

on_drone_state() special-cases disarm: when required_state == "ARMED" and the drone transitions to DISARMED, it clears _manual_start and calls _ensure_stopped("drone_disarmed") (module.py:324-328) — even if the stream was manually started.
4

force_start bypasses everything

With FORCE_START=true, can_stream is always true and the drone-state gate is ignored entirely. This is the shipped production default (see the config note below), so in practice the stream comes up as soon as valid room details exist.
The manual-start / auto-start / disarm-clear semantics are subtle and load-bearing. Reproduce them exactly during any refactor or streams will fail to start (or fail to stop on landing). In particular: disarm clears _manual_start, and force_start short-circuits the whole state gate.
Room details are persisted to room.json under VIDEO_ROOM_CONFIG_DIRECTORY (.janus_room_details, bind-mounted at /.janus_room_details in docker-compose.yml) by StoreVideoRoomDetails. When FAST_INIT=true, start() reads that file and attempts an immediate _transition_state(init=True) so a rebooting drone resumes its stream without waiting for a fresh Gateway push (module.py:112-116, 282-307).

​
Redis state sync

The module also mirrors its on/off state to the frontend over two global (non-IP-namespaced) Redis channels — do not rename these: These are the two channels that are not keyed off the drone’s WireGuard IP; every other drone channel is. See the Redis Message Bus for the full channel map.

​
CameraFactory and the driver set

CameraFactory (docker/core/src/modules/video_stream/factory.py) maps CAMERA_TYPE to a driver class. Each driver implements get_stream_command(room_details, need_flip) returning a GStreamer pipeline string that ends in whipsink. CAMERA_TYPE=AUTO instead probes each driver’s check_presence() in registration order and uses the first that responds.
Code default vs shipped default. Config.DEFAULTS sets CAMERA_TYPE=ZR30, VIDEO_STREAM_DRONE_STATE=ARMED, FORCE_START=False (docker/core/src/shared/config.py:24). The shipped .env.example overrides these to CAMERA_TYPE=UDP, VIDEO_STREAM_DRONE_STATE=CONNECTED, FORCE_START=true, FAST_INIT=true, SKIP_CHECKS=true, VIDEO_ENCODER=software, AUDIO_ENABLED=false. The real fleet runs the UDP driver with force_start, not ZR30 gated on arm.
Most drivers subclass ZR30_Camera and differ only in their RTSP source and pipeline tuning. The full registry:
ZR30_Camera (cameras/zr30.py) talks the SIYI TCP command protocol on port 37260 (SIYIIP_PORT) using CRC-16 CCITT/XMODEM checksums computed by calculate_checksum. check_presence() sends query_hardware_id_msg and asserts the returned hardware ID equals b"78" (ZR30); the A8 subclass asserts b"73". configure() sends video_mode_msg to lock the main stream to 1920×1080 H264 @ ~2 Mbps.The stream pipeline pulls the camera’s own H264 over RTSP, decodes with software avdec_h264, then re-encodes with x264enc bitrate=2000 speed-preset=ultrafast tune=zerolatency, wraps in rtph264pay, and terminates at:
The SIYI command bytes are hand-rolled little-endian frames — see the # NOTE blocks in zr30.py before changing them.
UDP_Camera (cameras/udp.py) is what production actually runs. In __init__ it builds a HardwareDetector and AudioDetector. check_presence() / configure() probe the RTSP source with a gst-launch … fakesink and retry up to UDP_RETRY_ATTEMPTS (default 20) with UDP_RETRY_BACKOFF (1.2×).get_stream_command():
  • Video: rtspsrc … protocols=tcp latency=0 drop-on-latency=true → rtph264depay → h264parse → {decoder} → {encoder} bitrate=5000 → rtph264pay aggregate-mode=zero-latency mtu=1300 pt=97 → whipsink name=mux … stun-server=<STUN_SERVER> async-handling=true. Decoder/encoder come from HardwareDetector (nvv4l2h264dec/nvv4l2h264enc on Jetson, avdec_h264/x264enc otherwise).
  • Audio (only if AUDIO_ENABLED=true and a ReSpeaker is accessible): alsasrc at 6-channel S16LE 16 kHz → downmix → 48 kHz stereo → opusenc bitrate=128000 → rtpopuspay → mux.sink_0. Audio and video are muxed into the single whipsink named mux.
Note the extensive audio-device liveness checks (_test_audio_device_access, _cleanup_audio_device) that pkill stuck alsasrc processes and fuser -k the ReSpeaker device before falling back to video-only.

​
Hardware vs software encode

HardwareDetector (utils/hardware_detector.py) chooses the H264 encoder/decoder from the VIDEO_ENCODER env var and what gst-inspect-1.0 reports as installed: The paired AudioDetector selects the OPUS encoder (AUDIO_ENCODER) and locates the ReSpeaker via arecord -l.
The application image is built aarch64-first (Jetson) with NVIDIA GStreamer plugins on the path, so hardware encode “just works” on the vehicle. On x86/SITL there are no nvv4l2* elements, so set VIDEO_ENCODER=software (the SITL default) to force x264enc/avdec_h264.There are two separate builds of the image — ARM for the Jetson (buildspec/app.yml) and x86 for Fargate (buildspec/fargate-app.yml). It is not a multi-arch manifest: do not point a vehicle and a Fargate task at one tag.

​
The gst-launch subprocess model

VideoStreamer (docker/core/src/modules/video_stream/streamer.py) does not run GStreamer in-process. It shells out to gst-launch-1.0 -e <pipeline> via subprocess.Popen, deliberately avoiding native crashes in the Python GI bindings. It detects the PLAYING state by parsing stdout for "Setting pipeline to PLAYING" / "Pipeline is live", with a 3-second fallback that assumes success if the process is still alive, and a 10-second hard timeout (streamer.py:31-67). A daemon monitor thread tails stdout for ERROR/WARNING. stop_stream() does terminate() → wait 5s → kill().
Because pipeline health is inferred from stdout text and a timer — not a real GStreamer bus — a pipeline that connects but produces no frames can still register as “streaming”. When debugging a black stream, run the driver’s get_stream_command() output directly with gst-launch-1.0 -e inside the container to see the true bus messages.

​
WHIP ingest client

Two things talk to the WHIP server, both at SKYHUB_SERVER_URL (default http://whip.skyhub-dev.internal:7080, prod …skyhub-prod.internal:7080):
  1. whipsink in the GStreamer pipeline POSTs the actual WebRTC media (SDP offer + RTP) to /whip/endpoint/<room_id> with auth-token=<room_mgmt_token>.
  2. WhipServerService (whip.py) is the REST control client the module uses to provision endpoints before streaming. The write calls (create_endpoint / delete_endpoint) carry Authorization: Bearer <room_mgmt_token>; the availability/free/list/room_number calls are unauthenticated. All calls use a 2 s timeout.
_ensure_endpoint() (module.py:540) runs this dance on every start: reuse the endpoint if it exists and is free, otherwise delete stale ones and create_endpoint a fresh UUID. See WHIP Ingest Server for the server that answers these.

​
The overlay path is intentionally dormant (CANVAS approach)

The module has a full overlay code path — OverlayVideoStreamer runs a dual pipeline (appsink → Python numpy callbacks → appsrc) and _start_overlay_stream() engages it whenever frame callbacks are registered and the camera supports them (module.py:502). In practice nothing registers a callback. ModuleLoader.setup_video_overlay() is a deliberate no-op that only logs (module_loader.py:304). ArUco precision-landing does not burn its marker box into the video; instead it captures RTSP independently and publishes tracking coordinates to the {ip}:aruco_tracking Redis channel, which the frontend renders as a canvas overlay on top of the WebRTC <video>. This “CANVAS approach” preserves video quality and spares Jetson GPU/CPU. Until 0.2.0 the point was moot in a second way: cv2 was missing from the vehicle image entirely, so UDP_OVERLAY / OverlayVideoStreamer could not have run even if something had registered a callback. That was a packaging bug and is fixed. CANVAS is the separate, deliberate reason not to re-enable server-side compositing.
Do not re-add server-side overlay rendering to “fix” a missing box on the video — the box is supposed to be a frontend canvas layer fed by Redis. See e-ArUco Precision Landing for the tracking-data contract.

​
Configuration reference

Config resolves via Config (docker/core/src/shared/config.py) — env var, else the DEFAULTS dict. Values below are the code defaults; the shipped .env.example overrides several (see the note above).

Gateway: Video Rooms

How the Gateway creates the Janus room and pushes /video_room_details / /video_room_state to the drone.

WHIP Ingest Server

The ingest endpoint that whipsink publishes to and that maps into a Janus publisher.

Janus WebRTC SFU

The VideoRoom the Dashboard subscribes to over WebRTC.

Dashboard: State & Video

The frontend JanusService / WebRTC subscriber and canvas overlay.

e-ArUco Precision Landing

The ArUco canvas-overlay tracking contract and the down-camera capture path.

Redis Message Bus

The video_stream_state / video_stream_status_request global channels.