The video_stream module is the on-drone half of SkyHub’s live video path. It runs inside the core service as a ROS2 drone_node module, turns whatever camera the vehicle carries into a single H264 GStreamer pipeline, and pushes that stream to the WHIP ingest server — which republishes it into a Janus VideoRoom that the Dashboard subscribes to over WebRTC. Nothing about the video travels through the Gateway or rosbridge; the only thing the control plane does is create the room and hand the drone its credentials. This page covers the drone side end to end: how the stream is gated on/off, the CameraFactory driver set, hardware vs software encode selection, the gst-launch subprocess model, the WHIP endpoint client, and the separate camera-proxy RTSP re-server. It is written against docker/core/src/modules/video_stream/.
Out-of-band by design. The Gateway creates the Janus room and pushes {room_number, room_password, room_mgmt_token} to the drone over the rosbridge ROS topic video_room_details; from there the drone talks directly to the WHIP server. See Janus Video Rooms & On-Drone Video Control for the Gateway side of this handshake.

​
End-to-end pipeline

The drone-side entrypoint is the ROS2 DroneNode in docker/core/main.py. It subscribes to three std_msgs/String topics — video_room_source, video_room_details, video_room_state — plus /mavros/state, and forwards each into the module via ModuleLoader.dispatch_command(...) / dispatch_drone_state(...) (docker/core/main.py:154-187). The module itself never touches rosbridge; it only reacts to those dispatched events and to Redis.

​
Stream state machine (arm / disarm / manual / force-start)

Whether the pipeline is running is decided entirely inside Module._transition_state() (docker/core/src/modules/video_stream/module.py:399). Three inputs combine:
  • status — START or STOP, set by video_room_details/video_room_state.
  • drone — derived from /mavros/state against required_state (VIDEO_STREAM_DRONE_STATE, either ARMED or CONNECTED).
  • gates — force_start, _manual_start, or the drone reaching required_state.
The decisive line is module.py:415: 
can_stream = self.force_start or self._manual_start or self.required_state == self.current_state["drone"]
1

Auto-start on state

When status == START and the drone reaches required_state (e.g. ARMED), can_stream becomes true and, if room prerequisites are present, the stream starts.
2

Manual start persists

A video_room_details/video_room_state message carrying status: START sets _manual_start = True (module.py:359-382). A manual start streams regardless of drone state and keeps streaming even while disarmed — until an explicit STOP or a disarm clears it.
3

Disarm always stops

on_drone_state() special-cases disarm: when required_state == "ARMED" and the drone transitions to DISARMED, it clears _manual_start and calls _ensure_stopped("drone_disarmed") (module.py:324-328) — even if the stream was manually started.
4

force_start bypasses everything

With FORCE_START=true, can_stream is always true and the drone-state gate is ignored entirely. This is the shipped production default (see the config note below), so in practice the stream comes up as soon as valid room details exist.
The manual-start / auto-start / disarm-clear semantics are subtle and load-bearing. Reproduce them exactly during any refactor or streams will fail to start (or fail to stop on landing). In particular: disarm clears _manual_start, and force_start short-circuits the whole state gate.
Room details are persisted to room.json under VIDEO_ROOM_CONFIG_DIRECTORY (.janus_room_details, bind-mounted at /.janus_room_details in docker-compose.yml) by StoreVideoRoomDetails. When FAST_INIT=true, start() reads that file and attempts an immediate _transition_state(init=True) so a rebooting drone resumes its stream without waiting for a fresh Gateway push (module.py:112-116, 282-307).

​
Redis state sync

The module also mirrors its on/off state to the frontend over two global (non-IP-namespaced) Redis channels — do not rename these:
ChannelDirectionPurpose
video_stream_statecore → gamepad → UIPublished on every start/stop with {is_streaming, reason, room_id} (module.py:241)
video_stream_status_requestgamepad → coreA background listener replies with current state (module.py:207)
These are the two channels that are not keyed off the drone’s WireGuard IP; every other drone channel is. See the Redis Message Bus for the full channel map.

​
CameraFactory and the driver set

CameraFactory (docker/core/src/modules/video_stream/factory.py) maps CAMERA_TYPE to a driver class. Each driver implements get_stream_command(room_details, need_flip) returning a GStreamer pipeline string that ends in whipsink. CAMERA_TYPE=AUTO instead probes each driver’s check_presence() in registration order and uses the first that responds.
Code default vs shipped default. Config.DEFAULTS sets CAMERA_TYPE=ZR30, VIDEO_STREAM_DRONE_STATE=ARMED, FORCE_START=False (docker/core/src/shared/config.py:24). The shipped .env.example overrides these to CAMERA_TYPE=UDP, VIDEO_STREAM_DRONE_STATE=CONNECTED, FORCE_START=true, FAST_INIT=true, SKIP_CHECKS=true, VIDEO_ENCODER=software, AUDIO_ENABLED=false. The real fleet runs the UDP driver with force_start, not ZR30 gated on arm.
Most drivers subclass ZR30_Camera and differ only in their RTSP source and pipeline tuning. The full registry:
CAMERA_TYPEClass / fileSource & transportNotes
ZR30cameras/zr30.pySIYI RTSP <camera-ip>:8554/main.264Base driver. SIYI TCP probe (HW-ID 0x78), avdec_h264 (SW decode) → x264enc 2 Mbps, video-only
A8cameras/a8.pySIYI RTSP over UDPExtends ZR30; SIYI A8 mini probe (HW-ID 0x73), constrained-baseline x264, key-int-max=60
UDPcameras/udp.pyRTSP CAMERA_IP:CAMERA_PORT/CAMERA_PATHProduction default. HW-accel via HardwareDetector, muxes ReSpeaker audio, 5 Mbps, RTSP retry/backoff
UDP_OVERLAYcameras/udp_overlay.pySame as UDPUDP variant with an appsink frame-callback tap for overlay processing
UDPAVcameras/udpav.pyRaw udpsrc video :5010 + audio :5011Separate A/V UDP ports
TESTcameras/test.pyvideotestsrc + audiotestsrcAlways “present”; openh264enc. Used for SITL / source=TEST
INTELcameras/intel_d435.pyRealSense D435Stub — class IntelD435_Camera(CameraProvider): pass, not implemented
GOPROcameras/gopro.pyudpsrc port=8554 + tsdemuxUntested per source comments
A8_DIRECTcameras/a8_direct.pySIYI A8 RTSPZR30 subclass, check_presence() hard-returns True (no probe)
SIYI_DIRECTcameras/siyi_direct.pySIYI_IP RTSPGeneric SIYI, no HW-ID probe
SIYI_25cameras/siyi_25.pySIYI_IP RTSPSIYI variant
SIYI_RECODEcameras/siyi_recode.pySIYI_IP RTSPAdds optional DO_RATE re-rate
RELAYcameras/relay.pyRTSP 127.0.0.1:7663/streamConsumes a local relay stream
SHMcameras/shm.pyshmsrc socket-path=/tmp/video.socketShared-memory H264 source
SHM_LIGHTcameras/shm_light.pySame shm socketPassthrough (h264parse → identity → pay), no re-encode
ZR30_Camera (cameras/zr30.py) talks the SIYI TCP command protocol on port 37260 (SIYIIP_PORT) using CRC-16 CCITT/XMODEM checksums computed by calculate_checksum. check_presence() sends query_hardware_id_msg and asserts the returned hardware ID equals b"78" (ZR30); the A8 subclass asserts b"73". configure() sends video_mode_msg to lock the main stream to 1920×1080 H264 @ ~2 Mbps.The stream pipeline pulls the camera’s own H264 over RTSP, decodes with software avdec_h264, then re-encodes with x264enc bitrate=2000 speed-preset=ultrafast tune=zerolatency, wraps in rtph264pay, and terminates at:
whipsink auth-token=<room_mgmt_token> whip-endpoint=<SKYHUB_SERVER_URL>/whip/endpoint/<room_id>
The SIYI command bytes are hand-rolled little-endian frames — see the # NOTE blocks in zr30.py before changing them.
UDP_Camera (cameras/udp.py) is what production actually runs. In __init__ it builds a HardwareDetector and AudioDetector. check_presence() / configure() probe the RTSP source with a gst-launch … fakesink and retry up to UDP_RETRY_ATTEMPTS (default 20) with UDP_RETRY_BACKOFF (1.2×).get_stream_command():
  • Video: rtspsrc … protocols=tcp latency=0 drop-on-latency=true → rtph264depay → h264parse → {decoder} → {encoder} bitrate=5000 → rtph264pay aggregate-mode=zero-latency mtu=1300 pt=97 → whipsink name=mux … stun-server=<STUN_SERVER> async-handling=true. Decoder/encoder come from HardwareDetector (nvv4l2h264dec/nvv4l2h264enc on Jetson, avdec_h264/x264enc otherwise).
  • Audio (only if AUDIO_ENABLED=true and a ReSpeaker is accessible): alsasrc at 6-channel S16LE 16 kHz → downmix → 48 kHz stereo → opusenc bitrate=128000 → rtpopuspay → mux.sink_0. Audio and video are muxed into the single whipsink named mux.
Note the extensive audio-device liveness checks (_test_audio_device_access, _cleanup_audio_device) that pkill stuck alsasrc processes and fuser -k the ReSpeaker device before falling back to video-only.

​
Hardware vs software encode

HardwareDetector (utils/hardware_detector.py) chooses the H264 encoder/decoder from the VIDEO_ENCODER env var and what gst-inspect-1.0 reports as installed:
VIDEO_ENCODEREncoderDecoderSettings (encoder)
hardwarenvv4l2h264enc (or nvh264enc)nvv4l2h264decdisable-dpb=true enable-max-performance=true maxperf-enable=true poc-type=2 insert-sps-pps=true
softwarex264encavdec_h264speed-preset=veryfast tune=zerolatency key-int-max=30 bframes=0
auto (default)probes nvv4l2h264enc→nvh264enc, else x264encprobes NV decoders, else avdec_h264per encoder above
The paired AudioDetector selects the OPUS encoder (AUDIO_ENCODER) and locates the ReSpeaker via arecord -l.
The core image is built aarch64-first (Jetson) with NVIDIA GStreamer plugins on the path, so hardware encode “just works” on the vehicle. On x86/SITL there are no nvv4l2* elements, so set VIDEO_ENCODER=software (the SITL default) to force x264enc/avdec_h264.

​
The gst-launch subprocess model

VideoStreamer (docker/core/src/modules/video_stream/streamer.py) does not run GStreamer in-process. It shells out to gst-launch-1.0 -e <pipeline> via subprocess.Popen, deliberately avoiding native crashes in the Python GI bindings. It detects the PLAYING state by parsing stdout for "Setting pipeline to PLAYING" / "Pipeline is live", with a 3-second fallback that assumes success if the process is still alive, and a 10-second hard timeout (streamer.py:31-67). A daemon monitor thread tails stdout for ERROR/WARNING. stop_stream() does terminate() → wait 5s → kill().
Because pipeline health is inferred from stdout text and a timer — not a real GStreamer bus — a pipeline that connects but produces no frames can still register as “streaming”. When debugging a black stream, run the driver’s get_stream_command() output directly with gst-launch-1.0 -e inside the container to see the true bus messages.

​
WHIP ingest client

Two things talk to the WHIP server, both at SKYHUB_SERVER_URL (default http://whip.skyhub-dev.internal:7080, prod …skyhub-prod.internal:7080):
  1. whipsink in the GStreamer pipeline POSTs the actual WebRTC media (SDP offer + RTP) to /whip/endpoint/<room_id> with auth-token=<room_mgmt_token>.
  2. WhipServerService (whip.py) is the REST control client the module uses to provision endpoints before streaming. The write calls (create_endpoint / delete_endpoint) carry Authorization: Bearer <room_mgmt_token>; the availability/free/list/room_number calls are unauthenticated. All calls use a 2 s timeout.
MethodHTTPPurpose
create_endpointPOST /whip/createRegister {id, room, token, pin} for a room
is_endpoint_availableGET /whip/endpoint/{id}404 → not created yet
is_endpoint_freeGET /whip/free/{id}Whether the publisher slot is idle
delete_endpointDELETE /whip/endpoint/{id}Tear down one endpoint
delete_with_my_room_numberDELETE /whip/room_number/{room}Clear stale endpoints for a room before (re)creating
list_endpointsGET /whip/endpointsDebug listing
_ensure_endpoint() (module.py:540) runs this dance on every start: reuse the endpoint if it exists and is free, otherwise delete stale ones and create_endpoint a fresh UUID. See WHIP Ingest Server for the server that answers these.

​
camera-proxy: dual RTSP re-server

camera-proxy (compose profile camera, built from Dockerfile.proxy, entrypoint docker/core/rtsp_server.py) is a separate container from video_stream. It uses GstRtspServer to re-serve the single physical SIYI RTSP feed as two local mount points on SERVER_PORT (8554):
MountPipelineConsumer
rtsp://0.0.0.0:8554/streamHQ passthrough: rtspsrc → rtph264depay → h264parse → rtph264pay (no re-encode)High-quality recording / storage
rtsp://0.0.0.0:8554/fast_streamHW re-encode: decode → videorate max-rate=25 → {encoder} bitrate=5000 iframeinterval=15 idrinterval=1 → rtph264pay mtu=1300Low-latency source for YOLO detection
The /fast_stream mount is the RTSP_URL that the Isaac YOLO detection service reads. Keeping detection on this down-scaled re-encode (rather than the raw camera feed) is what makes person tracking affordable on the Jetson.

​
The overlay path is intentionally dormant (CANVAS approach)

The module has a full overlay code path — OverlayVideoStreamer runs a dual pipeline (appsink → Python numpy callbacks → appsrc) and _start_overlay_stream() engages it whenever frame callbacks are registered and the camera supports them (module.py:502). In practice nothing registers a callback. ModuleLoader.setup_video_overlay() is a deliberate no-op that only logs (module_loader.py:304). ArUco precision-landing does not burn its marker box into the video; instead it captures RTSP independently and publishes tracking coordinates to the {ip}:aruco_tracking Redis channel, which the frontend renders as a canvas overlay on top of the WebRTC <video>. This “CANVAS approach” preserves video quality and spares Jetson GPU/CPU.
Do not re-add server-side overlay rendering to “fix” a missing box on the video — the box is supposed to be a frontend canvas layer fed by Redis. See YOLO Detection, ArUco Landing & Docking for the tracking-data contract.

​
Configuration reference

Config resolves via Config (docker/core/src/shared/config.py) — env var, else the DEFAULTS dict. Values below are the code defaults; the shipped .env.example overrides several (see the note above).
VariableCode defaultPurpose
CAMERA_TYPEZR30 (env: UDP)Selects the CameraFactory driver
CAMERA_IP / CAMERA_PORT / CAMERA_PATH<camera-ip> / 8554 / main.264SIYI/RTSP source for the UDP driver + camera-proxy
SKYHUB_SERVER_URLhttp://whip.skyhub-dev.internal:7080WHIP ingest base URL
VIDEO_STREAM_DRONE_STATEARMED (env: CONNECTED)required_state gate: start on ARMED or CONNECTED
FORCE_STARTFalse (env: true)Bypass the drone-state gate entirely
FAST_INITFalse (env: true)Read room.json and start on boot
SKIP_CHECKSFalse (env: true)Skip camera.prepare() presence/config probe
AUTOPLAY / TEST_MODE / FLIP_VIDEO / DISABLE_VIDEOTrue / False / False / FalseInitial status, test source, vertical flip, disable
VIDEO_ENCODER / AUDIO_ENCODERauto (env video: software)Force hardware/software or auto-detect
AUDIO_ENABLEDtrue (env: false)Enable ReSpeaker audio muxing (UDP driver)
VIDEO_STREAM_ENABLEDTrueWhether ModuleLoader loads the module at all
SERVER_PORT / SERVER_PATH8554 / streamcamera-proxy listen port / storage mount name

Gateway: Video Rooms

How the Gateway creates the Janus room and pushes /video_room_details / /video_room_state to the drone.

WHIP Ingest Server

The ingest endpoint that whipsink publishes to and that maps into a Janus publisher.

Janus WebRTC SFU

The VideoRoom the Dashboard subscribes to over WebRTC.

Dashboard: State & Video

The frontend JanusService / WebRTC subscriber and canvas overlay.

Detection & Landing

YOLO on /fast_stream and the ArUco canvas-overlay tracking contract.

Redis Message Bus

The video_stream_state / video_stream_status_request global channels.