HomeModule and is wired through AppStateService.controllerData$.
This page is about the cockpit UI and its command payloads. The lower-level transport mechanics live on sibling pages, and this page links to them rather than repeating them:
Vehicle Commands & Gamepad (redispad)
The
VehicleCommandService WebSocket, reconnect, and message routing.App State & Video (Janus/WebRTC)
JanusService internals and the AppStateService store.Real-time Telemetry Client
The Socket.IO telemetry stream that drives mode/altitude readouts.
Real-time Transport Channels
Why there are three concurrent channels with opposite directions.
The three channels touching this screen
The cockpit deliberately straddles all three of the Dashboard’s real-time transports. Conflating them is the single most common source of bugs, so keep the directions straight:| Channel | Service | Direction | Carries |
|---|---|---|---|
| Socket.IO telemetry | WebsocketService | vehicle → UI | mode, altitude, GPS (drives the flight-control readouts) |
| redispad WebSocket | VehicleCommandService | UI → vehicle | gamepad axes, buttons, gimbal, camera_command, guided_control, quick set_mode: LAND |
| redispad WebSocket (2nd socket) | ArucoOverlayService | vehicle → UI | aruco_tracking frames for the landing overlay |
| WebRTC media | JanusService | vehicle → UI | the H264 video tracks |
Live video — the Janus subscriber lifecycle
VideoWindowComponent (src/app/home/video-window/video-window.component.ts) owns the <video> element and drives JanusService (src/app/services/janus.service.ts). The room comes straight off the selected drone: roomId = drone.video_room_id, pin = drone.video_room_password — both minted server-side when the vehicle’s Janus room is created (see Janus Video Rooms
Gateway video rooms
Drone selected → schedule start
selectedDrone$ fires (deduped by drone id). The component disconnects any prior session, then scheduleVideoStart(roomId, password, 500) debounces a start so a stale timer can’t open a session for the previous drone (video-window.component.ts:246).Init + attach as publisher
onStart() calls janusService.initJanus() (Janus debug logging is forced off to prevent memory accumulation), then new Janus({ server: janusGatewayUrl }) and attachVideoRoomPlugin(), which joins the room with ptype: 'publisher' and the pin (janus.service.ts:54).Discover publishers → subscribe
On the
joined event, extractStreams() reads msg.publishers and subscribeTo() attaches a second handle as ptype: 'subscriber' (createNewSubscription). This is the standard Janus VideoRoom two-handle pattern.Answer the offer
The subscriber handle receives an SDP offer (
jsep); createAnswer() replies and sends { request: 'start' } back to the room (janus.service.ts:112).Connect states and gotchas
- No
video_room_id→ after a 5s grace period the component toasts “Video streaming not configured for this drone” and does nothing else (handleDroneSelection,video-window.component.ts:218). - 15s fallback timeout — if no track and no error arrive within 15s,
isConnectingis cleared so the Start Video button reappears instead of spinning forever (video-window.component.ts:450). - Start Video button calls
droneService.startVideoStream(video_room_id)→GET /video_room/{id}/start?update=true, then re-schedules a Janus connect after 1s (video-window.component.ts:540). Sibling REST calls:/video_room/{id}/stopand/video_room/{id}/restart. - Vehicle stream sync — the vehicle emits
video_stream_state { is_streaming }over redispad; the component auto-connects when the vehicle starts streaming and tears down when it stops (handleVideoStreamState,video-window.component.ts:295). Avideo_stream_status_requestis sent 500ms after drone selection to sync initial state.
Gimbal, zoom, record & photo
Two very different payload shapes travel the redispad channel from this panel, both viaAppStateService.setControllerData(...):
- Camera (camera_command)
camera_status message. ControllerDataSenderService routes it into websocketService.cameraStatusSubject, and VideoWindowComponent reads is_recording, recording_elapsed_seconds and zoom_percentage from it to keep the record timer and zoom slider truthful (controller-data-sender.service.ts:90, video-window.component.ts:120). A local 1s timer increments the displayed record time between backend syncs.
Because gimbal frames are plain
GamepadData buttons, they ride the exact same gate as manual flight input — but buttons are never gated (only axes are). So gimbal, photo and center always reach the vehicle even when guided control is off. See the gate below.ArUco precision-landing overlay
ArucoOverlayService (src/app/services/aruco-overlay.service.ts) opens the second redispad socket, filters for aruco_tracking messages, and draws marker geometry onto a <canvas> layered over the <video> — the tracking is not burned into the video stream, it is a frontend overlay.
- Composite markers — an outer ArUco tag and an inner AprilTag. The currently-active marker (chosen by altitude via
marker_switch_altitude) is drawn green, the other red; corners, a center-to-crosshair dashed line, and an info panel (distance, yaw, confidence, yaw/XY alignment) are rendered. - Performance — the socket and its per-frame draw run via
ngZone.runOutsideAngular(canvas draws, no template bindings), and aResizeObserverkeeps the canvas bitmap matched to the video’s displayed size. When the video is hidden (0×0),overlayVisiblegoes false and message processing pauses. - Staleness — if no frame arrives for
dataStaleTimeoutMs(2000ms,aruco-tracking.model.ts), the overlay is cleared so it never shows a frozen marker. - SITL is skipped —
VideoWindowComponentonly callsarucoOverlayService.connect()whendrone.type?.toLowerCase() !== 'sitl'(video-window.component.ts:214).
YOLO Detection, ArUco Landing & Docking
detection & landing
Manual flight — three input paths, one gate
Three input sources all converge onAppStateService.controllerData$, are bridged to the vehicle by ControllerDataSenderService, and gated by guided control:
| Path | Source file | Rate | Notes |
|---|---|---|---|
| Physical gamepad | gamepad.service.ts | 60Hz (17ms) | Polls navigator.getGamepads(); deadzone 0.1; R1/button 5 plays a shutter sound on edge |
| Keyboard | gamepad.service.ts (same service) | 60Hz | Arrow keys → progressive-acceleration yaw/throttle on axes; AWSD/Q → gimbal buttons only when video is fullscreen; A → GoTo mode when in GUIDED |
| Virtual joysticks | virtual-joystick.component.ts | 80ms | Dual on-screen sticks; maps to a 4-element axes array (axes[0..3]) for quad; rover adds throttle button 7 |
ngZone.runOutsideAngular and only re-enter the zone for template-bound side effects (e.g. a docking-mode toast). Losing that pattern reintroduces heavy change-detection churn on every frame.
The guided-control gate
ControllerDataSenderService (src/app/services/controller-data-sender.service.ts, wired once from AppComponent.ngOnInit) subscribes to controllerData$ and decides what actually reaches the vehicle:
src/app/services/controller-data-sender.service.ts
Enabling guided control (the handshake)
GuidedControlService (src/app/services/guided-control.service.ts) owns the enable/disable/status handshake over redispad:
- The Enable Manual Control button in
MissionControlComponentopens a real-vs-virtual selection dialog, which is subscription-gated (SubscriptionDialogComponent.hasSubscription). Selecting a mode callsguidedControlService.enable()before activating frontend controls (manual-control-dialog.component.ts). enable()/disable()wait up to 3s for aguided_control_statusreply, then optimistically assume success (backwards compatibility).- Guided control auto-disables when the vehicle leaves
GUIDEDmode:currentMode$is watched and any non-GUIDEDmode flips the state off and hides the virtual joystick (guided-control.service.ts:64).
The dock / disconnect button
Gamepad button0 (the “X”/dock button) is the disconnect-all signal. MissionControlComponent.dock() and disconnectGamepad() synthesize a { '0': 1 } frame; the vehicle side acts on it, and the frontend enters docking mode (the video crosshair turns to the docking color). Any subsequent keyboard/gamepad/virtual input clears docking mode with a toast (gamepad.service.ts:88, :260).
Flight-control bar (mode / arm / takeoff / land)
MissionControlComponent (src/app/home/mission-control/mission-control.component.ts) is the primary flight-control strip. Its mode buttons are vehicle-type specific:
| Vehicle | Mode options |
|---|---|
rover | MANUAL, GUIDED, AUTO, RTL |
copter (quad) | POSHOLD, GUIDED, AUTO, RTL |
- The active mode is read from telemetry, not from the button press —
diagnosticsSubjectmessages are parsed withwebsocketService.getMode(level, 'Mode')so the highlighted button reflects the vehicle’s real state. - Mode / Arm / Takeoff / Land / Start-Mission all route through confirmation or altitude dialogs held in
AppStateService(e.g.setModeConfirmationDialogVisible,setTakeoffAltitudeDialogVisible) rather than firing REST calls directly from this component. The underlying REST actions (/drone/action/{arm|takeoff|set_mode|start_mission}) are documented on.Drone Management & Control Actions
drone actions - Takeoff vs Land share one button, toggled by
isArmedAndFlying()=systemStatus() === 'ACTIVE' && altitude() > 0.5. - GoTo (
onGoTo) setscontrolVehicleModeso the next map click sets a destination — the map-click dispatch lives on.Map & Live Vehicle Tracking
map tracking - A low-latency quick land also exists on the command channel:
VehicleCommandService.land()sends{ set_mode: 'LAND' }straight over redispad (vehicle-command.service.ts:145).
Configuration
These come fromsrc/environments/environment.ts and are the knobs most relevant to this screen:
| Key | Purpose | Value |
|---|---|---|
janusGatewayUrl | Janus WebRTC SFU signaling URL | wss://prod.skyhub.ai:8188 |
janusIceServers | ICE/STUN servers for WebRTC | ['stun:stun.l.google.com:19302'] |
ws_proxy | redispad base for commands + ArUco | wss://prod.skyhub.ai:7070 |
production | Toggles verbose video-stream console logging | false in dev |
Media (video/audio) does not flow through and .
janusGatewayUrl directly — UDP relays via the jumphost in the 20000–21100 range, and the on-drone GStreamer pipeline publishes to WHIP first. See WHIP Ingest Server
WHIP
Video Streaming (RTSP → WHIP/WebRTC)
on-drone video streaming

