.bin DataFlash log is parsed into flight statistics; and a formatted HTML report (with a downloadable ZIP archive) is emailed to the operator. The whole pipeline is driven by VPN-authenticated callbacks from the on-drone gamepad service plus a handful of JWT-authenticated UI endpoints.
Three collaborating services, all wired as singletons in . The HTTP request/response shapes are catalogued on .
src/application/app.py:ExecutionService(src/service/execution_service.py) — flight lifecycle + the wideupdate_flight_stats()sink.LogAnalysisService(src/service/log_analysis_service.py) — pymavlink parsing of ArduPilot.binlogs into aFlightStatsdataclass.ReportService(src/service/report_service.py) — HTML/text report generation + SMTP send, with a 2-worker background thread pool for archive + report.
ArchiveService) and S3 asset plumbing (AssetService) are covered on S3 Assets, HLS Video & Execution Archives
the assets & archives page
Executions, Assets & Reports API
the API reference
The post-flight lifecycle
The drone (specifically the on-drone gamepad service, over the WireGuard VPN) drives the lifecycle through callbacks authenticated by source IP — seeVPN IP Authentication & Jumphost
VPN middleware
generic_drone_service.get_drone_by_ip(request.vpn_ip).
Arm -> start_execution
POST /executions/start calls ExecutionService.start_execution(user_id, drone_id, mission_id). mission_id falls back to the drone’s currently loaded drone.mission_id (NULL for manual flights). A MissionExecution row is inserted with status = in_progress and started_at = now(UTC).Disarm -> complete_execution
POST /executions/{id}/complete verifies ownership, then complete_execution(execution_id, status, notes) sets ended_at, computes duration_seconds = int(ended_at - started_at), and applies the final status (completed / aborted / error). The route then branches on pending uploads (see below).Log upload -> set_log_asset + async analysis
After the gamepad uploads the
.bin log to S3, POST /executions/{id}/log verifies the asset belongs to the same drone/user in the route (src/routes/execution_routes.py:319) before linking it via set_log_asset(execution_id, asset_id, user_id=...) (which verifies the execution belongs to user_id) and fires _analyze_log_async(execution_id, s3_key) on a 4-worker ThreadPoolExecutor (src/routes/execution_routes.py:32, prefix log_analysis).The report-dispatch decision
Reports are deliberately deferred until all asset uploads finish, because the ZIP archive and the email’s “Download all assets” link should include everything captured during the flight.- No pending assets at completion →
report_service.enqueue_archive_and_report(execution_id, app)submits to the background pool and returns immediately (src/routes/execution_routes.py:265). - Assets still uploading →
execution_service.mark_report_pending(execution_id). Later,asset_routes._dispatch_pending_report_if_ready()(src/routes/asset_routes.py:16) runs after each upload completes; whenreport_pendingis set andget_pending_asset_count() == 0, it callsreport_service.send_report()synchronously in the upload request thread and thenclear_report_pending().
ExecutionService
Beyond the lifecycle methods above,ExecutionService is the query and mutation surface for executions.
| Method | Purpose |
|---|---|
get_execution(id, user_id) | Ownership-scoped single fetch. |
get_execution_with_assets(id, user_id) | Detail dict: execution + drone_name/mission_name + asset_counts + presigned asset URLs + log_asset. |
get_executions_by_drone(user_id, drone_id, limit, offset) | Paginated summaries for one drone. |
get_executions_by_user(user_id, limit, offset) | Paginated summaries across all drones. |
get_in_progress_execution(user_id, drone_id) | The open flight, if any. |
attach_asset_to_execution(asset_id, execution_id) | Link an asset row. |
set_log_asset(id, asset_id, user_id=None) | Set log_asset_id (ownership-checked when user_id given). |
has_pending_assets(id) / get_pending_asset_count(id) | Count assets still in pending status. |
mark_report_sent / mark_report_pending / clear_report_pending | Report-state flags on the row. |
delete_executions_by_drone(id) / delete_executions_by_user(id) | Bulk cleanup. |
_get_asset_counts_batch(execution_ids) (src/service/execution_service.py:555) runs one GROUP BY execution_id, asset_type and maps results into {"images", "videos", "logs"} per execution, so listing pages avoid N+1 queries.
update_flight_stats — the wide sink
update_flight_stats(execution_id, **~34 optional kwargs) (src/service/execution_service.py:231) is the single place log-derived numbers land in the DB. Every argument defaults to None and is written only when it is not None, so a partial re-analysis never clobbers a previously-populated column. It stamps log_analyzed_at and commits.
To add a new flight statistic, you must touch four places: add the field to the FlightStats dataclass, extract it in analyze_log_data(), add the keyword + column-assignment to update_flight_stats(), and add the DB column (via a migration) on MissionExecution. The route helper _persist_flight_stats() (src/routes/execution_routes.py:54) forwards fields verbatim, so also add it there.
LogAnalysisService
analyze_log_from_s3(s3_key) downloads the .bin from settings.ASSET_BUCKET and hands the bytes to analyze_log_data(bytes). Because pymavlink needs a filesystem path, the bytes are written to a NamedTemporaryFile(suffix=".bin"), opened with mavutil.mavlink_connection, and the temp file is always removed in a finally block. The parser walks messages with recv_match(blocking=False) and aggregates by message type:
| DataFlash message | Fields read | Populates |
|---|---|---|
MSG / EV | "arming/disarming motors" text; Id 10=armed, 11=disarmed | flight_time_seconds, armed_count, takeoff_time, landing_time |
GPS | Lat, Lng, Spd, NSats, HDop, Status | total_distance_meters (haversine), avg/max_speed_ms, gps_avg/min_satellites, gps_avg_hdop, gps_fix_type(_name) |
CTUN | Alt, ThH | max_altitude_meters, hover_throttle_pct (0-1 → %) |
POS | RelHomeAlt | max_altitude_meters (relative) |
BAT | Volt, EnrgTot, RemPct | battery_start/min/end_voltage, battery_energy_wh, battery_remaining_pct |
VIBE | VibeX/Y/Z, Clip0/1/2 | vibe_*_avg/max, clip_0/1/2 (cumulative max) |
ERR | Subsys, ECode | error_events, error_count |
MODE | Mode, Rsn | mode_changes, mode_change_count, failsafe_count |
RCOU | C1..C8 PWM | motor_balance_avg/max/min, motor_count |
ESC | Instance, RPM | motor-balance fallback (used in SITL when there is no RCOU) |
COPTER_MODE_NAMES first, then ROVER_MODE_NAMES; failsafe_count counts mode changes whose reason code is >= 3 (see MODE_REASON_NAMES). Error subsystems use ERR_SUBSYS_NAMES, GPS fix types use GPS_FIX_TYPE_NAMES (0=NO_GPS … 6=RTK_FIXED). Motor arrays store None for inactive channels so a genuine zero output is distinguishable from “no data”.
Robustness — the parser never crashes the caller
Robustness — the parser never crashes the caller
Log files from the field are frequently truncated or corrupted, so the analyzer is defensive:
contextlib.redirect_stderrswallows pymavlink’s noisy “bad header” warnings (logged at DEBUG).- Per-message parse errors are counted and skipped; after 1000 errors it stops early.
- It catches
ImportError(pymavlink missing),SystemExit(pymavlink can callsys.exit()on badly corrupted files), and any otherException, returningNonerather than raising.
_analyze_log_async and the synchronous analyze_execution_log route — also catches SystemExit around the call, so a corrupt log cannot take down a worker thread or the request.Gotcha — takeoff_time / landing_time are boot-relative, not UTC
Gotcha — takeoff_time / landing_time are boot-relative, not UTC
The
FlightStats.takeoff_time / landing_time field docstrings say “UTC timestamp”, but _format_timestamp() (src/service/log_analysis_service.py:779) returns seconds since system boot formatted as "X.XXs" — absolute UTC is not recoverable from a TimeUS-only message. Treat these as elapsed-time strings, not wall-clock times. The authoritative wall-clock times are MissionExecution.started_at / ended_at, set by the lifecycle callbacks.Manual re-analysis
POST /executions/{id}/analyze-log (JWT) runs analyze_log_from_s3 synchronously (the user is waiting) and returns stats.to_dict(). GET /executions/{id}/flight-stats reads the already-persisted columns back off the execution row.
ReportService
ReportService renders and delivers the mission report. All heavy lifting flows through _send_report_impl().
generate_report(execution_id, execution=None)builds the data dict. User-controlled fields —mission_name,drone_name,pilot_email,notes— are run throughhtml.escape()to prevent XSS in the email body. It also fetches the archive download URL viaArchiveService.get_archive_download_url()whenarchive_s3_keyis set.generate_html_report()produces a self-contained, inline-CSS HTML email: a status-coloured header, Flight Performance and Battery stat rows, a System Health block (GPS / vibration / motors / hover-throttle, each bucketed into Excellent → Poor with threshold logic), an Events block (up to 5 errors + a mode-change/failsafe summary), an asset-count grid, and a “Download ZIP” call-to-action (valid 7 days).generate_text_report()is the plaintext equivalent.get_report_recipients(execution)returnsuser.get_report_recipients()— the user’s primary email first, then de-duplicatedreport_extra_emails(src/models/user.py:36). Recipients are managed viaGET/PUT /user/report-emails.
The background pipeline
src/service/report_service.py
enqueue_archive_and_report(execution_id, app) submits a task that runs inside app.app_context(): it calls ArchiveService.generate_archive() first (so the email’s download link resolves), then _send_report_impl(). Because ReportService and ArchiveService reference each other, the archive service is imported lazily via _get_archive_service() to break the import cycle — preserve that indirection.
_send_report_impl(execution_id, generate_archive=False) loads the execution with all relationships in one query, resolves recipients, optionally (re)generates the archive, builds the subject Mission Report: {mission_name} - {drone_name}, and sends to every recipient through EmailService.send_email() (SMTP).
Sending to an arbitrary address
POST /executions/{id}/send-report {email} → send_report_to_email(execution_id, email) regenerates the archive only if one doesn’t already exist, then sends to that single validated address.
Gotchas for future editors
Thumbnail S3 path is inconsistent between services
Thumbnail S3 path is inconsistent between services
ExecutionService._enrich_asset_with_urls (src/service/execution_service.py:540) presigns image thumbnails at thumbnails/{s3_key}, but AssetService writes thumbnails to drone/{drone_id}/thumbnail/{file}. These do not point at the same object — the enriched thumbnail_download_url from the executions detail endpoint can 404. Any refactor unifying asset URLs must reconcile the two schemes. See S3 Assets & Archives
assets & archives
A GPS quality label never matches
A GPS quality label never matches
The HTML report’s GPS bucketing checks
gps_fix in ("RTK_FIXED", "RTK_FLOAT", "3D_DGPS"), but GPS_FIX_TYPE_NAMES maps code 4 to "DGPS" (not "3D_DGPS"), so the 3D_DGPS branch is dead. Cosmetic only — it just downgrades a DGPS fix from “Good” to “Fair” in the email.Two independent thread pools
Two independent thread pools
Log analysis and report/archive generation use separate module-global pools:
log_analysis (4 workers, in execution_routes.py) and archive_gen (2 workers, in report_service.py). Both have unbounded task queues. Under a burst of simultaneous landings, archive/report work serializes behind 2 workers while analysis has 4 — keep this in mind when tuning concurrency.Endpoint map
All paths below (and the inline paths elsewhere on this page) are relative to the/api/v1 prefix that every blueprint is registered under (src/main.py:224) — e.g. the real request path for /executions/start is POST /api/v1/executions/start.
- Drone callbacks (VPN IP auth)
- UI (JWT Bearer)
Authenticated by WireGuard source IP (
@check_vpn_ip); drone resolved via get_drone_by_ip(request.vpn_ip).| Method | Path | Trigger |
|---|---|---|
| POST | /executions/start | On ARM |
| POST | /executions/{id}/complete | On disarm/land — enqueues or defers report |
| POST | /executions/{id}/log | Link .bin asset + async analysis |
| GET | /executions/current | Poll the open execution |
Related reading: S3 Assets, HLS Video & Execution Archives (ZIP generation and asset URLs), DroneControlService & Rosbridge Dispatch (the arm/disarm commands that bracket a flight), Database Schema Overview (the
mission_execution columns), and the Executions API reference.
