UAV Log Database Specs Document
Overall concept: A software library providing management of a local collection of UAV logs. Preferably as a Python module.

Applications

Standalone

As a console program (or with rudimentary UI), it could search through multiple logs, stored in a tree folder structure, return files satisfying search criteria and post basic descriptive information.

Used through its API

The Log Database library could serve other applications as:
  1. Plain log storage management: Provide a log catalogue and serve log data for logbook applications such as DRONEE.
  1. Log analysis: Quickly serve large data series for more involved log examination algorithms, akin to Maverick and Log Analyzer.
  1. One-off research: Provide a useful API for log data, to enable one-off, tailor-made analyses on a temporary log collection. Such, on-the-fly investigations could use the API to make inferences on mean flight time across UAV classes, which APM version had the most crashes or generate diagnostic signals and test their accuracy on real logs.

High-Level Requirements

  • Database-only functionality
  • Its core is about storing logs and searching and serving log data, not processing them.
  • Local storage
  • The logs are local to the PC and databases are portable by physical means (USB sticks etc).
  • Log-agnostic
  • Not specific to ardupilot logs (dataflash or tlogs)
  • Metadata-based
  • Since any log file format is admissible, vital/useful information may not be stored inside them. Metadata fields are useful in accepting and storing external quantities (notes, pilot, platform, events, etc)
  • Extensible
  • It provides an API for implementing log-processing plugins.

Specifications

Standalone Application

  • Parse logs folder
  • Populate Metadata files, and only if not previously done soe
  • Accept log queries on metadata
  • Answer queries as file paths (on console or by writing on file)
  • Provide summary of log library

API

Queries/Filters

Parsers
  • Total number of logs
  • Total flight time
  • Number of different vehicles
  • (Solidify this list)
Getters (for any individual log):
  • Any quantity timeseries, properly timestamped
  • Metadata, useful for filtering and cross-correlation, as said before (UAV ID, battery ID, pilot, date, etc).
  • The whole log, for database migration but with some filters applied (possibly based on the aforementioned metadata, e.g. get me all the logs of this FW version).
  • (Solidify this list)

Supported Log Types

  • ArduPilot Dataflash
  • MAVLink telemetry
  • Other UAV logs (?)

Structure

UML-like Class Diagram

Technologies Used