bag_writer.h File Reference

Abstract VLink bag recorder with split, compression, schema embedding and a global hook. More...

#include <chrono>
#include <cstdint>
#include <memory>
#include <mutex>
#include <string>
#include <string_view>
#include <unordered_map>
#include <unordered_set>
#include <vector>
#include "../base/functional.h"
#include "../base/macros.h"
#include "../base/message_loop.h"
#include "../impl/types.h"

Include dependency graph for bag_writer.h:

This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Classes
class	vlink::BagWriter
	Asynchronous VLink message recorder built on top of MessageLoop. More...
struct	vlink::BagWriter::Config
	Recording behaviour, split policy and resource budgets. More...

Namespaces
	vlink

Detailed Description

Abstract VLink bag recorder with split, compression, schema embedding and a global hook.

BagWriter is the polymorphic base for VLink offline recording. It exposes a push() entry point, which either enqueues serialised messages onto a private MessageLoop or persists them synchronously according to Config::sync_mode. Two backends ship with VLink:

• VDBWriter for SQLite-backed .vdb / .vdbx containers; default codec is LZAV for kCompressAuto and kCompressLzav selectors.
• VCAPWriter for MCAP-format .vcap / .vcapx containers; kCompressAuto and kCompressZstd select Zstandard when Zstd support is compiled in.

Default asynchronous writer state machine:

*                async_run()                push()               close()/dtor
*   +---------+ ----------> +-----------+ ---------> +---------+ ----------> +---------+
*   |  Open   |             |  Running  | <--------- | Pending |             |  Closed |
*   +---------+             +-----------+   ack      +---------+             +---------+
*                                ^                                              ^
*                                |                                              |
*                                +--- split_by_size / split_by_time -- rotate --+
* 

On-disk layout produced by the writers:

*   +---------+----------------+---------------+----------------+--------+
*   | Header  |  URL index     |  Schema index | Message stream | Footer |
*   +---------+----------------+---------------+----------------+--------+
*      tag        url_metas       schema_data       payloads      finalisation
*      app
*      timezone
* 

Feature highlights:

• Writer-wide asynchronous or synchronous record policy selected by Config::sync_mode.
• File splitting by byte size and/or by wall-clock interval.
• Optional WAL mode for SQLite crash resilience.
• URL-level loss reporting via set_url_loss().
• Schema embedding through push_schema() for offline introspection.
• Process-global writer triggered by the VLINK_BAG_PATH environment variable.

Example

vlink::BagWriter::Config cfg;
cfg.compress      = vlink::BagWriter::kCompressAuto;
cfg.split_by_size = 1024LL * 1024LL * 512;            // 512 MiB per split
 
auto writer = vlink::BagWriter::create("/data/drive_log.vdb", cfg);
writer->async_run();
 
vlink::Frame frame;
frame.timestamp   = -1;                       // < 0 => writer auto-assigns from its clock (0 is verbatim)
frame.url         = "dds://camera/front";
frame.ser_type    = "demo.proto.Image";
frame.schema_type = vlink::SchemaType::kProtobuf;
frame.action_type = vlink::ActionType::kPublish;
frame.data        = bytes;
writer->push(frame);
writer->wait_for_idle();
writer->quit();
writer->wait_for_quit();
writer->close();
if (writer->fail()) { handle_recording_error(); }

Global writer

// Set VLINK_BAG_PATH=/data/global.vdb before process launch.
if (auto* gw = vlink::BagWriter::global_get(); gw != nullptr) {
  vlink::Frame frame;
  frame.timestamp   = -1;                       // < 0 => auto-assign (0 would be recorded verbatim)
  frame.url         = "intra://debug";
  frame.ser_type    = "raw";
  frame.schema_type = vlink::SchemaType::kRaw;
  frame.action_type = vlink::ActionType::kPublish;
  frame.data        = bytes;
  gw->push(frame);
}

Note

push() is thread-safe. Config::sync_mode selects synchronous writes for the writer's entire lifetime; otherwise writes are queued on the recording loop.