Concrete SQLite-backed BagWriter implementation with WAL caching and batch commits. More...

#include <vdb_writer.h>

Inheritance diagram for vlink::VDBWriter:

Collaboration diagram for vlink::VDBWriter:

Public Member Functions
	VDBWriter (const std::string &path, const Config &config={})
	Opens or creates a SQLite bag file for recording. More...

	~VDBWriter () override
	Commits any cached writes and closes the SQLite database. More...

void	register_split_callback (SplitCallback &&callback, bool before) override
	Registers a callback invoked at each file-split boundary. More...

void	register_schema_callback (SchemaCallback &&callback) override
	Registers a resolver that maps a serialisation-type string to `SchemaData`. More...

bool	push_schema (const SchemaData &schema_data, bool immediate=false) override
	Embeds `schema_data` in the `schemas` table for offline introspection. More...

bool	is_dumping () const override
	Returns the current value of the internal dumping flag. More...

bool	is_split_mode () const override
	Reports whether split-file recording is active. More...

int	get_split_index () const override
	Returns the index of the split part currently being written. More...

Public Member Functions inherited from vlink::BagWriter
	BagWriter (const std::string &path, const Config &config={})
	Constructs the base writer and opens the output file. More...

virtual	~BagWriter ()
	Halts the loop, flushes pending writes and closes the file. More...

virtual void	bind_plugin_interface (const std::shared_ptr< BagPluginInterface > &plugin_interface)
	Attaches a custom frame-forwarding plugin to this writer. More...

virtual void	clear_plugin_interface ()
	Detaches the currently bound plugin, if any. More...

int64_t	push (const Frame &frame, bool immediate=false)
	Records a single frame to the bag. More...

BagWriter &	operator<< (const Frame &frame)
	Streaming shorthand for `push(frame)`. More...

BagWriter &	operator<< (const SchemaData &schema_data)
	Streaming shorthand for `push_schema(schema_data)`. More...

bool	fail () const noexcept
	Returns whether a previous `operator<<` observed a write failure. More...

	operator bool () const noexcept
	Reports whether the streaming write state is still good (no latched failure). More...

void	clear () noexcept
	Clears a latched `fail()` state so streaming writes can resume being observed. More...

virtual void	set_url_loss (const std::string &url, double loss)
	Records the expected loss ratio for `url` as bag metadata. More...

Public Member Functions inherited from vlink::MessageLoop
	MessageLoop ()
	Constructs a loop with the default `kNormalType` queue. More...

	MessageLoop (Type type)
	Constructs a loop with the given queue type. More...

virtual	~MessageLoop ()
	Destructor; requests quit and joins the dispatcher thread if needed. More...

void	set_name (const std::string &name)
	Sets a human-readable name visible to profiling tools. More...

const std::string &	get_name () const
	Returns the loop name set via `set_name`. More...

Type	get_type () const
	Returns the queue type this loop was constructed with. More...

Strategy	get_strategy () const
	Returns the active back-pressure strategy. More...

void	set_strategy (Strategy strategy)
	Replaces the back-pressure strategy. More...

void	register_begin_handler (Callback &&callback)
	Registers a callback fired once at loop thread startup. More...

void	register_end_handler (Callback &&callback)
	Registers a callback fired once when the loop thread exits. More...

void	register_idle_handler (Callback &&callback)
	Registers a callback fired every time the queue becomes empty. More...

bool	run ()
	Runs the loop on the calling thread until `quit` is requested. More...

bool	async_run ()
	Starts the loop on a new background thread. More...

bool	spin ()
	Alias of `run` blocking the calling thread. More...

bool	spin_once (bool block=true)
	Processes one batch of pending tasks and timers on the calling thread. More...

bool	quit (bool force=false)
	Requests the loop to exit. More...

bool	wait_for_quit (int ms=Timer::kInfinite, bool check=true)
	Waits until the loop has fully exited. More...

bool	post_task (Callback &&callback)
	Posts a task for execution on the loop thread. More...

TaskHandle	post_task_handle (Callback &&callback, const PostTaskOptions &options={})
	Tracked variant of `post_task` returning a `TaskHandle`. More...

bool	post_task_with_priority (Callback &&callback, uint16_t priority)
	Posts a task with an explicit priority on a `kPriorityType` loop. More...

TaskHandle	post_task_with_priority_handle (Callback &&callback, uint16_t priority, const PostTaskOptions &options={})
	Tracked variant of `post_task_with_priority` returning a `TaskHandle`. More...

template<typename CallbackT , typename = std::enable_if_t<!std::is_convertible_v<CallbackT, Schedule::RetCallback>>>
Schedule::Status	exec_task (const Schedule::Config &config, CallbackT &&callback)
	Schedules a `void-returning` callable and returns a chainable `Schedule::Status`. More...

template<typename CallbackT , typename = std::enable_if_t<std::is_convertible_v<CallbackT, Schedule::RetCallback>>>
Schedule::RetStatus	exec_task (const Schedule::Config &config, CallbackT &&callback)
	Schedules a `bool-returning` callable and returns a chainable `Schedule::RetStatus`. More...

bool	wakeup ()
	Wakes the loop thread if it is suspended in its idle wait. More...

void	reset_lockfree_capacity ()
	Recreates the lock-free queue, clearing all queued tasks and counters. More...

bool	is_running () const
	Reports whether the loop is currently running. More...

bool	is_ready_to_quit () const
	Reports whether `quit` has been requested and the loop is winding down. More...

bool	is_busy () const
	Reports whether the loop is currently executing a task. More...

size_t	get_task_count () const
	Returns the current pending task count. More...

virtual bool	wait_for_idle (int ms=Timer::kInfinite, bool check=true)
	Waits until the loop has drained its queue and is not executing a task. More...

virtual size_t	get_max_timer_count () const
	Returns the maximum number of timers that can be attached. More...

virtual uint32_t	get_max_elapsed_time () const
	Returns the maximum allowed task execution time in milliseconds. More...

virtual bool	is_in_same_thread () const
	Reports whether the calling thread is owned by this loop. More...

std::shared_ptr< AliveState >	get_alive_state () const
	Returns the shared lifetime flag used by cross-thread bridges. More...

template<class FunctionT , class... ArgsT, typename ResultT = std::invoke_result_t<FunctionT, ArgsT...>>
std::future< ResultT >	invoke_task (FunctionT &&function, ArgsT &&... args)
	Dispatches a callable to the loop thread and returns a `std::future` for the result. More...

template<class FunctionT , class... ArgsT, typename ResultT = std::invoke_result_t<FunctionT, ArgsT...>>
std::future< ResultT >	invoke_task_with_priority (FunctionT &&function, uint16_t priority, ArgsT &&... args)
	Priority variant of `invoke_task`; requires a `kPriorityType` loop. More...

Protected Member Functions
int64_t	record (const Frame &frame, bool immediate) override

int64_t	get_record_timestamp () const override

size_t	get_max_task_count () const override
	Returns the maximum queue depth. More...

void	on_begin () override
	Hook invoked once on the loop thread before the first task runs. More...

void	on_end () override
	Hook invoked once on the loop thread after the last task runs. More...

Protected Member Functions inherited from vlink::BagWriter
std::string	convert_recorded_url (const std::string &url) const

std::vector< std::string >	recorded_urls_for_origin (const std::string &url) const

std::string	recover_recorded_url (const std::string &url) const

void	get_url_meta (const std::string &url, const std::string &ser, int &url_index, int &ser_index) const

void	get_url_meta (int url_index, int ser_index, std::string &url, std::string &ser) const

std::mutex &	sample_mutex ()

std::unordered_map< std::string, double > &	url_loss_map_ref ()

std::unordered_map< std::string, double > &	total_url_loss_map_ref ()

void	flush_plugin ()

void	detach_plugin ()

Protected Member Functions inherited from vlink::MessageLoop
virtual void	on_idle ()
	Hook invoked on the loop thread each time the queue becomes empty. More...

virtual void	on_task_changed (Callback &&callback, uint32_t start_time)
	Dispatches a ready task on the loop thread. More...

virtual void	on_task_timeout (Callback &&callback, uint32_t elapsed_time)
	Hook invoked when a task exceeds `get_max_elapsed_time()`. More...

Additional Inherited Members
Public Types inherited from vlink::BagWriter
enum	CompressType : uint8_t { kCompressNone = 0 , kCompressAuto = 1 , kCompressZstd = 2 , kCompressLz4 = 3 , kCompressLzav = 4 }
	Compression codec selector understood by the writer backends. More...

using	SplitCallback = MoveFunction< void(int split_index, const std::string &split_filename)>
	Notification fired when the writer rotates to a new split file. More...

using	SchemaCallback = MoveFunction< SchemaData(const std::string &ser_type, SchemaType schema_type)>
	Schema resolver used by the writer when a previously unseen URL is recorded. More...

using	SystemClock = std::chrono::time_point< std::chrono::system_clock, std::chrono::milliseconds >
	System clock alias used when formatting timestamps into split file names. More...

Public Types inherited from vlink::MessageLoop
enum	Type : uint8_t { kNormalType = 0 , kLockfreeType = 1 , kPriorityType = 2 }
	Internal queue implementation type. More...

enum	Strategy : uint8_t { kOptimizationStrategy = 0 , kPopStrategy = 1 , kBlockStrategy = 2 }
	Back-pressure strategy applied when the bounded queue is at capacity. More...

enum	Priority : uint16_t { kNoPriority = 0 , kLowestPriority = 1 , kTimerPriority = 50 , kNormalPriority = 100 , kHighestPriority = std::numeric_limits<uint16_t>::max() }
	Built-in priority levels for `kPriorityType` loops; higher values dispatch first. More...

using	Callback = MoveFunction< void()>
	Callback type for tasks and event handlers. More...

Static Public Member Functions inherited from vlink::BagWriter
static std::shared_ptr< BagWriter >	create (const std::string &path, const Config &config={})
	Builds the concrete writer matching the extension of `path`. More...

static std::shared_ptr< BagWriter >	filter_get (const std::string &path)
	Returns the cached writer for `path`, lazily creating and starting one. More...

static BagWriter *	global_get ()
	Returns the singleton writer driven by the `VLINK_BAG_PATH` environment variable. More...

Static Protected Member Functions inherited from vlink::BagWriter
static const std::string &	get_default_tag_name ()

static const std::string &	get_default_app_name ()

static SchemaPluginInterface *	get_schema_interface ()

static int32_t	get_default_timezone_diff ()

static std::string_view	convert_action (ActionType type)

static std::string	get_format_date (SystemClock *current=nullptr, bool file_format=false)

Detailed Description

Concrete SQLite-backed BagWriter implementation with WAL caching and batch commits.

Prefer BagWriter::create() for format-agnostic instantiation; instantiate this class directly only when a SQLite-specific feature is required.

Constructor & Destructor Documentation

◆ VDBWriter()

vlink::VDBWriter::VDBWriter	(	const std::string &	path,
		const Config &	config = `{}`
	)

explicit

Opens or creates a SQLite bag file for recording.

Parameters

path	Filesystem path of the .vdb or .vdbx target.
config	Recording configuration (split policy, compression, cache thresholds).

◆ ~VDBWriter()

vlink::VDBWriter::~VDBWriter ( )

override

Commits any cached writes and closes the SQLite database.

Member Function Documentation

◆ get_max_task_count()

size_t vlink::VDBWriter::get_max_task_count ( ) const

overrideprotectedvirtual

Returns the maximum queue depth.

Returns: kMaxTaskSize (10000) by default.

Reimplemented from vlink::MessageLoop.

◆ get_record_timestamp()

int64_t vlink::VDBWriter::get_record_timestamp ( ) const

overrideprotectedvirtual

Implements vlink::BagWriter.

◆ get_split_index()

int vlink::VDBWriter::get_split_index ( ) const

overridevirtual

Returns the index of the split part currently being written.

Returns: Zero-based split index.

Implements vlink::BagWriter.

◆ is_dumping()

bool vlink::VDBWriter::is_dumping ( ) const

overridevirtual

Returns the current value of the internal dumping flag.

Returns: true while messages are actively being persisted.

Implements vlink::BagWriter.

◆ is_split_mode()

bool vlink::VDBWriter::is_split_mode ( ) const

overridevirtual

Reports whether split-file recording is active.

Returns: true when emitting a .vdbx manifest with multiple parts.

Implements vlink::BagWriter.

◆ on_begin()

void vlink::VDBWriter::on_begin ( )

overrideprotectedvirtual

Hook invoked once on the loop thread before the first task runs.

Subclasses override to perform per-thread initialisation.

Reimplemented from vlink::MessageLoop.

◆ on_end()

void vlink::VDBWriter::on_end ( )

overrideprotectedvirtual

Hook invoked once on the loop thread after the last task runs.

Subclasses override to perform per-thread cleanup.

Reimplemented from vlink::MessageLoop.

◆ push_schema()

bool vlink::VDBWriter::push_schema	(	const SchemaData &	schema_data,
		bool	immediate = `false`
	)

overridevirtual

Embeds schema_data in the schemas table for offline introspection.

Parameters

schema_data	Schema descriptor to embed.
immediate	`true` merges synchronously; `false` enqueues the write.

Returns: false only when an immediate merge failed.

Implements vlink::BagWriter.

◆ record()

int64_t vlink::VDBWriter::record	(	const Frame &	frame,
		bool	immediate
	)

overrideprotectedvirtual

Implements vlink::BagWriter.

◆ register_schema_callback()

void vlink::VDBWriter::register_schema_callback ( SchemaCallback && callback )

overridevirtual

Registers a resolver that maps a serialisation-type string to SchemaData.

Parameters

callback Function consulted before inserting a URL row.

Implements vlink::BagWriter.

◆ register_split_callback()

void vlink::VDBWriter::register_split_callback	(	SplitCallback &&	callback,
		bool	before
	)

overridevirtual

Registers a callback invoked at each file-split boundary.

Parameters

callback	Receives (split_index, filename) before or after the split.
before	`true` fires before the new file is opened; `false` fires after.

Implements vlink::BagWriter.

The documentation for this class was generated from the following file:

include/vlink/extension/vdb_writer.h

Public Member Functions

Protected Member Functions

Additional Inherited Members

Detailed Description

Constructor & Destructor Documentation

◆ VDBWriter()

◆ ~VDBWriter()

Member Function Documentation

◆ get_max_task_count()

◆ get_record_timestamp()

◆ get_split_index()

◆ is_dumping()

◆ is_split_mode()

◆ on_begin()

◆ on_end()

◆ push_schema()

◆ record()

◆ register_schema_callback()

◆ register_split_callback()