LLive Reference Manual & Live Capture Guide

The LLive class is the live hardware ingestion and network streaming capture engine of the now2sdk framework, derived from the LObject class. It manages real-time audio and video capture from:

1. Physical Video Capture Cards: Blackmagic Design (BMD) DeckLink hardware inputs.
2. Network Streams: Network Device Interface (NDI) sources detected on the local subnetwork.
3. Internal Synthesizers: Generates SMPTE test pattern cards and sine wave audio signals.

Like LFile, LLive processes capture feeds through an internal FFmpeg-based pipeline (providing real-time resizing, color conversion, volume gain adjustments, and custom LFilter integration) and pushes processed frames to registered downstream sinks.

1. Device, Channel, and Format Ingestion Architecture

Ingestion configurations are split into a three-step hierarchy: Device Selection -> Connection / Channel Selection -> Video Format Selection.

graph TD
    A[LLive Ingestion Engine] --> B(Device Selection)
    B -->|Index < BMD Devices| C[DeckLink Capture Card]
    B -->|Index == BMD Devices| D[NDI Receiver]
    B -->|Index == BMD Devices + 1| E[Test Signal Generator]
    
    C --> F(Channel Connection)
    F -->|Set SDI, HDMI, etc.| G[DeckLink Input Connection]
    
    D --> H(NDI Stream Source)
    H -->|Set NDI Feed Name| I[NDI Network Capture]
    
    G --> J(Display Mode / Format)
    I --> J
    E --> J
    
    J -->|Set Hardware Format Index| K[Frame Ingest Thread]

Ingestion Components:

• LLive Ingest Loop: Multi-threaded capture threads (decklinkLoop, ndiLoop, and m_testThread) read raw hardware buffers asynchronously, converting them into standard AVFrame layouts.
• Auto Signal Resolution Norm: Supports direct GPU bypass mode (gpu = true) or CPU scaling conversions to output target width, height, frame rate, and pixel formats via videoFormatProps and audioFormatProps.
• Embedded Timecode Decoders: Decodes broadcast timecodes from sources (e.g., NDI metadata, or BMD RP188 broadcast SMPTE timecode packets) and embeds them directly in the frame headers.

2. API Reference & Ingestion Methods

📌 Device Management

void DeviceGetCount(int& count);

System query listing active live inputs and sets count. $\text{Total Ingest Count} = \text{DeckLink Cards Count} + 2\ (\text{NDI Receiver} + \text{Test Signal})$

void DeviceGetByIndex(int index, std::string& name, std::string& desc);

Queries identification parameters for a specific device index:

• If index is less than the DeckLink count: returns the hardware model name (e.g. "DeckLink Quad 2").
• If index equals the DeckLink count: returns "NDI Receiver".
• If index equals the DeckLink count + 1: returns "Test Signal".

void DeviceSet(int index);

Selects the active input device. If it points to a DeckLink index, allocates IDeckLink and IDeckLinkInput interface resources.

📌 Channel / Connection Management

void DeviceChannelGetCount(int deviceIndex, int& count);

Retrieves the number of ingestion lines/channels available for the selected device index:

• DeckLink: Returns the number of physical connections (SDI, HDMI, Component, Composite, S-Video) supported by the card attributes.
• NDI: Searches the network and returns the number of active NDI stream sources.
• Test Signal: Returns 1.

void DeviceChannelGetByIndex(int deviceIndex, int channelIndex, std::string& name, std::string& desc);

Queries naming variables for a channel index (e.g. "SDI", "HDMI", or NDI stream name "WORKSTATION-1 (OBS)").

void DeviceChannelSet(int channelIndex);

Binds the active connection channel.

• For DeckLink cards: Queries IDeckLinkConfiguration and sets the hardware multiplexer to target connections:
  • bmdVideoConnectionSDI
  • bmdVideoConnectionHDMI
  • bmdVideoConnectionComponent
  • bmdVideoConnectionComposite
  • bmdVideoConnectionSVideo

📌 Format Management

void DeviceFormatVideoGetCount(int deviceIndex, int channelIndex, int& count);

Queries the number of video standard profiles supported by the channel:

• DeckLink: Queries display mode iterators (IDeckLinkDisplayModeIterator) and appends "Auto Detection" at index 0.
• NDI: Returns 1 ("Auto Detect").
• Test Signal: Returns 1 ("1080p25").

void DeviceFormatVideoGetByIndex(int deviceIndex, int channelIndex, int formatIndex, std::string& name, videoFormatProps& props);

Retrieves standard profile parameters (name, width, height, and target frame rate) for the format index.

void DeviceFormatVideoSet(int formatIndex);

Binds the hardware capture engine to initialize on-air capture using the selected format index.

📌 Ingest Controls & Normalization

bool Start();

Spawns the background ingestion threads and starts hardware data capture. Returns true if successful.

void Stop();

Halts ingestion threads, ceases callbacks, and releases NDI/DeckLink hardware handles.

void setVideoFormat(const videoFormatProps& props);
void setAudioFormat(const audioFormatProps& props);

Sets output normalization target formats. Frames are scaled and audio is resampled to match these properties before passing to sinks.

void setVideoFilter(LFilter* filter);
void setAudioFilter(LFilter* filter);

Binds a custom video or audio LFilter graph to be dynamically parsed and integrated into the stream processing chain.

3. Properties & Configuration Options

All options listed below are configured at runtime via the setProps(const std::string& key, const std::string& value) interface:

4. Qt UI Device/Format Integration Example

This example demonstrates how to integrate LLive in a Qt UI. It handles scanning hardware, populating comboboxes for device selection, and dynamically updating connection channels and supported video formats:

mainwindow.cpp

#include "`mainwindow.h`"
#include "./ui_mainwindow.h"
#include "LLive.h"

MainWindow::MainWindow(QWidget *parent)
    : QMainWindow(parent),
      ui(new Ui::MainWindow),
      m_lLive(new LLive())
{
    ui->setupUi(this);

    // 1. Initial Device scanning on startup
    refreshLiveSources();

    // 2. Connect reactive UI signals for cascade dropdown updates
    connect(ui->deviceComboBox, QOverload<int>::of(&QComboBox::currentIndexChanged), 
            this, &MainWindow::onDeviceChanged);
            
    connect(ui->deviceChannelComboBox, QOverload<int>::of(&QComboBox::currentIndexChanged), 
            this, &MainWindow::onChannelChanged);
            
    connect(ui->deviceFormatComboBox, QOverload<int>::of(&QComboBox::currentIndexChanged), 
            this, &MainWindow::onFormatChanged);
            
    connect(ui->startLiveBtn, &QPushButton::clicked, 
            this, &MainWindow::onStartLiveClicked);
}

MainWindow::~MainWindow() {
    m_lLive->Stop();
    delete m_lLive;
    delete ui;
}

// ─── Step 1: Scan and Populate Devices ───
void MainWindow::refreshLiveSources() {
    ui->deviceComboBox->blockSignals(true);
    ui->deviceComboBox->clear();
    
    int deviceCount = 0;
    m_lLive->DeviceGetCount(deviceCount); // Queries DeckLink + NDI + Test generators
    
    for (int i = 0; i < deviceCount; i++) {
        std::string name, desc;
        m_lLive->DeviceGetByIndex(i, name, desc);
        ui->deviceComboBox->addItem(QString::fromStdString(name));
    }
    
    ui->deviceComboBox->blockSignals(false);
    
    if (deviceCount > 0) {
        ui->deviceComboBox->setCurrentIndex(0);
        onDeviceChanged(0);
    }
}

// ─── Step 2: Update Channel Connections Reactively ───
void MainWindow::onDeviceChanged(int index) {
    if (index < 0) return;

    // Halt current capture before switching devices
    int status = 0;
    m_lLive->statusGet(status);
    bool wasRunning = (status == 1);
    if (wasRunning) m_lLive->Stop();

    // Bind selected index to SDK
    m_lLive->DeviceSet(index);

    // Update Connection Channels Cascade
    ui->deviceChannelComboBox->blockSignals(true);
    ui->deviceChannelComboBox->clear();
    
    int channelCount = 0;
    m_lLive->DeviceChannelGetCount(index, channelCount); // Queries NDI streams or DeckLink line inputs
    
    for (int i = 0; i < channelCount; i++) {
        std::string name, desc;
        m_lLive->DeviceChannelGetByIndex(index, i, name, desc);
        ui->deviceChannelComboBox->addItem(QString::fromStdString(name));
    }
    
    ui->deviceChannelComboBox->blockSignals(false);
    
    if (channelCount > 0) {
        ui->deviceChannelComboBox->setCurrentIndex(0);
        onChannelChanged(0);
    }

    if (wasRunning) m_lLive->Start();
}

// ─── Step 3: Update Video Formats Cascade ───
void MainWindow::onChannelChanged(int index) {
    if (index < 0) return;
    int deviceIndex = ui->deviceComboBox->currentIndex();

    int status = 0;
    m_lLive->statusGet(status);
    bool wasRunning = (status == 1);
    if (wasRunning) m_lLive->Stop();

    // Bind channel/connection to hardware (SDI/HDMI setup)
    m_lLive->DeviceChannelSet(index);

    // Update Supported Format Presets
    ui->deviceFormatComboBox->blockSignals(true);
    ui->deviceFormatComboBox->clear();
    
    int formatCount = 0;
    m_lLive->DeviceFormatVideoGetCount(deviceIndex, index, formatCount);
    
    for (int i = 0; i < formatCount; ++i) {
        std::string name;
        videoFormatProps props;
        m_lLive->DeviceFormatVideoGetByIndex(deviceIndex, index, i, name, props);
        ui->deviceFormatComboBox->addItem(QString::fromStdString(name), i);
    }
    
    ui->deviceFormatComboBox->blockSignals(false);
    
    if (formatCount > 0) {
        ui->deviceFormatComboBox->setCurrentIndex(0);
        onFormatChanged(0);
    }

    if (wasRunning) m_lLive->Start();
}

// ─── Step 4: Bind Format Selection ───
void MainWindow::onFormatChanged(int index) {
    if (index < 0) return;
    
    int status = 0;
    m_lLive->statusGet(status);
    bool wasRunning = (status == 1);
    if (wasRunning) m_lLive->Stop();
    
    // Bind hardware format profile index to engine
    m_lLive->DeviceFormatVideoSet(index);
    
    if (wasRunning) m_lLive->Start();
}

// ─── Step 5: Start/Stop Ingest ───
void MainWindow::onStartLiveClicked() {
    int status = 0;
    m_lLive->statusGet(status);
    bool isRunning = (status == 1);

    if (!isRunning) {
        // Set properties before launching threads
        m_lLive->setProps("timecode", "true");
        m_lLive->setProps("timecodeSource", "1"); // Use source embedded timecode
        m_lLive->setProps("timecodeStart", "01:00:00:00");
        m_lLive->setProps("audio_gain", "85");
        m_lLive->setProps("gpu", "false"); // Use CPU filter pipeline
        
        int formatIndex = ui->deviceFormatComboBox->currentData().toInt();
        m_lLive->DeviceFormatVideoSet(formatIndex);
        
        if (m_lLive->Start()) {
            ui->startLiveBtn->setText("STOP LIVE INPUT");
        }
    } else {
        m_lLive->Stop();
        ui->startLiveBtn->setText("START LIVE INPUT");
    }
}

4. Professional Jitter-Free Audio Sync & Drop Detection

now2sdk v0.0.2 implements hardware-level capture synchronization and monitoring for high-reliability broadcast scenarios:

Jitter-Free DeckLink Audio-Video Sync

In standard capture setups, thread scheduling variations on the host CPU can cause video frames and audio packets to be stamped at different relative times, resulting in audio stuttering and lip-sync mismatch.

LLive solves this by:

• Capturing a single, high-precision unified system timestamp (steady_clock) inside the low-level hardware callback (VideoInputFrameArrived).
• Stamping both the video frame item and the incoming audio packet with the exact same time stamp.
• Propagating the synchronized PTS through the decoding pipeline, guaranteeing a consistent ±0ms synchronization window for lip-sync when mixed downstream.

Hardware Frame Drop Detection

To simplify troubleshooting of PCIe slot bandwidth limits or rendering bottlenecks:

• LLive tracks consecutive frames' internal hardware timecodes via the DeckLink SDK's GetStreamTime() API.
• If a gap is detected (e.g. if the CPU skips processing a callback or a frame is lost at the PCIe hardware interface level), LLive calculates the exact number of lost frames.
• Warnings are logged directly to standard output: [DECKLINK DROP DETECTED] X frames dropped! (Current StreamTime: ..., Expected: ...)