LSwitcher Reference Manual & Vision Mixer Guide

The LSwitcher class is the central production switcher and vision mixer engine of the now2sdk framework, derived from the LObject base class. It operates as a real-time vision mixer, receiving two source inputs (Preview and Program buses) through downstream SwitcherSink objects, transitioning them using standard broadcast effects, overlaying transparent Character Generator (CG) layers, and outputting the synchronized program stream.

Since LSwitcher inherits from LObject, its mixed video/audio output can be routed directly to any downstream target (e.g. LPreview preview windows, LRecorder capture engines, or LStream network publishers).

1. Vision Mixer & Transition Architecture

graph TD
    subgraph A/B Input Buses
        SourceA[Preview/Program Source A - LObject] -->|pushVideoFrame / pushAudioFrame| SinkA[SwitcherSink A]
        SourceB[Preview/Program Source B - LObject] -->|pushVideoFrame / pushAudioFrame| SinkB[SwitcherSink B]
    end
    
    subgraph Mixing Thread Loop
        SinkA -->|getLatestFrame / popAudioSamplesForPts| MixEngine[switcherLoop Thread]
        SinkB -->|getLatestFrame / popAudioSamplesForPts| MixEngine
        
        MixEngine -->|Apply scale_mode & scaling_quality| Scaler[SwsContext Scale & Fit]
        Scaler -->|Normalized YUV420p Frames| Blender[Transitions Blending Processor]
        
        subgraph Blending Processor (Transition Type)
            Blender -->|fade| FadeProc[Cross-opacity fade]
            Blender -->|wipe_lr / wipe_rl / wipe_tb / wipe_bt / diagonal / box / circle| WipeProc[Geometrical pixel-wipe split]
            Blender -->|slide_lr / slide_rl / slide_tb / slide_bt| SlideProc[A/B coordinate offset slide]
            Blender -->|squeeze_lr / squeeze_rl / squeeze_tb / squeeze_bt| SqueezeProc[Anamorphic compress]
            Blender -->|dip| DipProc[Interpolation to Custom HEX color]
        end
    end
    
    subgraph Downstream Overlays
        Stinger[LStinger Video Cache] -->|stingerActive| StingerOverlay[Mathematical Alpha Blend]
        CG[LCharacter Graphic Templates] -->|YUVA Layer| CGOverlay[YUVA Blend Processor]
        
        FadeProc --> StingerOverlay
        WipeProc --> StingerOverlay
        SlideProc --> StingerOverlay
        SqueezeProc --> StingerOverlay
        DipProc --> StingerOverlay
        
        StingerOverlay --> CGOverlay
    end
    
    subgraph Playout
        CGOverlay -->|Mixed YUV420p AVFrame*| OutV[distributeVideoFrame]
        MixEngine -->|Mixed S16 Audio PCM| OutA[distributeAudioFrame]
    end

Key Technical Details:

• Background Mixing Thread: Operates a dedicated execution loop (switcherLoop) that scales, crops, and blends frames at a precise target video standard frame rate (e.g. 50.0 FPS).
• Multi-Format Scaling & Fit Modes: Automatically scales mismatched input resolutions (e.g., mixing a 720p file with a 1080p camera) according to the configured scale_mode (Stretch, Fit/Letterbox with black bars, or Fill/Crop).
• CG Character Generator Compositing: Accepts multiple downstream graphics layers (LCharacter) and composites them mathematically over YUV420P frames using multi-threaded YUVA alpha blending.
• Monotonic TOD Timecode Engine: Generates standard timecode strings (HH:MM:SS:FF) stamped onto output frame dictionaries, either synced to system Time of Day (TOD) or calculated from local elapsed frames.
• Synchronized Audio Cross-Fading: Cross-fades and mixes PCM audio streams from Preview and Program sources based on cross-fade progress, applying master volume control and pop-sync logic to eliminate audio drift.

2. API Reference & Public Methods

📌 Bus Routing & Controls

void setPreview(LObject* obj);

Binds the input object to the Preview bus.

void setProgram(LObject* obj);

Binds the input object to the Program bus.

void setCutTransition();

Performs an instantaneous cut switch. To prevent T-Bar slider jump glitches, this logically swaps the underlying A and B layer pointers without resetting transition progress.

📌 Transition Configurations

void setTransition(float progress);

Manually positions the transition blending state (where 0.0f shows Program source exclusively and 1.0f shows Preview).

void setAutoTransition(double durationMs);

Asynchronously animates the transition between Program and Preview over a defined duration. When finished, it swaps the layer pointers and resets the progress values.

void setTransitionType(const std::string& type, const std::string& arg = "");

Sets the active transition type. Transition options are detailed in Section 3.

📌 Stinger & LStinger Class Integration

A stinger is a short, transparent graphic animation (usually a PNG image sequence or transparent ProRes 4444 video) that covers the entire screen at a specific midpoint, allowing the mixer to switch the background feeds invisibly.

Rather than making developers instantiate and handle sync manually, LSwitcher abstracts this by managing an internal LStinger instance:

sequenceDiagram
    participant App as C++ Application
    participant Sw as LSwitcher
    participant St as LStinger (Internal)
    participant Sink as SwitcherSink (Stinger)

    App->>Sw: setStinger("/path/to/stinger_%05d.png")
    Sw->>St: load("/path/to/stinger_%05d.png", targetW, targetH, targetFps)
    St->>St: Parse PNG sequence & cache YUVA frames in RAM
    App->>Sw: setStingerDuration(1200) // 1.2 seconds
    Sw->>St: setFPS(totalFrames / 1.2) // Recalculate playback speed
    App->>Sw: triggerStinger()
    Sw->>St: playOnce()
    Note over Sw,St: Stinger playback loop starts distributing frames...
    St->>Sink: pushVideoFrame(YUVA Frame)
    Note over Sw: switcherLoop detects stingerActive...
    Sw->>Sw: Check if stingerElapsed >= cutPoint (e.g. 600ms)
    Sw->>Sw: Swaps Preview & Program buses invisibly
    Sw->>Sw: Alpha-blends YUVA stinger frame over output YUV420p frame

Stinger Interface Methods:

• void setStinger(const std::string& path) Points to the transparent stinger resource (e.g. "/templates/transitions/logo_wipe/frame_%05d.png"). This triggers the internal LStinger object to parse and cache frames into RAM.
• void setStingerDuration(int ms) Updates stinger playback length. It divides the loaded stinger frame count by the duration to set the target LStinger FPS. It also updates the transition cut midpoint (m_stingerCutPointMs = ms / 2).
• void triggerStinger() Starts stinger animation playback and schedules the invisible source cut.

📌 Character Generator (CG) Layers

• void addCG(LCharacter* cg) Attaches a downstream graphics layer to be composited over the final program feed.
• void removeCG(LCharacter* cg) Removes a character generator template layer.
• int getCGCount() const
• LCharacter* getCGByIndex(int index) Queries overlays by index.

📌 Audio Mixer Controls

• void setProgAudioGain(int gain) Sets the target volume level percentage for the active Program source (e.g., 100 for normal volume).
• void setPrevAudioGain(int gain) Sets the volume level percentage for the active Preview source (normally 0 when muted).

📌 Switcher Metrics

struct SwitcherStats {
    double fps;             // Live output frame rate
    std::string onAir;      // Current state: "SubA", "SubB", or "Transition"
    float progress;         // Current T-bar progress percentage (0.0 to 100.0)
    int64_t frameCount;     // Total elapsed video frames mixed
    std::string timecode;   // Formatted HH:MM:SS:FF timecode string
    float audioLevelA;      // Preview source volume peak (0.0 to 1.0)
    float audioLevelB;      // Program source volume peak (0.0 to 1.0)
    float audioLevelOut;    // Mixed program output volume peak (0.0 to 1.0)
    int audioBufferMs;      // Maximum latency buffer queue size in ms
    double avSyncMs;        // Output A/V synchronization drift in ms
};

void getStats(SwitcherStats& stats);

Chain Pull-Rendering & Dynamic Fallback Lifecycle

now2sdk v0.0.2 introduces a high-performance Chain Pull-Rendering architecture that aligns the render cycles of composition layers to avoid phase offsets, latency build-up, and frame-drops.

virtual AVFrame* renderVideoFrame() override;

Called on-demand by downstream consumers (such as LMixer or video outputs). It triggers a single synchronized blending/rendering cycle and returns the final composite AVFrame pointer instantly.

void addPullConsumer();
void removePullConsumer();

Tracks the active count of pull-based downstream objects.

• Pull Mode: When at least one pull consumer is registered, the switcher joins/halts its internal timer thread (switcherLoop) to prevent frame rate collisions and runs on-demand.
• Self-Timed Mode (Fallback): When the last pull consumer is removed, it automatically respawns the internal 50 FPS high-precision timer thread to ensure preview widgets and standalone playouts continue rendering smoothly.

3. Transitions List & Algorithms

The following table lists all transition types supported via setTransitionType(const std::string& type, const std::string& arg):

4. Properties & Configuration Options

All switcher options listed below are configured at runtime via the setProps(const std::string& key, const std::string& value) interface:

5. Vision Switcher Panel Integration Example

This example adapts the Switcher Example project to demonstrate how to configure LSwitcher, route media/capture inputs, select transition patterns, perform auto-fades, configure and load stingers, and display output statistics in real time.

#include "mainwindow.h"
#include "./ui_mainwindow.h"
#include "LFile.h"
#include "LLive.h"
#include "LSwitcher.h"
#include <QTimer>
#include <QFileDialog>
#include <QMessageBox>

MainWindow::MainWindow(QWidget *parent)
    : QMainWindow(parent),
      ui(new Ui::MainWindow),
      m_filePlayer(new LFile()),
      m_cameraInput(new LLive()),
      m_switcher(new LSwitcher()),
      m_uiTimer(new QTimer(this)),
      m_activeProg(0),
      m_activePrev(1)
{
    ui->setupUi(this);

    // 1. Setup global formats (1080p50 Broadcast standard)
    videoFormatProps format;
    format.setVideoFormat = vF::HD1080_50p;
    m_switcher->setVideoFormat(format);
    
    // Apply normalization properties
    m_switcher->setProps("scale_mode", "1");        // Letterbox fit
    m_switcher->setProps("scaling_quality", "3");   // Bicubic quality
    m_switcher->setProps("timecode", "true");
    m_switcher->setProps("timecodeSource", "1");    // Frame counter timecode

    // 2. Setup inputs
    m_filePlayer->fileNameSet("/media/clips/background_loop.mp4");
    m_filePlayer->setProps("loop", "true");
    m_filePlayer->play();

    m_cameraInput->setProps("device_type", "decklink");
    m_cameraInput->DeviceSet(0); // Select first decklink card
    m_cameraInput->Start();

    // 3. Connect inputs to Switcher buses (0 = File, 1 = Camera)
    m_switcher->setProgram(m_filePlayer);
    m_switcher->setPreview(m_cameraInput);

    // 4. Default transition configuration
    m_switcher->setTransitionType("fade");
    m_switcher->setProgAudioGain(100);
    m_switcher->setPrevAudioGain(0); // Mute preview audio channel

    // 5. Connect UI Triggers
    connect(ui->cutBtn, &QPushButton::clicked, this, &MainWindow::onCutClicked);
    connect(ui->autoBtn, &QPushButton::clicked, this, &MainWindow::onAutoClicked);
    connect(ui->stingerSelectBtn, &QPushButton::clicked, this, &MainWindow::onSelectStingerClicked);
    connect(ui->stingerBtn, &QPushButton::clicked, this, &MainWindow::onStingerClicked);
    connect(ui->tbarSlider, &QSlider::valueChanged, this, &MainWindow::onTBarSliderMoved);
    connect(ui->transTypeCombo, &QComboBox::currentTextChanged, this, &MainWindow::onTransTypeChanged);

    // 6. Connect dynamic update timers
    connect(m_uiTimer, &QTimer::timeout, this, &MainWindow::onUIUpdate);
    m_uiTimer->start(40); // 25 fps UI refreshes
}

MainWindow::~MainWindow() {
    m_uiTimer->stop();
    m_filePlayer->stop();
    m_cameraInput->Stop();
    delete m_uiTimer;
    delete m_filePlayer;
    delete m_cameraInput;
    delete m_switcher;
    delete ui;
}

// ─── Instant Cut Switch ───
void MainWindow::onCutClicked() {
    m_switcher->setCutTransition();
    std::swap(m_activeProg, m_activePrev);
    
    // Re-bind source routing to prevent lockups
    m_switcher->setProgram(getSourceObject(m_activeProg));
    m_switcher->setPreview(getSourceObject(m_activePrev));
    m_switcher->setTransition(0.0f);
}

// ─── Timed Crossfade transition ───
void MainWindow::onAutoClicked() {
    m_switcher->setAutoTransition(1000.0); // 1-second auto cross-fade
    std::swap(m_activeProg, m_activePrev);
}

// ─── Select Stinger Assets Sequence ───
void MainWindow::onSelectStingerClicked() {
    QString dirPath = QFileDialog::getExistingDirectory(
        this, "Select Stinger PNG Sequence Directory", "/home/user");
    if (!dirPath.isEmpty()) {
        // Format naming: frame_%05d.png
        QString formatPattern = dirPath + "/logo_wipe_%05d.png";
        m_stingerPath = formatPattern.toStdString();
        
        // Load sequence into the switcher's internal LStinger cache
        m_switcher->setStinger(m_stingerPath);
        m_switcher->setStingerDuration(1200); // Set speed to cover 1.2s
        
        ui->stingerPathLabel->setText(QFileInfo(dirPath).fileName());
    }
}

// ─── Transparent Stinger Wipe Trigger ───
void MainWindow::onStingerClicked() {
    if (m_stingerPath.empty()) {
        QMessageBox::warning(this, "Stinger", "Select a stinger PNG sequence folder first.");
        return;
    }
    
    // Cue stinger mode and trigger internal LStinger playback loop
    m_switcher->setTransitionType("stinger");
    m_switcher->triggerStinger();
    
    std::swap(m_activeProg, m_activePrev);
}

// ─── Manual T-Bar mixing ───
void MainWindow::onTBarSliderMoved(int value) {
    float progress = static_cast<float>(value) / 100.0f;
    m_switcher->setTransition(progress);

    // If fully transition to other end, cut and switch buses
    if (progress >= 1.0f) {
        m_switcher->setCutTransition();
        m_switcher->setTransition(0.0f);
        std::swap(m_activeProg, m_activePrev);
        
        m_switcher->setProgram(getSourceObject(m_activeProg));
        m_switcher->setPreview(getSourceObject(m_activePrev));
        
        // Block signals to avoid slider infinite loops
        ui->tbarSlider->blockSignals(true);
        ui->tbarSlider->setValue(0);
        ui->tbarSlider->blockSignals(false);
    }
}

// ─── Transition type selection ───
void MainWindow::onTransTypeChanged(const QString& typeName) {
    std::string type = typeName.toLower().toStdString();
    if (type == "dip to black") {
        m_switcher->setTransitionType("dip", "#000000");
    } else if (type == "dip to white") {
        m_switcher->setTransitionType("dip", "#ffffff");
    } else {
        m_switcher->setTransitionType(type);
    }
}

// Helper routing
LObject* MainWindow::getSourceObject(int id) {
    return (id == 0) ? m_filePlayer : m_cameraInput;
}

// ─── Refresh stats panel labels ───
void MainWindow::onUIUpdate() {
    LSwitcher::SwitcherStats stats;
    m_switcher->getStats(stats);

    ui->fpsLabel->setText(QString("%1 FPS").arg(stats.fps, 0, 'f', 2));
    ui->timecodeLabel->setText(QString::fromStdString(stats.timecode));
    ui->onAirLabel->setText(QString::fromStdString(stats.onAir));

    // Update VU Audio levels
    ui->pgmVuBar->setValue(static_cast<int>(stats.audioLevelOut * 100));
}