LStreamPlay Reference Manual & Network Playout Guide

The LStreamPlay class is the network stream receiver and playback decoding engine of the now2sdk framework, derived from the LObject base class. It is designed to capture, demux, decode, scale, and play out network streams (such as RTSP, RTMP, SRT, UDP, and HTTP live feeds) in real time.

By subclassing LObject, it acts as a stream source, allowing developers to route its decoded video and audio frames to downstream sinks like LPreview widgets, switcher buses, or recording blocks.

1. Stream Playback & Synchronization Architecture

graph TD
    A[Network Stream Source - RTMP/RTSP/SRT/UDP/HTTP] -->|TCP/UDP Packets| B[av_read_frame Context]
    
    subgraph Decode & Recovery Loop
        B -->|Packet Read Error / EOF| C{Stream Healthy?}
        C -->|No| D[Close context & Trigger Reconnect Loop]
        D -->|Sleep 1s| E[Retry avformat_open_input]
        E -->|Success| F[Flush sinks & Reset PTS offset]
        E -->|Failure| D
        
        B -->|Success| G[Demux Packet]
    end
    
    subgraph Video Playout Pipeline
        G -->|Video Packet| H[Video Decoder - e.g. h264/hevc]
        H -->|AVFrame*| I{PTS Sync Check}
        I -->|ptsTime > elapsed| J[Calculate Delay & Thread Sleep]
        I -->|Drift > 2s| K[Force Resync & Reset clock]
        J --> L[SwsContext Scale & Convert]
        K --> L
        L -->|YUV420P AVFrame*| M[distributeVideoFrame]
    end
    
    subgraph Audio Playout Pipeline
        G -->|Audio Packet| N[Audio Decoder - e.g. AAC/MP3]
        N -->|AVFrame*| O{Audio Output Enabled?}
        O -->|Yes| P[SwrContext Resample to S16 Stereo]
        O -->|No| Q[Drop Audio Frame]
        P --> R{audio_gain != 100?}
        R -->|Yes| S[Apply PCM Sample multiplier & clamp]
        R -->|No| T[Update Peak Meters]
        S --> T
        T -->|S16 Stereo AVFrame*| U[distributeAudioFrame]
    end

Key Technical Details:

• Asynchronous Reconnect Engine: When a live network feed drops, the internal decode thread decodeLoopInternal automatically flags the stream as unhealthy (isHealthy() = false), tears down decoder contexts, and enters a 1-second polling reconnect loop using avformat_open_input. Once the connection recovers, it automatically flushes down-stream sinks and resets playout timelines.
• PTS-Based Drift Alignment: Matches the elapsed presentation timestamps (PTS) of decoded video frames against high-precision monotonic system elapsed time. If video is ahead, the thread sleeps. If a frame drift larger than 2.0 seconds is detected (e.g. from network starvation or stream gap), it forces an immediate clock resync to prevent lock-ups.
• Direct Scaling & Format Normalization: Bypasses full filtergraphs for latency optimization. Employs sws_getCachedContext to dynamically rescale incoming frames to target dimensions and format formats (normally normalizing to YUV420P format format).
• Audio Track Switching: Supports multi-lingual live feeds by allowing developers to switch the active audio stream index on the fly. Doing so dynamically releases the old decoder, allocates the new codec context, and re-initializes resampling buffers.
• Peak Volume Accumulators: Calculates left/right audio channels' maximum peaks by reading raw S16 PCM samples. These peaks are cached and accessible via getAudioPeak() for UI-bound VU meters.

2. API Reference & Public Methods

📌 Playback Control

void streamUrlSet(const std::string& url);

Sets the destination stream source URL. Alias: fileNameSet().

static bool checkStream(const std::string& url);

Static helper method that probes connection status by attempting to open the source input. Returns true if responsive.

bool play();

Spawns the background thread loop to begin connecting, demuxing, and decoding frames.

void pause(double ms = 0);

Pauses distribution to all attached sinks. If a value ms > 0 is specified, an auto-resume timer is registered.

void resume();

Resumes playout and adjusts internal PTS timelines to avoid frame-skip compensation.

void stop();

Safely terminates decode threads, joins execution, and frees all FFmpeg contexts.

bool isHealthy() const;

Returns connection stability status (true if connected and decoding, false if disconnected or reconnecting).

📌 Delay Configuration

void setAudioDelay(int ms);

Adds a delay offset (in milliseconds) to resampled audio frames to help align lip sync.

void setVideoDelay(int ms);

Adds a delay offset (in milliseconds) to video frames prior to distribution.

📌 Format Normalization Controls

void setVideoFormat(const videoFormatProps& props);
void getVideoFormat(videoFormatProps& props);

Configures target scaling output dimensions, framerates, and standards.

void setAudioFormat(const audioFormatProps& props);
void getAudioFormat(audioFormatProps& props);

Configures target audio samplerate, channel layout, and standard formats.

📌 Stream Metrics & Getters

double getFPS() override;

Returns the target frame rate or stream's native frame rate if target format is disabled.

int getWidth();
int getHeight();

Returns the native dimensions of the decoded video stream.

void getAudioPeak(float& left, float& right);
void resetAudioPeak();

Retrieves the maximum peaks calculated during the latest audio block playout (ranging from 0.0f to 1.0f).

3. Properties & Configuration Options

All dynamic properties listed below are configured at runtime via the setProps(const std::string& key, const std::string& value) interface:

4. Live Stream Player Integration Example

This example demonstrates how to integrate LStreamPlay into a Qt application, bind it to an LPreview preview rendering widget, switch audio tracks, alter volume levels dynamically, and monitor the stream's network connection status.

mainwindow.h

#pragma once
#include <QMainWindow>
#include "LStreamPlay.h"
#include "LPreview.h"

QT_BEGIN_NAMESPACE
namespace Ui { class MainWindow; }
QT_END_NAMESPACE

class MainWindow : public QMainWindow {
    Q_OBJECT
public:
    MainWindow(QWidget *parent = nullptr);
    ~MainWindow();

private slots:
    void onPlayClicked();
    void onStopClicked();
    void onVolumeChanged(int value);
    void onAudioTrackChanged(int index);
    void onScaleTypeChanged(const QString& scale);
    void checkConnectionStatus();

private:
    Ui::MainWindow *ui;
    LStreamPlay* m_player;
    LPreview* m_preview;
    QTimer* m_healthTimer;
    bool m_isPlaying;
};

mainwindow.cpp

#include "mainwindow.h"
#include "./ui_mainwindow.h"
#include <QTimer>
#include <QMessageBox>

MainWindow::MainWindow(QWidget *parent)
    : QMainWindow(parent),
      ui(new Ui::MainWindow),
      m_player(new LStreamPlay()),
      m_preview(new LPreview()),
      m_healthTimer(new QTimer(this)),
      m_isPlaying(false)
{
    ui->setupUi(this);

    // 1. Initialize Player and Preview Widgets
    // We target HD1080_50p progressive format for smooth scaling
    videoFormatProps vFormat;
    vFormat.setVideoFormat = vF::HD1080_50p;
    m_player->setVideoFormat(vFormat);

    audioFormatProps aFormat;
    aFormat.setAudioFormat = aF::_16B_48K_2CH;
    m_player->setAudioFormat(aFormat);

    // Bind preview renderer to Qt widget parent and route the player source
    m_preview->setProps("ui_framework", "qt");
    m_preview->setProps("audio_meter", "true");
    m_preview->previewEnable(ui->previewFrame, true, true);
    m_preview->previewObject(m_player);

    // 2. Connect UI Signal Triggers
    connect(ui->playBtn, &QPushButton::clicked, this, &MainWindow::onPlayClicked);
    connect(ui->stopBtn, &QPushButton::clicked, this, &MainWindow::onStopClicked);
    connect(ui->volumeSlider, &QSlider::valueChanged, this, &MainWindow::onVolumeChanged);
    connect(ui->audioTrackCombo, &QComboBox::currentIndexChanged, this, &MainWindow::onAudioTrackChanged);
    connect(ui->scaleCombo, &QComboBox::currentTextChanged, this, &MainWindow::onScaleTypeChanged);

    // 3. Setup stream health monitoring timer
    connect(m_healthTimer, &QTimer::timeout, this, &MainWindow::checkConnectionStatus);
}

MainWindow::~MainWindow() {
    m_healthTimer->stop();
    m_player->stop();
    delete m_preview;
    delete m_player;
    delete ui;
}

void MainWindow::onPlayClicked() {
    if (m_isPlaying) return;

    QString streamUrl = ui->urlLineEdit->text(); // e.g. "rtsp://192.168.1.100:554/stream1"
    if (streamUrl.isEmpty()) {
        QMessageBox::warning(this, "Player Control", "Please enter a valid RTSP/RTMP stream URL.");
        return;
    }

    // Probes connection to verify host availability beforehand
    if (!LStreamPlay::checkStream(streamUrl.toStdString())) {
        QMessageBox::critical(this, "Network Error", "Unable to connect or handshake with the stream URL.");
        return;
    }

    m_player->streamUrlSet(streamUrl.toStdString());

    // Apply baseline scaling properties
    m_player->setProps("scaling_quality", "4"); // Lanczos scaling
    m_player->setProps("scale_type", ui->scaleCombo->currentText().toLower().toStdString());

    if (m_player->play()) {
        m_isPlaying = true;
        ui->playBtn->setEnabled(false);
        ui->stopBtn->setEnabled(true);
        m_healthTimer->start(1000); // Check stream status every second
    }
}

void MainWindow::onStopClicked() {
    if (!m_isPlaying) return;

    m_healthTimer->stop();
    m_player->stop();
    m_isPlaying = false;
    
    ui->playBtn->setEnabled(true);
    ui->stopBtn->setEnabled(false);
    ui->statusLabel->setText("Status: Stopped");
    ui->statusLabel->setStyleSheet("color: #3f3f46; font-weight: normal;");
}

void MainWindow::onVolumeChanged(int value) {
    // Volume scale from slider (0 - 150)
    m_player->setProps("audio_gain", std::to_string(value));
}

void MainWindow::onAudioTrackChanged(int index) {
    // Switch between audio track indexes (e.g. 0 = English, 1 = Spanish)
    m_player->setProps("audio_track", std::to_string(index));
}

void MainWindow::onScaleTypeChanged(const QString& scale) {
    // Set scale mode (e.g. "letter-box", "crop", "stretch")
    m_player->setProps("scale_type", scale.toLower().toStdString());
}

void MainWindow::checkConnectionStatus() {
    if (!m_isPlaying) return;

    // Monitor stream health indicator
    if (m_player->isHealthy()) {
        ui->statusLabel->setText("Status: Stream Connected");
        ui->statusLabel->setStyleSheet("color: #16a34a; font-weight: bold;"); // Green
    } else {
        ui->statusLabel->setText("Status: Reconnecting...");
        ui->statusLabel->setStyleSheet("color: #ea580c; font-weight: bold;"); // Orange
    }

    // Refresh peaks for VU meters
    float leftPeak = 0.0f, rightPeak = 0.0f;
    m_player->getAudioPeak(leftPeak, rightPeak);
    ui->leftVuBar->setValue(static_cast<int>(leftPeak * 100));
    ui->rightVuBar->setValue(static_cast<int>(rightPeak * 100));
}