PicoStream

High-performance data acquisition from PicoScope 5000a series to HDF5, with decoupled live visualisation. Designed for robust, high-speed logging where data integrity is critical.

Quick Start

Install with uv:

uv pip install -e .

Install the PicoSDK (Linux users: see Arch AUR wiki for configuration).

Launch the GUI:

just gui
# or: uv run python -m picostream.main

Or use the CLI:

uv run picostream -s 62.5 -o data.hdf5 --plot

Key Features

• Producer-consumer architecture with large shared memory buffer pool prevents data loss
• Decoupled live plotting reads from HDF5 file, cannot interfere with acquisition
• Live-only mode for continuous monitoring without filling disk
• Numba-accelerated decimation for efficient visualisation of large datasets
• GUI and CLI interfaces for interactive and scripted workflows

Usage

GUI Application

just gui

Configure acquisition parameters, start/stop capture, and view live data. Settings persist between sessions.

Building Standalone Executable

just build-gui

Executable appears in dist/PicoStream (or dist/PicoStream.exe on Windows).

Command-Line Interface

Standard acquisition (saves all data):

uv run picostream -s 62.5 -o my_data.hdf5 --plot

Live-only mode (limits file size):

uv run picostream -s 62.5 --plot --max-buff-sec 60

View existing file:

uv run python -m picostream.dfplot /path/to/data.hdf5

Run uv run picostream --help for all options.

Documentation

Architecture

PicoStream uses a producer-consumer pattern to ensure data integrity:

• Producer (PicoDevice): Interfaces with PicoScope hardware, streams ADC data into shared memory buffers in a dedicated thread
• Consumer (Consumer): Retrieves filled buffers from queue, writes to HDF5, returns empty buffers to pool in separate thread
• Buffer Pool: Large shared memory buffers (100+ MB) prevent data loss if disk I/O slows
• Live Plotter (HDF5LivePlotter): Reads from HDF5 file on disk, completely decoupled from acquisition

This ensures the critical acquisition path is never blocked by disk I/O or GUI rendering.

Data Analysis with PicoStreamReader

The PicoStreamReader class provides efficient access to HDF5 files with on-the-fly decimation.

import numpy as np
from picostream.reader import PicoStreamReader

with PicoStreamReader('my_data.hdf5') as reader:
    sample_rate_sps = 1e9 / reader.sample_interval_ns
    print(f"File contains {reader.num_samples:,} samples at {sample_rate_sps / 1e6:.2f} MS/s")

    # Iterate through file with 10x decimation
    for times, voltages_mv in reader.get_block_iter(
        chunk_size=10_000_000, decimation_factor=10, decimation_mode='min_max'
    ):
        print(f"Processed {voltages_mv.size} decimated points")

API Reference: PicoStreamReader

Initialisation

__init__(self, hdf5_path: str)

Initialises the reader. File is opened when used as context manager.

Metadata Attributes

Available after opening:

• num_samples: int - Total raw samples in dataset
• sample_interval_ns: float - Time interval between samples (nanoseconds)
• voltage_range_v: float - Configured voltage range (e.g., 20.0 for ±20V)
• max_adc_val: int - Maximum ADC count value (e.g., 32767)
• analog_offset_v: float - Configured analogue offset (Volts)
• downsample_mode: str - Hardware downsampling mode ('average' or 'aggregate')
• hardware_downsample_ratio: int - Hardware downsampling ratio

Methods

get_block_iter(self, chunk_size: int = 1_000_000, decimation_factor: int = 1, decimation_mode: str = "mean") -> Generator

Generator yielding (times, voltages) tuples for entire dataset. Recommended for large files.

• chunk_size: Number of raw samples per chunk
• decimation_factor: Decimation factor
• decimation_mode: 'mean' or 'min_max'

get_next_block(self, chunk_size: int, decimation_factor: int = 1, decimation_mode: str = "mean") -> Tuple | None

Retrieves next sequential block. Returns None at end of file. Use reset() to restart.

get_block(self, size: int, start: int = 0, decimation_factor: int = 1, decimation_mode: str = "mean") -> Tuple

Retrieves specific block from file.

• size: Number of raw samples
• start: Starting sample index

reset(self) -> None

Resets internal counter for get_next_block().

API Reference: HDF5LivePlotter

PyQt5 widget for real-time visualisation of HDF5 files. Can be used standalone or embedded in custom applications.

Initialisation

__init__(self, hdf5_path: str = "/tmp/data.hdf5", update_interval_ms: int = 50, display_window_seconds: float = 0.5, decimation_factor: int = 150, max_display_points: int = 4000)

• hdf5_path: Path to HDF5 file
• update_interval_ms: Update frequency (milliseconds)
• display_window_seconds: Duration of data to display (seconds)
• decimation_factor: Decimation factor for display
• max_display_points: Maximum points to display (prevents GUI slowdown)

Methods

set_hdf5_path(self, hdf5_path: str) -> None

Updates monitored HDF5 file path.

start_updates(self) -> None

Begins periodic plot updates.

stop_updates(self) -> None

Stops periodic updates.

save_screenshot(self) -> None

Saves PNG screenshot. Called automatically when 'S' key is pressed.

Usage Example

from PyQt5.QtWidgets import QApplication
from picostream.dfplot import HDF5LivePlotter

app = QApplication([])
plotter = HDF5LivePlotter(
    hdf5_path="my_data.hdf5",
    display_window_seconds=1.0,
    decimation_factor=100
)
plotter.show()
plotter.start_updates()
app.exec_()

Changelog

Version 0.2.0

• Added live-only mode with --max-buff-sec option
• Added GUI application for interactive control
• Improved plotter to handle buffer resets gracefully
• Added total sample count tracking across buffer resets
• Skip verification step in live-only mode for better performance

Version 0.1.0

• Initial release with core streaming and plotting functionality

Acknowledgements

This package began as a fork of JoshHarris2108/pico_streaming (unlicensed). Thanks to Josh for the original architecture.