# PicoStream

High-performance data acquisition from PicoScope 5000a series to HDF5, with decoupled live visualisation. Designed for robust, high-speed logging where data integrity is critical.

## Quick Start

Install with `uv`:

```bash
uv pip install -e .
```

Install the [PicoSDK](https://www.picotech.com/downloads) (Linux users: see [Arch AUR wiki](https://aur.archlinux.org/packages/picoscope) for configuration).

Launch the GUI:

```bash
just gui
# or: uv run python -m picostream.main
```

Or use the CLI:

```bash
uv run picostream -s 62.5 -o data.hdf5 --plot
```

## Key Features

- **Producer-consumer architecture** with large shared memory buffer pool prevents data loss
- **Decoupled live plotting** reads from HDF5 file, cannot interfere with acquisition
- **Live-only mode** for continuous monitoring without filling disk
- **Numba-accelerated decimation** for efficient visualisation of large datasets
- **GUI and CLI interfaces** for interactive and scripted workflows

## Usage

### GUI Application

```bash
just gui
```

Configure acquisition parameters, start/stop capture, and view live data. Settings persist between sessions.

### Building Standalone Executable

```bash
just build-gui
```

Executable appears in `dist/PicoStream` (or `dist/PicoStream.exe` on Windows).

### Command-Line Interface

Standard acquisition (saves all data):

```bash
uv run picostream -s 62.5 -o my_data.hdf5 --plot
```

Live-only mode (limits file size):

```bash
uv run picostream -s 62.5 --plot --max-buff-sec 60
```

View existing file:

```bash
uv run python -m picostream.dfplot /path/to/data.hdf5
```

Run `uv run picostream --help` for all options.

## Documentation

### Architecture

PicoStream uses a producer-consumer pattern to ensure data integrity:

- **Producer (`PicoDevice`)**: Interfaces with PicoScope hardware, streams ADC data into shared memory buffers in a dedicated thread
- **Consumer (`Consumer`)**: Retrieves filled buffers from queue, writes to HDF5, returns empty buffers to pool in separate thread
- **Buffer Pool**: Large shared memory buffers (100+ MB) prevent data loss if disk I/O slows
- **Live Plotter (`HDF5LivePlotter`)**: Reads from HDF5 file on disk, completely decoupled from acquisition

This ensures the critical acquisition path is never blocked by disk I/O or GUI rendering.

### Data Analysis with `PicoStreamReader`

The `PicoStreamReader` class provides efficient access to HDF5 files with on-the-fly decimation.

```python
import numpy as np
from picostream.reader import PicoStreamReader

with PicoStreamReader('my_data.hdf5') as reader:
    sample_rate_sps = 1e9 / reader.sample_interval_ns
    print(f"File contains {reader.num_samples:,} samples at {sample_rate_sps / 1e6:.2f} MS/s")

    # Iterate through file with 10x decimation
    for times, voltages_mv in reader.get_block_iter(
        chunk_size=10_000_000, decimation_factor=10, decimation_mode='min_max'
    ):
        print(f"Processed {voltages_mv.size} decimated points")
```

### API Reference: `PicoStreamReader`

#### Initialisation

**`__init__(self, hdf5_path: str)`**

Initialises the reader. File is opened when used as context manager.

#### Metadata Attributes

Available after opening:

- `num_samples: int` - Total raw samples in dataset
- `sample_interval_ns: float` - Time interval between samples (nanoseconds)
- `voltage_range_v: float` - Configured voltage range (e.g., `20.0` for ±20V)
- `max_adc_val: int` - Maximum ADC count value (e.g., 32767)
- `analog_offset_v: float` - Configured analogue offset (Volts)
- `downsample_mode: str` - Hardware downsampling mode (`'average'` or `'aggregate'`)
- `hardware_downsample_ratio: int` - Hardware downsampling ratio

#### Methods

**`get_block_iter(self, chunk_size: int = 1_000_000, decimation_factor: int = 1, decimation_mode: str = "mean") -> Generator`**

Generator yielding `(times, voltages)` tuples for entire dataset. Recommended for large files.

- `chunk_size`: Number of raw samples per chunk
- `decimation_factor`: Decimation factor
- `decimation_mode`: `'mean'` or `'min_max'`

**`get_next_block(self, chunk_size: int, decimation_factor: int = 1, decimation_mode: str = "mean") -> Tuple | None`**

Retrieves next sequential block. Returns `None` at end of file. Use `reset()` to restart.

**`get_block(self, size: int, start: int = 0, decimation_factor: int = 1, decimation_mode: str = "mean") -> Tuple`**

Retrieves specific block from file.

- `size`: Number of raw samples
- `start`: Starting sample index

**`reset(self) -> None`**

Resets internal counter for `get_next_block()`.

### API Reference: `HDF5LivePlotter`

PyQt5 widget for real-time visualisation of HDF5 files. Can be used standalone or embedded in custom applications.

#### Initialisation

**`__init__(self, hdf5_path: str = "/tmp/data.hdf5", update_interval_ms: int = 50, display_window_seconds: float = 0.5, decimation_factor: int = 150, max_display_points: int = 4000)`**

- `hdf5_path`: Path to HDF5 file
- `update_interval_ms`: Update frequency (milliseconds)
- `display_window_seconds`: Duration of data to display (seconds)
- `decimation_factor`: Decimation factor for display
- `max_display_points`: Maximum points to display (prevents GUI slowdown)

#### Methods

**`set_hdf5_path(self, hdf5_path: str) -> None`**

Updates monitored HDF5 file path.

**`start_updates(self) -> None`**

Begins periodic plot updates.

**`stop_updates(self) -> None`**

Stops periodic updates.

**`save_screenshot(self) -> None`**

Saves PNG screenshot. Called automatically when 'S' key is pressed.

#### Usage Example

```python
from PyQt5.QtWidgets import QApplication
from picostream.dfplot import HDF5LivePlotter

app = QApplication([])
plotter = HDF5LivePlotter(
    hdf5_path="my_data.hdf5",
    display_window_seconds=1.0,
    decimation_factor=100
)
plotter.show()
plotter.start_updates()
app.exec_()
```

## Changelog

### Version 0.2.0
- Added live-only mode with `--max-buff-sec` option
- Added GUI application for interactive control
- Improved plotter to handle buffer resets gracefully
- Added total sample count tracking across buffer resets
- Skip verification step in live-only mode for better performance

### Version 0.1.0
- Initial release with core streaming and plotting functionality

## Acknowledgements

This package began as a fork of [JoshHarris2108/pico_streaming](https://github.com/JoshHarris2108/pico_streaming) (unlicensed). Thanks to Josh for the original architecture.