aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorSam Scholten2025-10-15 10:35:24 +1000
committerSam Scholten2025-10-15 11:04:02 +1000
commit55001da9a7add8ae57c66f7c921e4f70309df511 (patch)
tree9109491dc9f2a5a58660d76b37473d866a145c72
parent14042a8f9e36f2bf61d23f60dfba223eafb58157 (diff)
downloadpicostream-55001da9a7add8ae57c66f7c921e4f70309df511.tar.gz
picostream-55001da9a7add8ae57c66f7c921e4f70309df511.zip
docs: improve README
-rw-r--r--README.md231
1 files changed, 150 insertions, 81 deletions
diff --git a/README.md b/README.md
index d21e84b..d332ae0 100644
--- a/README.md
+++ b/README.md
@@ -1,147 +1,216 @@
# PicoStream
-PicoStream is a high-performance Python application for streaming data from a PicoScope to an HDF5 file, with an optional, decoupled live visualization. It is designed for robust, high-speed data logging where data integrity is critical.
+High-performance data acquisition from PicoScope 5000a series to HDF5, with decoupled live visualisation. Designed for robust, high-speed logging where data integrity is critical.
-## Features
+## Quick Start
-- **Robust Architecture**: Separates data acquisition from disk I/O using a producer-consumer pattern with a large, shared memory buffer pool to prevent data loss.
-- **Zero-Risk Live Plotting**: The plotter reads from the HDF5 file, not the live data stream. This ensures that a slow or crashing GUI cannot interfere with data acquisition.
-- **Efficient Visualization**: Uses `pyqtgraph` and a Numba-accelerated min-max decimation algorithm to display large datasets with minimal CPU impact.
-- **Live-Only Mode**: Optional buffer size limiting for pure real-time monitoring without generating large files.
-- **Flexible Data Reading**: The included `PicoStreamReader` class allows for easy post-processing, including on-the-fly decimation ('mean' or 'min_max').
+Install with `uv`:
-## Installation
+```bash
+uv pip install -e .
+```
-1. `pip install -e .`
+Install the [PicoSDK](https://www.picotech.com/downloads) (Linux users: see [Arch AUR wiki](https://aur.archlinux.org/packages/picoscope) for configuration).
-2. Install the official PicoSDK from Pico Technology. Linux can take some additional work, see the AUR wiki for details.
+Launch the GUI:
-## Usage
+```bash
+just gui
+# or: uv run python -m picostream.main
+```
+
+Or use the CLI:
-The primary way to use the package is through the `picostream` command-line tool.
+```bash
+uv run picostream -s 62.5 -o data.hdf5 --plot
+```
+
+## Key Features
-### Acquisition
+- **Producer-consumer architecture** with large shared memory buffer pool prevents data loss
+- **Decoupled live plotting** reads from HDF5 file, cannot interfere with acquisition
+- **Live-only mode** for continuous monitoring without filling disk
+- **Numba-accelerated decimation** for efficient visualisation of large datasets
+- **GUI and CLI interfaces** for interactive and scripted workflows
-The following commands show different acquisition modes:
+## Usage
+
+### GUI Application
```bash
-# Standard acquisition (saves all data)
-picostream -s 62.5 -o my_data.hdf5 --plot
+just gui
+```
-# Live-only mode (limited buffer)
-picostream -s 62.5 --plot --max-buff-sec 60
+Configure acquisition parameters, start/stop capture, and view live data. Settings persist between sessions.
+
+### Building Standalone Executable
+
+```bash
+just build-gui
```
-Run `picostream --help` for a full list of options.
+Executable appears in `dist/PicoStream` (or `dist/PicoStream.exe` on Windows).
-### Live-Only Mode
+### Command-Line Interface
-For pure real-time monitoring without saving large amounts of data, use the `--max-buff-sec` option:
+Standard acquisition (saves all data):
```bash
-picostream -s 62.5 --plot --max-buff-sec 60
+uv run picostream -s 62.5 -o my_data.hdf5 --plot
```
-This limits the HDF5 file to the last 60 seconds of data, automatically overwriting older data. Perfect for continuous monitoring applications where you only need recent data visible.
+Live-only mode (limits file size):
-**Benefits:**
-- Prevents disk space issues during long acquisitions
-- Maintains full plotting functionality
-- File size stays constant after initial buffer fill
-- No performance impact on acquisition
-
-### Viewing an Existing File
+```bash
+uv run picostream -s 62.5 --plot --max-buff-sec 60
+```
-The plotter can be run as a standalone tool to view any compatible HDF5 file.
+View existing file:
```bash
-python -m picostream.dfplot /path/to/your/data.hdf5
+uv run python -m picostream.dfplot /path/to/data.hdf5
```
-## Data Analysis with `PicoStreamReader`
+Run `uv run picostream --help` for all options.
-The output HDF5 file contains raw ADC counts and metadata. The `PicoStreamReader` class in `picostream.reader` is the recommended way to read and process this data.
+## Documentation
-### Example: Processing a File in Chunks
+### Architecture
-Here is an example of how to use `PicoStreamReader` to iterate through a large file and perform analysis without loading the entire dataset into memory.
+PicoStream uses a producer-consumer pattern to ensure data integrity:
+
+- **Producer (`PicoDevice`)**: Interfaces with PicoScope hardware, streams ADC data into shared memory buffers in a dedicated thread
+- **Consumer (`Consumer`)**: Retrieves filled buffers from queue, writes to HDF5, returns empty buffers to pool in separate thread
+- **Buffer Pool**: Large shared memory buffers (100+ MB) prevent data loss if disk I/O slows
+- **Live Plotter (`HDF5LivePlotter`)**: Reads from HDF5 file on disk, completely decoupled from acquisition
+
+This ensures the critical acquisition path is never blocked by disk I/O or GUI rendering.
+
+### Data Analysis with `PicoStreamReader`
+
+The `PicoStreamReader` class provides efficient access to HDF5 files with on-the-fly decimation.
```python
import numpy as np
from picostream.reader import PicoStreamReader
-# Use the reader as a context manager
with PicoStreamReader('my_data.hdf5') as reader:
- # Metadata is available as attributes after opening
sample_rate_sps = 1e9 / reader.sample_interval_ns
- print(f"File contains {reader.num_samples:,} samples.")
- print(f"Sample rate: {sample_rate_sps / 1e6:.2f} MS/s")
+ print(f"File contains {reader.num_samples:,} samples at {sample_rate_sps / 1e6:.2f} MS/s")
- # Example: Iterate through the file with 10x decimation
- print("\nProcessing data with 10x 'min_max' decimation...")
+ # Iterate through file with 10x decimation
for times, voltages_mv in reader.get_block_iter(
chunk_size=10_000_000, decimation_factor=10, decimation_mode='min_max'
):
- # The 'times' and 'voltages_mv' arrays are now decimated.
- # Process the smaller data chunk here.
- print(f" - Processed a chunk of {voltages_mv.size} decimated points.")
-
- print("\nFinished processing.")
+ print(f"Processed {voltages_mv.size} decimated points")
```
-## API Reference: `PicoStreamReader`
+### API Reference: `PicoStreamReader`
+
+#### Initialisation
-The `PicoStreamReader` provides a simple and efficient interface for accessing data.
+**`__init__(self, hdf5_path: str)`**
-### Initialization
+Initialises the reader. File is opened when used as context manager.
-#### `__init__(self, hdf5_path: str)`
-Initializes the reader. The file is opened and metadata is read when the object is used as a context manager.
+#### Metadata Attributes
-### Metadata Attributes
+Available after opening:
-These attributes are populated from the HDF5 file's metadata when the reader is opened.
+- `num_samples: int` - Total raw samples in dataset
+- `sample_interval_ns: float` - Time interval between samples (nanoseconds)
+- `voltage_range_v: float` - Configured voltage range (e.g., `20.0` for ±20V)
+- `max_adc_val: int` - Maximum ADC count value (e.g., 32767)
+- `analog_offset_v: float` - Configured analogue offset (Volts)
+- `downsample_mode: str` - Hardware downsampling mode (`'average'` or `'aggregate'`)
+- `hardware_downsample_ratio: int` - Hardware downsampling ratio
-- `num_samples: int`: Total number of raw samples in the dataset.
-- `sample_interval_ns: float`: The time interval between samples in nanoseconds.
-- `voltage_range_v: float`: The configured voltage range (e.g., `20.0` for ±20V).
-- `max_adc_val: int`: The maximum ADC count value (e.g., 32767).
-- `analog_offset_v: float`: The configured analog offset in Volts.
-- `downsample_mode: str`: The hardware downsampling mode used during acquisition (`'average'` or `'aggregate'`).
-- `hardware_downsample_ratio: int`: The hardware downsampling ratio used.
+#### Methods
-### Data Access Methods
+**`get_block_iter(self, chunk_size: int = 1_000_000, decimation_factor: int = 1, decimation_mode: str = "mean") -> Generator`**
-#### `get_block_iter(self, chunk_size: int = 1_000_000, decimation_factor: int = 1, decimation_mode: str = "mean") -> Generator`
-Returns a generator that yields data blocks as `(times, voltages)` tuples for the entire dataset. This is the recommended method for processing large files.
-- `chunk_size`: The number of *raw* samples to read from the file for each chunk.
-- `decimation_factor`: The factor by which to decimate the data.
-- `decimation_mode`: The decimation method (`'mean'` or `'min_max'`).
+Generator yielding `(times, voltages)` tuples for entire dataset. Recommended for large files.
-#### `get_next_block(self, chunk_size: int, decimation_factor: int = 1, decimation_mode: str = "mean") -> Tuple | None`
-Retrieves the next sequential block of data. Returns `None` when the end of the file is reached. Use `reset()` to start over.
+- `chunk_size`: Number of raw samples per chunk
+- `decimation_factor`: Decimation factor
+- `decimation_mode`: `'mean'` or `'min_max'`
-#### `get_block(self, size: int, start: int = 0, decimation_factor: int = 1, decimation_mode: str = "mean") -> Tuple`
-Retrieves a specific block of data from the file.
-- `size`: The number of *raw* samples to retrieve.
-- `start`: The starting sample index.
+**`get_next_block(self, chunk_size: int, decimation_factor: int = 1, decimation_mode: str = "mean") -> Tuple | None`**
-#### `reset(self) -> None`
-Resets the internal counter for `get_next_block()` to the beginning of the file.
+Retrieves next sequential block. Returns `None` at end of file. Use `reset()` to restart.
+**`get_block(self, size: int, start: int = 0, decimation_factor: int = 1, decimation_mode: str = "mean") -> Tuple`**
+
+Retrieves specific block from file.
+
+- `size`: Number of raw samples
+- `start`: Starting sample index
+
+**`reset(self) -> None`**
+
+Resets internal counter for `get_next_block()`.
+
+### API Reference: `HDF5LivePlotter`
+
+PyQt5 widget for real-time visualisation of HDF5 files. Can be used standalone or embedded in custom applications.
+
+#### Initialisation
+
+**`__init__(self, hdf5_path: str = "/tmp/data.hdf5", update_interval_ms: int = 50, display_window_seconds: float = 0.5, decimation_factor: int = 150, max_display_points: int = 4000)`**
+
+- `hdf5_path`: Path to HDF5 file
+- `update_interval_ms`: Update frequency (milliseconds)
+- `display_window_seconds`: Duration of data to display (seconds)
+- `decimation_factor`: Decimation factor for display
+- `max_display_points`: Maximum points to display (prevents GUI slowdown)
+
+#### Methods
+
+**`set_hdf5_path(self, hdf5_path: str) -> None`**
+
+Updates monitored HDF5 file path.
+
+**`start_updates(self) -> None`**
+
+Begins periodic plot updates.
+
+**`stop_updates(self) -> None`**
+
+Stops periodic updates.
+
+**`save_screenshot(self) -> None`**
+
+Saves PNG screenshot. Called automatically when 'S' key is pressed.
+
+#### Usage Example
+
+```python
+from PyQt5.QtWidgets import QApplication
+from picostream.dfplot import HDF5LivePlotter
+
+app = QApplication([])
+plotter = HDF5LivePlotter(
+ hdf5_path="my_data.hdf5",
+ display_window_seconds=1.0,
+ decimation_factor=100
+)
+plotter.show()
+plotter.start_updates()
+app.exec_()
+```
## Changelog
### Version 0.2.0
-- **New Feature**: Added live-only mode with `--max-buff-sec` option for buffer size limiting
-- **Enhancement**: Improved plotter to handle buffer resets gracefully
-- **Enhancement**: Added total sample count tracking across buffer resets
-- **Enhancement**: Skip verification step in live-only mode for better performance
-- **Documentation**: Updated README with live-only mode usage examples
+- Added live-only mode with `--max-buff-sec` option
+- Added GUI application for interactive control
+- Improved plotter to handle buffer resets gracefully
+- Added total sample count tracking across buffer resets
+- Skip verification step in live-only mode for better performance
### Version 0.1.0
- Initial release with core streaming and plotting functionality
## Acknowledgements
-This package began as a fork of [JoshHarris2108/pico_streaming](https://github.com/JoshHarris2108/pico_streaming) (which is unlicensed). I acknowledge and appreciate Josh's original idea/architecture.
+This package began as a fork of [JoshHarris2108/pico_streaming](https://github.com/JoshHarris2108/pico_streaming) (unlicensed). Thanks to Josh for the original architecture.