Input/Output¶

Overview¶

ConfUSIus is designed to handle large-scale fUSI datasets efficiently. This guide explains how to work with different data formats and convert them to work with ConfUSIus.

Working with Xarray¶

ConfUSIus uses Xarray as its core data structure for representing multi-dimensional fUSI data. Xarray provides several advantages over raw NumPy arrays:

• Named dimensions: Access data using meaningful names (e.g., time, z, y, x) instead of remembering axis indices.
• Coordinates: Associate physical coordinates with each dimension (e.g., time in seconds, depth in millimeters).
• Metadata storage: Keep acquisition parameters, units, and other metadata alongside your data.
• Unified API: Use the same operations regardless of the underlying storage format.

Xarray-Compatible Formats¶

Xarray can read and write data from multiple storage formats, including:

• Zarr: Chunked, compressed, cloud-native format (recommended for large datasets).
• HDF5: Hierarchical format widely used in research.
• netCDF: Self-describing format common in scientific computing (e.g., basis of the MINC 1.0 format).

Additionally, ConfUSIus provides utilities to read and write NIfTI files (the standard neuroimaging format for BIDS) as Xarray DataArrays, enabling seamless integration with BIDS-compliant fUSI datasets.

Recommended Formats for fUSI¶

ConfUSIus works with all Xarray-compatible formats, but two formats are particularly well-suited for fUSI workflows:

fUSI Data Types¶

fUSI workflows involve two main categories of data:

Converting Beamformed IQ Data¶

Most fUSI acquisition systems output beamformed IQ data in proprietary binary formats. ConfUSIus currently provides built-in conversion utilities for AUTC and EchoFrame systems to transform their data into Zarr for efficient processing.

from confusius.io import convert_autc_dats_to_zarr

convert_autc_dats_to_zarr(
    dats_root="path/to/data_folder",
    output_path="sub-01_task-awake_iq.zarr",
    # Optional: specify block start times, transmit frequency, axis coordinates, and
    # other metadata via keyword arguments (see API for details).
    block_times=block_times,
    compound_sampling_frequency=500.0,
    transmit_frequency=15.625e6,
    sound_velocity=1510.0,
)

from confusius.io import convert_echoframe_dat_to_zarr

convert_echoframe_dat_to_zarr(
    dat_path="path/to/data.dat",
    meta_path="path/to/metadata.mat",
    output_path="sub-01_task-awake_iq.zarr",
    # Optional: specify block start times. Other metadata (e.g., transmit frequency,
    # axis coordinates) will be automatically extracted from the metadata file.
    block_times=block_times,
)

Other Systems¶

If you're working with IQ data from a system other than AUTC or EchoFrame, load it using your own loader and use validate_iq to ensure it meets ConfUSIus requirements before processing:

import xarray as xr
from confusius.validation import validate_iq

# Load IQ data from an unsupported format (example using your own loading function).
iq = your_loader_function("path/to/iq_data")

# Validate the IQ data structure and required attributes.
try:
    iq_validated = validate_iq(iq)
    print("✓ IQ data is valid and ready for processing")
except ValueError as e:
    print(f"✗ Validation failed: {e}")

The validate_iq function checks that your data:

• Has the correct dimensions: (time, z, y, x).
• Is complex-valued (numpy.complex64 or numpy.complex128).
• Contains required attributes:
  • compound_sampling_frequency: Effective IQ sampling frequency in Hz.
  • transmit_frequency: Ultrasound transmit frequency in Hz.
  • sound_velocity: Speed of sound in m/s.

Loading Data¶

All ConfUSIus loaders return lazy DataArrays backed by Dask—data stays on disk until an operation requires it.

da = cf.load("sub-01_task-awake_pwd.nii.gz").compute()

Loading Zarr Files¶

Once your data is in Zarr format, load it with confusius.load:

import confusius as cf

# Load beamformed IQ data (returns the first variable as a DataArray by default).
iq_data = cf.load("sub-01_task-awake_iq.zarr")

# Or specify the variable name if there are multiple variables in the Zarr store.
iq_data = cf.load("sub-01_task-awake_iq.zarr", variable="iq")

print(iq_data)

import xarray as xr

ds = xr.open_zarr("sub-01_task-awake_iq.zarr")

This returns an DataArray lazily loaded from the Zarr store, ready for processing:

<xarray.DataArray 'iq' (time: 1168500, z: 1, y: 118, x: 52)> Size: 57GB
dask.array<open_dataset-iq, shape=(1168500, 1, 118, 52), dtype=complex64, chunksize=(300, 1, 118, 52), chunktype=numpy.ndarray>
Coordinates:
  * time     (time) float64 9MB 5.551 5.553 5.555 ... 2.355e+03 2.355e+03
  * z        (z) float64 8B 0.0
  * y        (y) float64 944B 4.656 4.705 4.753 4.802 ... 10.23 10.28 10.33
  * x        (x) float64 416B -2.671 -2.57 -2.469 -2.369 ... 2.268 2.369 2.469
Attributes:
    transmit_frequency:           15625000.0
    probe_n_elements:             128
    probe_pitch:                  0.0001
    sound_velocity:               1510.0
    plane_wave_angles:            [-10.0, -9.310344696044922, -8.620689392089...
    compound_sampling_frequency:  500.0
    pulse_repetition_frequency:   15000.0
    beamforming_method:           Fourier

Notice that the data remains on disk (shown by dask.array<...>) until you explicitly compute operations on it.

Loading Iconeus SCAN Files¶

Use load_scan to load Iconeus .scan files (HDF5 files produced by IcoScan) as lazy Xarray DataArrays. Three acquisition modes are supported, each yielding a DataArray with different dimensions:

All spatial coordinates are in millimeters; the time coordinate is in seconds.

import confusius as cf

da = cf.load("sub-01_task-awake_pwd.source.scan")

print(da.dims)
# Output: ('time', 'z', 'y', 'x')

import confusius as cf

da = cf.load("sub-01_acq-anat_pwd.source.scan")

print(da.dims)
# Output: ('pose', 'z', 'y', 'x')

import confusius as cf

da = cf.load("sub-01_task-awake_pwd.source.scan")

print(da.dims)
# Output: ('time', 'pose', 'z', 'y', 'x')

The DataArray is loaded lazily: data remains on disk until explicitly computed. SCAN files stay open while the Dask graph is un-computed, so keep the DataArray in scope or call .compute() before discarding it.

Provenance metadata from the file is stored in da.attrs: scan_mode, subject, session, scan, project, date, neuroscan_version, and machine_sn.

Converting SCAN Data to NIfTI¶

Since load_scan returns a standard Xarray DataArray with ConfUSIus-compatible dimensions and coordinates, you can save it directly to NIfTI using save_nifti or the Xarray accessor.

For 2Dscan data, save it directly:

import confusius as cf

da = cf.load("sub-01_task-awake_pwd.scan")
cf.save(da, "sub-01_task-awake_pwd.nii.gz")
# Or equivalently:
da.fusi.save("sub-01_task-awake_pwd.nii.gz")

For 3Dscan and 4Dscan data, consolidate the poses into a single volume before saving, or save each pose separately if you want to retain the multi-pose structure:

import confusius as cf

anat = cf.load("sub-01_acq-anat_pwd.scan")
volume = cf.multipose.consolidate_poses(anat)
cf.save(volume, "sub-01_acq-anat_pwd.nii.gz")

import confusius as cf

anat = cf.load("sub-01_acq-anat_pwd.scan")
for pose in anat.pose:
    pose_da = anat.sel(pose=pose)
    cf.save(pose_da, f"sub-01_acq-anat_pose-{pose.values}.nii.gz")

Loading NIfTI Files¶

Use confusius.load to load NIfTI files as lazy Xarray DataArrays:

import confusius as cf

# Load with automatic BIDS sidecar metadata.
da = cf.load("sub-01_task-awake_pwd.nii.gz")
print(da.dims)
# Output: ('time', 'z', 'y', 'x')

ConfUSIus automatically loads a JSON sidecar file with the same basename (e.g., sub-01_task-awake_pwd.json) if present, following the BIDS specification for fUSI metadata.

Saving Data¶

You can save DataArrays to NIfTI and Zarr using confusius.save or the Xarray accessor:

import confusius as cf

# Save to NIfTI with automatic JSON sidecar creation.
cf.save(data_array, "output.nii.gz")

# Save to Zarr.
cf.save(data_array, "output.zarr")

import confusius

# Save to NIfTI with automatic JSON sidecar creation.
data_array.fusi.save("output.nii.gz")

# Save to Zarr.
data_array.fusi.save("output.zarr")

When saving to NIfTI, a JSON sidecar file will be automatically created. Spatial coordinates and units are encoded in the NIfTI header itself; the sidecar stores custom attributes and BIDS timing fields (e.g., RepetitionTime, DelayAfterTrigger, VolumeTiming).

Format Conversion Reference¶

Quick reference for converting between formats: