modos.storage#
Attributes#
Classes#
Functions#
|
Initialize object's directory and metadata structure. |
|
|
|
Add input metadata dictionary to an existing zarr group. |
|
Add a numpy array to an existing zarr group. |
|
Recursively list all zarr groups and arrays |
Module Contents#
- class modos.storage.Storage[source]#
Bases:
abc.ABC
Helper class that provides a standard way to create an ABC using inheritance.
- property path: pathlib.Path[source]#
- Abstractmethod:
- Return type:
- property zarr: zarr.hierarchy.Group[source]#
- Abstractmethod:
- Return type:
- abstract exists(target)[source]#
- Parameters:
target (pathlib.Path)
- Return type:
- abstract put(source, target)[source]#
- Parameters:
source (pathlib.Path)
target (pathlib.Path)
- abstract remove(target)[source]#
- Parameters:
target (pathlib.Path)
- abstract list(target)[source]#
- Parameters:
target (Optional[pathlib.Path])
- Return type:
Generator[pathlib.Path, None, None]
- class modos.storage.LocalStorage(path)[source]#
Bases:
Storage
Helper class that provides a standard way to create an ABC using inheritance.
- Parameters:
path (pathlib.Path)
- property zarr: zarr.hierarchy.Group[source]#
- Return type:
- property path: pathlib.Path[source]#
- Return type:
- exists(target)[source]#
- Parameters:
target (pathlib.Path)
- Return type:
- list(target=None)[source]#
- Parameters:
target (Optional[pathlib.Path])
- remove(target)[source]#
- Parameters:
target (pathlib.Path)
- put(source, target)[source]#
- Parameters:
source (pathlib.Path)
target (pathlib.Path)
- class modos.storage.S3Path[source]#
Pydantic Model for S3 URLs. Performs validation against amazon’s official naming rules [1] [2]
Examples
>>> S3Path(url="s3://test/ex") S3Path(url='s3://test/ex') >>> S3Path(url='s3://?invalid-bucket-name!/def') Traceback (most recent call last): ... pydantic_core._pydantic_core.ValidationError: 1 validation error for S3Path
- _s3_pattern: ClassVar[re.Pattern[str]][source]#
- class modos.storage.S3Storage(path, s3_endpoint, s3_kwargs=None)[source]#
Bases:
Storage
Helper class that provides a standard way to create an ABC using inheritance.
- property path: pathlib.Path[source]#
- Return type:
- property zarr: zarr.hierarchy.Group[source]#
- Return type:
- exists(target=ZARR_ROOT)[source]#
- Parameters:
target (pathlib.Path)
- Return type:
- list(target=None)[source]#
- Parameters:
target (Optional[pathlib.Path])
- Return type:
Generator[pathlib.Path, None, None]
- remove(target)[source]#
- Parameters:
target (pathlib.Path)
- put(source, target)[source]#
- Parameters:
source (pathlib.Path)
target (pathlib.Path)
- modos.storage.init_zarr(zarr_store)[source]#
Initialize object’s directory and metadata structure.
- Parameters:
zarr_store (zarr.storage.Store)
- Return type:
- modos.storage.add_metadata_group(parent_group, metadata)[source]#
Add input metadata dictionary to an existing zarr group.
- Parameters:
parent_group (zarr.hierarchy.Group)
metadata (dict)
- Return type:
None
- modos.storage.add_data(group, data)[source]#
Add a numpy array to an existing zarr group.
- Parameters:
group (zarr.hierarchy.Group)
- Return type:
None
- modos.storage.list_zarr_items(group)[source]#
Recursively list all zarr groups and arrays
- Parameters:
group (zarr.hierarchy.Group)
- Return type: