modos.storage#

Attributes#

`ZARR_ROOT`
`S3_ADDRESSING_STYLE`

Classes#

`Storage`	Helper class that provides a standard way to create an ABC using
`LocalStorage`	Helper class that provides a standard way to create an ABC using
`S3Path`	Pydantic Model for S3 URLs. Performs validation against amazon's official naming rules [1] [2]
`S3Storage`	Helper class that provides a standard way to create an ABC using

Functions#

`init_zarr`(zarr_store)	Initialize object's directory and metadata structure.
`connect_s3`(endpoint, s3_kwargs)
`add_metadata_group`(parent_group, metadata)	Add input metadata dictionary to an existing zarr group.
`add_data`(group, data)	Add a numpy array to an existing zarr group.
`list_zarr_items`(group)	Recursively list all zarr groups and arrays

Module Contents#

modos.storage.ZARR_ROOT[source]#

modos.storage.S3_ADDRESSING_STYLE[source]#

class modos.storage.Storage[source]#

Bases: abc.ABC

Helper class that provides a standard way to create an ABC using inheritance.

property path: pathlib.Path[source]#

Abstractmethod:
Return type:: pathlib.Path

property zarr: zarr.hierarchy.Group[source]#

Abstractmethod:
Return type:: zarr.hierarchy.Group

abstract exists(target)[source]#

Parameters:: target (pathlib.Path)
Return type:: bool

abstract put(source, target)[source]#

Parameters:

source (pathlib.Path)
target (pathlib.Path)

abstract remove(target)[source]#

Parameters:: target (pathlib.Path)

abstract list(target)[source]#

Parameters:: target (Optional[pathlib.Path])
Return type:: Generator[pathlib.Path, None, None]

abstract open(target)[source]#

Parameters:: target (pathlib.Path)
Return type:: io.BufferedReader

empty()[source]#

Return type:: bool

class modos.storage.LocalStorage(path)[source]#

Bases: Storage

Helper class that provides a standard way to create an ABC using inheritance.

Parameters:: path (pathlib.Path)

_path[source]#

property zarr: zarr.hierarchy.Group[source]#

Return type:: zarr.hierarchy.Group

property path: pathlib.Path[source]#

Return type:: pathlib.Path

exists(target)[source]#

Parameters:: target (pathlib.Path)
Return type:: bool

list(target=None)[source]#

Parameters:: target (Optional[pathlib.Path])

open(target)[source]#

Parameters:: target (pathlib.Path)
Return type:: io.BufferedReader

remove(target)[source]#

Parameters:: target (pathlib.Path)

put(source, target)[source]#

Parameters:

source (pathlib.Path)
target (pathlib.Path)

class modos.storage.S3Path[source]#

Pydantic Model for S3 URLs. Performs validation against amazon’s official naming rules [1] [2]

Examples

>>> S3Path(url="s3://test/ex")
S3Path(url='s3://test/ex')
>>> S3Path(url='s3://?invalid-bucket-name!/def') 
Traceback (most recent call last):
  ...
pydantic_core._pydantic_core.ValidationError: 1 validation error for S3Path