modos.storage#

Attributes#

`ZARR_ROOT`
`S3_ADDRESSING_STYLE`

Classes#

`ElementType`	Enumeration of all element types.
`Storage`	Helper class that provides a standard way to create an ABC using
`LocalStorage`	Helper class that provides a standard way to create an ABC using
`S3Path`	Pydantic Model for S3 URLs. Performs validation against amazon's official naming rules [1] [2]
`S3Storage`	Helper class that provides a standard way to create an ABC using

Functions#

`init_zarr`(zarr_store)	Initialize object's directory and metadata structure.
`connect_s3`(endpoint, s3_kwargs)
`add_metadata_group`(parent_group, metadata)	Add input metadata dictionary to an existing zarr group.
`add_data`(group, data)	Add a numpy array to an existing zarr group.
`list_zarr_items`(group)	Recursively list all zarr groups and arrays

Module Contents#

class modos.storage.ElementType[source]#

Bases: str, enum.Enum

Enumeration of all element types.

SAMPLE = 'sample'#

ASSAY = 'assay'#

DATA_ENTITY = 'data'#

REFERENCE_GENOME = 'reference'#

REFERENCE_SEQUENCE = 'sequence'#

get_target_class()[source]#

Return the target class for the element type.

Return type:: type

classmethod from_object(obj)[source]#: Return the element type from an object.

classmethod from_model_name(name)[source]#

Return the element type from an object name.

Parameters:: name (str)

modos.storage.ZARR_ROOT[source]#

modos.storage.S3_ADDRESSING_STYLE[source]#

class modos.storage.Storage[source]#

Bases: abc.ABC

Helper class that provides a standard way to create an ABC using inheritance.

property path: pathlib.Path[source]#

Abstractmethod:
Return type:: pathlib.Path

property zarr: zarr.hierarchy.Group[source]#

Abstractmethod:
Return type:: zarr.hierarchy.Group

abstract exists(target)[source]#

Parameters:: target (pathlib.Path)
Return type:: bool

abstract put(source, target)[source]#

Parameters:

source (pathlib.Path)
target (pathlib.Path)

abstract remove(target)[source]#

Parameters:: target (pathlib.Path)

abstract list(target)[source]#

Parameters:: target (Optional[pathlib.Path])
Return type:: Generator[pathlib.Path, None, None]

empty()[source]#

Return type:: bool

class modos.storage.LocalStorage(path)[source]#

Bases: Storage

Helper class that provides a standard way to create an ABC using inheritance.

Parameters:: path (pathlib.Path)

property zarr: zarr.hierarchy.Group[source]#

Return type:: zarr.hierarchy.Group

property path: pathlib.Path[source]#

Return type:: pathlib.Path

exists(target)[source]#

Parameters:: target (pathlib.Path)
Return type:: bool

list(target=None)[source]#

Parameters:: target (Optional[pathlib.Path])

remove(target)[source]#

Parameters:: target (pathlib.Path)

put(source, target)[source]#

Parameters:

source (pathlib.Path)
target (pathlib.Path)

class modos.storage.S3Path[source]#

Pydantic Model for S3 URLs. Performs validation against amazon’s official naming rules [1] [2]

Examples

>>> S3Path(url="s3://test/ex")
S3Path(url='s3://test/ex')
>>> S3Path(url='s3://?invalid-bucket-name!/def') 
Traceback (most recent call last):
  ...
pydantic_core._pydantic_core.ValidationError: 1 validation error for S3Path

_s3_pattern: ClassVar[re.Pattern[str]][source]#

url: str[source]#

s3_url_parts()[source]#

property bucket: str[source]#

Return type:: str

property key: str[source]#

Return type:: str

class modos.storage.S3Storage(path, s3_endpoint, s3_kwargs=None)[source]#

Bases: Storage

Helper class that provides a standard way to create an ABC using inheritance.

Parameters:

path (str)
s3_endpoint (pydantic.HttpUrl)
s3_kwargs (Optional[dict[str, Any]])

property path: pathlib.Path[source]#

Return type:: pathlib.Path

property zarr: zarr.hierarchy.Group[source]#

Return type:: zarr.hierarchy.Group

exists(target=ZARR_ROOT)[source]#

Parameters:: target (pathlib.Path)
Return type:: bool

list(target=None)[source]#

Parameters:: target (Optional[pathlib.Path])
Return type:: Generator[pathlib.Path, None, None]

remove(target)[source]#

Parameters:: target (pathlib.Path)

put(source, target)[source]#

Parameters:

source (pathlib.Path)
target (pathlib.Path)

modos.storage.init_zarr(zarr_store)[source]#

Initialize object’s directory and metadata structure.

Parameters:: zarr_store (zarr.storage.Store)
Return type:: zarr.hierarchy.Group

modos.storage.connect_s3(endpoint, s3_kwargs)[source]#

Parameters:

endpoint (pydantic.HttpUrl)
s3_kwargs (dict[str, Any])

Return type:

s3fs.S3FileSystem

modos.storage.add_metadata_group(parent_group, metadata)[source]#

Add input metadata dictionary to an existing zarr group.

Parameters:

parent_group (zarr.hierarchy.Group)
metadata (dict)

Return type:

None

modos.storage.add_data(group, data)[source]#

Add a numpy array to an existing zarr group.

Parameters:: group (zarr.hierarchy.Group)
Return type:: None

modos.storage.list_zarr_items(group)[source]#

Recursively list all zarr groups and arrays

Parameters:: group (zarr.hierarchy.Group)
Return type:: list[zarr.hierarchy.Group | zarr.core.Array]