neurotic.datasets.metadata

The neurotic.datasets.metadata module implements a class for reading metadata files.

class neurotic.datasets.metadata.MetadataSelector(file=None, local_data_root=None, remote_data_root=None, initial_selection=None)

A class for managing metadata.

A metadata file can be specified at initialization, in which case it is read immediately. The file contents are stored as a dictionary in all_metadata.

>>> metadata = MetadataSelector(file='metadata.yml')
>>> print(metadata.all_metadata)

File contents can be reloaded after they have been changed, or after changing file, using the load() method.

>>> metadata = MetadataSelector()
>>> metadata.file = 'metadata.yml'
>>> metadata.load()

A particular metadata set contained within the file can be selected at initialization with initial_selection or later using the select() method. After making a selection, the selected metadata set is accessible at metadata.selected_metadata, e.g.

>>> metadata = MetadataSelector(file='metadata.yml')
>>> metadata.select('Data Set 5')
>>> print(metadata.selected_metadata['data_file'])

A compact indexing method is implemented that allows the selected metadata set to be accessed directly, e.g.

>>> print(metadata['data_file'])

This allows the MetadataSelector to be passed to functions expecting a simple dictionary corresponding to a single metadata set, and the selected metadata set will be used automatically.

Files associated with the selected metadata set can be downloaded individually or all together, e.g.

>>> metadata.download('video_file')

or

>>> metadata.download_all_data_files()

The absolute path to a local file or the full URL to a remote file associated with the selected metadata set can be resolved with the abs_path() and abs_url() methods, e.g.

>>> print(metadata.abs_path('data_file'))
>>> print(metadata.abs_url('data_file'))

abs_path(file)

Convert the relative path of file to an absolute path using data_dir.

abs_url(file)

Convert the relative path of file to a full URL using remote_data_dir.

all_metadata = None

A dictionary containing the entire file contents, set by load().

download(file, **kwargs)

Download a file associated with the selected metadata set.

See neurotic.datasets.download.download() for possible keyword arguments.

download_all_data_files(**kwargs)

Download all files associated with the selected metadata set.

See neurotic.datasets.download.download() for possible keyword arguments.

keys

The available metadata keys.

load()

Read the metadata file.

select(selection)

Select a metadata set.

selected_metadata

The access point for the selected metadata set.