model_manager package

Submodules

model_manager.config module

Configuration and global variables for the model manager

model_manager.exceptions module

Some exceptions to help with model manager debugging

exception model_manager.exceptions.ModelFilesCorrupted

Bases: Exception

ModelFilesCorrupted indicates that files inside the models directory are not tracked properly

exception model_manager.exceptions.NoOverwrite

Bases: Exception

NoOverwrite indicates that a model file with that model name already exists and the user didn’t use the overwrite parameter

exception model_manager.exceptions.ShouldHaveModel

Bases: Exception

ShouldHaveModel indicates that the model manager should have the model in question

model_manager.model_manager module

A simple and generalized tool for saving and loading of ML models

class model_manager.model_manager.ModelManager(model_dir: str = '/home/docs/checkouts/readthedocs.org/user_builds/digitalpathology-readthedocs/checkouts/latest/docs/source/models')

Bases: object

A system facilitating the saving/loading of ML models of any kind.

Saved models are represented by two files: a serialized model object and a json file containing information about the model. These files are stored in the same directory.

Example use:

model_manager = ModelManager() model_manager.save_model(

model_name = “my_favorite_model”, model = model, model_info = {“description”: “this model is simply the coolest”}

) my_model = model_manager.load_model(“my_favorite_model”)

get_model_info(model_name: str) → dict

get_model_info gets the json information associated with the model

Parameters

model_name (str) – the name of the model whose info to get

Returns

the information associated with the model

Return type

dict

has_model(model_name: str) → bool

has_model returns whether the model manager has the model in question

Parameters

model_name (str) – the name of the model in question

Returns

True if the model manager has a model with that name

Return type

bool

load_model(model_name: str) → Any

load_model loads a model from model_dir

Parameters

model_name (str) – the name of the model to be loaded

Returns

the deserialized model

Return type

Any

property models: List[str]

models is a list of the models accessible by ModelManager

Raises

exceptions.ModelFilesCorrupted – if the model files are of unexpected structure

Returns

a list of model names

Return type

List[str]

save_model(model_name: str, model: Any, model_info: Optional[dict] = None, overwrite_model: bool = False, dependency_modules: Optional[Iterable[module]] = None) → None

save_model saves a model to model_dir with info in a separate json file

Parameters

  • model_name (str) – the name for the model to be saved

  • model (Any) – the model to be saved

  • model_info (dict, optional) – a json-serializable dictionary containing any information the user wishes to store about the model, with the model, defaults to None

  • overwrite_model (bool, optional) – whether to overwrite the model’s files, defaults to False

Raises

exceptions.NoOverwrite – if the model file exists can overwrite wasn’t specified

should_have_model(model_name: str, exception: Optional[str] = None) → None

should_have_model throws an exception if this doesn’t have model_name

Parameters

  • model_name (str) – the name of the model in question

  • exception (str, optional) – an optional error message other than model_name, defaults to None

Raises

exceptions.ShouldHaveModel – if the model manager doesn’t have the model in question

model_manager.model_manager.get_model_name(filepath: str) → str

get_model_name extracts the model name from model file or model info file

Parameters

filepath (str) – the filepath from which to extract the model name

Returns

the filepath’s model name

Return type

str

model_manager.util module

Utility functions for the model manager

model_manager.util.iterate_by_n(collection: Iterable[Any], n: int, yield_remainder: bool = False, error_if_remainder: bool = False) → Iterator[Any]

iterate_by_n returns a generator function returning sets of n items from collection

Parameters

  • collection (Iterable) – the collection to iterate over

  • n (int) – the step size

  • ignore_remainder (bool, optional) – whether to ignore the last len(collection)%n items, defaults to True

Raises

Exception – when there is a remainder and ignore_remainder=False

Yield

a set of n contiguous items from collection

Return type

Iterator[Any]

model_manager.util.open_file(filepath: str, binary: bool = True) → _io.TextIOWrapper

open_file opens a file with ‘w’ or ‘x’ based on whether the file exists

Parameters

  • filepath (str) – the file to be opened

  • binary (bool, optional) – whether to open the file in binary mode, defaults to True

Returns

the opened file

Return type

io.TextIOWrapper

Module contents