API Reference¶
-
class
mincepy.Archive[source]¶ An archive provides the persistent storage for the historian. It is responsible for storing, searching and loading data records and their metadata.
-
class
MetaEntry(obj_id, meta)¶ Create new instance of MetaEntry(obj_id, meta)
-
meta¶ Alias for field number 1
-
obj_id¶ Alias for field number 0
-
-
class
RefEdge(source, target)¶ Create new instance of RefEdge(source, target)
-
source¶ Alias for field number 0
-
target¶ Alias for field number 1
-
-
bulk_write(ops: Sequence[mincepy.operations.Operation])[source]¶ Made a collection of write operations to the database
-
construct_archive_id(value) → IdT[source]¶ If it’s possible, construct an archive value from the passed value. This is useful as a convenience to the user if, say, the archive id can be constructed from a string. Raise TypeError or ValueError if this is not possible for the given value.
-
count(obj_id=None, type_id=None, created_by=None, copied_from=None, version=-1, state=None, snapshot_hash=None, meta=None, limit=0)[source]¶ Count the number of entries that match the given query
-
create_file(filename: str = None, encoding: str = None)[source]¶ Create a new file object specific for this archive type
-
find(obj_id: Union[IdT, Iterable[IdT], Dict[KT, VT]] = None, type_id=None, created_by=None, copied_from=None, version=None, state=None, state_types=None, snapshot_hash=None, meta=None, limit=0, sort=None, skip=0) → Iterator[mincepy.records.DataRecord][source]¶ Find records matching the given criteria
Parameters: - type_id – the type id to look for
- created_by – find records with the given type id
- copied_from – find records copied from the record with the given id
- version – restrict the search to this version, -1 for latest
- state – find objects with this state filter
- state_types – file objects with this state types filter
- snapshot_hash – find objects with this snapshot hash
- meta – find objects with this meta filter
- limit – limit the results to this many records
- obj_id – an optional restriction on the object ids to search. This ben be either: 1. a single object id 2. an iterable of object ids in which is treated as {‘$in’: list(obj_ids)} 3. a general query filter to be applied to the object ids
- sort – sort the results by the given criteria
- skip – skip the this many entries
-
get_reference_graph(sids: Sequence[mincepy.records.SnapshotId[~IdT][IdT]]) → Sequence[Iterable[mincepy.archives.RefEdge]][source]¶ Given one or more object ids the archive will supply the corresponding reference graph(s)
-
get_snapshot_refs(obj_id: IdT) → Sequence[mincepy.records.SnapshotId[~IdT][IdT]][source]¶ Returns a list of time ordered snapshot references
-
classmethod
get_types() → Sequence[T_co][source]¶ This method allows the archive to return either types or type helper that the historian should support. A common example is the type helper for the object id type
-
history(obj_id: IdT, idx_or_slice) → [<class 'mincepy.records.DataRecord'>, typing.Sequence[mincepy.records.DataRecord]][source]¶ Load the snapshot records for a particular object, can return a single or multiple records
-
load(reference: mincepy.records.SnapshotId[~IdT][IdT]) → mincepy.records.DataRecord[source]¶ Load a snapshot of an object with the given reference
-
meta_create_index(keys, unique=False, where_exist=False)[source]¶ Create an index on the metadata. Takes either a single key or list of (key, direction) pairs
Parameters: - keys – the key or keys to create the index on
- unique – if True, create a uniqueness constraint on this index
- where_exist – if True the index only applies for documents where the key(s) exist
-
meta_find(filter: dict, obj_id: Union[IdT, Iterable[IdT], Mapping[KT, VT_co]] = None) → Iterator[mincepy.archives.MetaEntry][source]¶ Yield metadata satisfying the given criteria. The search can optionally be restricted to a set of passed object ids.
Parameters: - filter – a query filter for the search
- obj_id – an optional restriction on the object ids to search. This ben be either: 1. a single object id 2. an iterable of object ids in which is treated as {‘$in’: list(obj_ids)} 3. a general query filter to be applied to the object ids
-
meta_get_many(obj_ids: Iterable[IdT]) → Dict[KT, VT][source]¶ Get the metadata for multiple objects. Returns a dictionary mapping the object id to the metadata dictionary
-
meta_set(obj_id: IdT, meta: Optional[Mapping[KT, VT_co]])[source]¶ Set the metadata on on the object with the corresponding id
-
meta_set_many(metas: Mapping[IdT, Optional[Mapping[KT, VT_co]]])[source]¶ Set the metadata on multiple objects. This method expects to get multiple tuples containing the object id and corresponding metadata
-
meta_update(obj_id: IdT, meta: Mapping[KT, VT_co])[source]¶ Update the metadata on the object with the corresponding id
-
class
-
class
mincepy.BaseArchive[source]¶ -
construct_archive_id(value) → IdT[source]¶ If it’s possible, construct an archive value from the passed value. This is useful as a convenience to the user if, say, the archive id can be constructed from a string. Raise TypeError or ValueError if this is not possible for the given value.
-
history(obj_id: IdT, idx_or_slice) → [<class 'mincepy.records.DataRecord'>, typing.Sequence[mincepy.records.DataRecord]][source]¶ Load the snapshot records for a particular object, can return a single or multiple records
-
meta_get_many(obj_ids: Iterable[IdT]) → Dict[IdT, dict][source]¶ Get the metadata for multiple objects. Returns a dictionary mapping the object id to the metadata dictionary
-
meta_set_many(metas: Mapping[IdT, Mapping[KT, VT_co]])[source]¶ Set the metadata on multiple objects. This method expects to get multiple tuples containing the object id and corresponding metadata
-
-
class
mincepy.SimpleHelper[source]¶ -
-
load_instance_state(obj, saved_state, loader)[source]¶ Take the given blank object and load the instance state into it
-
-
class
mincepy.SnapshotLoader(historian)[source]¶ Responsible for loading snapshots. This object should not be reused and only one external call to load should be made. This is because it keeps an internal cache.
-
class
mincepy.LiveDepositor(historian)[source]¶ Depositor with strategy that all objects that get referenced should be saved
-
ref(obj) → Optional[mincepy.records.SnapshotId][source]¶ Get a persistent reference for the given object
-
-
exception
mincepy.ModificationError[source]¶ Raised when a modification of the history encountered a problem
-
exception
mincepy.ObjectDeleted[source]¶ Raise when the user tries to interact with a deleted object
-
class
mincepy.Historian(archive: mincepy.archives.Archive, equators=())[source]¶ The historian acts as a go-between between your python objects and the archive which is a persistent store of the records. It will keep track of all live objects (i.e. those that have active references to them) that have been loaded and/or saved as well as enabling the user to lookup objects in the archive.
-
create_file(filename: str = None, encoding: str = None) → mincepy.builtins.File[source]¶ Create a new file. The historian will supply file type compatible with the archive in use.
-
current_transaction() → Optional[mincepy.transactions.Transaction][source]¶ Get the current transaction if there is one, otherwise returns None
-
delete(obj_or_identifier)[source]¶ Delete an object.
Raises: mincepy.NotFound – if the object cannot be found (potentially because it was already deleted)
-
find(obj_type=None, obj_id=None, version: int = -1, state=None, meta: dict = None, sort=None, limit=0, skip=0) → Iterator[Any][source]¶ Find objects. This call will search the archive for objects matching the given criteria. In many cases the main arguments of interest will be state and meta which allow you to apply filters on the stored state of the object and metadata respectively. To understand how the state is stored in the database (and therefore how to apply filters to it) it may be necessary to look at the details of the save_instance_state() method for that type. Metadata is always a dictionary containing primitives (strings, dicts, lists, etc).
For the most part, the filter syntax of mincepy conforms to that of MongoDB with convenience functions locate in
mincepy.qopsthat can make it easier to to build a query.Examples:
Find all :py:class:`~mincepy.testing.Car`s that are brown or red:
>>> import mincepy >>> historian.find(state=dict(colour=mincepy.q.in_('brown', 'red')))
Find all people that are older than 34 and live in Edinburgh:
>>> historian.find(state=dict(age=mincepy.q.gt_(34)), meta=dict(city='Edinburgh'))
Parameters: - obj_type – the object type to look for
- obj_id – an object or multiple object ids to look for
- version – the version of the object to retrieve, -1 means latest
- state (must be subclass of historian.primitive) – the criteria on the state of the object to apply
- meta – the search criteria to apply on the metadata of the object
- sort – the sort criteria
- limit – the maximum number of results to return, 0 means unlimited
- skip – the page to get results from
-
find_records(obj_type=None, obj_id=None, version: int = -1, state=None, meta: dict = None, sort=None, limit=0, skip=0) → Iterator[mincepy.records.DataRecord][source]¶ Find records
Parameters: - obj_type – the object type to look for
- obj_id – an object or multiple object ids to look for
- version – the version of the object to retrieve, -1 means latest
- state (must be subclass of historian.primitive) – the criteria on the state of the object to apply
- meta – the search criteria to apply on the metadata of the object
- sort – the sort criteria
- limit – the maximum number of results to return, 0 means unlimited
- skip – the page to get results from
-
get_current_record(obj) → mincepy.records.DataRecord[source]¶ Get a record for an object known to the historian
-
get_obj_id(obj) → Any[source]¶ Get the object ID for a live object.
Returns: the object id or None if the object is not known to the historian
-
get_snapshot_id(obj) → mincepy.records.SnapshotId[source]¶ Get the current snapshot reference for a live object
-
get_snapshot_ref(obj) → mincepy.records.SnapshotId[source]¶ Get a reference for this data record
Deprecated since version 0.13.2: This will be removed in 0.14.0. Use .get_snapshot_id instead
-
history(obj_or_obj_id, idx_or_slice='*', as_objects=True) → [typing.Sequence[mincepy.historians.ObjectEntry], typing.Sequence[mincepy.records.DataRecord]][source]¶ Get a sequence of object ids and instances from the history of the given object.
Parameters: - obj_or_obj_id – The instance or id of the object to get the history for
- idx_or_slice – The particular index or a slice of which historical versions to get
- as_objects – if True return the object instances, otherwise returns the DataRecords
Example: >>> historian = get_historian() >>> car = Car(‘ferrari’, ‘white’) >>> car_id = historian.save(car) >>> car.colour = ‘red’ >>> historian.save(car) >>> history = historian.history(car_id) >>> len(history) 2 >>> history[0].obj.colour == ‘white’ True >>> history[1].obj.colour == ‘red’ True >>> history[1].obj is car
-
is_known(obj) → bool[source]¶ Check if an object has ever been saved and is therefore known to the historian
Returns: True if ever saved, False otherwise
-
is_primitive(obj) → bool[source]¶ Check if the object is one of the primitives and should be saved by value in the archive
-
is_saved(obj) → bool[source]¶ Test if an object is saved with this historian. This is equivalent to historian.get_obj_id(obj) is not None.
-
classmethod
is_trackable(obj)[source]¶ Determine if an object is trackable i.e. we can treat these as live objects and automatically keep track of their history when saving. Ultimately this is determined by whether the type is weak referencable or not.
-
meta¶ Access to functions that operate on the metadata
-
migrations¶ Access the migration possibilities
-
primitives¶ A tuple of all the primitive types
-
replace(old, new)[source]¶ Replace a live object with a new version.
This is especially useful if you have made a copy of an object and modified it but you want to continue the history of the object as the original rather than a brand new object. Then just replace the old object with the new one by calling this function.
-
save(*objs)[source]¶ Save multiple objects producing corresponding object identifiers. This returns a sequence of ids that is in the same order as the passed objects.
Parameters: objs – the object(s) to save. Can also be a tuple of (obj, meta) to optionally include metadata to be saved with the object(s)
-
save_one(obj, meta: dict = None)[source]¶ Save the object returning an object id. If metadata is supplied it will be set on the object.
Developer note: this is the front end point-of-entry for a user/client code saving an object however subsequent objects being saved in this transaction will only go through _save_object and therefore any code common to all objects being saved should possibly go there.
-
sync(obj) → bool[source]¶ Update an object with the latest state in the database. If there is no new version in the archive then the current version remains unchanged including any modifications.
Returns: True if the object was updated, False otherwise
-
to_obj_id(obj_or_identifier)[source]¶ This call will try and get an object id from the passed parameter. There are three possibilities:
- Passed an object ID in which case it will be returned unchanged
- Passed a live object instance, in which case the id of that object will be returned
- Passed a type that can be understood by the archive as an object id e.g. a string of version, in which case the archive will attempt to convert it
Returns None if neither of these cases were True.
-
-
class
mincepy.ObjectEntry(ref, obj)¶ Create new instance of ObjectEntry(ref, obj)
-
obj¶ Alias for field number 1
-
ref¶ Alias for field number 0
-
-
class
mincepy.Savable[source]¶ Interface for an object that can save an load its instance state
-
class
mincepy.TypeHelper[source]¶ This interface provides the basic methods necessary to enable a type to be compatible with the historian.
-
ensure_up_to_date(saved_state, version: Optional[int], loader: mincepy.depositors.Loader)[source]¶ Apply any migrations that are necessary to this saved state. If no migrations are necessary then None is returned
-
get_version() → Optional[int][source]¶ Gets the version of the latest migration, returns None if there is not migration
-
load_instance_state(obj, saved_state, loader: mincepy.depositors.Loader)[source]¶ Take the given blank object and load the instance state into it
-
-
class
mincepy.WrapperHelper(obj_type: Type[mincepy.types.SavableObject])[source]¶ Wraps up an object type to perform the necessary Historian actions
-
load_instance_state(obj, saved_state: mincepy.types.Savable, loader)[source]¶ Take the given blank object and load the instance state into it
-
-
class
mincepy.BaseHelper[source]¶ A base helper that defaults to yielding hashables directly on the object and testing for equality using == given two objects. This behaviour is fairly standard and therefor more type helpers will want to subclass from this class.
-
mincepy.get_historian() → mincepy.historians.Historian[source]¶ Get the currently set global historian. If one doesn’t exist create_default_historian will be called to try and make a new one
-
mincepy.set_historian(new_historian: Optional[mincepy.historians.Historian], apply_plugins=True)[source]¶ Set the current global historian. Optionally load all plugins. To reset the historian pass None.
-
mincepy.create_historian(archive_uri: str, apply_plugins=True) → mincepy.historians.Historian[source]¶ Convenience function to create a standard historian directly from an archive URI
-
class
mincepy.ObjRef(obj=None)[source]¶ A reference to an object instance
-
load_instance_state(saved_state, loader)[source]¶ Take the given object and load the instance state into it
-
-
class
mincepy.DataRecord[source]¶ An immutable record that describes a snapshot of an object
Create new instance of DataRecord(obj_id, type_id, creation_time, version, state, state_types, snapshot_hash, snapshot_time, extras)
-
child_builder(**kwargs) → mincepy.utils.NamedTupleBuilder[mincepy.records.DataRecord][mincepy.records.DataRecord][source]¶ Get a child builder from this DataRecord instance. The following attributes will be copied over:
- obj_id
- type_id
- creation_time
- created_by
and version will be incremented by one.
-
copy_builder(**kwargs) → mincepy.utils.NamedTupleBuilder[mincepy.records.DataRecord][mincepy.records.DataRecord][source]¶ Get a copy builder from this DataRecord instance. The following attributes will be copied over:
- type_id
- state [deepcopy]
- snapshot_hash
- extras [deepcopy] - the COPIED_FROM entry will be set to a reference to this object
the version will be set to 0 and the creation time to now.
-
created_by¶ Convenience property to get the creator from the extras
-
classmethod
defaults() → dict[source]¶ Returns a dictionary of default values, the caller owns the dict and is free to modify it
-
get_copied_from() → Optional[mincepy.records.SnapshotId][source]¶ Get the reference of the data record this object was originally copied from
-
get_extra(name)[source]¶ Convenience function to get an extra from the record, returns None if the extra doesn’t exist
-
get_reference() → mincepy.records.SnapshotId[source]¶ Get a reference for this data record
Deprecated since version 0.13.2: This will be removed in 0.14.0. Use .snapshot_id instead
-
get_references() → Iterable[Tuple[Sequence[Union[str, int]], mincepy.records.SnapshotId]][source]¶ Get all the references to other objects contained in this record
-
get_state_schema() → Mapping[tuple, mincepy.records.SchemaEntry][source]¶ Get the schema for the state. This contains the types and versions for each member of the state
-
classmethod
new_builder(**kwargs) → mincepy.utils.NamedTupleBuilder[mincepy.records.DataRecord][mincepy.records.DataRecord][source]¶ Get a builder for a new data record, the version will be set to 0
-
snapshot_id¶ The snapshot id for this record
-
-
mincepy.SnapshotRef¶ alias of
mincepy.records.SnapshotId
-
class
mincepy.SnapshotId(obj_id, version: int)[source]¶ A snapshot id identifies a particular version of an object (and the corresponding record), it it therefore composed of the object id and the version number.
Create a snapshot id by passing an object id and version
-
TYPE_ID= UUID('633c7035-64fe-4d87-a91e-3b7abd8a6a28')¶ The type id for references
-
-
class
mincepy.BaseSavableObject[source]¶ A helper class that makes a class compatible with the historian by flagging certain attributes which will be saved/loaded/hashed and compared in __eq__. This should be an exhaustive list of all the attributes that define this class. If more complex functionality is needed then the standard SavableComparable interface methods should be overwritten.
-
load_instance_state(saved_state, _loader)[source]¶ Take the given object and load the instance state into it
-
-
class
mincepy.ConvenienceMixin[source]¶ A mixin that adds convenience methods to your savable object
-
mincepy.AsRef(name: str) → mincepy.base_savable.AttrSpec[source]¶ Create an attribute specification for an attribute that should be stored by reference
-
class
mincepy.LiveRefList(init_list=None)[source]¶ A live list that uses references to store objects
-
class
mincepy.RefList(init_list=None)[source]¶ A list that stores all entries as references in the database except primitives
-
DATA_TYPE¶ alias of
builtins.list
-
-
class
mincepy.RefDict(*args, **kwargs)[source]¶ A dictionary that stores all values as references in the database.
-
DATA_TYPE¶ alias of
builtins.dict
-
-
class
mincepy.LiveRefDict(*args, **kwargs)[source]¶ A live dictionary that uses references to refer to contained objects
-
mincepy.BaseFile¶ alias of
mincepy.builtins.File
-
class
mincepy.File(filename: str = None, encoding=None)[source]¶ -
-
open(mode='r', **kwargs) → BinaryIO[source]¶ Open returning a file like object that supports close() and read()
-
read_text(encoding=None) → str[source]¶ Read the contents of the file as text. This function is named as to mirror pathlib.Path
-
-
class
mincepy.mongo.MongoArchive(database: pymongo.database.Database)[source]¶ MongoDB implementation of the mincepy archive
-
ID_TYPE¶ alias of
bson.objectid.ObjectId
-
bulk_write(ops: Sequence[mincepy.operations.Operation])[source]¶ Made a collection of write operations to the database
-
construct_archive_id(value) → bson.objectid.ObjectId[source]¶ If it’s possible, construct an archive value from the passed value. This is useful as a convenience to the user if, say, the archive id can be constructed from a string. Raise TypeError or ValueError if this is not possible for the given value.
-
count(obj_id: Optional[bson.objectid.ObjectId] = None, type_id=None, _created_by=None, _copied_from=None, version=-1, state=None, snapshot_hash=None, meta=None, limit=0)[source]¶ Count the number of entries that match the given query
-
create_file(filename: str = None, encoding: str = None)[source]¶ Create a new file object specific for this archive type
-
find(obj_id: Union[bson.objectid.ObjectId, Iterable[bson.objectid.ObjectId], Dict[KT, VT]] = None, type_id: Union[bson.objectid.ObjectId, Iterable[bson.objectid.ObjectId], Dict[KT, VT]] = None, _created_by=None, _copied_from=None, version=None, state=None, state_types=None, snapshot_hash=None, meta=None, limit=0, sort=None, skip=0)[source]¶ Find records matching the given criteria
Parameters: - type_id – the type id to look for
- created_by – find records with the given type id
- copied_from – find records copied from the record with the given id
- version – restrict the search to this version, -1 for latest
- state – find objects with this state filter
- state_types – file objects with this state types filter
- snapshot_hash – find objects with this snapshot hash
- meta – find objects with this meta filter
- limit – limit the results to this many records
- obj_id – an optional restriction on the object ids to search. This ben be either: 1. a single object id 2. an iterable of object ids in which is treated as {‘$in’: list(obj_ids)} 3. a general query filter to be applied to the object ids
- sort – sort the results by the given criteria
- skip – skip the this many entries
-
get_reference_graph(sids: Sequence[mincepy.records.SnapshotId[bson.objectid.ObjectId][bson.objectid.ObjectId]]) → Sequence[Iterable[mincepy.archives.RefEdge]][source]¶ Given one or more object ids the archive will supply the corresponding reference graph(s)
-
get_snapshot_refs(obj_id: bson.objectid.ObjectId)[source]¶ Returns a list of time ordered snapshot references
-
classmethod
get_types() → Sequence[T_co][source]¶ This method allows the archive to return either types or type helper that the historian should support. A common example is the type helper for the object id type
-
load(reference: mincepy.records.SnapshotId) → mincepy.records.DataRecord[source]¶ Load a snapshot of an object with the given reference
-
meta_create_index(keys, unique=True, where_exist=False)[source]¶ Create an index on the metadata. Takes either a single key or list of (key, direction) pairs
Parameters: - keys – the key or keys to create the index on
- unique – if True, create a uniqueness constraint on this index
- where_exist – if True the index only applies for documents where the key(s) exist
-
meta_find(filter: dict, obj_id: Union[bson.objectid.ObjectId, Iterable[bson.objectid.ObjectId], Dict[KT, VT]] = None) → Iterator[Tuple[bson.objectid.ObjectId, Dict[KT, VT]]][source]¶ Yield metadata satisfying the given criteria. The search can optionally be restricted to a set of passed object ids.
Parameters: - filter – a query filter for the search
- obj_id – an optional restriction on the object ids to search. This ben be either: 1. a single object id 2. an iterable of object ids in which is treated as {‘$in’: list(obj_ids)} 3. a general query filter to be applied to the object ids
-
meta_get(obj_id: Union[bson.objectid.ObjectId, Iterable[bson.objectid.ObjectId]])[source]¶ Get the metadata for an objects.
-
meta_get_many(obj_ids: Iterable[bson.objectid.ObjectId]) → Dict[bson.objectid.ObjectId, dict][source]¶ Get the metadata for multiple objects. Returns a dictionary mapping the object id to the metadata dictionary
-
-
class
mincepy.mongo.GridFsFile(file_bucket: gridfs.GridFSBucket, filename: str = None, encoding: str = None)[source]¶ -
load_instance_state(saved_state, loader: mincepy.depositors.Loader)[source]¶ Take the given object and load the instance state into it
-