KERI DB API

keri.db.basing

KERI keri.db.basing module

class keri.db.basing.Baser(headDirPath=None, reopen=False, **kwa)[source]

Baser sets up named sub databases with Keri Event Logs within main database

see superclass LMDBer for inherited attributes

kevers

read-through cache of Kever instances indexed by identifier prefix qb64

Type:: dbdict

prefixes

local prefixes corresponding to habitats for this db

Type:: OrderedSet

groups

group hab identifier prefixes for this db

Type:: OrderedSet

.evts is named subDB instance of SerderSuber whose values are serialized: key events subkey ‘evts.’ dgKey (prefix + digest) DB is keyed by identifier prefix plus digest of serialized event Only one value per DB key is allowed

.fels is named subDB instance of OnSuber for first seen event logs

as indices mapping first-seen ordinal fn to event digests. Actual serialized key events are stored in .evts by SAID digest Indexed in first-seen accepted order for replay and cloning. subkey ‘fels.’ Key: identifier prefix + monotonically increasing fn. Value: qb64 str of event digest used to lookup event in .evts. Only one value per DB key is allowed. Append-only ordering of accepted first-seen events.

Type:: FEL

.kels is named subDB instance of OnIoDupSuber for key event logs as indices: mapping composite key “<pre><sep><on>” to serialized key event digests. Actual serialized key events are stored in .evts by SAID digest. subkey ‘kels.’ Key: identifier prefix + sequence number. Value: qb64 digest used to lookup event in .evts. More than one value per DB key is allowed.

.dtss is named subDB instance of CesrSuber

stamps of when the event was first escrowed and then later first seen by log. Used for escrow timeouts and extended validation. subkey ‘dtss.’ dgKey (prefix + digest) Value: Dater instance Only one value per DB key is allowed.

Type:: klas=Dater

.aess is named subDB instance of CatCesrSuber

for authorizing event source seal couples that map digest of key event to seal source couple of authorizer’s (delegator or issuer) event. subkey ‘aess.’ dgKey (prefix + digest) Value: (Number, Diger) tuple; Number serialized as Huge (fixed 24-char), used to lookup authorizer’s source event in .kels. Only one value per DB key is allowed.

Type:: klas=(Number, Diger)

.sigs is named subDB instance of CesrIoSetSuber

fully qualified indexed event signatures from the controller. subkey ‘sigs.’ dgKey (prefix + digest) More than one value per DB key is allowed.

Type:: klas=Siger

.wigs is named subDB instance of CesrIoSetSuber

indexed witness signatures of events that may come directly or be derived from a witness receipt message. Witnesses always have nontransferable identifier prefixes. The index is the offset of the witness into the witness list of the most recent establishment event wrt the receipted event. subkey ‘wigs.’ dgKey (prefix + digest) More than one value per DB key is allowed.

Type:: klas=Siger

.rcts is named subDB instance of CatCesrIoSetSuber

for event receipt couplets from nontransferable signers. These are endorsements from nontransferable signers who are not witnesses May be watchers or other Each entry is a (Prefixer, Cigar) duple. subkey ‘rcts.’ dgKey (prefix + digest) Multiple values per key stored as ordered set (duplicates ignored, insertion order preserved).

Type:: klas=(Prefixer, Cigar)

.ures is named subDB instance of CatCesrIoSetSuber: (klas=(Diger, Prefixer, Cigar)) for unverified event receipt escrowed triples from nontransferable signers. Each triple is (receipted event digest, receiptor prefix, receipt signature). Used to escrow receipt couples until the receipted event appears. subkey ‘ures.’ snKey (prefix + sequence number) More than one value per DB key is allowed.

.vrcs is named subDB instance of CatCesrIoSetSuber: (klas=(Prefixer, Number, Diger, Siger)) for verified transferable- validator receipt quadruples. Each stored value is a typed CESR tuple (Prefixer, Number, Diger, Siger) representing a validator’s AID, its latest establishment-event sequence number, digest, and its indexed signature over the event. Values preserved in insertion order. Represents fully validated receipts moved out of escrow. subkey ‘vrcs.’ dgKey (prefix + digest) Multiple values per key stored as ordered set.

.vres is named subDB instance of CatCesrIoSetSuber for escrowed: transferable-receipt quintuples. Each value is a typed CESR tuple (Diger, Prefixer, Number, Diger, Siger) representing a validator’s receipt escrow entry. Holds unverified transferable receipts until validated and moved into .vrcs. subkey ‘vres.’ snKey (prefix + sequence number) Values stored in insertion order.

.pses is named subDB instance of OnIoDupSuber for partially-signed: event escrows under composite keys “<pre><sep><on>”. Tracks events with at least one verified signature but not yet fully validated due to missing signatures or dependent events. subkey ‘pses.’ Key: identifier prefix + sequence number. Values stored in insertion order.

.pwes is named subDB instance of OnIoDupSuber for partially witnessed: key event escrows under composite keys “<pre><sep><on>” to serialized event digest. Escrows events with verified signatures but not yet verified witness receipts. subkey ‘pwes.’ Key: identifier prefix + sequence number. More than one value per DB key is allowed.

.pdes is named subDB instance of OnIoDupSuber for partially delegated: key event escrows that map prefix + sequence number to serialized event digest. Used in conjunction with .udes which escrows the associated seal source couple. subkey ‘pdes.’ snKey (prefix + sequence number) More than one value per DB key is allowed.

.udes is named subDB instance of CatCesrSuber

for unverified delegation seal source couple escrows that map (prefix, digest) of delegated event to delegating seal source couple (sn, dig) that provides the source delegator event seal. Each couple is (Number(num=sn).qb64b, Diger.qb64b) used to lookup the source event in the delegator’s KEL. Once accepted, entries move into .aess. subkey ‘udes.’ dgKey (prefix + digest) Only one value per DB key is allowed.

Type:: klas=(Number, Diger)

.uwes is named subDB instance of B64OnIoSetSuber for unverified event: indexed escrowed couples from witness signers. Each couple is (edig, wig) where edig is receipted event digest and wig is the indexed witness signature derived from the witness nontrans prefix and offset into the witness list of the latest establishment event. subkey ‘uwes.’ Key: receipted event controller prefix + sequence number. Multiple values per key are stored in insertion order as a set.

.ooes is named subDB instance of OnIoDupSuber for out-of-order event: escrows under composite keys “<pre><sep><on>”. Tracks events whose prior event has not yet been accepted into the KEL. subkey ‘ooes.’ Key: identifier prefix + sequence number. Values stored in insertion order.

.dels is named subDB instance of OnIoDupSuber for duplicitous event: log tables that map identifier prefix plus sequence number to serialized event digests. subkey ‘dels.’ snKey (prefix + sequence number) Values are qb64 digests used to lookup event in .evts. More than one value per DB key is allowed (insertion ordered).

.ldes is named subDB instance of OnIoDupSuber for likely duplicitous: escrowed event tables that map identifier prefix plus sequence number to serialized event digests. subkey ‘ldes.’ snKey (prefix + sequence number) Values are qb64 digests used to lookup event in .evts. More than one value per DB key is allowed (insertion ordered).

.qnfs is named subDB instance of IoSetSuber for queued not-first-seen: event escrows. Maps (prefix, said) to event digest. subkey ‘qnfs.’ dupsort=True More than one value per DB key is allowed.

.fons is named subDB instance of CesrSuber

prefix and digest to fn value (first seen ordinal number) of the associated event. Given pre and event digest, retrieve fn here then fetch event from .fels. Ensures any event looked up this way was first seen at some point, even if later superseded by a recovery rotation. Direct lookup in .evts could return escrowed events that may never have been accepted as first seen. subkey ‘fons.’ dgKey (prefix + digest) Only one value per DB key is allowed.

Type:: klas=Number

.migs is named subDB instance of CesrSuber

completed migrations. Maps migration module name to the Dater timestamp of when it was run. subkey ‘migs.’ Key: migration name str. Only one value per DB key is allowed.

Type:: klas=Dater

.vers is named subDB instance of Suber storing the current database: schema version string. subkey ‘vers.’

.esrs is named subDB instance of Komer

tracking the source of each event. When .local is Truthy the event was sourced in a protected way (generated locally or via a protected path). When .local is Falsey the event was NOT sourced in a protected way. The value of .local determines what validation logic to run. Used to track source when processing escrows that would otherwise be decoupled from the original source of the event. subkey ‘esrs.’ dgKey (prefix + digest) Only one value per DB key is allowed.

Type:: schema=EventSourceRecord

.misfits is named subDB instance of OnIoSetSuber for misfit escrows.: Events with remote (nonlocal) sources that are inappropriate (i.e. would be dropped) unless promoted to local source via extra after-the-fact authentication. Escrow processing determines if and how to promote event source to local and then reprocess. subkey ‘mfes.’ snKey (prefix + sequence number) Value: qb64b digest of event.

.delegables is named subDB instance of IoSetSuber for delegable event: escrows of key events with a local delegator that need approval. Approval is via anchoring of the delegated event seal in the delegator’s KEL. Event source must be local. A nonlocal (remote) source must first pass through .misfits and be promoted to local. subkey ‘dees.’ snKey (prefix + sequence number) Value: qb64b digest of event.

.states is named subDB instance of Komer

mapping a prefix to its latest key state. Used as read-through cache backing .kevers to reload Kever instances from persistent storage. subkey ‘stts.’ Key: identifier prefix. Only one value per DB key is allowed.

Type:: schema=KeyStateRecord

.wits is named subDB instance of CesrIoSetSuber

storing the current witness set for an identifier. subkey ‘wits.’ Key: identifier prefix. Multiple values per key (one per witness).

Type:: klas=Prefixer

.habs is named subDB instance of Komer

habitat names to habitat application state including identifier prefix. subkey ‘habs.’ Key: habitat name str. Only one value per DB key is allowed.

Type:: schema=HabitatRecord

.names is named subDB instance of Suber

(namespace, name) to identifier prefix. Provides namespace-scoped name lookup for habitats. subkey ‘names.’ Key: namespace + ‘^’ + name.

Type:: sep=’^’

.sdts is named subDB instance of CesrSuber

SAID to Dater CESR serialization of ISO-8601 datetime (sad date-time stamp). subkey ‘sdts.’ Key: said (bytes) of SAD. Only one value per DB key is allowed.

Type:: klas=Dater

.ssgs is named subDB instance of CesrIoSetSuber

transferable indexed signatures. Maps quadruple key (diger.qb64, prefixer.qb64, number.qb64, diger.qb64) to Siger of the transferable signer’s signature. Diger is the SAID of the SAD; prefixer, number, and diger indicate the key state establishment event for the signer. subkey ‘ssgs.’ Key: join(diger.qb64b, prefixer.qb64b, number.qb64b, diger.qb64b) Multiple values per key (one per signer, insertion ordered).

Type:: klas=Siger

.scgs is named subDB instance of CatCesrIoSetSuber: (klas=(Verfer, Cigar)) for SAD nontransferable signatures. Maps SAD SAID to (Verfer, Cigar) couple for each nontransferable signer. For nontransferable signers, qb64 of Verfer equals Prefixer. subkey ‘scgs.’ Key: said (bytes) of SAD. Multiple values per key (one per nontransferable signer, insertion ordered).

.rpys is named subDB instance of SerderSuber for reply messages. Maps: reply SAID to serialization of the reply message (versioned SAD). Use .sdts, .ssgs, and .scgs for associated datetimes and signatures. subkey ‘rpys.’ Key: said bytes. Only one value per DB key is allowed.

.rpes is named subDB instance of CesrIoSetSuber

reply escrows. Maps reply route to Diger of the escrowed reply message. Routes such as ‘/end/role’ and ‘/loc/scheme’. subkey ‘rpes.’ Key: route bytes. Multiple values per key.

Type:: klas=Diger

.eans is named subDB instance of CesrSuber

role authorizations. Maps cid.role.eid to SAID of the reply SAD that authN by controller cid authZ endpoint provider eid in the given role. Routes /end/role/add and /end/role/cut to nullify. subkey ‘eans.’ Key: cid.role.eid. Only one value per DB key is allowed.

Type:: klas=Diger

.lans is named subDB instance of CesrSuber

authorizations. Maps eid.scheme to SAID of the reply SAD that authN by endpoint provider eid designates scheme URL. Route /loc/scheme; null URL nullifies. subkey ‘lans.’ Key: eid.scheme. Only one value per DB key is allowed.

Type:: klas=Diger

.ends is named subDB instance of Komer

(cid, role, eid) to EndpointRecord attributes about endpoint authorization. cid is controller prefix, role is endpoint role (e.g. watcher), eid is controller prefix of endpoint provider. Data extracted from reply /end/role/add or /end/role/cut. subkey ‘ends.’ Key: cid.role.eid.

Type:: schema=EndpointRecord

.locs is named subDB instance of Komer

endpoint prefix eid and network location scheme to endpoint location details. Data extracted from reply /loc/scheme. subkey ‘locs.’ Key: eid.scheme.

Type:: schema=LocationRecord

.obvs is named subDB instance of Komer

observed OIDs by watcher. Maps (cid, aid, oid) to ObservedRecord. subkey ‘obvs.’ Key: cid.aid.oid.

Type:: schema=ObservedRecord

.tops is named subDB instance of Komer

witness identifier prefix to the topic index of the last recieved mailbox message. subkey ‘witm.’ Key: witness prefix identifier.

Type:: schema=TopicsRecord

.gpse is named subDB instance of CatCesrIoSetSuber: (klas=(Number, Diger)) for group multisig partial signature escrows. subkey ‘gpse.’ Multiple values per key.

.gdee is named subDB instance of CatCesrIoSetSuber: (klas=(Number, Diger)) for group multisig delegate escrows. subkey ‘gdee.’ Multiple values per key.

.gpwe is named subDB instance of CatCesrIoSetSuber: (klas=(Number, Diger)) for group multisig partial witness escrows. subkey ‘gdwe.’ Multiple values per key.

.cgms is named subDB instance of CesrSuber

group multisig events. Maps key to Diger of completed event. subkey ‘cgms.’ Only one value per DB key is allowed.

Type:: klas=Diger

.epse is named subDB instance of SerderSuber for exchange message: partial signature escrows. Maps key to serialized Serder of the escrowed exchange message. subkey ‘epse.’

.epsd is named subDB instance of CesrSuber

message partial signature escrow datetimes. Maps key to Dater timestamp of the escrowed message. subkey ‘epsd.’

Type:: klas=Dater

.exns is named subDB instance of SerderSuber for accepted exchange: messages. Maps key to serialized Serder of the exchange message. subkey ‘exns.’

.erpy is named subDB instance of CesrSuber

pointer to a provided reply message associated with an exchange message. subkey ‘erpy.’ Only one value per DB key is allowed.

Type:: klas=Saider

.esigs is named subDB instance of CesrIoSetSuber

exchange message transferable indexed signatures. subkey ‘esigs.’ Multiple values per key.

Type:: klas=Siger

.ecigs is named subDB instance of CatCesrIoSetSuber: (klas=(Verfer, Cigar)) for exchange message nontransferable signatures. Maps key to (Verfer, Cigar) couples. subkey ‘ecigs.’ Multiple values per key.

.epath is named subDB instance of IoSetSuber for exchange message: pathed attachments. subkey ‘epath.’ Multiple values per key.

.essrs is named subDB instance of CesrIoSetSuber

exchange message event source records. subkey ‘essrs.’ Multiple values per key.

Type:: klas=Texter

.chas is named subDB instance of CesrIoSetSuber

accepted signed 12-word challenge response exn messages. Keyed by prefix of signer. subkey ‘chas.’ Multiple values per key.

Type:: klas=Diger

.reps is named subDB instance of CesrIoSetSuber

successful signed 12-word challenge response exn messages. Keyed by prefix of signer. subkey ‘reps.’ Multiple values per key.

Type:: klas=Diger

.wkas is named subDB instance of IoSetKomer

for authorized well-known OOBIs. subkey ‘wkas.’ Multiple values per key.

Type:: schema=WellKnownAuthN

.kdts is named subDB instance of CesrSuber

state SAID to ISO-8601 datetime stamp (ksn date-time stamp). subkey ‘kdts.’ Key: said (bytes). Only one value per DB key is allowed.

Type:: klas=Dater

.ksns is named subDB instance of Komer

key state notice messages. Maps key state SAID to KeyStateRecord. Use .kdts for associated datetimes and signatures. subkey ‘ksns.’

Type:: schema=KeyStateRecord

.knas is named subDB instance of CesrSuber

SAID records of successfully saved key state notices. Maps (prefix, aid) to SAID of key state. subkey ‘knas.’ Only one value per DB key is allowed.

Type:: klas=Diger

.wwas is named subDB instance of CesrSuber

watched SAID records. Maps (cid, aid, oid) to SAID of the reply message for successfully saved watched AIDs. subkey ‘wwas.’ Only one value per DB key is allowed.

Type:: klas=Diger

.oobis is named subDB instance of Komer

for configured OOBIs to be processed asynchronously. Keyed by OOBI URL. sep=’>’ prevents splitting as ‘>’ is not valid in URLs. subkey ‘oobis.’

Type:: schema=OobiRecord, sep=’>’

.eoobi is named subDB instance of Komer

for OOBIs that failed to load and are pending retry. Keyed by OOBI URL. subkey ‘eoobi.’

Type:: schema=OobiRecord, sep=’>’

.coobi is named subDB instance of Komer

for OOBIs with outstanding client requests. Keyed by OOBI URL. subkey ‘coobi.’

Type:: schema=OobiRecord, sep=’>’

.roobi is named subDB instance of Komer

for resolved OOBIs that have been successfully processed. Keyed by OOBI URL. subkey ‘roobi.’

Type:: schema=OobiRecord, sep=’>’

.woobi is named subDB instance of Komer

for well-known OOBIs used for MFA against a resolved OOBI. Keyed by OOBI URL. subkey ‘woobi.’

Type:: schema=OobiRecord, sep=’>’

.moobi is named subDB instance of Komer

for multifactor well-known OOBI records. Keyed by OOBI URL. subkey ‘moobi.’

Type:: schema=OobiRecord, sep=’>’

.mfa is named subDB instance of Komer

for multifactor well-known OOBI auth records pending processing. Keyed by controller URL. subkey ‘mfa.’

Type:: schema=OobiRecord, sep=’>’

.rmfa is named subDB instance of Komer

for resolved multifactor well-known OOBI auth records. Keyed by controller URL. subkey ‘rmfa.’

Type:: schema=OobiRecord, sep=’>’

.schema is named subDB instance of SchemerSuber storing JSON schema: SADs keyed by SAID of the schema. subkey ‘schema.’

.cfld is named subDB instance of Suber for contact field values for: remote identifiers. Keyed by prefix/field. subkey ‘cfld.’

.hbys is named subDB instance of Suber for global settings of the: Habery environment. subkey ‘hbys.’

.cons is named subDB instance of Suber for signed contact data. Keyed: by prefix. subkey ‘cons.’

.ccigs is named subDB instance of CesrSuber

transferable signatures on contact data. Keyed by prefix. subkey ‘ccigs.’ Only one value per DB key is allowed.

Type:: klas=Cigar

.imgs is raw LMDB sub database for chunked image data for contact: information of remote identifiers. Keyed by prefix/chunk-index. subkey b’imgs.’ Raw bytes values; accessed directly via env.open_db.

.ifld is named subDB instance of Suber for identifier field values: for local identifiers. Keyed by prefix/field. subkey ‘ifld.’

.sids is named subDB instance of Suber for signed local identifier: data. Keyed by prefix. subkey ‘sids.’

.icigs is named subDB instance of CesrSuber

transferable signatures on local identifier data. Keyed by prefix. subkey ‘icigs.’ Only one value per DB key is allowed.

Type:: klas=Cigar

.iimgs is raw LMDB sub database for chunked image data for local: identifier information. Keyed by prefix/chunk-index. subkey b’iimgs.’ Raw bytes values; accessed directly via env.open_db.

.dpwe is named subDB instance of SerderSuber for delegated partial: witness escrows. Maps key to serialized Serder of the escrowed delegated event. subkey ‘dpwe.’

.dune is named subDB instance of SerderSuber for delegated unanchored: escrows. Maps key to serialized Serder of the unanchored delegated event awaiting delegation anchor. subkey ‘dune.’

.dpub is named subDB instance of SerderSuber for delegate publication: escrows used to send delegator info to the delegate’s witnesses. subkey ‘dpub.’

.cdel is named subDB instance of CesrOnSuber

completed group delegated AIDs. Maps ordinal key to Diger of the completed delegation event. subkey ‘cdel.’

Type:: klas=Diger

.meids is named subDB instance of CesrIoSetSuber

multisig embed payload SAID to the SAIDs of the exn messages that contained it. Aggregates identical message bodies across group multisig participants reaching consensus on events or credentials. subkey ‘meids.’ Multiple values per key.

Type:: klas=Diger

.maids is named subDB instance of CesrIoSetSuber

mapping multisig embed payload SAID to the AIDs of the group multisig participants that contributed it. subkey ‘maids.’ Multiple values per key.

Type:: klas=Prefixer

.kramCTYP is named subDB instance of Komer

KRAM cache type records. Maps expression string to drift and lag parameters. subkey ‘ctyp.’

Type:: schema=CacheTypeRecord

.kramMSGC is named subDB instance of Komer

KRAM message cache. Maps (AID, MID) to message datetime, drift, and lag values. subkey ‘msgc.’

Type:: schema=MsgCacheRecord

.kramTMSC is named subDB instance of Komer

KRAM transactioned message cache. Maps (AID, XID, MID) to datetimes, drift, and lag values. subkey ‘tmsc.’

Type:: schema=TxnMsgCacheRecord

.kramPMKM is named subDB instance of SerderSuber for KRAM partially signed: multi-key messages. Maps (AID, MID) key to the associated SerderKERI message. subkey ‘pmkm.’

.kramPMKS is named subDB instance of CesrIoSetSuber

KRAM partially signed multi-key signatures. Maps (AID, MID) key to associated Siger instances. subkey ‘pmks.’ Multiple values per key.

Type:: klas=Siger

.kramPMSK is named subDB instance of CatCesrSuber

for KRAM partially signed multi-key sender key state records. Maps (AID, MID) key to (sn, event SAID) couple identifying the sender’s key state. subkey ‘pmsk.’ Only one value per DB key is allowed.

Type:: klas=(Number, Diger)

.kramTRQS is named subDB instance of CatCesrIoSetSuber for KRAM partially: signed multi-key trans receipt quadruple attachments. subkey ‘trqs.’ DB is keyed by (AID, MID): sender identifier prefix plus message SAID Value is (Prefixer, Number, Diger, Siger) tuple. Sourced from parser kwa key ‘trqs’. Multiple values per key stored as ordered set (duplicates ignored). Entries persist until removed by the KRAM pruner.

.kramTSGS is named subDB instance of CatCesrIoSetSuber for KRAM partially: signed multi-key trans last sig group attachments. Each group is stored per-siger as a flat (Prefixer, Seqner, Saider, Siger) tuple. subkey ‘tsgs.’ DB is keyed by (AID, MID): sender identifier prefix plus message SAID Value is (Prefixer, Number, Diger, Siger) tuple. Sourced from parser kwa key ‘tsgs’. Multiple values per key stored as ordered set (duplicates ignored). Entries persist until removed by the KRAM pruner.

.kramSSCS is named subDB instance of CatCesrIoSetSuber for KRAM partially: signed multi-key first seen seal couple attachments from issuing or delegating events. subkey ‘sscs.’ DB is keyed by (AID, MID): sender identifier prefix plus message SAID Value is (Number, Diger) tuple. Sourced from parser kwa key ‘sscs’. Multiple values per key stored as ordered set (duplicates ignored). Entries persist until removed by the KRAM pruner.

.kramSSTS is named subDB instance of CatCesrIoSetSuber for KRAM partially: signed multi-key source seal triple attachments from issued or delegated events. subkey ‘ssts.’ DB is keyed by (AID, MID): sender identifier prefix plus message SAID Value is (Prefixer, Number, Diger) tuple. Sourced from parser kwa key ‘ssts’. Multiple values per key stored as ordered set (duplicates ignored). Entries persist until removed by the KRAM pruner.

.kramFRCS is named subDB instance of CatCesrIoSetSuber for KRAM partially: signed multi-key first seen replay couple attachments. subkey ‘frcs.’ DB is keyed by (AID, MID): sender identifier prefix plus message SAID Value is (Number, Dater) tuple. Sourced from parser kwa key ‘frcs’. Multiple values per key stored as ordered set (duplicates ignored). Entries persist until removed by the KRAM pruner.

.kramTDCS is named subDB instance of CatCesrIoSetSuber for KRAM partially: signed multi-key typed digest seal couple attachments. subkey ‘tdcs.’ DB is keyed by (AID, MID): sender identifier prefix plus message SAID Value is (Verser, Diger) tuple. Sourced from parser kwa key ‘tdcs’. Multiple values per key stored as ordered set (duplicates ignored). Entries persist until removed by the KRAM pruner.

.kramPTDS is named subDB instance of IoSetSuber for KRAM partially signed: multi-key pathed stream attachments. subkey ‘ptds.’ DB is keyed by (AID, MID): sender identifier prefix plus message SAID Value is raw bytes of pathed CESR stream. Sourced from parser kwa key ‘ptds’. Multiple values per key stored as ordered set (duplicates ignored). Entries persist until removed by the KRAM pruner.

.kramBSQS is named subDB instance of CatCesrIoSetSuber for KRAM partially: signed multi-key blind state quadruple attachments. subkey ‘bsqs.’ DB is keyed by (AID, MID): sender identifier prefix plus message SAID Value is (Diger, Noncer, Noncer, Labeler) tuple. Sourced from parser kwa key ‘bsqs’. Multiple values per key stored as ordered set (duplicates ignored). Entries persist until removed by the KRAM pruner.

.kramBSSS is named subDB instance of CatCesrIoSetSuber for KRAM partially: signed multi-key bound state sextuple attachments. subkey ‘bsss.’ DB is keyed by (AID, MID): sender identifier prefix plus message SAID Value is (Diger, Noncer, Noncer, Labeler, Number, Noncer) tuple. Sourced from parser kwa key ‘bsss’. Multiple values per key stored as ordered set (duplicates ignored). Entries persist until removed by the KRAM pruner.

.kramTMQS is named subDB instance of CatCesrIoSetSuber for KRAM partially: signed multi-key type media quadruple attachments. subkey ‘tmqs.’ DB is keyed by (AID, MID): sender identifier prefix plus message SAID Value is (Diger, Noncer, Labeler, Texter) tuple. Sourced from parser kwa key ‘tmqs’. Multiple values per key stored as ordered set (duplicates ignored). Entries persist until removed by the KRAM pruner.

Properties:: kevers (statedict): read through cache of kevers of states for KELs in db

__init__(headDirPath=None, reopen=False, **kwa)[source]

Setup named sub databases.

Parameters:

database (headDirPath is optional str head directory pathname for main) – When system employs more than one keri database, name allows differentiating each instance by name
boolean (temp is) – True then open in temporary directory, clear on close Othewise then open persistent directory, do not clear on close
.temp (assign to) – True then open in temporary directory, clear on close Othewise then open persistent directory, do not clear on close
database – If not provided use default .HeadDirpath
directory (mode is int numeric os dir permissions for database)
reopen (bool) – True means database will be reopened by this init

clean()[source]

Clean database by creating re-verified cleaned cloned copy and then replacing original with cleaned cloned copy

Database usage should be offline during cleaning as it will be cloned in readonly mode

clearEscrows()[source]: Clear all escrows

cloneAllPreIter()[source]

Returns iterator of first seen event messages with attachments for all identifier prefixes starting at key. If key == b’’ then start at first key in databse. Use key to resume replay. Essentially a replay in first seen order with attachments of entire set of FELs.

Returns:: over all items in db
Return type:: msgs (Iterator)

cloneDelegation(kever)[source]

Recursively clone delegation chain from AID of Kever if one exits.

Parameters:: kever (Kever) – Kever from which to clone the delegator’s AID.

cloneEvtMsg(pre, fn, dig)[source]

Clones Event as Serialized CESR Message with Body and attached Foot

Parameters:

pre (bytes) – identifier prefix of event
fn (int) – first seen number (ordinal) of event
dig (bytes) – digest of event

Returns:

message body with attachments

Return type:

bytearray

clonePreIter(pre, fn=0)[source]

Returns iterator of first seen event messages with attachments for the identifier prefix pre starting at first seen order number, fn. Essentially a replay in first seen order with attachments

Parameters:

prefix (pre is bytes of itdentifier)
fn=0 (fn is int fn to resume replay. Earliset is)

Returns:

over all items with pre starting at fn

Return type:

msgs (Iterator)

complete(name=None)[source]

Returns list of tuples of migrations completed with date of completion

Parameters:: name (str) – optional name of migration to check completeness
Returns:: tuples of migration,date of completed migration names and the date of completion
Return type:: list

property current

Current property determines if we are at the current database migration state.

If the database version matches the library version return True If the current database version is behind the current library version, check for migrations

If there are migrations to run, return False

If there are no migrations to run, reset database version to library version and return True

If the current database version is ahead of the current library version, raise exception

fetchAllSealingEventByEventSeal(pre, seal, sn=0)[source]

Search through a KEL for the event that contains a specific anchored SealEvent type of provided seal but in dict form and is also fully witnessed. Searchs from sn forward (default = 0).Searches all events in KEL of pre including disputed and/or superseded events. Returns the Serder of the first event with the anchored SealEvent seal,

None if not found

Parameters:

pre (bytes|str) – identifier of the KEL to search
seal (dict) – dict form of Seal of any type SealEvent to find in anchored seals list of each event
sn (int) – beginning sn to search

fetchLastSealingEventByEventSeal(pre, seal, sn=0)[source]

Search through a KEL for the last event at any sn but that contains a specific anchored event seal of namedtuple SealEvent type that matches the provided seal in dict form and is also fully witnessed. Searchs from provided sn forward (default = 0). Searches only last events in KEL of pre so does not include disputed and/or superseded events.

Returns:

instance of the first event with the matching: anchoring SealEvent seal,

None if not found

Return type:

srdr (Serder)

Parameters:

pre (bytes|str) – identifier of the KEL to search
seal (dict) – dict form of Seal of any type SealEvent to find in anchored seals list of each event
sn (int) – beginning sn to search

fetchLastSealingEventBySeal(pre, seal, sn=0)[source]

Only searches last event at any sn therefore does not search any disputed or superseded events. Search through last event at each sn in KEL for the event that contains an anchored Seal with same Seal type as provided seal but in dict form. Searchs from sn forward (default = 0). Returns the Serder of the first found event with the anchored Seal seal,

None if not found

Parameters:

pre (bytes|str) – identifier of the KEL to search
seal (dict) – dict form of Seal of any type to find in anchored seals list of each event
sn (int) – beginning sn to search

findAnchoringSealEvent(pre, seal, sn=0)

Search through a KEL for the event that contains a specific anchored SealEvent type of provided seal but in dict form and is also fully witnessed. Searchs from sn forward (default = 0).Searches all events in KEL of pre including disputed and/or superseded events. Returns the Serder of the first event with the anchored SealEvent seal,

None if not found

Parameters:

pre (bytes|str) – identifier of the KEL to search
seal (dict) – dict form of Seal of any type SealEvent to find in anchored seals list of each event
sn (int) – beginning sn to search

fullyWitnessed(serder)[source]

Verify the witness threshold on the event

Parameters:: serder (Serder) – event serder to validate witness threshold

Returns:

getEvtLastPreIter(pre, sn=0)[source]

Returns iterator of event messages without attachments in sn order from the KEL of identifier prefix pre. Essentially a replay of all event messages without attachments for each sn from the KEL of pre including superseded duplicates

Parameters:

pre (bytes|str) – identifier prefix
sn (int) – sequence number (default 0) to begin interation

getEvtPreIter(pre, sn=0)[source]

Returns iterator of event messages without attachments in sn order from the KEL of identifier prefix pre. Essentially a replay of all event messages without attachments for each sn from the KEL of pre including superseded duplicates

Parameters:

pre (bytes|str) – identifier prefix
sn (int) – sequence number (default 0) to begin interation

property kevers: Returns .db.kevers

migrate()[source]

Run all migrations required

Run all migrations that are required from the current version of database up to the current version

of the software that have not already been run.

Sets the version of the database to the current version of the software after successful completion of required migrations

reload()[source]: Reload stored prefixes and Kevers from .habs

reopen(**kwa)[source]

Open sub databases

Notes:

dupsort=True for sub DB means allow unique (key,pair) duplicates at a key. Duplicate means that is more than one value at a key but not a redundant copies a (key,value) pair per key. In other words the pair (key,value) must be unique both key and value in combination. Attempting to put the same (key,value) pair a second time does not add another copy.

Duplicates are inserted in lexocographic order by value, insertion order.

resolveVerifiers(pre=None, sn=0, dig=None)[source]

Returns the Tholder and Verfers for the provided identifier prefix. Default pre is own .pre

Parameters:

pre (str)
sn (int)
dig (str)

rotationMembers(pre: str)[source]

Find rotation members of a multisig group aid.

Using the digs index to lookup member pres of a group aid

Parameters:: pre (str) – qb64 identifier prefix to find members
Returns:: qb64 identifier prefixes of rotation members for provided aid
Return type:: list

signingMembers(pre: str)[source]

Find signing members of a multisig group aid.

Using the pubs index to find members of a signing group

Parameters:: pre (str) – qb64 identifier prefix to find members
Returns:: qb64 identifier prefixes of signing members for provided aid
Return type:: list

class keri.db.basing.BaserDoer(baser, **kwa)[source]

Basic Baser Doer ( LMDB Database )

Attributes: (inherited)

done (bool): completion state:: True means completed Otherwise incomplete. Incompletion maybe due to close or abort.

.baser is Baser or LMDBer subclass

Properties: (inherited)

.tyme is float relative cycle time of associated Tymist .tyme obtained: via injected .tymth function wrapper closure.
.tymth is function wrapper closure returned by Tymist .tymeth() method.: When .tymth is called it returns associated Tymist .tyme. .tymth provides injected dependency on Tymist tyme base.
.tock is float, desired time in seconds between runs or until next run,: non negative, zero means run asap

Properties:

.wind injects ._tymth dependency from associated Tymist to get its .tyme

.__call__ makes instance callable: Appears as generator function that returns generator

.do is generator method that returns generator

.enter is enter context action method

.recur is recur context action method or generator method

.exit is exit context method

.close is close context method

.abort is abort context method

Hidden:

._tymth is injected function wrapper closure returned by .tymen() of: associated Tymist instance that returns Tymist .tyme. when called.

._tock is hidden attribute for .tock property

__init__(baser, **kwa)[source]

Inherited Parameters:: tymist is Tymist instance tock is float seconds initial value of .tock

Parameters:: instance (baser is Baser)

keri.db.basing.MIGRATIONS = [('0.6.8', ['hab_data_rename']), ('1.0.0', ['add_key_and_reg_state_schemas']), ('1.2.0', ['rekey_habs'])]

class komerdict(dict): “”” Subclass of dict that has db as attribute and employs read through cache from db Baser.stts of kever states to reload kever from state in database when not found in memory as dict item.

add method that answers is a given pre a group hab pre .localGroup(pre)

Todo add wrapper decorator to update attributes on class that injects instance attributes when class is instanced one of the injected parameters is function that that maps returned Komer to object class parameters are subdb (must be Komer) and function that maps retrieved dataclass record from dataabase to class instance. if no mapping function then just return the dataclass record as value. “””

keri.db.basing.openDB(*, cls=None, name='test', **kwa)[source]: Returns contextmanager generated by openLMDB but with Baser instance as default

keri.db.basing.reopenDB(db, clear=False, **kwa)[source]

Context manager wrapper LMDB DB instances. Repens and closes db.path and db.env LMDB

Parameters:

db (LMDBer) – instance with LMDB environment at .env
clear (bool) – True means clear directory after close

Usage:

with reopenDB(baser) as env:: env. ….

class keri.db.basing.statedict(*pa, **kwa)[source]

Subclass of dict that has db as attribute and employs read through cache from db Baser.stts of kever states to reload kever from state in database when not found in memory as dict item.

get(k, default=None)[source]

Override of dict get method

Parameters:

k (str) – key for dict
default – default value to return if not found

Returns:

converted from underlying dict or database

Return type:

kever

keri.db.dbing

keri.db.dbing module

import lmdb db = lmdb.open(“/tmp/keri_db_setup_test”) db.max_key_size() 511

# create named dbs (core and tables): gDbEnv.open_db(b’core’) gDbEnv.open_db(b’hid2did’) # table of dids keyed by hids gDbEnv.open_db(b’did2offer’, dupsort=True) # table of offer expirations keyed by offer relative dids gDbEnv.open_db(b’anon’, dupsort=True) # anonymous messages gDbEnv.open_db(b’expire2uid’, dupsort=True) # expiration to uid anon

The dupsort, integerkey, integerdup, and dupfixed parameters are ignored if the database already exists. The state of those settings are persistent and immutable per database. See _Database.flags() to view the state of those options for an opened database. A consequence of the immutability of these flags is that the default non-named database will never have these flags set.

So only need to set dupsort first time opened each other opening does not need to call it

May want to use buffers for reads of immutable serializations such as events and sigs. Anything not read modify write but read only.

“{:032x}”.format(1024) ‘00000000000000000000000000000400’

h = [“00”, “01”, “02”, “0a”, “0f”, “10”, “1a”, “11”, “1f”, “f0”, “a0”] h.sort() h [‘00’, ‘01’, ‘02’, ‘0a’, ‘0f’, ‘10’, ‘11’, ‘1a’, ‘1f’, ‘a0’, ‘f0’]

l [‘a’, ‘aa’, ‘b’, ‘ba’, ‘aaa’, ‘baa’] l.sort() l [‘a’, ‘aa’, ‘aaa’, ‘b’, ‘ba’, ‘baa’]

class keri.db.dbing.LMDBer(readonly=False, **kwa)[source]

LBDBer base class for LMDB manager instances. Creates a specific instance of an LMDB database directory and environment.

Attributes: (inherited): name (str): unique path component used in directory or file path name base (str): another unique path component inserted before name temp (bool): True means use /tmp directory headDirPath is head directory path path is full directory path perm is numeric os permissions for directory and/or file(s) filed (bool): True means .path ends in file; False means .path ends in directory mode (str): file open mode if filed fext (str): file extension if filed file (File) opened is Boolean, True means directory created and if file then file is opened. False otherwise

env

LMDB main (super) database environment

Type:: lmdb.env

readonly

True means open LMDB env as readonly

Type:: bool

Properties:

File/Directory Creation Mode Notes:

.Perm provides default restricted access permissions to directory and/or files stat.S_ISVTX | stat.S_IRUSR | stat.S_IWUSR | stat.S_IXUSR 0o1700==960

stat.S_ISVTX is Sticky bit. When this bit is set on a directory it means: that a file in that directory can be renamed or deleted only by the owner of the file, by the owner of the directory, or by a privileged process. When this bit is set on a file it means nothing

stat.S_IRUSR Owner has read permission. stat.S_IWUSR Owner has write permission. stat.S_IXUSR Owner has execute permission.

__init__(readonly=False, **kwa)[source]

Setup main database directory at .dirpath. Create main database environment at .env using .path.

Parameters:

name (str) – directory path name differentiator directory/file When system employs more than one keri installation, name allows differentiating each instance by name
base (str) – optional directory path segment inserted before name that allows further differentiation with a hierarchy. “” means optional.
temp (bool) – assign to .temp True then open in temporary directory, clear on close Otherwise then open persistent directory, do not clear on close
headDirPath (str) – optional head directory pathname for main database Default .HeadDirPath
mode (str) – optional numeric os dir permissions for database directory and database files. Default .DirMode
reopen (bool) – True means (re)opened by this init False means not (re)opened by this init but later
clear (bool) – True means remove directory upon close if reopon False means do not remove directory upon close if reopen
reuse (bool) – True means reuse self.path if already exists False means do not reuse but remake self.path
clean (bool) – True means path uses clean tail variant False means path uses normal tail variant
filed (bool) – True means .path is file path not directory path False means .path is directory path not file path
mode – File open mode when filed
fext (str) – File extension when filed
readonly (bool) – True means open database in readonly mode False means open database in read/write mode

addIoDupVal(db, key, val)[source]

Add val bytes as dup in insertion order to key in db for val not empty. Adds to existing values at key if any Returns True if written else False if val is already a dup Actual value written include prepended proem ordinal Assumes DB opened with dupsort=True

Duplicates at a given key preserve insertion order of duplicate. Because lmdb is lexocographic an insertion ordering proem is prepended to all values that makes lexocographic order that same as insertion order.

Duplicates are ordered as a pair of key plus value so prepending proem to each value changes duplicate ordering. Proem is 33 characters long. With 32 character hex string followed by ‘.’ for essentiall unlimited number of values which will be limited by memory.

With prepended proem ordinal must explicity check for duplicate values before insertion. Uses a python set for the duplicate inclusion test. Set inclusion scales with O(1) whereas list inclusion scales with O(n).

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort=True
key (bytes) – within sub db’s keyspace
val (bytes) – value to be written unless empty

addIoSetVal(db, key, val, *, sep=b'.')[source]

Add val to insertion ordered set of values all with the same apparent effective key if val not already in set of vals at key. When val None returns False

Uses hidden ordinal key suffix for insertion ordering. The suffix is appended and stripped transparently.

Returns:

True if val added to set.: False if already in set or key is empty or None or val is None

Return type:

result (bool)

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort==False
key (bytes|None) – Apparent effective key
val (bytes|None) – serialized value to add
sep (bytes) – separator character for split

addOnIoDupVal(db, key, on=0, val=b'', sep=b'.')[source]

Add val bytes as dup at onkey consisting of key + sep + serialized on in db. Adds to existing values at key if any Returns True if written else False if dup val already exists

Duplicates are inserted in lexocographic order not insertion order. Lmdb does not insert a duplicate unless it is a unique value for that key.

Does inclusion test to dectect of duplicate already exists Uses a python set for the duplicate inclusion test. Set inclusion scales with O(1) whereas list inclusion scales with O(n).

Returns:

True if duplicate val added at onkey idempotent: False if duplicate val preexists at onkey

Return type:

result (bool)

Parameters:

db (SubDB) – opened named sub db with dupsort=True
key (bytes) – key within sub db’s keyspace plus trailing part on
on (int) – ordinal number at which to add
val (bytes) – serialized value to add at onkey as dup
sep (bytes) – separator character for split

addOnIoSetVal(db, key, *, on=0, val=None, sep=b'.')[source]

Add val to insertion ordered set of values at onkey = key + on, when val not already in set of vals at key and key is not empty or None and val is not None.

Returns:

True if val added to set.: False if already in set or key is empty or None or val is None

Return type:

result (bool)

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort==False
key (bytes|None) – base key
on (int) – ordinal number at which to add to key form effective key
val (bytes|None) – serialized value to add
sep (bytes) – separator character for split

With appended suffix ordinal must explicity check for duplicate values in set before insertion. Uses a python set for the duplicate inclusion test. Set inclusion scales with O(1) whereas list inclusion scales with O(n).

Uses hidden ordinal key suffix for insertion ordering which is transparently suffixed and unsuffixed Assumes DB opened with dupsort=False

addVal(db, key, val)[source]

Add val bytes as dup to key in db Adds to existing values at key if any Returns True if written else False if dup val already exists

Duplicates are inserted in lexocographic order not insertion order. Lmdb does not insert a duplicate unless it is a unique value for that key.

Does inclusion test to dectect of duplicate already exists Uses a python set for the duplicate inclusion test. Set inclusion scales with O(1) whereas list inclusion scales with O(n).

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort=True
key (bytes) – within sub db’s keyspace
val (bytes) – value to be written

appendOnIoDupVal(db, key, val, *, sep=b'.')[source]

Appends val in order after last previous key with same pre in db where full key has key prefix and serialized on suffix attached with sep and value has ordinal proem prefixed.

Returns ordinal number on, of appended entry. Appended on is 1 greater than previous latest on at pre. Uses onKey(pre, on) for entries.

Works with either dupsort==True or False since always creates new full key.

Append val to end of db entries with same pre but with on incremented by 1 relative to last preexisting entry at pre.

Returns:

ordinal number of newly appended val

Return type:

on (int)

Parameters:

db (subdb) – named sub db in lmdb
key (bytes) – key within sub db’s keyspace plus trailing part on
val (bytes) – serialized value to append
sep (bytes) – separator character for split

appendOnIoSetVals(db, key, vals, *, sep=b'.')[source]

Appends set vals in order after last previous onkey = key + sep + on as new entry at at new onkey. New on for new onkey is one greater than last prior on for given key in db. The onkey of the appended entry is one greater than last prior on for key in db.

Returns:

ordinal number of new onkey for newly appended set of vals.: Raises ValueError when unsuccessful append including when key is empty or None or vals is empty or None

Return type:

on (int)

Parameters:

db (subdb) – named sub db in lmdb
key (bytes) – key within sub db’s keyspace plus trailing part on
vals (NonStrIterable) – values to append as set at new on
sep (bytes) – separator character for split

Starts at onkey = key + MaxOn and then walks backwards to find last prior entry at key. Then increments on and appends new entry with val Otherwise create new zeroth on entry at key.

Uses hidden ordinal key suffix for insertion ordering which is transparently suffixed and unsuffixed Assumes DB opened with dupsort=False

appendOnVal(db, key, val, *, sep=b'.')[source]

Appends val in order after last previous onkey = key + sep + on as new entry at at new onkey. New on for new onkey is one greater than last prior on for given key in db. The onkey of the appended entry is one greater than last prior on for key in db.

Returns:

ordinal number of new onkey for newly appended val.: Raises ValueError when unsuccessful append including when key is empty or None or val is None

Return type:

on (int)

Parameters:

db (subdb) – named sub db in lmdb
key (bytes) – key within sub db’s keyspace plus trailing part on
val (bytes) – serialized value to append
sep (bytes) – separator character for split

close(clear=False)[source]: Close lmdb at .env and if clear or .temp then remove lmdb directory at .path :param clear is boolean: :param True means clear lmdb directory:

cntAll(db)[source]

Return count of values in db, or zero otherwise

Parameters:: False (db is opened named sub db with either dupsort=True or)

cntIoDups(db, key)[source]

Get count of dup values at key in db, or zero otherwise Assumes DB opened with dupsort=True Count doesn’t need to add strip proem from dups just count dups

Duplicates at a given key preserve insertion order of duplicate. Because lmdb is lexocographic an insertion ordering proem is prepended to all values that makes lexocographic order that same as insertion order.

Duplicates are ordered as a pair of key plus value so prepending proem to each value changes duplicate ordering. Proem is 33 characters long. With 32 character hex string followed by ‘.’ for essentiall unlimited number of values which will be limited by memory.

With prepended proem ordinal must explicity check for duplicate values before insertion. Uses a python set for the duplicate inclusion test. Set inclusion scales with O(1) whereas list inclusion scales with O(n).

Returns:

number of total dup values at key if any, 0 if not.

Return type:

cnt (int)

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort=True
key (bytes) – within sub db’s keyspace

cntIoSet(db, key, *, ion=0, sep=b'.')[source]

Count set entries at onkey = key + sep + on for ion >= ion. Count beginning with entry at insertion offset ion. Count is zero if key not in db or ion greater than whats in set.

Uses hidden ordinal key suffix for insertion ordering.: The suffix is suffixed and unsuffixed transparently.

Returns:

count values in set at apparent effective key

Return type:

count (int)

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort==False
key (bytes) – Apparent effective key
ion (int) – starting ordinal value, default 0
sep (bytes) – separator character for split

cntOnAll(db, key=b'', on=0, *, sep=b'.')[source]

Counts all entries one for each onkey for all on >= on where for each on, onkey = key + sep + on. When key empty then count whole database.

Returns (int): count of of all ordinal keyed vals with key but different on tail in db starting at ordinal number on of key for on >= on. Full key is composed of key+sep+on

When dupsort==true then duplicates are included in count since .iternext includes duplicates. when key is empty then counts whole db

Parameters:

db (lmdbsubdb) – named sub db of lmdb
key (bytes) – key within sub db’s keyspace plus trailing part on when key is empty then retrieves whole db
on (int) – ordinal number at which to initiate count
sep (bytes) – separator character for split

cntOnAllIoSet(db, key=b'', *, on=0, sep=b'.')[source]

Counts all entries of each set at each onkey for all on >= on where for each on, onkey = key + sep + on. Count includes all set members at all matching onkeys. When on = 0, default, then count all set members for all on for key When key is empty then count all on for all key i.e. whole db

Returns:

count of set members for onkey for on >= on. When on is: None then count of all on for key. When key is empty then count of all on for all key for whole db.

Return type:

count (int)

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort==False
key (bytes) – base key
on (int) – ordinal number at which to add to key form onkey
sep (bytes) – separator character for split

UUses hidden ordinal key suffix for insertion ordering which is transparently suffixed and unsuffixed

cntOnIoDups(db, key, on=0, sep=b'.')[source]

Get count of IoDup values at key + on in db, or zero otherwise Assumes DB opened with dupsort=True Count doesnt need to add/strip proem from dups just count them.

Duplicates at a given key preserve insertion order of duplicate. Because lmdb is lexocographic an insertion ordering proem is prepended to all values that makes lexocographic order that same as insertion order.

Duplicates are ordered as a pair of key plus value so prepending proem to each value changes duplicate ordering. Proem is 33 characters long. With 32 character hex string followed by ‘.’ for essentiall unlimited number of values which will be limited by memory.

With prepended proem ordinal must explicity check for duplicate values before insertion. Uses a python set for the duplicate inclusion test. Set inclusion scales with O(1) whereas list inclusion scales with O(n).

Returns:

number of total IoDup values at key if any, 0 if not.

Return type:

cnt (int)

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort=True
key (bytes) – within sub db’s keyspace
on (int) – ordinal number at which to retrieve
sep (bytes) – separator character for split

cntOnIoSet(db, key, *, on=0, ion=0, sep=b'.')[source]

Count set values at onkey made from onkey = key + on starting at ion offset within set at onkey. Count = 0 if onkey not in db.

Returns:

count values in set at effective onkey

Return type:

count (int)

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort==False
key (bytes) – base key
on (int|None) – ordinal number at which to add to key form onkey
ion (int) – starting ordinal value, default 0
sep (bytes) – separator character for split

Uses hidden ordinal key suffix for insertion ordering which is transparently suffixed and unsuffixed Assumes DB opened with dupsort=False

cntTop(db, top=b'')[source]

Counts all entries in branch of db given by top key. When top is empty then counts all entries in whole db.

Returns:

number of counted entries in branch if any

Return type:

count (int)

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort==False
top (bytes) – truncated top key, a key space prefix to get all the items from multiple branches of the key space. If top key is empty then counts all items in database

Works for both dupsort==False and dupsort==True

cntVals(db, key)[source]

Counts dup values at key in db.

Return count of dup values at key in db, or zero otherwise

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort=True
keyspace (key is bytes of key within sub db's)

delIoDupVal(db, key, val)[source]

Deletes dup io val at key in db. Performs strip search to find match. Strips proems and then searches. Returns True if delete else False if val not present Assumes DB opened with dupsort=True

Duplicates at a given key preserve insertion order of duplicate. Because lmdb is lexocographic an insertion ordering proem is prepended to all values that makes lexocographic order that same as insertion order Duplicates are ordered as a pair of key plus value so prepending proem to each value changes duplicate ordering. Proem is 33 characters long. With 32 character hex string followed by ‘.’ for essentially unlimited number of values which will be limited by memory.

Does a linear search so not very efficient when not deleting from the front. This is hack for supporting escrow which needs to delete individual dup. The problem is that escrow is not fixed buts stuffs gets added and deleted which just adds to the value of the proem. 2**16 is an impossibly large number so the proem will not max out practically. But its not an elegant solution.

Returns:

True if dup item (key, val) exists in db: False otherwise

Return type:

result (bool)

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort=True
key (bytes) – within sub db’s keyspace
val (bytes) – effective value to be deleted

delIoDupVals(db, key)[source]

Deletes all values at key in db if key present. Returns True If key exists

Duplicates at a given key preserve insertion order of duplicate. Because lmdb is lexocographic an insertion ordering proem is prepended to all values that makes lexocographic order that same as insertion order.

Duplicates are ordered as a pair of key plus value so prepending proem to each value changes duplicate ordering. Proem is 33 characters long. With 32 character hex string followed by ‘.’ for essentiall unlimited number of values which will be limited by memory.

With prepended proem ordinal must explicity check for duplicate values before insertion. Uses a python set for the duplicate inclusion test. Set inclusion scales with O(1) whereas list inclusion scales with O(n).

Returns:

True if key exists in db: False if key not exists in db

Return type:

result (bool)

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort=True
key (bytes) – within sub db’s keyspace

delOnIoDupVal(db, key, on=0, val=b'', sep=b'.')[source]

Deletes dup ioval at key onkey consisting of key + sep + serialized on in db. Returns True if deleted else False if dup val not present Assumes DB opened with dupsort=True

Duplicates are inserted in lexocographic order not insertion order. Lmdb does not insert a duplicate unless it is a unique value for that key.

Does inclusion test to dectect of duplicate already exists Uses a python set for the duplicate inclusion test. Set inclusion scales with O(1) whereas list inclusion scales with O(n).

Returns:

True if duplicate val found and deleted: False if duplicate val does not exist at onkey

Return type:

result (bool)

Parameters:

dupsort=True (db is opened named sub db with)
key (bytes) – key within sub db’s keyspace plus trailing part on
on (int) – ordinal number at which to retrieve
val (bytes) – serialized dup value to del at onkey
sep (bytes) – separator character for split

delOnIoDups(db, key, on=0, sep=b'.')[source]

Deletes all dup iovals at onkey consisting of key + sep + serialized on in db.

Assumes DB opened with dupsort=True

Duplicates are inserted in lexocographic order not insertion order. Lmdb does not insert a duplicate unless it is a unique value for that key.

Does inclusion test to dectect of duplicate already exists Uses a python set for the duplicate inclusion test. Set inclusion scales with O(1) whereas list inclusion scales with O(n).

Returns:

True if onkey present so all dups at onkey deleted: False if onkey not present

Return type:

result (bool)

Parameters:

dupsort=True (db is opened named sub db with)
key (bytes) – key within sub db’s keyspace plus trailing part on
on (int) – ordinal number at which to retrieve
sep (bytes) – separator character for split

delTop(db, top=b'')

Deletes all values in branch of db given top key. Top empty deletes whole db.

Returns:

True if values were deleted at key. False otherwise: if no values at key

Return type:

result (bool)

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort==False
top (bytes) – truncated top key, a key space prefix to get all the items from multiple branches of the key space. If top key is empty then deletes all items in database

Works for both dupsort==False and dupsort==True Because cursor.iternext() advances cursor after returning item its safe to delete the item within the iteration loop.

delVal(db, key)

Removes value at key in db. :returns:

True If key exists in database and item deleted
False If key empty or missing from database

Return type:: result (bool)

Raises KeyError if problem with key

Parameters:

dupsort=False (db is opened named sub db with)
keyspace (key is bytes of key within sub db's)

delVals(db, key, val=b'')[source]

Deletes all values at key in db if val=b’’ else deletes the dup that equals val Returns True If key (and val if not empty) exists in db Else False

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort=True
keyspace (key is bytes of key within sub db's)
delete (val is bytes of dup val at key to)

getIoDupItemIter(db, key, *, ion=0)[source]

Get iterator of all duplicate items at key in db in insertion order for insertion ordering ordinal ion >= ion. When key is empty then returns empty iterator

Raises StopIterationError when done.

Removes prepended proem ion ordinal from each val before returning Assumes DB opened with dupsort=True

Duplicates at a given key preserve insertion order of duplicate. Because lmdb is lexocographic an insertion ordering proem is prepended to all values that makes lexocographic order that same as insertion order.

Duplicates are ordered as a pair of key plus value so prepending proem to each value changes duplicate ordering. Proem is 33 characters long. With 32 character hex string followed by ‘.’ for essentiall unlimited number of values which will be limited by memory.

With prepended proem ordinal must explicity check for duplicate values before insertion. Uses a python set for the duplicate inclusion test. Set inclusion scales with O(1) whereas list inclusion scales with O(n).

Returns:

dup values at key

Return type:

vals (Iterator[bytes])

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort=True
key (bytes) – within sub db’s keyspace
ion (int) – starting ordinal value, default 0

getIoDupValLast(db, key)[source]

Get last added dup value at key in db in insertion order Returns None no entry at key Removes prepended proem ordinal from val before returning Assumes DB opened with dupsort=True

Duplicates at a given key preserve insertion order of duplicate. Because lmdb is lexocographic an insertion ordering proem is prepended to all values that makes lexocographic order that same as insertion order.

Duplicates are ordered as a pair of key plus value so prepending proem to each value changes duplicate ordering. Proem is 33 characters long. With 32 character hex string followed by ‘.’ for essentiall unlimited number of values which will be limited by memory.

With prepended proem ordinal must explicity check for duplicate values before insertion. Uses a python set for the duplicate inclusion test. Set inclusion scales with O(1) whereas list inclusion scales with O(n).

Returns:

last dup value at key

Return type:

last (bytes)

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort=True
key (bytes) – within sub db’s keyspace

getIoDupVals(db, key)[source]

Get Iterable of duplicate values at key in db in insertion order Returns empty list if no entry at key Removes prepended proem ordinal from each val before returning Assumes DB opened with dupsort=True

Duplicates at a given key preserve insertion order of duplicate. Because lmdb is lexocographic an insertion ordering proem is prepended to all values that makes lexocographic order that same as insertion order.

Duplicates are ordered as a pair of key plus value so prepending proem to each value changes duplicate ordering. Proem is 33 characters long. With 32 character hex string followed by ‘.’ for essentiall unlimited number of values which will be limited by memory.

With prepended proem ordinal must explicity check for duplicate values before insertion. Uses a python set for the duplicate inclusion test. Set inclusion scales with O(1) whereas list inclusion scales with O(n).

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort=True
key (bytes) – within sub db’s keyspace

getIoSetItemIter(db, key, *, ion=0, sep=b'.')[source]

Get iterator over items in IoSet at effecive key for ion >= ion. When key is empty then returns empty iterator

Raises StopIterationError when done.

Uses hidden ordinal key suffix for insertion ordering.: The suffix is suffixed and unsuffixed transparently.

Returns:

iterator over insertion ordered set: items at same apparent effective key. Empty iterator when key is empty

Return type:

items (Iterator[memoryview])

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort==False
key (bytes) – Apparent effective key. raises StopIterationError when key is empty
ion (int) – starting ordinal value, default 0
sep (bytes) – separator character for split

getIoSetLastItem(db, key, *, sep=b'.')[source]

Gets last added ioset entry item at effective key if any else empty tuple.

Uses hidden ordinal key suffix for insertion ordering.: The suffix is suffixed and unsuffixed transparently.

Returns:

last added entry item at apparent: effective key if any, otherwise empty tuple if no entry at key or if key empty

Return type:

last ((bytes, memoryview))

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort==False
key (bytes) – Apparent effective key (unsuffixed)
sep (bytes) – separator character for split

getIoSetLastItemIterAll(db, key=b'', *, sep=b'.')[source]

Iterates over every last added ioset entry at every effective key starting at key greater or equal to key. When key is empty then iterates over whole db.

Raises StopIterationError when done.

Uses hidden ordinal key suffix for insertion ordering.: The suffix is suffixed and unsuffixed transparently.

Returns:

last added entry item at tuple (key, val): at apparent effective key for all key >= key. When key empty then iterates over all keys in db

Return type:

last (Iterator[memoryview])

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort==False
key (bytes) – Apparent effective key
sep (bytes) – separator character for split

getIoSetLastIterAll(db, key=b'', *, sep=b'.')[source]

Iterates over every last added ioset entry at every effective key starting at key greater or equal to key. When key is empty then iterates over whole db.

Raises StopIterationError when done.

Uses hidden ordinal key suffix for insertion ordering.: The suffix is suffixed and unsuffixed transparently.

Returns:

last added entry val at apparent effective: key for all key >= key. When key empty then iterates over all keys in db

Return type:

last (Iterator[memoryview])

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort==False
key (bytes) – Apparent effective key
sep (bytes) – separator character for split

getOnAllIoSetItemBackIter(db, key=b'', on=None, *, sep=b'.')[source]

Iterates backwards over all set items for all on <= on for key. When on is None, iterates backwards over all set items for all on for key When key is empty then iterates backwards over whole db

Returned items are triples of (key, on, val)

Raises StopIterationError when done or when key empty or None

Backwards means decreasing numerical value of ion, for each on and decreasing numerical value on for each key and decreasing lexocographic order of each key.

Returns:

triples of (key, on, val)

Return type:

items (Iterator[(bytes, int, memoryview)])

Parameters:

db (subdb) – named sub db in lmdb
key (bytes) – base key. When empty then whole db
on (int|None) – ordinal number at which to initiate retrieval when on is None then all on starting at greatest
sep (bytes) – separator character for split

Uses hidden ordinal key suffix for insertion ordering which is transparently suffixed and unsuffixed Assumes DB opened with dupsort=False

getOnAllIoSetItemIter(db, key=b'', on=0, *, sep=b'.')[source]

Iterates over each item of each set for all on >= on for key. When on == 0, default, then iterates over all items for all on for key. When key is empty then iterates over all items for whole db.

Each effecive onkey = key + sep + on. Items are triples of (key, on, val)

Entries are sorted by onKey(key, on) where on is ordinal number int and key is prefix sans on.

The set at each entry is sorted internally by hidden suffixed insertion ordering ordinal

Raises StopIteration Error when done.

Returns:

iterator of triples: (key, on, val) where key forms base key, on is int, and val is entry value at with insertion ordering suffix removed from effective key.

Return type:

items (Iterator[(key, int, bytes)])

Parameters:

db (subdb) – named sub db in lmdb
key (bytes) – key within sub db’s keyspace plus trailing part on when key is empty then retrieves whole db
on (int) – ordinal number at which to initiate retrieval
sep (bytes) – separator character for split

Uses hidden ordinal key suffix for insertion ordering which is transparently suffixed and unsuffixed Assumes DB opened with dupsort=False

getOnAllIoSetLastItemBackIter(db, key=b'', on=None, *, sep=b'.')[source]

Iterates backwards over last set items for all on <= on for key. When on is None iterates backwards over last set items for all on for key When key is empty then iterates backwards over last set items for whole db starting at last item in db

Returned items are triples of (key, on, val)

Raises StopIterationError when done or when key empty or None

Backwards means decreasing numerical value of each ion, for each on and decreasing numerical value of each on for each key and decreasing lexocographic value of each key.

Returns:

triples of (key, on, val)

Return type:

items (Iterator[(bytes, int, memoryview)])

Parameters:

db (subdb) – named sub db in lmdb
key (bytes) – base key. When empty then whole db
on (int|None) – ordinal number at which to initiate retrieval when on is None then all on starting at greatest
sep (bytes) – separator character for split

Uses hidden ordinal key suffix for insertion ordering which is transparently suffixed and unsuffixed Assumes DB opened with dupsort=False

getOnAllIoSetLastItemIter(db, key=b'', on=0, *, sep=b'.')[source]

Iterates over last items of each set for all on >= on at key When on ==0, default, iterates over last items of each set for all on at key When key is empty then iterates over last items of all sets in whole db

Each effecive onkey = key + sep + on. Items are triples of (key, on, val)

Entries are sorted by onKey(key, on) where on is ordinal number int and key is prefix sans on.

The set at each entry is sorted internally by hidden suffixed insertion ordering ordinal

Raises StopIteration Error when done.

Returns:

triples of (key, on, val)

Return type:

last (Iterator[(bytes, int, memoryview)])

Parameters:

db (subdb) – named sub db in lmdb
key (bytes) – base key, empty defaults to whole database
on (int) – ordinal number at which to initiate retrieval
sep (bytes) – separator character for split

Uses hidden ordinal key suffix for insertion ordering which is transparently suffixed and unsuffixed Assumes DB opened with dupsort=False

getOnAllItemIter(db, key=b'', on=0, *, sep=b'.')[source]

Gets iterator of triples (key, on, val), at each key over all ordinal numbered keys with same key and on >= on. When on = 0, default, then iterates over all on at key When key is empty then iterates over all on for all keys, whole db. Returned items are triples of (key, on, val).

Entries are sorted by onKey(key, on) where on is ordinal number int and key is prefix sans on.

When dupsort==true then duplicates are included in items since .iternext includes duplicates.

Raises StopIterationError when done or key empty

Returns:

triples of (key, on, val): for onkey = key + sep + on for on >= on at key. When on is None then iterates over all on at key.

Return type:

items (Iterator[(bytes, int, bytes|memoryview)])

Parameters:

db (subdb) – named sub db in lmdb
key (bytes) – base key when key is empty then retrieves whole db
on (int) – ordinal number at which to initiate retrieval
sep (bytes) – separator character for split

getOnIoDupItemBackIter(db, key=b'', on=0, *, sep=b'.')[source]

Returns iterator going backwards of triples (key, on, val), of insertion ordered item at each key over all ordinal numbered keys with same full key of key + sep + on in db. Values are sorted by onKey(key, on) where on is ordinal number int and key is prefix sans on. Values duplicates are sorted internally by hidden prefixed insertion order proem ordinal Backwards means decreasing numerical value of duplicate proem, for each on, decreasing numerical value on for each key and decresing lexocogrphic order of each key prefix.

Returned items are triples of (key, on, val)

when key is empty then retrieves whole db

Raises StopIteration Error when empty.

Returns:

triples of key, on, val

Return type:

items (Iterator[(key, on, val)])

Parameters:

db (subdb) – named sub db in lmdb
key (bytes) – key within sub db’s keyspace plus trailing part on when key is empty then retrieves whole db
on (int) – ordinal number at which to initiate retrieval
sep (bytes) – separator character for split

getOnIoDupItemIter(db, key, on=0, ion=0, sep=b'.')[source]

Iterates over all dup items at onkey = key + sep + on in db.

Assumes DB opened with dupsort=True Return iterator of all duplicate values at key in db in insertion order Raises StopIteration Error when no remaining dup items = empty. Removes prepended proem ordinal from each val before returning Assumes DB opened with dupsort=True

Duplicates at a given key preserve insertion order of duplicate. Because lmdb is lexocographic an insertion ordering proem is prepended to all values that makes lexocographic order that same as insertion order.

Duplicates are ordered as a pair of key plus value so prepending proem to each value changes duplicate ordering. Proem is 33 characters long. With 32 character hex string followed by ‘.’ for essentiall unlimited number of values which will be limited by memory.

With prepended proem ordinal must explicity check for duplicate values before insertion. Uses a python set for the duplicate inclusion test. Set inclusion scales with O(1) whereas list inclusion scales with O(n).

iodups (Iterator): iterator over insertion ordered set of entries: at same apparent effective key made from key + sep + on. Uses hidden ordinal value proem for insertion ordering. The proem is appended and stripped transparently. When key is empty then returns empty iterator

Raises StopIteration Error when empty.

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort==True
key (bytes) – base key. When key is empty then returns empty iterator
on (int) – ordinal number at which to add to key form effective key
ion (int) – starting insertion ordinal value, default 0
sep (bytes) – separator character for split

getOnIoDupItemIterAll(db, key=b'', on=0, *, sep=b'.')[source]

Returns iterator of triples (key, on, val), at each key over all ordinal numbered keys starting at key + sep + on for all on >= on but same key in db. Values are sorted by onKey(key, on) where on is ordinal number int and key is prefix sans on. Values duplicates are sorted internally by hidden prefixed insertion order proem ordinal Returned items are triples of (key, on, val) when key is empty then retrieves whole db

Raises StopIteration Error when empty.

Returns:

triples of key, on, val

Return type:

items (Iterator[(key, on, val)])

Parameters:

db (subdb) – named sub db in lmdb
key (bytes) – key within sub db’s keyspace plus trailing part on when key is empty then retrieves whole db
on (int) – ordinal number at which to initiate retrieval
sep (bytes) – separator character for split

getOnIoDupIterAll(db, key=b'', on=0, *, sep=b'.')[source]

Returns iterator of val at each key over all ordinal numbered keys starting at key + sep + on for all on >= on but same key in db. Values are sorted by onKey(key, on) where on is ordinal number int and key is prefix sans on. Values duplicates are sorted internally by hidden prefixed insertion order proem ordinal Returned items are triples of (key, on, val) When dupsort==true then duplicates are included in items since .iternext includes duplicates. when key is empty then retrieves whole db

Raises StopIteration Error when empty.

Returns:

triples of key, on, val

Return type:

items (Iterator[(key, on, val)])

Parameters:

db (subdb) – named sub db in lmdb
key (bytes) – key within sub db’s keyspace plus trailing part on when key is empty then retrieves whole db
on (int) – ordinal number at which to initiate retrieval
sep (bytes) – separator character for split

getOnIoDupLast(db, key, on: int = 0, *, sep=b'.')[source]

Get last added dup value at onkey = key + sep + on in db in insertion order Returns None no entry at key

Removes prepended proem ordinal from val before returning Assumes DB opened with dupsort=True

Duplicates at a given key preserve insertion order of duplicate. Because lmdb is lexocographic an insertion ordering proem is prepended to all values that makes lexocographic order that same as insertion order.

Duplicates are ordered as a pair of key plus value so prepending proem to each value changes duplicate ordering. Proem is 33 characters long. With 32 character hex string followed by ‘.’ for essentiall unlimited number of values which will be limited by memory.

With prepended proem ordinal must explicity check for duplicate values before insertion. Uses a python set for the duplicate inclusion test. Set inclusion scales with O(1) whereas list inclusion scales with O(n).

Returns:

last dup value at onkey

Return type:

last (bytes)

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort=True
key (bytes) – base key within sub db’s keyspace
on (int) – ordinal number to form onkey to get last from dups at onkey
sep (bytes) – separator character for split

getOnIoDupLastItemIter(db, key=b'', on=0, *, sep=b'.')[source]

Returns iterator of triples (key, on, val), of last insertion ordered duplicate at each key over all ordinal numbered onkeys in db with same key where onkey = key + sep + on starting at on=on for on >= on. Values are sorted by onKey(key, on) where on is ordinal number int and key is prefix sans on. Values duplicates are sorted internally by hidden prefixed insertion order proem ordinal Returned items are triples of (key, on, val)

when key is empty then retrieves whole db

Raises StopIteration Error when empty.

Returns:

triples of key, on, val

Return type:

items (Iterator[(key, on, val)])

Parameters:

db (subdb) – named sub db in lmdb
key (bytes) – key within sub db’s keyspace plus trailing part on when key is empty then retrieves whole db
on (int) – ordinal number at which to initiate retrieval
sep (bytes) – separator character for split

getOnIoDupLastValIter(db, key=b'', on=0, *, sep=b'.')[source]

Returns iterator of val of last insertion ordered duplicate at each key over all ordinal numbered onkeys in db with same key where onkey = key + sep + on starting at on=on for on >= on. Values are sorted by onKey(key, on) where on is ordinal number int and key is prefix sans on.

Values duplicates are sorted internally by hidden prefixed insertion order proem ordinal

when key is empty then retrieves whole db

Raises StopIteration Error when empty. :returns: last dup val at each onkey :rtype: val (Iterator[bytes])

Parameters:

db (subdb) – named sub db in lmdb
key (bytes) – key within sub db’s keyspace plus trailing part on when key is empty then retrieves whole db
on (int) – ordinal number at which to initiate retrieval
sep (bytes) – separator character for split

getOnIoDupValBackIter(db, key=b'', on=0, *, sep=b'.')[source]

Returns iterator going backwards of values, of insertion ordered item at each key over all ordinal numbered keys with same full key of key + sep + on in db. Values are sorted by onKey(key, on) where on is ordinal number int and key is prefix sans on. Values duplicates are sorted internally by hidden prefixed insertion order proem ordinal Backwards means decreasing numerical value of duplicate proem, for each on, decreasing numerical value on for each key and decresing lexocogrphic order of each key prefix.

Returned items are vals

when key is empty then retrieves whole db

Raises StopIteration Error when empty.

Returns:

at key including duplicates in backwards order

Return type:

val (Iterator[bytes])

Parameters:

db (subdb) – named sub db in lmdb
key (bytes) – key within sub db’s keyspace plus trailing part on when key is empty then retrieves whole db
on (int) – ordinal number at which to initiate retrieval
sep (bytes) – separator character for split

getOnIoDupVals(db, key, on=0, sep=b'.')[source]

Returns list of all dup IoVals at onkey = key + sep + on in db where on is serialized. This provides ordinal ordering of keys and insertion ordering of dups.

Assumes DB opened with dupsort=True

Return list of duplicate values at key + sep + on in db in insertion order Returns empty list if no entry at key Removes prepended proem ordinal from each val before returning Assumes DB opened with dupsort=True

Duplicates at a given key preserve insertion order of duplicate. Because lmdb is lexocographic an insertion ordering proem is prepended to all values that makes lexocographic order that same as insertion order.

Duplicates are ordered as a pair of key plus value so prepending proem to each value changes duplicate ordering. Proem is 33 characters long. With 32 character hex string followed by ‘.’ for essentiall unlimited number of values which will be limited by memory.

With prepended proem ordinal must explicity check for duplicate values before insertion. Uses a python set for the duplicate inclusion test. Set inclusion scales with O(1) whereas list inclusion scales with O(n).

Returns:

of dup vals ot onkey when onkey present: empty list if onkey not present

Return type:

vals (list)

Parameters:

dupsort=True (db is opened named sub db with)
keyspace (key is bytes of key within sub db's)
on (int) – ordinal number at which to retrieve
sep (bytes) – separator character for split

getOnIoSetItemIter(db, key, *, on=0, ion=0, sep=b'.')[source]

Get iterator of all set vals at onkey = key + sep + on in db starting at insertion order ion within set This provides ordinal ordering of keys and inserion ordering of set vals. When key is empty then returns empty iterator

Returns:

iterator over insertion ordered set of values: at same apparent effective key made from key + on. Uses hidden ordinal key suffix for insertion ordering. The suffix is appended and stripped transparently. When key is empty then returns empty iterator

Return type:

ioset (Iterator)

Raises StopIteration Error when empty.

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort==False
key (bytes) – base key. When key is empty then returns empty iterator
on (int) – ordinal number at which to add to key form effective key
ion (int) – starting insertion ordinal value, default 0
sep (bytes) – separator character for split

Uses hidden ordinal key suffix for insertion ordering which is transparently suffixed and unsuffixed Assumes DB opened with dupsort=False

getOnIoSetLastItem(db, key, on=0, *, sep=b'.')[source]

Gets item (key, val) of last member of the insertion ordered set at key + sep + on

Returns:

last set item triple at onkey: (keys, on, val) Empty tuple () if onkey not in db or key empty.

Return type:

last (tuple[tuple, int, str])

Parameters:

db (subdb) – named sub db in lmdb
key (bytes) – key within sub db’s keyspace plus trailing part on when key is empty then retrieves whole db
on (int) – ordinal number at which to initiate retrieval
sep (bytes) – separator character for split

Uses hidden ordinal key suffix for insertion ordering which is transparently suffixed and unsuffixed Assumes DB opened with dupsort=False

getOnItem(db, key, on=0, *, sep=b'.')[source]

Gets item (key, on, val) at onkey = key + sep + on. When onkey is missing from db or key is empty or None returns None

Returns:

item (tuple[bytes, int, bytes|memoryview]|None): entry item at onkey
tuple of form (key, on, val). None if no entry at key

Parameters:
db (lmdbsubdb): named sub db of lmdb key (bytes): base key on (int): ordinal number at which to retrieve sep (bytes): separator character for split

getOnTopIoDupItemIter(db, top=b'', *, sep=b'.')[source]

Iterates over top branch of all insertion ordered set values where each key startwith top. When top is empty then iterates over whole db. Assumes every effective key in db has trailing on element, onkey = key + sep + on, so can return on in item. Also assumes every value includes hiddion insertion ordinal ion proem that is prepended and stripped transparently.

Items are triples of (key, on, val)

when key is empty then retrieves whole db

Raises StopIteration Error when empty.

Returns:

triples of key, on, val

Return type:

items (Iterator[(key, on, val)])

Parameters:

db (subdb) – named sub db in lmdb
key (bytes) – key within sub db’s keyspace plus trailing part on when key is empty then retrieves whole db
sep (bytes) – separator character for split

Uses python .startswith to match which always returns True if top is empty string so empty will matches all keys in db.

Uses hidden ordinal value proem for insertion ordering which is transparently prepended and stripped Assumes DB opened with dupsort=True

getOnTopIoSetItemIter(db, top=b'', *, sep=b'.')[source]

Iterates over top branch of all insertion ordered set values where each key startwith top. When top is empty then iterates over whole db. Assumes every effective key in db has trailing on element, onkey = key + sep + on, so can return on in item. Also assumes every effective key includes hiddion insertion ordinal ion suffix that is suffixed and unsuffixed transparently.

Items are triples of (keys, on, val)

Returns:

iterator of triples (key, on, val): where key base key, on is int, and val is entry value of with insertion ordering suffix removed from effective key.

Return type:

items (Iterator[(str, int, memoryview)])

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort==False
top (bytes) – truncated top key, a key space prefix to get all the items from multiple branches of the key space. If top key is empty then gets all items in database.
key (bytes) – base key
sep (bytes) – separator character for split

Uses python .startswith to match which always returns True if top is empty string so empty will matches all keys in db.

Uses hidden ordinal key suffix for insertion ordering which is transparently suffixed and unsuffixed Assumes DB opened with dupsort=False

getOnTopItemIter(db, top=b'', *, sep=b'.')[source]

Iterates over top branch of all entries where each top key startwith top. When top key is empty, gets all items in database. Assumes every effective key in db has trailing on element, onkey = key + sep + on, so can return on in item.

Returns:

iterator of triples: (keys, on, val)

Return type:

items (Iterator[(tuple, int, bytes)])

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort==False
top (bytes) – truncated top key, a key space prefix to get all the items from multiple branches of the key space. If top key is empty then gets all items in database.

getOnVal(db, key, on=0, *, sep=b'.')[source]

Gets value at onkey= key + sep + on When onkey is missing from db or key is empty or None returns None

Returns:

entry at onkey = key + sep + on: None if onkey missing from db or key empty or None

Return type:

val (bytes|memoryview|None)

Parameters:

db (lmdbsubdb) – named sub db of lmdb
key (bytes) – key within sub db’s keyspace plus trailing part on
on (int) – ordinal number at which to retrieve
sep (bytes) – separator character for split

getTopIoDupItemIter(db, top=b'')[source]

Iterates over top branch of db given by key of IoDup items where each value has 33 byte insertion ordinal number proem (prefixed) with separator. Automagically removes (strips) proem before returning items.

Assumes DB opened with dupsort=True

Returns:: iterator of (full key, val) tuples of all dup items over a branch of the db given by top key where returned full key is full database key for val not truncated top key. Item is (key, val) with proem stripped from val stored in db. If key = b’’ then returns list of dup items for all keys in db.
Return type:: items (abc.Iterator)

Because cursor.iternext() advances cursor after returning item its safe to delete the item within the iteration loop. curson.iternext() works for both dupsort==False and dupsort==True

Raises StopIteration Error when empty.

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort==False
top (bytes) – truncated top key, a key space prefix to get all the items from multiple branches of the key space. If top key is empty then gets all items in database

Uses python .startswith to match which always returns True if top is empty string so empty will matches all keys in db .

Duplicates at a given key preserve insertion order of duplicate. Because lmdb is lexocographic an insertion ordering proem is prepended to all values that makes lexocographic order that same as insertion order.

Duplicates are ordered as a pair of key plus value so prepending proem to each value changes duplicate ordering. Proem is 33 characters long. With 32 character hex string followed by ‘.’ for essentiall unlimited number of values which will be limited by memory.

With prepended proem ordinal must explicity check for duplicate values before insertion. Uses a python set for the duplicate inclusion test. Set inclusion scales with O(1) whereas list inclusion scales with O(n).

getTopIoSetItemIter(db, top=b'', *, sep=b'.')[source]

Iterates over top branch of all insertion ordered set values where each effective key has hidden suffix of serialization of insertion ordering ordinal ion. When top is empty then iterates over whole db.

Uses hidden ordinal key suffix for insertion ordering.: The suffix is suffixed and unsuffixed transparently.

Returns:

iterator of tuples (key, val) where: key is apparent key with hidden insertion ordering suffixe removed from effective key.

Return type:

items (Iterator[(key,val)])

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort==False
top (bytes) – truncated top key, a key space prefix to get all the items from multiple branches of the key space. If top key is empty then gets all items in database.
sep (bytes) – sep character for attached io suffix

Uses python .startswith to match which always returns True if top is empty string so empty will matches all keys in db.

getTopItemIter(db, top=b'')[source]

Iterates over branch of db given by top key. When top is empty then iterates over whole db.

Works for both dupsort==False and dupsort==True Because cursor.iternext() advances cursor after returning item its safe to delete the item within the iteration loop.

Raises StopIteration Error when empty.

Returns:

iterator of (full key, val) tuples over a: branch of the db given by top key where: full key is full database key for val not truncated top key

Return type:

items (Iterator)

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort==False
top (bytes) – truncated top key, a key space prefix to get all the items from multiple branches of the key space. If top key is empty then gets all items in database.

Uses python .startswith to match which always returns True if top is empty string so empty will matches all keys in db .

Works for both dupsort==False and dupsort==True Because cursor.iternext() advances cursor after returning item its safe to delete the item within the iteration loop.

getVal(db, key)[source]

Return val at key in db Returns None if no entry at key

Parameters:

dupsort=False (db is opened named sub db with)
keyspace (key is bytes of key within sub db's)

getValLast(db, key)[source]

Return last dup value at key in db in lexicographic order Returns None no entry at key Assumes DB opened with dupsort=True

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort=True
keyspace (key is bytes of key within sub db's)

getVals(db, key)[source]

Return list of values at key in db Returns empty list if no entry at key

Duplicates are retrieved in lexocographic order not insertion order.

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort=True
keyspace (key is bytes of key within sub db's)

getValsIter(db, key)[source]

Return iterator of all dup values at key in db Raises StopIteration error when done or if empty

Duplicates are retrieved in lexocographic order not insertion order.

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort=True
keyspace (key is bytes of key within sub db's)

getVer()[source]

Returns the value of the the semver formatted version in the __version__ key in this database

Returns:: semver formatted version of the database
Return type:: str

pinIoSetVals(db, key, vals, *, sep=b'.')[source]

Replace all vals at key with vals as insertion ordered set of values all with the same apparent effective key. Does not replace if key is empty or None or vals is empty or None

Uses hidden ordinal key suffix for insertion ordering. The suffix is appended and stripped transparently.

Returns:

True if vals replaced set.: False otherwise including key not in db, empty or None or vals empty or None

Return type:

result (bool)

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort==False
key (bytes|None) – Apparent effective key
vals (NonStrIterable|None) – serialized values to add to set of vals at key
sep (bytes) – separator character for split

pinOnIoSetVals(db, key, *, on=0, vals=None, sep=b'.')[source]

Replace all vals if any at onkey = key + sep + one with vals as insertion ordered set of values all with the same onkey. Does not replace if key is empty or None or vals is empty or None

Returns:

True if vals replaced set.: False otherwise including key not in db, empty or None or vals empty or None

Return type:

result (bool)

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort==False
key (bytes|None) – base key
vals (NonStrIterable|None) – serialized values to replace vals at key
on (int) – ordinal number to add to key form onkey
sep (bytes) – separator character for split

Assumes DB opened with dupsort=False

Set of values at a given effective key preserve insertion order. Because lmdb is lexocographic an insertion ordering suffix is appended to all keys that makes lexocographic order the same as insertion order.

Suffix is 33 characters long consisting of sep ‘.’ followed by 32 char hex string for essentially unlimited number of values in each set only limited by memory.

With appended suffix ordinal must explicity check for duplicate values in set before insertion. Uses a python set for the duplicate inclusion test. Set inclusion scales with O(1) whereas list inclusion scales with O(n).

Uses hidden ordinal key suffix for insertion ordering which is transparently suffixed and unsuffixed Assumes DB opened with dupsort=False

pinOnVal(db, key, on=0, val=None, *, sep=b'.')[source]

Replace value if any at location onkey = key + sep + on with val Replaces pre-existing value at onkey if any or different. When key empty or None or or val None returns false.

Returns:

True if successful replacement.: False if val already exists at key or if key empty or val None.

Return type:

result (bool)

Parameters:

db (lmdbsubdb) – named sub db of lmdb
key (bytes) – key within sub db’s keyspace plus trailing part on
on (int) – ordinal number at which write
val (bytes|None) – to be written at onkey. when None returns False
sep (bytes) – separator character for split

putIoDupVals(db, key, vals)[source]

Write each entry from list of bytes vals to key in db in insertion order Adds to existing values at key if any Returns True If at least one of vals is added as dup, False otherwise Assumes DB opened with dupsort=True

Duplicates at a given key preserve insertion order of duplicate. Because lmdb is lexocographic an insertion ordering proem is prepended to all values that makes lexocographic order that same as insertion order.

Duplicates are ordered as a pair of key plus value so prepending proem to each value changes duplicate ordering. Proem is 33 characters long. With 32 character hex string followed by ‘.’ for essentiall unlimited number of values which will be limited by memory.

With prepended proem ordinal must explicity check for duplicate values before insertion. Uses a python set for the duplicate inclusion test. Set inclusion scales with O(1) whereas list inclusion scales with O(n).

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort=True
key (bytes) – within sub db’s keyspace
vals (Iterable[bytes]) – of values to be written

putIoSetVals(db, key, vals, *, sep=b'.')[source]

Add each val in vals to insertion ordered set of values all with the same apparent effective key for each val that is not already in set of vals at key. Uses hidden ordinal key suffix for insertion ordering. The suffix is appended and stripped transparently.

Returns:

True if any val in vals is added to set.: False otherwise including key not in db, empty or None or vals empty or None

Return type:

result (bool)

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort==False
key (bytes|None) – Apparent effective key
vals (NonStrIterable|None) – serialized values to add to set of vals at key
sep (bytes) – separator character for split

putOnIoDupVals(db, key, on=0, vals=b'', *, sep=b'.')[source]

Write each entry from list of bytes vals to key made from key + sep + on where on is serialized in db in insertion order using IO proem prepended to each value. Adds to existing values at key if any Returns True If at least one of vals is added as dup, False otherwise Assumes DB opened with dupsort=True

Duplicates at a given key preserve insertion order of duplicate. Because lmdb is lexocographic an insertion ordering proem is prepended to all values that makes lexocographic order that same as insertion order.

Duplicates are ordered as a pair of key plus value so prepending proem to each value changes duplicate ordering. Proem is 33 characters long. With 32 character hex string followed by ‘.’ for essentiall unlimited number of values which will be limited by memory.

With prepended proem ordinal must explicity check for duplicate values before insertion. Uses a python set for the duplicate inclusion test. Set inclusion scales with O(1) whereas list inclusion scales with O(n).

Parameters:

dupsort=False (db is opened named sub db with)
keyspace (key is bytes of key within sub db's)
on (int) – ordinal number at which to add
written (vals is list of bytes of values to be) – sep (bytes): separator character for split

putOnIoSetVals(db, key, *, on=0, vals=None, sep=b'.')[source]

Add idempotently each val from list of bytes vals to set of entries at onkey = key + sep + on. Does not add if key is empty or None Each unique entry in set at each on is serialized in db in insertion order using hidden IO suffix for each onkey.

Returns:

True if any val in vals is added to set.: False otherwise including key not in db, empty or None or vals empty or None

Return type:

result (bool)

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort==False
key (bytes|None) – base key
on (int) – ordinal number to add to key form onkey
vals (NonStrIterable|None) – serialized values to add to set of vals at effective key if any. None returns False
sep (bytes) – separator character for split

Set of values at a given effective key preserve insertion order. Because lmdb is lexocographic an insertion ordering suffix is appended to all keys that makes lexocographic order the same as insertion order.

Suffix is 33 characters long consisting of sep ‘.’ followed by 32 char hex string for essentially unlimited number of values in each set only limited by memory.

With appended suffix ordinal must explicity check for duplicate values in set before insertion. Uses a python set for the duplicate inclusion test. Set inclusion scales with O(1) whereas list inclusion scales with O(n).

Uses hidden ordinal key suffix for insertion ordering which is transparently suffixed and unsuffixed Assumes DB opened with dupsort=False

putOnVal(db, key, on=0, val=None, *, sep=b'.')[source]

Write serialized bytes val to location at onkey consisting of key + sep + serialized on in db. Does not overwrite.

Returns:

True if successful write i.e onkey not already in db: False otherwise

Return type:

result (bool)

Parameters:

db (lmdbsubdb) – named sub db of lmdb
key (bytes) – key within sub db’s keyspace plus trailing part on
on (int) – ordinal number at which write
val (bytes|None) – to be written at onkey When None returns False
sep (bytes) – separator character for split

putVal(db, key, val)[source]

Write serialized bytes val to location key in db Does not overwrite. Returns True If val successfully written Else False Returns False if val at key already exitss

Parameters:

dupsort=False (db is opened named sub db with)
keyspace (key is bytes of key within sub db's)
written (val is bytes of value to be)

putVals(db, key, vals)[source]

Write each entry from list of bytes vals to key in db Adds to existing values at key if any Returns True If only one first written val in vals Else False Apparently always returns True (is this how .put works with dupsort=True)

Duplicates are inserted in lexocographic order not insertion order. Lmdb does not insert a duplicate unless it is a unique value for that key.

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort=True
key (bytes) – within sub db’s keyspace
vals (Iterable[bytes]) – of values to be written

remIoSet(db, key, *, sep=b'.')[source]

Removes all set values at apparent effective key. When key is empty or None or missing returns False.

Uses hidden ordinal key suffix for insertion ordering.: The suffix is suffixed and unsuffixed transparently.

Returns:

True if values were deleted at key.: False otherwise including key empty or None

Return type:

result (bool)

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort==False
key (bytes|None) – Apparent effective key
sep (bytes) – separator character for split

remIoSetVal(db, key, val=None, *, sep=b'.')[source]

Removes val if any as member of set at key if any. When value is None then removes all set members at key When key is empty or missing returns False. Uses hidden ordinal key suffix for insertion ordering. The suffix is suffixed and unsuffixed transparently.

Because the insertion order of val is not provided must perform a linear search over set of values.

Another problem is that vals may get added and deleted in any order so the max suffix ion may creep up over time. The suffix ordinal max > 2**16 is an impossibly large number, however, so the suffix will not max out practically.But its not the most elegant solution.

In some cases a better approach would be to use getIoSetItemsIter which returns the actual iokey not the apparent effective key so can delete using the iokey without searching for the value. This is most applicable when processing escrows where all the escrowed items are processed linearly and one needs to delete some of them in stride with their processing.

Returns:

True if val at key removed when val not None: or all entries at key removed when val None. False otherwise if no values at key or key is empty or val not found.

Return type:

result (bool)

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort==False
key (bytes) – val(int|None): value to remove if any. None means remove all entries at onkey
val (bytes|None) – value to delete
sep (bytes) – separator character for split

remOn(db, key, on=0, *, sep=b'.')[source]

Removes entry if any at onkey = key + sep + on. When key is missing or empty or None returns False.

Returns:

True if entry at onkey removed when not None.: False otherwise if no entry at onkey or key is empty.

Return type:

result (bool)

Parameters:

db (lmdbsubdb) – named sub db of lmdb
key (bytes) – key within sub db’s keyspace plus trailing part on
on (int) – ordinal number at which to delete
sep (bytes) – separator character for split

remOnAll(db, key=b'', on=0, *, sep=b'.')[source]

Removes entry at each onkey for all on >= on where for each on, onkey = key + sep + on When on is 0, default, then deletes all on at key. When key is empty then deletes whole db.

Returns:

True if any entries deleted: False otherwise

Return type:

result (bool)

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort==False
key (bytes) – base key
on (int) – ordinal number at which to add to key form effective key 0 means to delete all on at key
sep (bytes) – separator character for split

Uses hidden ordinal key suffix for insertion ordering which is transparently suffixed and unsuffixed Assumes DB opened with dupsort=False

remOnAllIoSet(db, key=b'', on=0, *, sep=b'.')[source]

Removes all set members at onkey for all on >= on where for each on, onkey = key + sep + on When on is 0, default, then deletes all on at key. When key is empty then deletes whole db.

Returns:

True if any entries deleted: False otherwise

Return type:

result (bool)

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort==False
key (bytes) – base key
on (int) – ordinal number at which to add to key form effective key 0 means to delete all on
sep (bytes) – separator character for split

Uses hidden ordinal key suffix for insertion ordering which is transparently suffixed and unsuffixed Assumes DB opened with dupsort=False

remOnIoSetVal(db, key, *, on=0, val=None, sep=b'.')[source]

Removes val if any as member of set at onkey = key + sep + on. When val is None then removes all set members at onkey. When key is empty or None or missing returns False.

Uses hidden ordinal key suffix for insertion ordering. The suffix is suffixed and unsuffixed transparently.

Because the insertion order of val is not provided must perform a linear search over set of values.

Another problem is that vals may get added and deleted in any order so the max suffix ion may creep up over time. The suffix ordinal max > 2**16 is an impossibly large number, however, so the suffix will not max out practically.But its not the most elegant solution.

In some cases a better approach would be to use getIoSetItemsIter which returns the actual iokey not the apparent effective key so can delete using the iokey without searching for the value. This is most applicable when processing escrows where all the escrowed items are processed linearly and one needs to delete some of them in stride with their processing.

Returns:

True if val at onkey removed when val not None: or all entries at onkey removed when val None. False otherwise if no values at onkey or key is empty or val not found.

Return type:

result (bool)

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort==False
key (bytes) – base key. When key is empty returns False
on (int) – ordinal number at which to add to key form effective key
val (int|None) – value to remove if any. None means remove all entries at onkey
sep (bytes) – separator character for split

Uses hidden ordinal key suffix for insertion ordering which is transparently suffixed and unsuffixed Assumes DB opened with dupsort=False

remTop(db, top=b'')[source]

Deletes all values in branch of db given top key. Top empty deletes whole db.

Returns:

True if values were deleted at key. False otherwise: if no values at key

Return type:

result (bool)

Parameters:

db (lmdb._Database) – instance of named sub db with dupsort==False
top (bytes) – truncated top key, a key space prefix to get all the items from multiple branches of the key space. If top key is empty then deletes all items in database

Works for both dupsort==False and dupsort==True Because cursor.iternext() advances cursor after returning item its safe to delete the item within the iteration loop.

remVal(db, key)[source]

Removes value at key in db. :returns:

True If key exists in database and item deleted
False If key empty or missing from database

Return type:: result (bool)

Raises KeyError if problem with key

Parameters:

dupsort=False (db is opened named sub db with)
keyspace (key is bytes of key within sub db's)

reopen(readonly=False, **kwa)[source]

Open if closed or close and reopen if opened or create and open if not if not preexistent, directory path for lmdb at .path and then Open lmdb and assign to .env

Parameters:

temp (bool) – assign to .temp True means open in temporary directory, clear on close False means open persistent directory, do not clear on close
headDirPath (str) – optional head directory pathname for main database Default .HeadDirpath
perm (int) – optional numeric os dir permissions for database directory and database files. Default .Perm
clear (bool) – True means remove directory upon close False means do not remove directory upon close
reuse (bool) – True means reuse self.path if already exists False means do not reuse but remake self.path
clean (bool) – True means path uses clean tail variant False means path uses normal tail variant
mode (str) – file open mode when .filed
fext (str) – File extension when .filed
readonly (bool) – True means open database in readonly mode False means open database in read/write mode

setVal(db, key, val)[source]

Write serialized bytes val to location key in db Overwrites existing val if any :returns:

True If val successfully written
False otherwise

Return type:

result (bool)

Parameters:

dupsort=False (db is opened named sub db with)
keyspace (key is bytes of key within sub db's)
written (val is bytes of value to be)

setVer(val)[source]

Set the version of the database in the __version__ key

Parameters:: val (str) – The new semver formatted version of the database

property version

Return the version of database stored in __version__ key.

This value is read through cached in memory

Returns:: the version of the database or None if not set in the database
Return type:: str

keri.db.dbing.clearDatabaserDir(path)[source]: Remove directory path

keri.db.dbing.dgKey(pre, dig)[source]: Returns bytes DB key from concatenation of ‘.’ with qualified Base64 prefix bytes pre and qualified Base64 bytes digest of serialized event If pre or dig are str then converts to bytes

keri.db.dbing.dtKey(pre, dts)[source]

Returns bytes DB key from concatenation of ‘|’ qualified Base64 prefix bytes pre and bytes dts datetime string of extended tz aware ISO8601 datetime of event

‘2021-02-13T19:16:50.750302+00:00’

keri.db.dbing.fetchTsgs(db, diger, snh=None)[source]

Fetch tsgs for diger from .db.ssgs. When sn then only fetch if sn <= snh :returns:

of tsg quadruple of form (prefixer, seqner, diger, sigers)

where:
prefixer (Prefixer): instance trans signer aid, seqner (Seqner): of sn of trans signer key state est event diger (Diger): of digest of trans signer key state est event sigers (list): of Siger instances of indexed signatures

Return type:

tsgs (list)

Parameters:

db – (Cesr
diger (Diger) – instance of said for reply SAD to which signatures are attached
snh (str) – 32 char zero pad lowercase hex of sequence number f”{sn:032x}”

keri.db.dbing.fnKey(pre, fn)[source]

Returns:

key formed by joining pre and hex str conversion of int: first seen ordinal number fn with sep character b”.”.

Return type:

fnkey (bytes)

Parameters:

pre (str | bytes) – key prefix to be joined with hex version of on using b”.” sep
fn (int) – first seen ordinal number to be converted to 32 hex bytes

keri.db.dbing.onKey(top, on, *, sep=b'.')[source]

Returns:

key formed by joining top key and hex str conversion of: int ordinal number on with sep character.

Return type:

onkey (bytes)

Parameters:

top (str | bytes) – top key prefix to be joined with hex version of on using sep
on (int) – ordinal number to be converted to 32 hex bytes
sep (bytes) – separator character for join

keri.db.dbing.openLMDB(*, cls=None, name='test', temp=True, **kwa)[source]

Context manager wrapper LMDBer instances. Defaults to temporary databases. Context ‘with’ statements call .close on exit of ‘with’ block

Parameters:

instance (cls is Class instance of subclass)
databasers (name is str name of LMDBer dirPath so can have multiple) – at different directory path names thar each use different name
Boolean (temp is) – Otherwise open in persistent directory, do not clear on close
directory (True means open in temporary) – Otherwise open in persistent directory, do not clear on close
close (clear on) – Otherwise open in persistent directory, do not clear on close

Usage:

with openDatabaser(name=”gen1”) as baser1:: baser1.env ….

with openDatabaser(name=”gen2, cls=Baser)

wl.close(clear=True if wl.temp else False)

keri.db.dbing.snKey(pre, sn)[source]

Returns:

key formed by joining pre and hex str conversion of int: sequence ordinal number sn with sep character b”.”.

Return type:

snkey (bytes)

Parameters:

pre (str | bytes) – key prefix to be joined with hex version of on using b”.” sep
sn (int) – sequence number to be converted to 32 hex bytes

keri.db.dbing.splitFnKey(key, *, sep=b'.'): Returns list of pre and int on from key Accepts either bytes or str key ordinal number appears in key in hex format

keri.db.dbing.splitKey(key, sep=b'.')[source]

Returns duple of pre and either dig or on, sn, fn str or dts datetime str by splitting key at bytes sep Accepts either bytes or str key and returns same type Raises ValueError if key does not split into exactly two elements

Parameters:

sep (key is database key with split at)
b'.' (sep is bytes separator character. default is)

keri.db.dbing.splitKeyDT(key)[source]

Returns list of pre and dts converted to datetime from key dts is TZ aware Iso8601 ‘2021-02-13T19:16:50.750302+00:00’

Accepts either bytes or str key

keri.db.dbing.splitKeyFN(key, *, sep=b'.'): Returns list of pre and int on from key Accepts either bytes or str key ordinal number appears in key in hex format

keri.db.dbing.splitKeyON(key, *, sep=b'.'): Returns list of pre and int on from key Accepts either bytes or str key ordinal number appears in key in hex format

keri.db.dbing.splitKeySN(key, *, sep=b'.'): Returns list of pre and int on from key Accepts either bytes or str key ordinal number appears in key in hex format

keri.db.dbing.splitOnKey(key, *, sep=b'.')[source]: Returns list of pre and int on from key Accepts either bytes or str key ordinal number appears in key in hex format

keri.db.dbing.splitSnKey(key, *, sep=b'.'): Returns list of pre and int on from key Accepts either bytes or str key ordinal number appears in key in hex format

keri.db.dbing.suffix(key: bytes | str | memoryview, ion: int, *, sep: bytes | str = b'.')[source]

Returns:

actual DB key after concatenating suffix as hex version of insertion ordering ordinal int ion using separator sep.

Return type:

iokey (bytes)

Parameters:

key (Union[bytes, str]) – apparent effective database key (unsuffixed)
ion (int)) – insertion ordering ordinal for set of vals
sep (bytes) – separator character(s) for concatenating suffix

keri.db.dbing.unsuffix(iokey: bytes | str | memoryview, *, sep: bytes | str = b'.')[source]

Returns:

(key, ion) by splitting iokey at rightmost separator sep: strip off suffix, where key is bytes apparent effective DB key and ion is the insertion ordering int converted from stripped of hex suffix

Return type:

result (tuple)

Parameters:

iokey (Union[bytes, str]) – apparent effective database key (unsuffixed)
sep (bytes) – separator character(s) for concatenating suffix

keri.db.escrowing

keri.core.escrowing module

class keri.db.escrowing.Broker(db, subkey, timeout=3600)[source]

Collection of databases for transaction state notices (TSNs) and handling TSN escrows.

__init__(db, subkey, timeout=3600)[source]

Initialize Broker with databases for transaction state notices and escrows. :param db: TEL event database to make sub databases under :type db: Reger :param subkey: parent LMDB subkey path to use for all sub databases :type subkey: str :param timeout: timeout in seconds for escrows, default is 3600 seconds (1 hour) :type timeout: int

db

TEL event database to make sub databases under

Type:: Reger

timeout

timeout in seconds for escrows, default is 3600 seconds (1 hour)

Type:: int

daterdb

database for datetime stamps by ksn SAID

Type:: CesrSuber

serderdb

database for reply messages by ksn SAID

Type:: SerderSuber

tigerdb

database for indexed signatures by ksn quadruple Key schema: (said, pre, sn, dig)

Type:: CesrIoSetSuber

cigardb

database for non-indexed signatures by ksn SAID

Type:: CatCesrIoSetSuber

escrowdb

database for escrows by route by (typ, pre, aid) tuple

Type:: CesrIoSetSuber

saiderdb

database for transaction state SAIDs by (pre, aid) tuple

Type:: CesrSuber

current(keys)[source]

Get successfully saved TSNs by keys.

Parameters:

(str – keys tuple of (prefix, aid) where prefix is the registry identifier and pre is the issuer
str) – keys tuple of (prefix, aid) where prefix is the registry identifier and pre is the issuer

Returns:

UTF-8 encoded string of the SAid of a TSN or None if not found.

Return type:

data (str | None)

escrowStateNotice(*, typ, pre, aid, serder, diger, dater, cigars=None, tsgs=None)[source]

Escrow reply by route

Parameters:

typ (str) – escrow type
pre (str) – identifier of key state
aid (str) – identifier of authorizer of key state
serder (Serder) – instance of reply msg (SAD)
diger (Diger) – instance from said in serder (SAD)
dater (Dater) – instance from date-time in serder (SAD)
cigars (list) – of Cigar instances that contain nontrans signing couple signature in .raw and public key in .verfer
tsgs (Iterable) – of quadruples of form (prefixer, seqner, diger, siger) where: prefixer is pre of trans endorser seqner is sequence number of trans endorser’s est evt for keys for sigs diger is digest of trans endorser’s est evt for keys for sigs siger is indexed sig from trans endorser’s key from est evt

processEscrowState(typ, processReply, extype: Type[Exception])[source]

Process escrows for reply messages

Process escrows for reply messages. Escrows are keyed by reply pre and val is reply said

triple (prefixer, seqner, diger) quadruple (prefixer, seqner, diger, siger)

Parameters:

typ (str) – escrow type
processReply (func) – function to call to process each message taken out of escrow
extype (Type[Exception]) – the expected exception type if the message should remain in escrow

removeState(diger)[source]: Remove all state associated with the given event TSN identified by SAID.

updateReply(aid, serder, diger, dater)[source]

Update Reply SAD in database given by by serder and associated databases for attached cig couple or sig quadruple. Overwrites val at key if already exists.

Parameters:

aid (str) – identifier of key state
serder (Serder) – instance of reply msg (SAD)
diger (Diger) – instance from said in serder (SAD)
dater (Dater) – instance from date-time in serder (SAD)

keri.db.koming

KERI keri.db.koming module

class keri.db.koming.DupKomer(db: LMDBer, *, subkey: str = 'recs.', klas: type[dataclass], kind: str | None = None, **kwa)[source]

Duplicate Keyspace Object Mapper factory class that supports multiple entries a given database key (lmdb dupsort == True).

Do not use if Komer dataclass instance serializes to greater than 511 bytes. This is a limitation of dupsort==True sub dbs in LMDB

__init__(db: LMDBer, *, subkey: str = 'recs.', klas: type[dataclass], kind: str | None = None, **kwa)[source]

Parameters:

db (LMDBer) – base db
schema (Type[dataclass]) – reference to Class definition for dataclass sub class
subkey (str) – LMDB sub database key
kind (str) – serialization/deserialization type

add(keys: str | bytes | memoryview | Iterable, val: dataclass)[source]

Add val to vals at key made from keys. Does not overwrite. Adds to existing dup values at key if any. Duplicate means another entry at the same key but the entry is still a unique value. Duplicates are inserted in lexocographic order not insertion order. Lmdb does not insert a duplicate unless it is a unique value for that key.

Parameters:

keys (str|bytes|memoryview|Iterable) – of key strs to be combined in order to form key
val (dataclass) – instance of type self.schema

Returns:

True means unique value among duplications,: False means duplicte of same value already exists.

Return type:

result (bool)

cnt(keys: str | bytes | memoryview | Iterable)[source]

Count entries (dups) at key made from keys.

Returns:: dup values at key made from keys, zero otherwise
Return type:: count (int)
Parameters:: keys (str|bytes|memoryview|Iterable) – of key strs to be combined in order to form key

get(keys: str | bytes | memoryview | Iterable)[source]

Gets dup vals list at key made from keys

Parameters:

keys (str|bytes|memoryview|Iterable) – of key strs to be combined in order to form key

Returns:

each item in list is instance of type self.schema: empty list if no entry at keys

Return type:

vals (list)

getIter(keys: str | bytes | memoryview | Iterable)[source]

Gets dup vals iterator at key made from keys

Duplicates are retrieved in lexocographic order not insertion order.

Parameters:: keys (str|bytes|memoryview|Iterable) – of key strs to be combined in order to form key
Returns:: vals each of type self.schema. Raises StopIteration when done
Return type:: iterator

getLast(keys: str | bytes | memoryview | Iterable)[source]

Gets last dup val at key made from keys

Parameters:

keys (str|bytes|memoryview|Iterable) – of key strs to be combined in order to form key

Returns:

instance of type self.schema: None if no entry at keys

Return type:

val (Type[dataclass])

pin(keys: str | bytes | memoryview | Iterable, vals: list)[source]

Pins (sets) vals at key made from keys. Overwrites. Removes all pre-existing dup vals and replaces them with vals

Parameters:

keys (str|bytes|memoryview|Iterable) – of key strs to be combined in order to form key
vals (list) – dataclass instances each of type self.schema as values

Returns:

True If successful, False otherwise.

Return type:

result (bool)

put(keys: str | bytes | memoryview | Iterable, vals: list)[source]

Puts all vals at key made from keys. Does not overwrite. Adds to existing dup values at key if any. Duplicate means another entry at the same key but the entry is still a unique value. Duplicates are inserted in lexocographic order not insertion order. Lmdb does not insert a duplicate unless it is a unique value for that key.

Parameters:

keys (str|bytes|memoryview|Iterable) – of key strs to be combined in order to form key
vals (list) – dataclass instances each of type self.schema as values

Returns:

True If successful, False otherwise.

Return type:

result (bool)

Apparently always returns True (how .put works with dupsort=True)

rem(keys: str | bytes | memoryview | Iterable, val=None)[source]

Removes entry at key made from keys

Parameters:

keys (str|bytes|memoryview|Iterable) – of key strs to be combined in order to form key
val (dataclass) – instance of dup val at key to delete if val is None then remove all values at key

Returns:

True if key exists so delete successful. False otherwise

Return type:

result (bool)

class keri.db.koming.IoSetKomer(db: LMDBer, *, subkey: str = 'recs.', klas: type[dataclass], kind: str | None = None, **kwa)[source]

Insertion Ordered Set Keyspace Object Mapper factory class that supports a set of distinct entries at a given effective database key but with dupsort==False. Effective data model is that there are multiple values in a set of values where every member of the set has the same key (duplicate key). The set of values is an ordered set using insertion order. Any given value may appear only once in the set (not a list).

This works similarly to the IO value duplicates for the LMDBer class with a sub db of LMDB (dupsort==True) but without its size limitation of 511 bytes for each value when dupsort==True. Here the key is augmented with a hidden numbered suffix that provides a an ordered set of values at each effective key (duplicate key). The suffix is appended and stripped transparently. The set of multiple items with duplicate keys are retrieved in insertion order when iterating or as a list of the set elements.

db

instance of LMDB database manager class

Type:: LMDBer

sdb

instance of named sub db lmdb for this Komer

Type:: lmdb._Database

schema

class reference of dataclass subclass

Type:: Type[dataclass]

kind

serialization/deserialization type from coring.Serials

Type:: str

serializer

serializer method

Type:: types.MethodType

deserializer

deserializer method

Type:: types.MethodType

sep

separator for combining keys tuple of strs into key bytes

Type:: str

__init__(db: LMDBer, *, subkey: str = 'recs.', klas: type[dataclass], kind: str | None = None, **kwa)[source]

Parameters:

db (LMDBer) – base db
clas (type[dataclass]) – reference to Class definition for dataclass sub class
subkey (str) – LMDB sub database key
kind (str) – serialization/deserialization type

add(keys: str | bytes | memoryview | Iterable, val: dataclass)[source]

Add val to vals at effective key made from keys and hidden ordinal suffix. that is not already in set of vals at key. Does not overwrite.

Parameters:

keys (str|bytes|memoryview|Iterable) – of key strs to be combined in order to form key
val (dataclass) – instance of type self.schema

Returns:

True means unique value among duplications,: False means duplicte of same value already exists.

Return type:

result (bool)

cnt(keys: str | bytes | memoryview | Iterable = '')[source]

Count of effective dup values at key made from keys. If keys is empty then returns count of all entries in db

Parameters:: keys (str|bytes|memoryview|Iterable) – of key strs to be combined in order to form key. If empty then returns coutn of all entries in db.

get(keys: str | bytes | memoryview | Iterable)[source]

Gets dup vals list at key made from keys

Parameters:

keys (str|bytes|memoryview|Iterable) – of key strs to be combined in order to form key

Returns:

each item in list is instance of type self.schema: empty list if no entry at keys

Return type:

vals (list)

getIter(keys: str | bytes | memoryview | Iterable)[source]

Gets vals iterator at effecive key made from keys and hidden ordinal suffix. All vals in set of vals that share same effecive key are retrieved in insertion order.

Parameters:: keys (str|bytes|memoryview|Iterable) – of key strs to be combined in order to form key
Returns:: str values. Raises StopIteration when done
Return type:: vals (Iterator)

getLast(keys: str | bytes | memoryview | Iterable)[source]

Gets last effective dup val at effective dup key made from keys

Parameters:

keys (str|bytes|memoryview|Iterable) – of key strs to be combined to form effective key

Returns:

instance of type self.schema: None if no entry at keys

Return type:

val (Type[dataclass])

getTopItemIter(keys: str | bytes | memoryview | Iterable = b'', *, topive=False)[source]

Get items iterator over top branch of db given by keys.

Returns:

of (key, val) tuples over the all the items in subdb whose effective key startswith key made from keys. Keys may be keyspace prefix in order to return branches of key space. When keys is empty then returns all items in subdb. Returned key in each item has ordinal suffix removed.

Return type:

items (Iterator)

Parameters:

keys (str|bytes|memoryview|Iterable) – may be a truncation of a full keys tuple in in order to address all the items from multiple branches of the key space. If keys is empty then gets all items in database. Either append “” to end of keys Iterable to ensure get properly separated top branch key or use top=True. In Python str.startswith(‘’) always returns True so if branch key is empty string it matches all keys in db with startswith.
topive (bool) – True means treat as partial key tuple from top branch of key space given by partial keys. Resultant key ends in .sep character. False means treat as full branch in key space. Resultant key does not end in .sep character. When last item in keys is empty str then will treat as partial ending in sep regardless of top value

pin(keys: str | bytes | memoryview | Iterable, vals: list)[source]

Pins (sets) vals at effective key made from keys and hidden ordinal suffix. Overwrites. Removes all pre-existing vals that share same effective keys and replaces them with vals

Parameters:

keys (str|bytes|memoryview|Iterable) – of key strs to be combined in order to form key
vals (list) – dataclass instances each of type self.schema as values

Returns:

True If successful, False otherwise.

Return type:

result (bool)

put(keys: str | bytes | memoryview | Iterable, vals: list)[source]

Puts all vals at key made from keys. Does not overwrite. Puts all vals at effective key made from keys and hidden ordinal suffix. that are not already in set of vals at key. Does not overwrite.

Parameters:

keys (str|bytes|memoryview|Iterable) – of key strs to be combined in order to form key
vals (list) – dataclass instances each of type self.schema as values

Returns:

True If successful, False otherwise.

Return type:

result (bool)

Apparently always returns True (how .put works with dupsort=True)

rem(keys: str | bytes | memoryview | Iterable, val=None)[source]

Removes entry at keys

Parameters:

keys (str|bytes|memoryview|Iterable) – of key strs to be combined in order to form key
val (dataclass) – instance of effective dup val at key to delete if val is None then remove all values at key

Returns:

True if key exists so delete successful. False otherwise

Return type:

result (bool)

class keri.db.koming.Komer(db: LMDBer, *, subkey: str = 'docs.', klas: type[dataclass], kind: str | None = None, **kwa)[source]

Keyspace dataclass Object Mapper factory class. Maps (serializes and deserializes) dataclass to/from database entry at key made from keys

__init__(db: LMDBer, *, subkey: str = 'docs.', klas: type[dataclass], kind: str | None = None, **kwa)[source]: Initialize instance :param db: base db :type db: LMDBer :param klas: reference to Class definition for dataclass sub class :type klas: Type[dataclass] :param subkey: LMDB sub database key :type subkey: str :param kind: serialization/deserialization type :type kind: str

cnt()[source]

Count all items in db

Returns:: of tuples of keys tuple and val dataclass instance for each entry in db. Raises StopIteration when done
Return type:: iterator

Example

if key in database is “a.b” and val is serialization of dataclass: with attributes x and y then returns ((“a”,”b”), dataclass(x=1,y=2))

cntAll()

Count all items in db

Returns:: of tuples of keys tuple and val dataclass instance for each entry in db. Raises StopIteration when done
Return type:: iterator

Example

if key in database is “a.b” and val is serialization of dataclass: with attributes x and y then returns ((“a”,”b”), dataclass(x=1,y=2))

get(keys: str | bytes | memoryview | Iterable)[source]

Gets val at keys

Parameters:: keys (str|bytes|memoryview|Iterable) – of key strs to be combined in order to form key
Returns:: None if no entry at keys
Return type:: val (dataclass)

Usage:

Use walrus operator to catch and raise missing entry if (val := mydb.get(keys)) is None:

raise ExceptionHere

use val here

getDict(keys: str | bytes | memoryview | Iterable)[source]

Gets dictified val at keys

Parameters:: keys (str|bytes|memoryview|Iterable) – of key strs to be combined in order to form key
Returns:: None if no entry at keys
Return type:: val (dict)

Usage:

Use walrus operator to catch and raise missing entry if (val := mydb.get(keys)) is None:

raise ExceptionHere

use val here

pin(keys: str | bytes | memoryview | Iterable, val: dataclass)[source]

Pins (sets) val at key made from keys. Overwrites.

Parameters:

keys (str|bytes|memoryview|Iterable) – of key strs to be combined in order to form key
val (dataclass) – instance of dataclass of type self.schema as value

Returns:

True If successful. False otherwise.

Return type:

result (bool)

put(keys: str | bytes | memoryview | Iterable, val: dataclass)[source]

Puts val at key made from keys. Does not overwrite

Parameters:

keys (str|bytes|memoryview|Iterable) – of key strs to be combined in order to form key
val (dataclass) – instance of dataclass of type self.schema as value

Returns:

True If successful, False otherwise, such as key: already in database.

Return type:

result (bool)

rem(keys: str | bytes | memoryview | Iterable)[source]

Removes entry at keys

Parameters:: keys (str|bytes|memoryview|Iterable) – of key strs to be combined in order to form key
Returns:: True if key exists so delete successful. False otherwise
Return type:: result (bool)

class keri.db.koming.KomerBase(db: LMDBer, *, subkey: str = 'docs.', klas: type[dataclass], kind: str | None = None, dupsort: bool = False, sep: str = None, **kwa)[source]

KomerBase is a base class for Komer (Keyspace Object Mapper) subclasses that each use a dataclass as the object mapped via serialization to an dber LMDB database subclass. Each Komer .schema is a dataclass class reference that is used to define the fields in each database entry. The base class is not meant to be instantiated. Use an instance of one of the subclasses instead.

db

instance of LMDB database manager class

Type:: LMDBer

sdb

instance of named sub db lmdb for this Komer

Type:: lmdb._Database

schema

class reference of dataclass subclass

Type:: Type[dataclass]

kind

serialization/deserialization type from coring.Serials

Type:: str

serializer

serializer method

Type:: types.MethodType

deserializer

deserializer method

Type:: types.MethodType

sep

separator for combining keys tuple of strs into key bytes

Type:: str

__init__(db: LMDBer, *, subkey: str = 'docs.', klas: type[dataclass], kind: str | None = None, dupsort: bool = False, sep: str = None, **kwa)[source]

Parameters:

db (LMDBer) – base db
klas (type[dataclass]) – reference to Class definition for dataclass sub class
subkey (str) – LMDB sub database key
kind (str) – serialization/deserialization type
dupsort (bool) – True means enable duplicates at each key False (default) means do not enable duplicates at each key
sep (str) – separator to convert keys iterator to key bytes for db key default is self.Sep == ‘.’

getFullItemIter(keys: str | bytes | memoryview | Iterable = b'', *, topive=False)[source]

Iterator over items in top branch of db that returns full items with subclass specific special hidden parts shown for debugging or testing.

Returns:

of (key, val) tuples over the all the items in subdb whose key startswith key made from keys. Keys may be keyspace prefix to return branches of key space. When keys is empty then returns all items in subdb

Return type:

items (Iterator)

Parameters:

keys (str|bytes|memoryview|Iterable) – may be a truncation of a full keys tuple in in order to get all the items from multiple branches of the key space. If keys is empty then gets all items in database.
topive (bool) – True means treat as partial key tuple from top branch of key space given by partial keys. Resultant key ends in .sep character. False means treat as full branch in key space. Resultant key does not end in .sep character. When last item in keys is empty str then will treat as partial ending in sep regardless of top value

getTopItemIter(keys: str | bytes | memoryview | Iterable = b'', *, topive=False)[source]

Iterator over items in top branch of db given by keys. Subclasses that do special hidden transforms on either the keyspace or valuespace should override this method to hide hidden parts from the returned items.

For example, adding either a hidden key space suffix or hidden val space proem to ensure insertion order.

To return full items with hidden parts shown for debugging or testing, use getFullItemIter instead.

Returns:

of (key, val) tuples over the all the items in subdb whose key startswith key made from keys. Keys may be keyspace prefix to return branches of key space. When keys is empty then returns all items in subdb

Return type:

items (Iterator)

Parameters:

keys (str|bytes|memoryview|Iterable) – tuple of bytes or strs that may be a truncation of a full keys tuple in in order to get all the items from multiple branches of the key space. If keys is empty then gets all items in database.
topive (bool) – True means treat as partial key tuple from top branch of key space given by partial keys. Resultant key ends in .sep character. False means treat as full branch in key space. Resultant key does not end in .sep character. When last item in keys is empty str then will treat as partial ending in sep regardless of top value

remTop(keys: str | bytes | memoryview | Iterable = b'', *, topive=False)

Removes all entries whose keys startswith keys. Enables removal of whole branches of db key space. To ensure that proper separation of a branch include empty string as last key in keys. For example (“a”,””) deletes ‘a.1’and ‘a.2’ but not ‘ab’

Parameters:

keys (str|bytes|memoryview|Iterable) – of key space elements to be combined in order to form key
topive (bool) – True means treat as partial key tuple from top branch of key space given by partial keys. Resultant key ends in .sep character. False means treat as full branch in key space. Resultant key does not end in .sep character. When last item in keys is empty str then will treat as partial ending in sep regardless of top value

Returns:

True if key exists so delete successful. False otherwise

Return type:

result (bool)

trim(keys: str | bytes | memoryview | Iterable = b'', *, topive=False)[source]

Removes all entries whose keys startswith keys. Enables removal of whole branches of db key space. To ensure that proper separation of a branch include empty string as last key in keys. For example (“a”,””) deletes ‘a.1’and ‘a.2’ but not ‘ab’

Parameters:

keys (str|bytes|memoryview|Iterable) – of key space elements to be combined in order to form key
topive (bool) – True means treat as partial key tuple from top branch of key space given by partial keys. Resultant key ends in .sep character. False means treat as full branch in key space. Resultant key does not end in .sep character. When last item in keys is empty str then will treat as partial ending in sep regardless of top value

Returns:

True if key exists so delete successful. False otherwise

Return type:

result (bool)

keri.db.migrations

keri.db.subing

KERI keri.db.subing module

Provide variety of mixin classes for LMDB sub-dbs with various behaviors. Takes of advantage of multiple inheritance to enable mixtures of behaviors with minimal code duplication (more DRY).

New style python classes use the C3 linearization algorithm. Multiple inheritance forms a directed acyclic graph called a diamond graph. This graph is linarized into the method resolution order. Use class.mro() or class.__mro__

(see https://www.geeksforgeeks.org/method-resolution-order-in-python-inheritance/) Basically: * children always precede their parents * immediate parent classes of a child are visited in the order listed in the child class statement. * a super class is visited only after all sub classes have been visited * linearized graph is monotonic (a class is only visted once)

Principally: SuberBase class provides trim, cnt, getTopItemIter, and getFullItemIter Suber subclass of SuberBase also provides, put, pin, get, and rem methods Suber is the simple class for managing a serialized value in a subdb with a set of keys as a tuple (iterable) that is converted to a .sep delimited key that defines the key space.

CesrSuber class extends Suber for values that are serializations of CESR serializable object instances. Ducktyped subclasses of Matter, Indexer, and Counter or the like.

IoSetSuber class extends Suber to allow a set of values to be stored in insertion order at each effective key. Only one copy of a unique value is allowed in the set at a given effective key. The effective key suffixes an ordinal to the key space to track insertion ordering. IoSetSuber adds additional methods to manage IoSets of values.

CatCesrSuber adds the ability to store multiple concatenated serializations at a value

CatCesrIoSetSuber combines the capabilities

Other special classer for special values

SerderSuber stores Serialized Serder Instances of in JSON, CBOR, or MGPK

Also for Secrets private keys SignerSuber CryptSignerSuber

Also for using the dupsort==true mechanism is DupSuber CesrDupSuber

Class Architecture

Suber is simple lexographic database with only one value per key OnSuber is simple lexographic database where trailing part of key is serialized

ordinal number so that the ordering within each key prefix is monotonically increasing numeric

B64Suber provides separated fields of B64 primitives for values. Useful when don’t: need to CESR ser/des the primitives or performance

The term ‘set’ of values means that no value may appear more than once in the set. Sets support idempotent adds and puts to db. This means one can add or put the same (key, val) pair multiple times and not change the db.

DupSuber provides set of lexicographic ordered values at each key. Each value has: a limited size (key + value <= 511 byes). The set is performant. Good for indices.
IoDupSuber provides set of insertion ordered values at each key. Each value has: a limited size (key + value <= 511 byes). The set is less perfromant than DupSuber but more performant than IoSetSuber. Good for insertion ordered indices
IoSetSuber proves set of insertion ordered values at each key. Value size is not limited: Good for any insertion ordered set where size may be too large for IoDupSuber
OnIoDupSuber provides set of insertion ordered values where the where trailing: part of key is serialized ordinal number so that the ordering within each key prefix is monotonically increasing numeric. Useful to provide omndices for sn ordering of superseding KEL events.

Each of these base types for managing the key space may be mixed with other Classes that provide different types of values these include.

Cesr CatCesr Serder etc.

class keri.db.subing.B64IoDupSuber(*pa, **kwa)[source]

Subclass of B64SuberBase and IoDupSuber that serializes and deserializes values as .sep joined strings of Base64 components in insertion ordered duplicates with leading value proem. Proem + .sep joined value and must fit in 511 bytes of keyspace as duplicate

Assumes dupsort==True

__init__(*pa, **kwa)[source]

Inherited Parameters:

db (LMDBer): base db subkey (str): LMDB sub database key dupsort (bool): True means enable duplicates at each key

False (default) means do not enable duplicates at each key. Set to True

sep (str): separator for combining keys tuple of strs into key bytes: for db key and also used to convert val iterator to val bytes Must not be Base64 character. default is self.Sep == ‘.’
verify (bool): True means reverify when ._des from db when applicable: False means do not reverify. Default False

class keri.db.subing.B64IoSetSuber(*pa, **kwa)[source]

Subclass of B64SuberBase and IoSetSuber that serializes and deserializes values as .sep joined strings of Base64 components in insertion order using hidden ion suffix in keyspace. Using IoSet removes 511 byte limitation of duplicates (dupsort=True).

Assumes dupsort==False

__init__(*pa, **kwa)[source]

Inherited Parameters:

db (LMDBer): base db subkey (str): LMDB sub database key dupsort (bool): True means enable duplicates at each key

False (default) means do not enable duplicates at each key. Set to False

sep (str): separator for combining keys tuple of strs into key bytes: for db key and also used to convert val iterator to val bytes Must not be Base64 character. default is self.Sep == ‘.’
verify (bool): True means reverify when ._des from db when applicable: False means do not reverify. Default False

class keri.db.subing.B64OnIoDupSuber(*pa, **kwa)[source]

Subclass of B64SuberBase and OnIoDupSuber that serializes and deserializes values as .sep joined strings of Base64 components in insertion ordered duplicates with leading value proem but where the trailing part of the key space is a serialized monotonically increasing ordinal number. Proem + .sep joined value and must fit n 511 bytes of keyspace as duplicate.

Insertion order is maintained by automagically prepending and stripping an ordinal ordering proem to/from each duplicate value at a given key.

OnIoDupSuber adds the convenience methods from OnSuberBase to OnIoDupSuber for those cases where the keyspace has a trailing ordinal part.

Assumes dupsort==True

__init__(*pa, **kwa)[source]

Inherited Parameters:

db (LMDBer): base db subkey (str): LMDB sub database key dupsort (bool): True means enable duplicates at each key

False (default) means do not enable duplicates at each key. Set to True

sep (str): separator for combining keys tuple of strs into key bytes: for db key and also used to convert val iterator to val bytes Must not be Base64 character. default is self.Sep == ‘.’
verify (bool): True means reverify when ._des from db when applicable: False means do not reverify. Default False

class keri.db.subing.B64OnIoSetSuber(*pa, **kwa)[source]

Subclass of B64SuberBase and OnIoSetSuber that serializes and deserializes values as .sep joined strings of Base64 components in insertion order using hidden ion suffix in keyspace. Using IoSet removes 511 byte limitation of duplicates (dupsort=True). The last keyspece element before the hidden insertion ordering suffix is an ordinal (hence On). This allows entries at a key prefix to be stored monotonically as per sequence numbering.

Insertion order within each set is maintained by automagically suffixing and unsuffixing the insertion ordering suffix.

OnIoSetSuber adds the convenience methods from OnSuberBase to OnIoSetSuber for those cases where the keyspace has a trailing ordinal part.

Assumes dupsort==False

__init__(*pa, **kwa)[source]

Inherited Parameters:

db (LMDBer): base db subkey (str): LMDB sub database key dupsort (bool): True means enable duplicates at each key

False (default) means do not enable duplicates at each key. Set to False

sep (str): separator for combining keys tuple of strs into key bytes: for db key and also used to convert val iterator to val bytes Must not be Base64 character. default is self.Sep == ‘.’
verify (bool): True means reverify when ._des from db when applicable: False means do not reverify. Default False

class keri.db.subing.B64Suber(*pa, **kwa)[source]

Subclass of B64SuberBase and Suber that serializes and deserializes values as .sep joined strings of Base64 components.

.sep must not be Base64 character.

Assumes dupsort==False

__init__(*pa, **kwa)[source]

Inherited Parameters:

db (LMDBer): base db subkey (str): LMDB sub database key dupsort (bool): True means enable duplicates at each key

False (default) means do not enable duplicates at each key. Set to False

sep (str): separator to convert keys iterator to key bytes for db key: default is self.Sep == ‘.’ also used to convert val iterator to val bytes Must not be Base64 character.
verify (bool): True means reverify when ._des from db when applicable: False means do not reverify. Default False

class keri.db.subing.B64SuberBase(*pa, **kwa)[source]

Base Class whose values are Iterables of Base64 str or bytes that are stored in db as .sep joined Base64 bytes. Separator character must not be valid Base64 character so the split will work unambiguously.

Automatically joins and splits along separator to Iterable (tuple) of Base64

Attributes:
db (LMDBer): base LMDB db sdb (lmdb._Database): instance of lmdb named sub db for this Suber sep (str): separator for combining keys tuple of strs into key bytes

for db key and also used to convert val iterator to val bytes Must not be Base64 character. default is self.Sep == ‘.’

__init__(*pa, **kwa)[source]

Inherited Parameters:

db (LMDBer): base db subkey (str): LMDB sub database key dupsort (bool): True means enable duplicates at each key

False (default) means do not enable duplicates at each key

sep (str): separator to convert keys iterator to key bytes for db key: default is self.Sep == ‘.’ Must not be Base64 character.
verify (bool): True means reverify when ._des from db when applicable: False means do not reverify. Default False

class keri.db.subing.CatCesrDupSuber(*pa, **kwa)[source]

Sub class of DupSuber whose values are CESR ducktypes of Matter subclasses. serialized to and deserialied from val instance .qb64b property which is a fully qualified serialization. Automatically serializes and deserializes from qb64b to/from Matter ducktyped instances DupSuber supports multiple entries at each key (duplicates) with dupsort==True

Do not use if serialized value is greater than 511 bytes. This is a limitation of dupsort==True sub dbs in LMDB

__init__(*pa, **kwa)[source]

Initialize Instance

Inherited Parameters:

db (LMDBer): base db subkey (str): LMDB sub database key dupsort (bool): True (forced default) means enable duplicates at each key

False means do not enable duplicates at each key

sep (str): separator to convert keys iterator to key bytes for db key: default is self.Sep == ‘.’
verify (bool): True means reverify when ._des from db when applicable: False means do not reverify. Default False
klas (Iterable|Type[coring.Matter]|None): of Class references to: subclasses of CESR compatible Type[coring.Matter etc]Class reference to subclass of Matter or Indexer or Counter or any ducktyped class of Matter None is replaced with default Matter

class keri.db.subing.CatCesrIoSetSuber(*pa, **kwa)[source]

Sub class of CatSuberBase and IoSetSuber where values stored in db are a concatenation of .qb64b property from one or more Cesr compatible subclass instances that automatically serializes and deserializes to/from qb64b . (qb64b is bytes of fully qualified serialization).

Extends IoSetSuber with mixin methods ._ser and ._des from CatSuberBase so that all IoSetSuber methods now work with an Iterable of CESR subclass for each val.

IoSetSuber stores at each effective key a set of distinct values that share that same effective key where each member of the set is retrieved in insertion order (dupsort==False) The methods allows an Iterable (set valued) of Iterables of Matter subclass instances to be stored at a given effective key in insertion order.

Actual keys include a hidden ordinal key suffix that tracks insertion order. The suffix is appended and stripped transparently from the keys. The set of items with duplicate effective keys are retrieved in insertion order when iterating or as a list of the set elements. The actual iokey for any item includes the ordinal suffix.

db

base LMDB db

Type:: LMDBer

sdb

instance of lmdb named sub db for this Suber

Type:: lmdb._Database

sep

separator for combining keys tuple of strs into key bytes

Type:: str

klas

of Class references to subclasses of Matter, each of to Type[coring.Matter]

Type:: Iterable

__init__(*pa, **kwa)[source]

Inherited Parameters:

db (LMDBer): base db subkey (str): LMDB sub database key dupsort (bool): True means enable duplicates at each key

False (default) means do not enable duplicates at each key

sep (str): separator to convert keys iterator to key bytes for db key: default is self.Sep == ‘.’
verify (bool): True means reverify when ._des from db when applicable: False means do not reverify. Default False
klas (Iterable|Type[coring.Matter]|None): of Class references to: subclasses of CESR compatible Type[coring.Matter etc]Class reference to subclass of Matter or Indexer or Counter or any ducktyped class of Matter None is replaced with default Matter

class keri.db.subing.CatCesrSuber(*pa, **kwa)[source]

Class whose values stored in db are a concatenation of the .qb64b property from one or more subclass instances (qb64b is bytes of fully qualified serialization) that support CESR encode/decode ducktyped subclass instance such as Matter, Indexer, Counter Automatically serializes and deserializes from qb64b to/from CESR instances

db

base LMDB db

Type:: LMDBer

sdb

instance of lmdb named sub db for this Suber

Type:: lmdb._Database

sep

separator for combining keys tuple of strs into key bytes

Type:: str

klas

of Class references to subclasses of CESR compatible , each of to Type[coring.Matter etc]

Type:: Iterable

__init__(*pa, **kwa)[source]

Inherited Parameters:

db (LMDBer): base db subkey (str): LMDB sub database key dupsort (bool): True means enable duplicates at each key

False (default) means do not enable duplicates at each key

sep (str): separator to convert keys iterator to key bytes for db key: default is self.Sep == ‘.’
verify (bool): True means reverify when ._des from db when applicable: False means do not reverify. Default False
klas (Iterable|Type[coring.Matter]|None): of Class references to: subclasses of CESR compatible Type[coring.Matter etc]Class reference to subclass of Matter or Indexer or Counter or any ducktyped class of Matter None is replaced with default Matter

class keri.db.subing.CatCesrSuberBase(*pa, klas: Iterable | Type[coring.Matter] | None = None, strict: bool = False, **kwa)[source]

Base Class whose values stored in db are a concatenation of the .qb64b property from one or more subclass instances (qb64b is bytes of fully qualified serialization) that support CESR encode/decode ducktyped subclass instance such as Matter, Indexer, Counter Automatically serializes and deserializes iterable of qb64b to/from CESR instances ._ser override .put .set input value to be instance that is serialized

Attributes:
db (LMDBer): base LMDB db sdb (lmdb._Database): instance of lmdb named sub db for this Suber sep (str): separator for combining keys tuple of strs into key bytes klas (Iterable): of Class references to subclasses of CESR compatible

, each of to Type[coring.Matter etc]

__init__(*pa, klas: Iterable | Type[coring.Matter] | None = None, strict: bool = False, **kwa)[source]

Inherited Parameters:

db (LMDBer): base db subkey (str): LMDB sub database key dupsort (bool): True means enable duplicates at each key

False (default) means do not enable duplicates at each key

sep (str): separator to convert keys iterator to key bytes for db key: default is self.Sep == ‘.’
verify (bool): True means reverify when ._des from db when applicable: False means do not reverify. Default False
klas (Iterable|Type[coring.Matter]|None): of Class references to: subclasses of CESR compatible Type[coring.Matter etc]Class reference to subclass of Matter or Indexer or Counter or any ducktyped class of Matter None is replaced with default Matter
strict (bool): True means enforce val in ._ser matches .klas: False means do not enforce. Default False

class keri.db.subing.CesrDupSuber(*pa, **kwa)[source]

Sub class of DupSuber whose values are CESR ducktypes of Matter subclasses. serialized to and deserialied from val instance .qb64b property which is a fully qualified serialization. Automatically serializes and deserializes from qb64b to/from Matter ducktyped instances DupSuber supports multiple entries at each key (duplicates) with dupsort==True

Do not use if serialized value is greater than 511 bytes. This is a limitation of dupsort==True sub dbs in LMDB

__init__(*pa, **kwa)[source]: Initialize Instance Inherited Parameters:

class keri.db.subing.CesrIoSetSuber(*pa, **kwa)[source]

Subclass of CesrSuber and IoSetSuber. Sub class of Suber where data is CESR encode/decode ducktyped subclass instance such as Matter, Indexer, Counter with .qb64b property when provided as fully qualified serialization Automatically serializes and deserializes from qb64b to/from CESR instances

Extends IoSetSuber with mixin methods ._ser and ._des from CesrSuberBase so that all IoSetSuber methods now work with CESR subclass for each val.

IoSetSuber stores at each effective key a set of distinct values that share that same effective key where each member of the set is retrieved in insertion order (dupsort==False) The methods allows an Iterable (set valued) of Iterables of Matter subclass instances to be stored at a given effective key in insertion order.

Actual keys include a hidden ordinal key suffix that tracks insertion order. The suffix is appended and stripped transparently from the keys. The set of items with duplicate effective keys are retrieved in insertion order when iterating or as a list of the set elements. The actual iokey for any item includes the ordinal suffix.

Attributes:
db (LMDBer): base LMDB db sdb (lmdb._Database): instance of lmdb named sub db for this Suber sep (str): separator for combining keys tuple of strs into key bytes klas (Iterable): of Class references to subclasses of CESR compatible

, each of to Type[coring.Matter etc]

__init__(*pa, **kwa)[source]

Inherited Parameters:

db (LMDBer): base db subkey (str): LMDB sub database key dupsort (bool): True means enable duplicates at each key

False (default) means do not enable duplicates at each key

sep (str): separator to convert keys iterator to key bytes for db key: default is self.Sep == ‘.’
verify (bool): True means reverify when ._des from db when applicable: False means do not reverify. Default False
klas (Type[coring.Matter]): Class reference to subclass of Matter or: Indexer or Counter or any ducktyped class of Matter

class keri.db.subing.CesrOnSuber(*pa, **kwa)[source]

Subclass of CesrSuberBase, OnSuberBase, and Suber that adds methods for keys with ordinal numbered tails and values that are Cesr serializations of Matter subclass ducktypes.

Each key consistes of pre joined with .sep to ordinal suffix

Assumes dupsort==False

__init__(*pa, **kwa)[source]

Inherited Parameters:

db (LMDBer): base db subkey (str): LMDB sub database key dupsort (bool): True means enable duplicates at each key

False (default) means do not enable duplicates at each key. Set to False

sep (str): separator to convert keys iterator to key bytes for db key: default is self.Sep == ‘.’
verify (bool): True means reverify when ._des from db when applicable: False means do not reverify. Default False

class keri.db.subing.CesrSuber(*pa, **kwa)[source]

Sub class of Suber where data is CESR encode/decode ducktyped subclass instance such as Matter, Indexer, Counter with .qb64b property when provided as fully qualified serialization. Extends Suber to support val that are ducktyped CESR serializable .qb64 .qb64b subclasses such as coring.Matter, coring.Indexer, coring.Counter. Automatically serializes and deserializes from qb64b to/from CESR instances

__init__(*pa, **kwa)[source]

Inherited Parameters:

db (LMDBer): base db subkey (str): LMDB sub database key dupsort (bool): True means enable duplicates at each key

False (default) means do not enable duplicates at each key

sep (str): separator to convert keys iterator to key bytes for db key: default is self.Sep == ‘.’
verify (bool): True means reverify when ._des from db when applicable: False means do not reverify. Default False
klas (Type[coring.Matter]): Class reference to subclass of Matter or: Indexer or Counter or any ducktyped class of Matter

class keri.db.subing.CesrSuberBase(*pa, klas: Type[coring.Matter] | None = None, strict: bool = False, **kwa)[source]

Sub class of SuberBase where data is CESR encode/decode ducktyped subclass instance such as Matter, Indexer, Counter with .qb64b property when provided as fully qualified serialization Automatically serializes and deserializes from qb64b to/from CESR instance ._ser override .put .set input value to be instance that is serialized

__init__(*pa, klas: Type[coring.Matter] | None = None, strict: bool = False, **kwa)[source]

Inherited Parameters:

db (LMDBer): base db subkey (str): LMDB sub database key dupsort (bool): True means enable duplicates at each key

False (default) means do not enable duplicates at each key

sep (str): separator to convert keys iterator to key bytes for db key: default is self.Sep == ‘.’
verify (bool): True means reverify when ._des from db when applicable: False means do not reverify. Default False

Parameters:

klas (Type[coring.Matter]) – Class reference to subclass of Matter or Indexer or Counter or any ducktyped class of Matter
strict (bool) – True means enforce val in ._ser matches .klas False means do not enforce. Default False

class keri.db.subing.CryptSignerSuber(*pa, klas: Type[signing.Signer] | None = None, **kwa)[source]

Sub class of SignerSuber where data is Signer subclass instance .qb64b property that has been encrypted if encrypter provided. which is a fully qualified serialization and uses the key which is the qb64b of the signer.verfer to get the transferable property of the verfer Automatically serializes and deserializes from qb64b to/from Signer instances

Assumes that last or only element of db key from keys for all entries is the qb64 of a public key for the associated Verfer instance. This allows returned Signer instance to have its .transferable property set correctly.

get(keys: str | Iterable, decrypter: signing.Decrypter = None)[source]

Gets Signer instance at keys. If decrypter then assumes value in db was encrypted and so decrypts value in db before converting to Signer.

Returns:

transferable determined by key which is verfer None if no entry at keys

Return type:

val (Signer)

Parameters:

keys (Union[str, iterable]) – key strs to be combined in order to form key. Last element of keys is verkey used to determin .transferable for Signer
decrypter (signing.Decrypter) – optional. If provided assumes value in db was encrypted and so decrypts before converting to Signer.

Usage:

Use walrus operator to catch and raise missing entry if (signer := mydb.get(keys)) is None:

raise ExceptionHere

use signer here

getTopItemIter(keys: str | bytes | memoryview | Iterable = '', decrypter: signing.Decrypter = None, *, topive=False)[source]

Iterates over all the items in top branch defined by keys where keys may be truncation of full branch.

Returns:: of tuples (key, val) over the all the items in subdb whose key startswith key made from keys. Keys may be keyspace prefix to return branches of key space. When keys is empty then returns all items in subdb
Return type:: items (Iterator)

decrypter (signing.Decrypter): optional. If provided assumes value in: db was encrypted and so decrypts before converting to Signer.

Parameters:

keys (Iterable) – tuple of bytes or strs that may be a truncation of a full keys tuple in in order to get all the items from multiple branches of the key space. If keys is empty then gets all items in database.
topive (bool) – True means treat as partial key tuple from top branch of key space given by partial keys. Resultant key ends in .sep character. False means treat as full branch in key space. Resultant key does not end in .sep character. When last item in keys is empty str then will treat as partial ending in sep regardless of top value

Uses python .startswith to match which always returns True if top is empty string so empty will matches all keys in db.

pin(keys: str | Iterable, val: coring.Matter, encrypter: signing.Encrypter = None)[source]

Pins (sets) qb64 of Matter instance val at key made from keys. Overwrites. If encrypter provided then encrypts first

Parameters:

keys (tuple) – of key strs to be combined in order to form key
val (Signer) – instance of self.klas
encrypter (signing.Encrypter) – optional

Returns:

True If successful. False otherwise.

Return type:

result (bool)

put(keys: str | Iterable, val: coring.Matter, encrypter: signing.Encrypter = None)[source]

Puts qb64 of Matter instance val at key made from keys. Does not overwrite If encrypter provided then encrypts first

Parameters:

keys (tuple) – of key strs to be combined in order to form key
val (Signer) – instance of self.klas
encrypter (signing.Encrypter) – optional

Returns:

True If successful, False otherwise, such as key: already in database.

Return type:

result (bool)

class keri.db.subing.DupSuber(db: Type[LMDBer], *, subkey: str = 'docs.', dupsort: bool = True, **kwa)[source]

Sub DB of LMDBer. Subclass of SuberBase that supports multiple entries at each key (duplicates) with dupsort==True

Do not use if serialized value is greater than 511 bytes. This is a limitation of dupsort==True sub dbs in LMDB

__init__(db: Type[LMDBer], *, subkey: str = 'docs.', dupsort: bool = True, **kwa)[source]

Parameters:

db (LMDBer) – base db
subkey (str) – LMDB sub database key
dupsort (bool) – True (forced default) means enable duplicates at each key False means do not enable duplicates at each key

Add val to vals at key made from keys. Does not overwrite. Adds to existing dup values at key if any. Duplicate means another entry at the same key but the entry is still a unique value. Duplicates are inserted in lexocographic order not insertion order. Lmdb does not insert a duplicate unless it is a unique value for that key.

Parameters:

keys (str | bytes | memoryview | Iterable) – of key strs to be combined in order to form key
val (str | bytes | memoryview) – value

Returns:

True means unique value among duplications,: False means duplicte of same value already exists.

Return type:

result (bool)

cnt(keys: str | bytes | memoryview | Iterable = '')[source]

Counts dup values at key made from keys, zero otherwise When keys is empty then counts all in db.

Parameters:: keys (str|bytes|memoryview|Iterable) – of key strs to be combined in order to form key. When keys empty then count all entires in db.

get(keys: str | bytes | memoryview | Iterable)[source]

Gets dup vals list at key made from keys

Parameters:

keys (str | bytes | memoryview | Iterable) – of key strs to be combined in order to form key

Returns:

each item in list is str: empty list if no entry at keys

Return type:

vals (list)

getIter(keys: str | bytes | memoryview | Iterable)[source]

Gets dup vals iterator at key made from keys

Duplicates are retrieved in lexocographic order not insertion order.

Parameters:: keys (str | bytes | memoryview | Iterable) – of key strs to be combined in order to form key
Returns:: vals each of str. Raises StopIteration when done
Return type:: iterator

getLast(keys: str | bytes | memoryview | Iterable)[source]

Gets last dup val at key made from keys

Parameters:: keys (tuple) – of key strs to be combined in order to form key
Returns:: value else None if no value at key
Return type:: val (str)

Pins (sets) vals at key made from keys. Overwrites. Removes all pre-existing dup vals and replaces them with vals

Parameters:

keys (str | bytes | memoryview | Iterable) – of key strs to be combined in order to form key
vals (str | bytes | memoryview | Iterable) – str or bytes values

Returns:

True If successful, False otherwise.

Return type:

result (bool)

Puts all vals at key made from keys. Does not overwrite. Adds to existing dup values at key if any. Duplicate means another entry at the same key but the entry is still a unique value. Duplicates are inserted in lexocographic order not insertion order. Lmdb does not insert a duplicate unless it is a unique value for that key.

Parameters:

keys (str | bytes | memoryview | Iterable) – of key strs to be combined in order to form key
vals (str | bytes | memoryview | Iterable) – str or bytes of each value to be written at key

Returns:

True If successful, False otherwise.

Return type:

result (bool)

Apparently always returns True (how .put works with dupsort=True)

Removes entry at keys

Parameters:

keys (tuple) – of key strs to be combined in order to form key
val (str|bytes|memoryview|None) – instance of dup val at key to delete if val is None, default, then remove all values at key

Returns:

True if key exists so delete successful. False otherwise

Return type:

result (bool)

class keri.db.subing.IoDupSuber(*pa, **kwa)[source]

Sub class of DupSuber that supports Insertion Ordering (IoDup) of duplicates By automagically prepending and stripping ordinal proem to/from each duplicate value at a given key.

IoDupSuber supports insertion ordered multiple entries at each key (duplicates) with dupsort==True

Do not use if serialized length key + proem + value, is greater than 511 bytes. This is a limitation of dupsort==True sub dbs in LMDB

IoDupSuber may be more performant then IoSetSuber for values that are indices to other sub dbs that fit the size constraint because LMDB support for duplicates is more space efficient and code performant.

Duplicates at a given key preserve insertion order of duplicate. Because lmdb is lexocographic an insertion ordering proem is prepended to all values that makes lexocographic order that same as insertion order.

Duplicates are ordered as a pair of key plus value so prepending proem to each value changes duplicate ordering. Proem is 33 characters long. With 32 character hex string followed by ‘.’ for essentiall unlimited number of values which will be limited by memory.

With prepended proem ordinal must explicity check for duplicate values before insertion. Uses a python set for the duplicate inclusion test. Set inclusion scales with O(1) whereas list inclusion scales with O(n).

__init__(*pa, **kwa)[source]: Inherited Parameters:

Add val idempotently at key made from keys in insertion order using hidden ordinal proem. Idempotently means do not add val that is already in dup vals at key. Does not overwrite.

Parameters:

keys (Iterable) – of key strs to be combined in order to form key
val (str | bytes | memoryview) – serialization

Returns:

True means unique value added among duplications,: False means duplicate of same value already exists.

Return type:

result (bool)

cnt(keys: str | bytes | memoryview | Iterable = '')[source]

Counts dup values at key made from keys with hidden ordinal proem. Zero otherwise. When keys empty then counts All entries in db not just those at a given key

Parameters:: keys (str|bytes|memoryview|Iterable) – of key parts to be combined in order to form key. When empty counts all entries in db.

get(keys: str | bytes | memoryview | Iterable)[source]

Gets vals dup list in insertion order using key made from keys and hidden ordinal proem on dups.

Parameters:

keys (Iterable) – of key strs to be combined in order to form key

Returns:

each item in list is str: empty list if no entry at keys

Return type:

vals (Iterable)

getItemIter(keys: str | bytes | memoryview | Iterable, *, ion=0)[source]

Gets vals dup iterator in insertion order using key made from keys and hidden ordinal proem on dups. All vals in dups that share same key are retrieved in insertion order.

Parameters:

keys (str | bytes | memoryview | Iterable) – of key parts
ion (int) – offset into set to start the count (0 based offset)

Returns:

entries in dups at key for ion >= ion.

Return type:

items (Iterator[str])

Raises StopIteration when done

getIter(keys: str | bytes | memoryview | Iterable, *, ion=0)[source]

Gets vals dup iterator in insertion order using key made from keys and hidden ordinal proem on dups. All vals in dups that share same key are retrieved in insertion order.

Parameters:

keys (str | bytes | memoryview | Iterable) – of key parts
ion (int) – offset into set to start the count (0 based offset)

Returns:

str values. Raises StopIteration when done

Return type:

vals (Iterator)

getLast(keys: str | bytes | memoryview | Iterable)[source]

Gets last val inserted at key made from keys in insertion order using hidden ordinal proem.

Parameters:: keys (Iterable) – of key strs to be combined in order to form key
Returns:: value str, None if no entry at keys
Return type:: val (str)

getTopItemIter(keys: str | bytes | memoryview | Iterable = '', *, topive=False)[source]

Iterates over all the items in top branch defined by keys where keys may be truncation of full branch.

Transparently prepends and strips insertion ordering proem from value

Return iterator over all the items including dup items for all keys in top branch defined by keys where keys may be truncation of full branch.

Returns:

of (key, val) tuples over the all the items in subdb whose key startswith key made from keys and val has its hidden dup ordinal proem removed. Keys may be keyspace prefix in order to return branches of key space. When keys is empty then returns all items in subdb.

Return type:

items (Iterator)

Parameters:

keys (str | bytes | memoryview | Iterable) – key or key parts that may be a truncation of a full keys tuple in in order to address all the items from multiple branches of the key space. If keys is empty then gets all items in database. Either append “” to end of keys Iterable to ensure get properly separated top branch key or use top=True. In Python str.startswith(‘’) always returns True so if branch key is empty string it matches all keys in db with startswith.
topive (bool) – True means treat as partial key tuple from top branch of key space given by partial keys. Resultant key ends in .sep character. False means treat as full branch in key space. Resultant key does not end in .sep character. When last item in keys is empty str then will treat as partial ending in sep regardless of top value

Uses python .startswith to match which always returns True if top is empty string so empty will matches all keys in db.

Pins (sets) vals at key made from keys in insertion order using hidden ordinal proem. Overwrites. Removes all pre-existing vals that share same keys and replaces them with vals

Parameters:

keys (Iterable) – of key strs to be combined in order to form key
vals (Iterable) – str serializations

Returns:

True If successful, False otherwise.

Return type:

result (bool)

Puts all vals idempotently at key made from keys in insertion order using hidden ordinal proem. Idempotently means do not put any val in vals that is already in dup vals at key. Does not overwrite.

Parameters:

keys (Iterable) – of key strs to be combined in order to form key
vals (Iterable) – of str serializations

Returns:

True If successful, False otherwise.

Return type:

result (bool)

Removes entry at key made from keys and dup val that matches val if any, notwithstanding hidden ordinal proem. If val is None, default removes all dup values at key if any.

Parameters:

keys (str | bytes | memoryview | Iterable) – of key parts to be combined in order to form key
val (str|bytes|memoryview|None) – value at key to delete. Subclass ._ser method may accept different value types if val is None then remove all values at key

Returns:

True if key with dup val exists so rem successful.: False otherwise

Return type:

result (bool)

class keri.db.subing.IoSetSuber(db: LMDBer, *, subkey: str = 'docs.', dupsort: bool = False, **kwa)[source]

Insertion Ordered Set Suber factory class that supports a set of distinct entries at a given effective database key but with dupsort==False. Effective data model is that there are multiple values in a set of values where every member of the set has the same key (duplicate key). The set of values is an ordered set using insertion order. Any given value may appear only once in the set (not a list).

This works similarly to the IO value duplicates for the LMDBer class with a sub db of LMDB (dupsort==True) but without its size limitation of 511 bytes for each value when dupsort==True. Here the key is augmented with a hidden numbered suffix that provides a an ordered set of values at each effective key (duplicate key). The suffix is appended and stripped transparently. The set of multiple items with duplicate keys are retrieved in insertion order when iterating or as a list of the set elements.

db

base LMDB db

Type:: LMDBer

sdb

instance of lmdb named sub db for this Suber

Type:: lmdb._Database

sep

separator for combining keys tuple of strs into key bytes

Type:: str

__init__(db: LMDBer, *, subkey: str = 'docs.', dupsort: bool = False, **kwa)[source]

Initialize instance

Inherited Parameters:

db (LMDBer): base db subkey (str): LMDB sub database key dupsort (bool): True means enable duplicates at each key

False (default) means do not enable duplicates at each key

sep (str): separator to convert keys iterator to key bytes for db key: default is self.Sep == ‘.’
verify (bool): True means reverify when ._des from db when applicable: False means do not reverify. Default False
klas (Type[coring.Matter]): Class reference to subclass of Matter or: Indexer or Counter or any ducktyped class of Matter

Add val idempotently to vals at effective key made from keys and hidden ordinal suffix. Idempotent means that added value is not already in set of vals at key. Does not overwrite or add same value at same key more than once. When val None returns False

Parameters:

keys (str|bytes|memoryview|Iterable) – of key parts to be combined in order to form key
val (str|bytes|memoryview|None) – value to add When val None returns False

Returns:

True means unique value added among duplications,: False means duplicate of same value already exists.

Return type:

result (bool)

cnt(keys: str | bytes | memoryview | Iterable = '', *, ion=0)[source]

Counts entries at effective key made from keys and hidden ordinal suffix. Zero otherwise. When keys empty then counts all in db.

Returns:

entries in set at keys starting with ion >= ion.: When keys is empty then counts all entries in db.

Return type:

cnt (int)

Parameters:

keys (str|bytes|memoryview|Iterable) – of key strs to be combined in order to form key
ion (int) – offset into set to start the count (0 based offset)

get(keys: str | bytes | memoryview | Iterable, *, ion=0)[source]

Gets vals set list at effective key made from keys and hidden ordinal suffix ion starting at ion >= ion. When keys is empty or missing then returns empty list

All vals in set of vals that share same effecive key are retrieved in insertion order starting at ion.

Parameters:

keys (str|bytes|memoryview|Iterable) – of key strs to be combined in order to form key
ion (int) – offset into set to start the count (0 based offset)

Returns:

each item in list is str: empty list if no entry at keys

Return type:

vals (list[str])

getItem(keys: str | bytes | memoryview | Iterable, *, ion=0)[source]

Gets item list in set at effective key made from keys and hidden ordinal suffix ion starting at ion >= ion. When keys is empty or missing then returns empty list

All vals in set of vals that share same effecive key are retrieved in insertion order starting at ion.

Parameters:

keys (str|bytes|memoryview|Iterable) – of key strs to be combined in order to form key
ion (int) – offset into set to start the count (0 based offset)

Returns:

each item in list is str: empty list if no entry at keys

Return type:

vals (list[str])

getItemIter(keys: str | bytes | memoryview | Iterable, *, ion=0)[source]

Iterates over set items at effecive key made from keys and hidden ordinal suffix ion starting at ion >= ion. When keys is empty or missing then returns empty iterator

All vals in set of vals that share same effecive key are retrieved in insertion order starting at ion.

Parameters:

keys (str|bytes|memoryview|Iterable) – of key strs to be combined in order to form key
ion (int) – offset into set to start the count (0 based offset)

Returns:

entries in set at key for ion >= ion.

Return type:

items (Iterator[str])

Raises StopIteration when done

getIter(keys: str | bytes | memoryview | Iterable, *, ion=0)[source]

Iterates over set values at effecive key made from keys and hidden ordinal suffix ion starting at ion >= ion. When keys is empty or missing then returns empty iterator

All vals in set of vals that share same effecive key are retrieved in insertion order starting at ion.

Parameters:

keys (str|bytes|memoryview|Iterable) – of key strs to be combined in order to form key
ion (int) – offset into set to start the count (0 based offset)

Returns:

str values. Raises StopIteration when done

Return type:

vals (Iterator[str])

getLast(keys: str | bytes | memoryview | Iterable)[source]

Gets last set val inserted at effecive key where effective key is made from keys and hidden ordinal suffix.

All vals in the set of vals that shares the same effecive key are retrieved in insertion order.

Parameters:: keys (str|bytes|memoryview|Iterable) – of key strs to be combined in order to form key
Returns:: value str, None if no entry at keys
Return type:: val (str)

getLastItem(keys: str | bytes | memoryview | Iterable)[source]

Gets last set item (key, val) inserted at effecive key where effective key is made from keys and hidden ordinal suffix when keys is empty or missing returns empty tuple.

All items in the set of items that shares the same effecive key are retrieved in insertion order.

Parameters:

keys (str|bytes|memoryview|Iterable) – of key strs to be combined in order to form key

Returns:

(key, val) tuple or empty tuple if no entry: at keys or keys is empty

Return type:

last ((str, str)|None)

getLastItemIter(keys: str | bytes | memoryview | Iterable = '')[source]

Iterates over last item inserted in each set starting at key made from keys for all keys in db where key >= key.

Each effective key is made from keys and hidden ordinal suffix.

All vals in the set of vals that shares the same effecive key are retrieved in insertion order.

raises StopIterationError when done

Parameters:: keys (str|bytes|memoryview|Iterable) – of key strs to be combined in order to form key. Iterates over all keys when key empty.
Returns:: (key, val) tuples
Return type:: items (Iterator[(str, str)])

getLastIter(keys: str | bytes | memoryview | Iterable = '')[source]

Iterates over last val inserted in each set starting at key made from keys for all keys in db where key >= key.

Each effective key is made from keys and hidden ordinal suffix.

All vals in the set of vals that shares the same effecive key are retrieved in insertion order.

raises StopIterationError when done

Parameters:: keys (str|bytes|memoryview|Iterable) – of key strs to be combined in order to form key. Iterates over all keys when key empty.
Returns:: value str
Return type:: vals (Iterator[str])

getTopItemIter(keys: str | bytes | memoryview | Iterable = '', *, topive=False)[source]

Iterates over all the items in top branch defined by keys where keys may be truncation of full branch.

Transparently suffix and unsuffix insertion ordering ordinal

raises StopIterationError when done

Returns:

of (key, val) tuples over the all the items in: subdb whose effective key startswith key made from keys. Keys may be keyspace prefix in order to return branches of key space. When keys is empty then returns all items in subdb. Returned key in each item has ordinal suffix removed.

Return type:

items (Iterator)

Parameters:

keys (Iterable) – tuple of bytes or strs that may be a truncation of a full keys tuple in in order to address all the items from multiple branches of the key space. If keys is empty then gets all items in database. Either append “” to end of keys Iterable to ensure get properly separated top branch key or use top=True. In Python str.startswith(‘’) always returns True so if branch key is empty string it matches all keys in db with startswith.
topive (bool) – True means treat as partial key tuple from top branch of key space given by partial keys. Resultant key ends in .sep character. False means treat as full branch in key space. Resultant key does not end in .sep character. When last item in keys is empty str then will treat as partial ending in sep regardless of top value

Uses python .startswith to match which always returns True if top is empty string so empty will matches all keys in db.

Pins (sets) vals at effective key made from keys and hidden ordinal suffix. Overwrites. Removes all pre-existing vals that share same effective keys and replaces them with vals

Parameters:

keys (str|bytes|memoryview|Iterable) – key(s) made into base key
vals (str|bytes|memoryview|Iterable|None) – serialized value to replace Value at onkey. None means empty iterable. Empty iterable or None returns False

Returns:

True If successful, False otherwise.

Return type:

result (bool)

Puts all vals at effective key made from keys and hidden ordinal suffix. that are not already in set of vals at key. Does not overwrite.

Parameters:

keys (str|bytes|memoryview|Iterable) – key(s) made into base key
vals (str|bytes|memoryview|NonStrIterable|None) – serialized values to add to set of vals at onkey if any. When not NonStrIterable converts to iterable. Empty iterable or None returns False

Returns:

True If successful, False otherwise.

Return type:

result (bool)

Removes entry at effective key made from keys and hidden ordinal suffix that matches val if any. When val is None, default, then removes all entries at keys

Parameters:

keys (str|bytes|memoryview|Iterable) – of key strs to be combined in order to form key
val (str|bytes|memoryview|None) – value at key to delete. Subclass ._ser method may accept different value types if val is None then remove all values at key

Returns:

True if effective key with val exists so rem successful.: False otherwise

Return type:

result (bool)

class keri.db.subing.OnIoDupSuber(*pa, **kwa)[source]

Sub class of IoDupSuber and OnSuberBase that supports Insertion Ordering (IoDup) of duplicates but where the trailing part of the key space is a serialized monotonically increasing ordinal number. This is useful for escrows of key events which are ordinally numbered such as sn but where duplicates of likely events are also maintained in insertion order.

Insertion order is maintained by automagically prepending and stripping an ordinal ordering proem to/from each duplicate value at a given key.

OnIoDupSuber adds the convenience methods from OnSuberBase to IoDupSuber for those cases where the keyspace has a trailing ordinal part.

There are two ordinals, one in the key space and a hidden one in the duplicate data value space.

OnIoDupSuber supports insertion ordered multiple entries at each key (duplicates) with dupsort==True

Do not use if serialized length key + proem + value, is greater than 511 bytes. This is a limitation of dupsort==True sub dbs in LMDB

OnIoDupSuber may be more performant then IoSetSuber for values that are indices to other sub dbs that fit the size constraint because LMDB support for duplicates is more space efficient and code performant.

Duplicates at a given key preserve insertion order of duplicate. Because lmdb is lexocographic an insertion ordering proem is prepended to all values that makes lexocographic order that same as insertion order.

Duplicates are ordered as a pair of key plus value so prepending proem to each value changes duplicate ordering. Proem is 33 characters long. With 32 character hex string followed by ‘.’ for essentiall unlimited number of values which will be limited by memory.

With prepended proem ordinal must explicity check for duplicate values before insertion. Uses a python set for the duplicate inclusion test. Set inclusion scales with O(1) whereas list inclusion scales with O(n).

__init__(*pa, **kwa)[source]: Inherited Parameters:

Add val idempotently at key made from keys with on suffix in insertion order using hidden ordinal proem. Idempotently means do not add any val that is already in dup vals at key. Does not overwrite.

Parameters:

keys (str | bytes | memoryview | Iterable) – top keys as prefix to be combined with serialized on suffix and sep to form onkey
on (int) – ordinal number used with onKey(pre,on) to form onkey.
val (str | bytes | memoryview) – serialization

Returns:

True means unique value added among duplications,: False means duplicate of same value already exists.

Return type:

result (bool)

append(keys: str | bytes | memoryview, val: str | bytes | memoryview)[source]

Returns:

ordinal number of newly appended val

Return type:

on (int)

Parameters:

keys (str | bytes | memoryview | Iterable) – top keys as prefix to be combined with serialized on suffix and sep to form key
val (str | bytes | memoryview) – serialization

cnt(keys: str | bytes | memoryview | Iterable = '', on: int = 0)[source]

Counts iodup values at onkey made from keys and on. When keys is empty then counts whole db.

Return count of dup values at key made from keys with hidden ordinal proem. Zero otherwise

Parameters:

keys (str | bytes | memoryview | Iterable) – of key parts to be combined in order to form key
on (int) – ordinal number used with onKey(pre,on) to form key.

get(keys: str | bytes | memoryview | Iterable, on: int = 0)[source]

Gets dup vals list at key made from keys

Parameters:

keys (str|bytes|memoryview|Iterable) – of key strs to be combined in order to form key
on (int) – ordinal number used with onKey(pre,on) to form key.

Returns:

values if any else empty tuuple

Return type:

vals (list[str])

getAllItemIter(keys: str | bytes | memoryview | Iterable = '', on: int = 0)[source]

Iterates over all items of each dup set for all on >= on for key. When on == 0, default, Iterates over alls items of each set for all on for key. When key is empty then iterates over all items in whole db

Items are triples of (keys, on, val)

Returns:

triples of (onkeys, on int,: deserialized val)

Return type:

items (Iterator[(top keys, on, val)])

Parameters:

keys (str | bytes | memoryview | iterator) – keys as prefix to be combined with serialized on suffix and sep to form onkey When keys is empty then retrieves whole database including duplicates
on (int) – ordinal number used with onKey(pre,on) to form key.
sep (bytes) – separator character for split

getAllIter(keys: str | bytes | memoryview | Iterable = '', on: int = 0)[source]

Iterates over all values of each dup set for all on >= on for key. When on == 0, default, Iterates over alls values of each dup set for all on for key. When key is empty then iterates over all items in whole db

Returns

vals (Iterator[bytes]): iterator of dup vals at each onkey but: increments of on >= on i.e. all key.on beginning with on

Parameters:

keys (str | bytes | memoryview | iterator) – keys as prefix to be combined with serialized on suffix and sep to form onkey When keys is empty then retrieves whole database including duplicates
on (int) – ordinal number used with onKey(pre,on) to form key.
sep (bytes) – separator character for split

getBackIter(keys: str | bytes | memoryview | Iterable = '', on: int = 0)[source]

Returns

val (Iterator[bytes]): deserialized val of of each: onkey in reverse order

Parameters:

keys (str | bytes | memoryview | iterator) – keys as prefix to be combined with serialized on suffix and sep to form onkey When keys is empty then retrieves whole database including duplicates
on (int) – ordinal number used with onKey(pre,on) to form key.
sep (bytes) – separator character for split

getItemBackIter(keys: str | bytes | memoryview | Iterable = '', on: int = 0)[source]

Returns:

triples of (onkeys, on int,: deserialized val) in reverse order

Return type:

items (Iterator[(top keys, on, val)])

Parameters:

keys (str | bytes | memoryview | iterator) – keys as prefix to be combined with serialized on suffix and sep to form onkey When keys is empty then retrieves whole database including duplicates
on (int) – ordinal number used with onKey(pre,on) to form key.
sep (bytes) – separator character for split

getItemIter(keys: str | bytes | memoryview | Iterable, on: int = 0, ion: int = 0)[source]

Iterates over dup items (key, on, val) at onkey made from keys and on in insertion order from offset ion >= ion into set using hidden ordinal proem. When effective onkey is empty or missing then returns empty iterator

Returns:

deserialized item elements of dups at onkey

Return type:

items (Iterator[str])

Parameters:

keys (str|bytes|memoryview|Iterable) – key(s) made into base key
on (int) – ordinal number tail used with onKey(pre,on) to form key.
ion (int) – starting insertion ordinal value, default 0

getIter(keys: str | bytes | memoryview | Iterable, on: int = 0, ion: int = 0)[source]

Iterates over dup vals at onkey made from keys and on in insertion order from offset ion >= ion into set using hidden ordinal proem. When effective onkey is empty or missing then returns empty iterator

Parameters:

keys (str|bytes|memoryview|Iterable) – key(s) made into base key
on (int) – ordinal number tail used with onKey(pre,on) to form key.
ion (int) – starting insertion ordinal value, default 0

Returns:

deserialized val of dups at onkey

Return type:

vals (Iterator[bytes])

getLast(keys: str | bytes | memoryview | Iterable, on: int = 0)[source]

Gets last val inserted at key made from keys in insertion order using hidden ordinal proem.

Parameters:

keys (Iterable) – of key strs to be combined in order to form key
on (int) – ordinal number used with onKey(pre,on) to form key.

Returns:

value str, None if no entry at effective key made from: keys and on

Return type:

last (str)

getLastItemIter(keys: str | bytes | memoryview | Iterable = '', on: int = 0)[source]

Returns

items (Iterator[(top keys, on, val)]): triples of (keys, on int,: deserialized val) last duplicate item as each onkey where onkey is the key+serialized on

Parameters:

keys (str | bytes | memoryview | iterator) – keys as prefix to be combined with serialized on suffix and sep to form key When keys is empty then retrieves whole database including duplicates
on (int) – ordinal number used with onKey(pre,on) to form key.
sep (bytes) – separator character for split

getLastIter(keys: str | bytes | memoryview | Iterable = '', on: int = 0)[source]

Returns

last (Iterator[bytes]): deserialized last duplicate val of of each: onkey

Parameters:

keys (str | bytes | memoryview | iterator) – top keys as prefix to be combined with serialized on suffix and sep to form key When keys is empty then retrieves whole database including duplicates
on (int) – ordinal number used with onKey(pre,on) to form key.
sep (bytes) – separator character for split

getTopItemIter(keys: str | bytes | memoryview | Iterable = '')[source]

Iterates over top branch of all insertion ordered dup values where each key startwith top. Assumes every effective key in db has trailing on element, onkey = key + sep + on, so can return on in item. Also assumes every effective key includes hiddion isertion ordinal ion suffix that is suffixed and unsuffixed transparently.

When top key is empty, gets all items in database.

Returns:

iterator of triples: (keys, on, val) where keys forms base key, on is int, and val is entry value at with insertion ordering suffix removed from effective key.

Return type:

items (Iterator[(tuple, int, str)])

Parameters:

keys (str|bytes|memoryview|Iterable) – keys as truncated top key, to get a key space prefix to get all the items from multiple branches of the key space. If top key is empty then gets all items in database.
on (int) – ordinal number used with onKey(pre,on) to form key.

Pins (sets) vals at key made from keys with on suffix in insertion order using hidden ordinal proem. Overwrites. Removes all pre-existing vals that share same keys and replaces them with vals

Parameters:

keys (Iterable) – of key strs to be combined in order to form key
on (int) – ordinal number used with onKey(pre,on) to form onkey.
vals (Iterable) – str serializations

Returns:

True If successful, False otherwise.

Return type:

result (bool)

Put all vals idempotently at key at key made from keys with on suffix in insertion order using hidden ordinal proem. Idempotently means do not put any val in vals that is already in dup vals at key. Does not overwrite.

Parameters:

keys (Iterable) – of key strs to be combined in order to form key
on (int) – ordinal number used with onKey(pre,on) to form onkey.
vals (Iterable) – of str serializations

Returns:

True If successful, False otherwise.

Return type:

result (bool)

Removes entry at key made from keys and dup val that matches val if any, notwithstanding hidden ordinal proem. Otherwise deletes all dup values at key if any.

Parameters:

keys (str | bytes | memoryview | iterator) – keys as prefix to be combined with serialized on suffix and sep to form onkey
on (int) – ordinal number used with onKey(pre,on) to form key.
val (str|bytes|memoryview|None) – value at key to delete. Subclass ._ser method may accept different value types if val is None then remove all values at key

Returns:

True if onkey with dup val exists so rem successful.: False otherwise

Return type:

result (bool)

class keri.db.subing.OnIoSetSuber(*pa, **kwa)[source]

Sub class of IoSetSuber and OnSuberBase that supports Insertion Ordered Set values (IoSet) with an exposed ordinal tail in each effective key.

To clarify, there are two ordinals, one exposed as the tail end of the effective key space followed by a hidden suffix for the set value space.

OnIoSetSuber adds the convenience methods from OnSuberBase to IoSetSuber for those cases where the keyspace has a trailing ordinal part.

Insertion order is maintained by automagically prepending and stripping an ordinal ordering suffix to/from each unique value at a given effective key.

An IoSet is a set of distinct entries at a given effective database key but with dupsort==False. Effective key means that there are multiple values in a set of values where every member of the set has the same key (duplicate key) but distinct values (a set not a list). The set of values is an ordered set using insertion order. Any given value may appear only once in the set (not a list).

This works similarly to the IO value duplicates for the LMDBer class with a sub db of LMDB (dupsort==True) but without its size limitation of 511 bytes.

Here the key is augmented with a hidden numbered suffix that provides a an insertion ordered set of values at each effective key (duplicate key). The suffix is appended and stripped transparently. The set of multiple items with duplicate keys are retrieved in insertion order when iterating or as a list of the set elements.

Combined with an On, ordinal numbered key space means that there are two ordinals at the tail (right) end of each key. The rightmost is the insertion ordering suffix ordinal. The next to the left is the exposed ordinal number tail used for external ordering. The insertion ordinal suffix is hidden. It is transparently added and stripped. To clarify, the exposed tail part of the key space is a serialized monotonically increasing ordinal number. Following that is a hidden insertion ordering ordinal suffix that defines the order within the set. Each memeber of a set has the same effective key including the exposed ordinal tail but a different hidden suffix. This is useful for escrows of key events which are ordinally numbered such as sn but where likely events with different overall value but same sn must be maintained in insertion order.

OnIoSetSuber may be less performant then IoDupSuber for values that are indices to other sub dbs that fit the size constraint because LMDB support for duplicates is more space efficient and code performant but has a 511 bytes maximum.

Duplicates at a given key preserve insertion order of duplicate. Because lmdb is lexocographic an insertion ordering proem is prepended to all values that makes lexocographic order that same as insertion order.

Duplicates are ordered as a pair of key plus value so prepending proem to each value changes duplicate ordering. Proem is 33 characters long. With 32 character hex string followed by ‘.’ for essentiall unlimited number of values which will be limited by memory.

To insure set semantics (i.e. no duplicate values in set) using a suffix ordinal insertions must explicity check for duplicate set values before insertion. A python set is used for the inclusion test. Set inclusion scales with O(1) whereas list inclusion scales with O(n).

__init__(*pa, **kwa)[source]

Initialize instance

Inherited Parameters:

Add val idempotently at key made from keys with exposed on tail in insertion order using hidden ordinal suffix. Idempotently means do not add any val that is already in set vals at effective key. Does not overwrite. When val None returns False

Returns:

True means unique value added to set,: False means value already in set.

Return type:

result (bool)

Parameters:

keys (str|bytes|memoryview|Iterable) – key(s) made into base key
on (int) – ordinal number tail used with onKey(pre,on) to form onkey.
val (str|bytes|memoryview|None) – value to add. when None returns False

Appends vals to next highest unused exposed ordinal tail and returns the ordinal. If vals None or empty raises ValueError.

Returns:

ordinal number tail of newly appended val

Return type:

on (int)

Parameters:

keys (str|bytes|memoryview|Iterable) – key(s) made into base key
vals (NonStrIterable|None) – values to append None means empty iterable. Empty iterable raises ValueError

cnt(keys: str | bytes | memoryview | Iterable = '', on: int = 0, ion: int = 0)[source]

Counts all entries in set at onkey = keys + sep + on starting at offset suffix ion into set.

Returns:

count values in set at effective onkey from insertion: ordering offset ion.

Return type:

count (int)

Parameters:

keys (str|bytes|memoryview|Iterable) – key(s) made into base key. When empty counts whole db.
on (int) – ordinal number used with onKey(pre,on) to form key.
ion (int) – starting insertion ordinal offset into set, default 0.

cntAll(keys: str | bytes | memoryview | Iterable = '', on: int = 0)[source]

Counts all set entries for all on >= on at key. When on == 0, default, then count all set members for all on for key When key is empty then count all on for all key i.e. whole db

Returns:

count of set members for onkey for on >= on. When on is: None then count of all on for key. When key is empty then count of all on for all key for whole db.

Return type:

count (int)

Parameters:

keys (str|bytes|memoryview|Iterable) – key(s) made into base key
on (int) – ordinal number used with onKey(pre,on) to form key.

get(keys: str | bytes | memoryview | Iterable, on: int = 0, ion: int = 0)[source]

Gets set vals list at onkey made from keys and on in insertion order from from offset ion into set using hidden ordinal suffix. When onkey is empty or missing then returns empty list

Returns:

values if any else empty tuuple

Return type:

vals (list[str])

Parameters:

keys (str|bytes|memoryview|Iterable) – key(s) made into base key When key is empty then returns empty iterator
on (int) – ordinal number tail used with onKey(pre,on) to form key.
ion (int) – starting insertion ordinal value, default 0

getAllBackIter(keys: str | bytes | memoryview | Iterable = '', on: int | None = None)[source]

Iterates backwards over all set values for all on <= on at key. When on is None iterates backwards over all set values for all on at key. When key empty then iterates backwards over whole db.

Backwards means decreasing numerical value of ion, for each on and decreasing numerical value on for each key and decreasing lexocographic order of each key.

Raises StopIterationError when done or when key empty or None

Returns: val (Iterator[str]): deserialized vals in backwards order

Parameters:

keys (str | bytes | memoryview | Iterable) – key(s) made into base key
on (int|None) – ordinal number used with onKey(pre,on) to form key.
sep (bytes) – separator character for split

getAllItemBackIter(keys: str | bytes | memoryview | Iterable = '', on: int | None = None)[source]

Iterates backwards over all set items for all on <= on at key. When on is None iterates backwards over all set items for all on at key. When key empty then iterates backwards over whole db.

Returned items are triples of (key, on, val)

Raises StopIterationError when done or when key empty or None

Backwards means decreasing numerical value of ion, for each on and decreasing numerical value on for each key and decreasing lexocographic order of each key.

Returns:

triples of (keys, on, val): in backwards order

Return type:

item (Iterator[(tuple, int, str)])

Parameters:

keys (str|bytes|memoryview|Iterable) – key(s) made into base key keys empty means whole db
on (int|None) – ordinal number used with onKey(pre,on) to form key. None means iterate over all on for key
sep (bytes) – separator character for split

getAllItemIter(keys: str | bytes | memoryview | Iterable = '', on: int = 0)[source]

Iterates over all items of each set for all on >= on for key. When on ==0, default Iterates over alls items of each set for all on for key. When key is empty then iterates over all items in whole db

Items are triples of (keys, on, val)

Returns:

iterator of triples: (keys, on, val) where keys forms base key, on is int, and val is entry value at with insertion ordering suffix removed from effective key.

Return type:

items (Iterator[(tuple, int, str)])

Parameters:

keys (str|bytes|memoryview|Iterable) – key(s) made into base key from which to make onkey = key + sep + on If keys is empty then gets all items in database.
on (int) – ordinal number used with onKey(pre,on) to form key. When on is None then iterates all on at key,

getAllIter(keys: str | bytes | memoryview | Iterable = '', on: int = 0)[source]

Iterates over alls values of each set for all on >= on for key. When on ==0, default, Iterates over alls items of each set for all on for key. When key is empty then iterates over all items in whole db

Items are triples of (keys, on, val) Returns

val (Iterator[str]): deserialized val of each matching entry

Parameters:

keys (str|bytes|memoryview|Iterable) – key(s) made into base key When keys is empty then retrieves whole database including all set values at each effective key.
on (int) – ordinal number used with onKey(pre,on) to form key. When on ==0 then iterates over all items of all sets for all on for all key >= key.
sep (bytes) – separator character for split

getAllLastBackIter(keys: str | bytes | memoryview | Iterable = '', on: int | None = None)[source]

Iterates backwards over last set values for all on <= on at key. When on is None iterates backwards over last set values for all on at key. When key empty then iterates backwards over whole db for last set values.

Backwards means decreasing numerical value of ion, for each on and decreasing numerical value on for each key and decreasing lexocographic order of each key.

Raises StopIterationError when done or when key empty or None

Returns: lastval (Iterator[str]): deserialized vals in backwards order

Parameters:

keys (str|bytes|memoryview|Iterable) – key(s) made into base key
on (int|None) – ordinal number used with onKey(pre,on) to form key.
sep (bytes) – separator character for split

getAllLastItemBackIter(keys: str | bytes | memoryview | Iterable = '', on: int | None = None)[source]

Iterates backwards over last set items for all on <= on at key. When on is None iterates backwards over last set items for all on at key. When key empty then iterates backwards over whole db for last set items.

Returned items are triples of (key, on, val)

Raises StopIterationError when done or when key empty or None

Backwards means decreasing numerical value of ion, for each on and decreasing numerical value on for each key and decreasing lexocographic order of each key.

Returns:

triples of (keys, on, val): in backwards order

Return type:

lastitem (Iterator[(tuple, int, str)])

Parameters:

keys (str|bytes|memoryview|Iterable) – key(s) made into base key keys empty means whole db
on (int|None) – ordinal number used with onKey(pre,on) to form key. None means iterate over all on for key
sep (bytes) – separator character for split

getAllLastItemIter(keys: str | bytes | memoryview | Iterable = '', on: int = 0)[source]

Iterates over last items of each set for all on >= on at key. When on is None iterates over last items of each set for all on at key. When key is empty iterates over last items of all sets at all keys in db.

Returns

last (Iterator[(tuple, int, str)]): last set item triples of: (keys, on, val) where onkey = key + sep + on

Parameters:

keys (str|bytes|memoryview|Iterable) – key(s) made into base key When keys is empty then retrieves whole database including all set items
on (int|None) – ordinal number used with onKey(pre,on) to form key. When None then all on for all key >= key
sep (bytes) – separator character for split

getAllLastIter(keys: str | bytes | memoryview | Iterable = '', on: int = 0)[source]

Iterates over last value of each set for all on >= on at key When on ==0, default, iterates over last value of each set for all on at key When key is empty iterates over last value of each set of all on for all keys, i.e. whole db.

Returns: last (Iterator[str]): deserialized last set val of of each onkey

Parameters:

keys (str|bytes|memoryview|Iterable) – key(s) made into base key When keys is empty then retrieves whole database including all set values
on (int) – ordinal number used with onKey(pre,on) to form key.
sep (bytes) – separator character for split

getItem(keys: str | bytes | memoryview | Iterable, on: int = 0, ion: int = 0)[source]

Gets list of items (key, on, val) from set of entries at onkey made from keys and on starting at offset ion into set. When onkey missing or key empty or missing returns empty list.

Returns

item (tuple[bytes, int, str|bytes|memoryview]|None): at onkey if any: of form (key, on, val) None if no entry at onkey or key empty or None

Parameters:

keys (str|bytes|memoryview|Iterable) – keys as prefix to be combined with serialized exposed on tail and sep to form onkey
on (int) – ordinal number used with onKey(key ,on) to form key.

getItemIter(keys: str | bytes | memoryview | Iterable, on: int = 0, ion: int = 0)[source]

Iterates over set items (key, on, val) at onkey made from keys and on in insertion order from from offset ion into set using hidden ordinal suffix. When onkey is empty or missing then returns empty iterator

Returns:

deserialized items elements of set at onkey

Return type:

items (Iterator[str])

Parameters:

keys (str|bytes|memoryview|Iterable) – key(s) made into base key
on (int) – ordinal number tail used with onKey(pre,on) to form key.
ion (int) – starting insertion ordinal value, default 0

getIter(keys: str | bytes | memoryview | Iterable, on: int = 0, ion: int = 0)[source]

Iterates over set vals at onkey made from keys and on in insertion order from from offset ion into set using hidden ordinal suffix. When onkey is empty or missing then returns empty iterator

Returns:

deserialized val elements of set at onkey

Return type:

val (Iterator[str])

Parameters:

keys (str|bytes|memoryview|Iterable) – key(s) made into base key
on (int) – ordinal number tail used with onKey(pre,on) to form key.
ion (int) – starting insertion ordinal value, default 0

getLast(keys: str | bytes | memoryview | Iterable, on: int = 0)[source]

Gets last val inserted at key made from keys in insertion order using hidden ordinal proem.

Returns:

value str, None if no entry at effective key made from: keys and on

Return type:

last (str|None)

Parameters:

keys (str|bytes|memoryview|Iterable) – key(s) made into base key
on (int) – ordinal number used with onKey(pre,on) to form key.

getLastItem(keys: str | bytes | memoryview | Iterable, on: int = 0)[source]

Gets last item inserted at key made from keys in insertion order using hidden ordinal proem.

Returns:

last set item triple at onkey: (keys, on, val) Empty tuple () if onkey not in db or key empty.

Return type:

last (tuple[tuple, int, str])

Parameters:

keys (str|bytes|memoryview|Iterable) – key(s) made into base key
on (int) – ordinal number used with onKey(pre,on) to form key.

getTopItemIter(keys: str | bytes | memoryview | Iterable = '')[source]

Iterates over top branch of all insertion ordered set values where each key startwith top. Assumes every effective key in db has trailing on element, onkey = key + sep + on, so can return on in item. Also assumes every effective key includes hiddion isertion ordinal ion suffix that is suffixed and unsuffixed transparently.

When top key is empty, gets all items in database.

Returns:

iterator of triples: (keys, on, val) where keys forms base key, on is int, and val is entry value at with insertion ordering suffix removed from effective key.

Return type:

items (Iterator[(tuple, int, str)])

Parameters:

keys (str|bytes|memoryview|Iterable) – keys as truncated top key, to get a key space prefix to get all the items from multiple branches of the key space. If top key is empty then gets all items in database.
on (int) – ordinal number used with onKey(pre,on) to form key.

Pins (sets) vals at key made from keys with exposed on tail in insertion order using hidden ordinal suffix. Overwrites. Removes all pre-existing vals that share same onkey and replaces them with vals

Returns:

True If successful, False otherwise.

Return type:

result (bool)

Parameters:

keys (str|bytes|memoryview|Iterable) – key(s) made into base key
on (int) – ordinal number tail used with onKey(pre,on) to form onkey.
vals (NonStrIterable|None) – serialized values to replace set of vals at onkey if any. None means empty iterable. When not NonStrIterable converts to iterable. Empty iterable returns False

Put all vals idempotently at key at key made from keys with exposed on tail in insertion order using hidden ordinal suffix. Idempotently means do not put any val in vals that is already in set vals at key. Does not overwrite.

Returns:

True If successful, False otherwise.

Return type:

result (bool)

Parameters:

keys (str|bytes|memoryview|Iterable) – key(s) made into base key
on (int) – ordinal number tail used with onKey(pre,on) to form onkey.
vals (str|bytes|memoryview|NonStrIterable|None) – serialized values to add to set of vals at onkey if any. When not NonStrIterable converts to iterable. Empty iterable or None returns False

Removes entry in set with val if any at key made and on in insertion order using hidden ordinal suffix. When val matches a value in the set at the effective key then only that value is removed. When val is None then removes all values from set effectively deleting entry at onkey = keys + sep + on if any in db.

Returns:

True if onkey with dup val exists so rem successful.: False otherwise

Return type:

result (bool)

Parameters:

keys (str|bytes|memoryview|Iterable) – key(s) made into base key
on (int) – ordinal number used with onKey(pre,on) to form key.
val (str|bytes|memoryview|None) – value at key to remove. Subclass ._ser method may accept different value types if val is None then remove all values at key

remAll(keys: str | bytes | memoryview | Iterable = '', on: int = 0)[source]

Removes all entries for all sets for all on >= on at key. When on i==0, default, then removes all entries for all sets for all on at key. When key is empty then removes whole db.

Returns:

True if onkey with dup val exists so rem successful.: False otherwise

Return type:

result (bool)

Parameters:

keys (str|bytes|memoryview|Iterable) – key(s) made into base key When key is empty then remove all entries in whole db
on (int) – base key. None means all on for key

class keri.db.subing.OnSuber(*pa, **kwa)[source]

Subclass of OnSuberBase andSuber that adds methods for keys with ordinal numbered exposed tail. Each key consistes of pre joined with .sep to ordinal tail

Assumes dupsort==False

__init__(*pa, **kwa)[source]

Inherited Parameters:

db (LMDBer): base db subkey (str): LMDB sub database key dupsort (bool): True means enable duplicates at each key

False (default) means do not enable duplicates at each key. Set to False

sep (str): separator to convert keys iterator to key bytes for db key: default is self.Sep == ‘.’
verify (bool): True means reverify when ._des from db when applicable: False means do not reverify. Default False

class keri.db.subing.OnSuberBase(*pa, **kwa)[source]

Subclass of SuberBase that adds methods for keys with exposed trialing ordinal key part that is 32 byte serializaton of monotonically increasing ordinal number on such as sn or fn. Useful for escrows that are ordered by ordinals such as first seen or sequence number.

Each key consistes of top key joined with .sep to ordinal tail Works with dupsort==True or False

__init__(*pa, **kwa)[source]

Inherited Parameters:

db (LMDBer): base db subkey (str): LMDB sub database key dupsort (bool): True means enable duplicates at each key

False (default) means do not enable duplicates at each key. Default False

sep (str): separator to convert keys iterator to key bytes for db key: Default ‘.’
verify (bool): True means reverify when ._des from db when applicable: False means do not reverify. Default False

Appends val to next highest unused exposed ordinal tail and returns the ordinal.

Returns:

ordinal number of newly appended val

Return type:

on (int)

Parameters:

keys (str|bytes|memoryview|Iterable) – top keys as prefix to be combined with serialized exposed on tail and sep to form key if key empty then raises ValueError
val (str|bytes|memoryview) – serialized value to append If None then raises ValueError

cnt(keys: str | bytes | memoryview | Iterable = '', on: int = 0)[source]

Counts all entries with same key over all all on >= on. If key not in db then count is 0

Returns

cnt (int): count of of all exposed on tail keyed vals with same: onkey prefix but different on in onkey in db starting at ordinal number on where key is formed with onKey(key,on). Count at each onkey includes duplicates if any.

Parameters:

keys (str|bytes|memoryview|Iterable) – top keys as prefix to be combined with serialized exposed on tail and sep to form top key When keys is empty then counts whole database including duplicates if any.
on (int) – ordinal number used with onKey(key,on) to form key.

cntAll(keys: str | bytes | memoryview | Iterable = '', on: int = 0)[source]

Counts all entries with same key over all all on >= on. When keys is empty then counts for on for all keys in whole database.

Returns

cnt (int): count of of all exposed on tail keyed vals with same: onkey prefix but different on in onkey in db starting at ordinal number on where key is formed with onKey(key,on). Count at each onkey includes duplicates if any.

Parameters:

keys (str|bytes|memoryview|Iterable) – top keys as prefix to be combined with serialized exposed on tail and sep to form top key When keys is empty then counts whole database including duplicates if any.
on (int) – ordinal number used with onKey(key,on) to form key.

get(keys: str | bytes | memoryview | Iterable, on: int = 0)[source]

Gets val at onkey made from keys and on.

Returns

val (str): serialization at onkey if any: None if no entry at onkey

Parameters:

keys (str|bytes|memoryview|Iterable) – keys as prefix to be combined with serialized exposed on tail and sep to form onkey
on (int) – ordinal number used with onKey(key ,on) to form key.

getAllItemIter(keys: str | bytes | memoryview | Iterable = '', on: int = 0)[source]

Iterates over all entries in db for all onkey made from keys and on for all on >= on. When on==0 default then gets all on. When keys empty then iterates over whole db.

Returns:

triples of key, on, val with same: key but increments of on >= on i.e. all onkey beginning with on

Return type:

items (Iterator[(key, on, val)])

Parameters:

keys (str|bytes|memoryview|iterator) – keys as prefix to be combined with serialized on exposed on tail and sep to form actual key
on (int) – ordinal number used with onKey(pre,on) to form key at at which to initiate retrieval. When on==0 then all on for key
sep (bytes) – separator character for split

getAllIter(keys: str | bytes | memoryview | Iterable = '', on: int = 0)[source]

Returns:

of val with same key but increments of: on >= on i.e. all key.on beginning with on

Return type:

items (Iterator[bytes])

Parameters:

keys (str | bytes | memoryview | iterator) – keys as prefix to be combined with serialized exposed on tail and sep to form actual key When keys is empty then retrieves whole database including duplicates if any
on (int) – ordinal number used with onKey(pre,on) to form key at at which to initiate retrieval
sep (bytes) – separator character for split

getItem(keys: str | bytes | memoryview | Iterable, on: int = 0)[source]

Gets item (key, on, val) at onkey made from keys and on. When onkey missing or key empty or None returns None

Returns

item (tuple[bytes, int, str|bytes|memoryview]|None): at onkey if any: of form (key, on, val) None if no entry at onkey or key empty or None

Parameters:

keys (str|bytes|memoryview|Iterable) – keys as prefix to be combined with serialized exposed on tail and sep to form onkey
on (int) – ordinal number used with onKey(key ,on) to form key.

getTopItemIter(keys: str | bytes | memoryview | Iterable = '')[source]

Iterates over top branch of all entries where each top key startwith key made from keys. Assumes every effective key in db has trailing on element, onkey = key + sep + on, so can return on in item. When top key is empty, gets all items in database.

Returns:

iterator of triples: (keys, on, val) where keys forms base key, on is int, and val is entry value at with insertion ordering suffix removed from effective key.

Return type:

items (Iterator[(tuple, int, str)])

Parameters:

keys (str|bytes|memoryview|Iterable) – keys as truncated top key, to get a key space prefix to get all the items from multiple branches of the key space. If top key is empty then gets all items in database.
on (int) – ordinal number used with onKey(pre,on) to form key.

Returns

result (bool): True if value is written or overwritten at onkey: False otherwise

Parameters:

keys (str|bytes|memoryview|Iterable) – keys as prefix to be combined with serialized exposed on tail and sep to form onkey
on (int) – ordinal number used with onKey(key ,on) to form key.
val (str|bytes|memoryview|None) – serialized value to pin when None returns False

Returns

result (bool): True if onkey made from key+sep+serialized on is: not found in database so value is written idempotently.

False otherwise

Parameters:

keys (str|bytes|memoryview|Iterable) – keys as prefix to be combined with serialized exposed on tail and sep to form onkey
on (int) – ordinal number used with onKey(key ,on) to form key.
val (str|bytes|memoryview|None) – serialized value to put When None returns False

rem(keys: str | bytes | memoryview | Iterable, on: int = 0)[source]

Removes entry at onkey = key + sep + on When key is missing or empty or None returns False

Returns

result (bool): True if onkey made from key+sep+serialized on is: found in database so entry at onkey is removed

False otherwise if no entry at onkey or key is empty.

Parameters:

keys (str|bytes|memoryview|Iterable) – keys as prefix to be combined with serialized exposed on tail and sep to form onkey
on (int) – ordinal number used with onKey(key ,on) to form key.

remAll(keys: str | bytes | memoryview | Iterable = '', on: int = 0)[source]

Removes entry for each on >= on at key. When on == 0, default, then removes each entry for all on at key. When key is empty then removes whole db.

Returns:

True if onkey with dup val exists so rem successful.: False otherwise

Return type:

result (bool)

Parameters:

keys (str|bytes|memoryview|Iterable) – key(s) made into base key When key is empty then remove all entries in whole db
on (int) – base key. None means all on for key

class keri.db.subing.SchemerSuber(*pa, klas: Type[scheming.Schemer] | None = None, **kwa)[source]

Sub class of SerderSuberBase and Suber where data is serialized Schemer instance Schemer ser/des is ducktype of Serder using .raw Automatically serializes and deserializes using Schemer methods

__init__(*pa, klas: Type[scheming.Schemer] | None = None, **kwa)[source]

Inherited Parameters:

db (LMDBer): base db subkey (str): LMDB sub database key dupsort (bool): True means enable duplicates at each key

False (default) means do not enable duplicates at each key

sep (str): separator to convert keys iterator to key bytes for db key: default is self.Sep == ‘.’
verify (bool): True means reverify when ._des from db when applicable: False means do not reverify. Default False
klas (Type[scheming.Schemer]): Class reference to ducktyped subclass: of Serder

Overridden Parameters:

klas (Type[scheming.Schemer]): Class reference to ducktyped subclass: of Serder intercepts passed in klas and forces it to Schemer

class keri.db.subing.SerderIoSetSuber(*pa, **kwa)[source]

Sub class of SerderSuberBase and IoSetSuber that allows multiple Serder instances to be stored at the same db key in insertion order. Example use case would be an escrow where the key is a sequence number based index (such as snKey).

Sub class of SerderSuberBase where data is serialized Serder Subclass instance given by .klas Automatically serializes and deserializes using .klas Serder methods

Extends IoSetSuber so that all IoSetSuber methods now work with Serder subclass for each val.

IoSetSuber stores at each effective key a set of distinct values that share that same effective key where each member of the set is retrieved in insertion order (dupsort==False) The methods allows an Iterable (set valued) of Iterables of separation subclass instances to be stored at a given effective key in insertion order.

Actual keys include a hidden ordinal key suffix that tracks insertion order. The suffix is appended and stripped transparently from the keys. The set of items with duplicate effective keys are retrieved in insertion order when iterating or as a list of the set elements. The actual iokey for any item includes the ordinal suffix.

Attributes:
db (LMDBer): base LMDB db sdb (lmdb._Database): instance of lmdb named sub db for this Suber sep (str): separator for combining keys tuple of strs into key bytes klas (Iterable): of Class references to subclasses of CESR compatible

, each of to Type[coring.Matter etc]

__init__(*pa, **kwa)[source]

Inherited Parameters:

db (LMDBer): base db subkey (str): LMDB sub database key dupsort (bool): True means enable duplicates at each key

False (default) means do not enable duplicates at each key

sep (str): separator to convert keys iterator to key bytes for db key: default is self.Sep == ‘.’
verify (bool): True means reverify when ._des from db when applicable: False means do not reverify. Default False

klas (Type[serdering.Serder]): Class reference to subclass of Serder

class keri.db.subing.SerderSuber(*pa, **kwa)[source]

Sub class of SerderSuberBase, Suber where data is serialized Serder Subclass instance given by .klas Automatically serializes and deserializes using .klas Serder methods

__init__(*pa, **kwa)[source]

Inherited Parameters:

db (LMDBer): base db subkey (str): LMDB sub database key dupsort (bool): True means enable duplicates at each key

False (default) means do not enable duplicates at each key

sep (str): separator to convert keys iterator to key bytes for db key: default is self.Sep == ‘.’
verify (bool): True means reverify when ._des from db when applicable: False means do not reverify. Default False

klas (Type[serdering.Serder]): Class reference to subclass of Serder

class keri.db.subing.SerderSuberBase(*pa, klas: Type[serdering.Serder] | None = None, **kwa)[source]

Sub class of SuberBase where data is serialized Serder Subclass instance given by .klas Automatically serializes and deserializes using .klas Serder methods

__init__(*pa, klas: Type[serdering.Serder] | None = None, **kwa)[source]

Inherited Parameters:

db (LMDBer): base db subkey (str): LMDB sub database key dupsort (bool): True means enable duplicates at each key

False (default) means do not enable duplicates at each key

sep (str): separator to convert keys iterator to key bytes for db key: default is self.Sep == ‘.’
verify (bool): True means reverify when ._des from db when applicable: False means do not reverify. Default False

Overridden Parameters:

klas (Type[serdering.Serder]): Class reference to subclass of Serder

class keri.db.subing.SignerSuber(*pa, klas: Type[signing.Signer] | None = None, **kwa)[source]

Sub class of CesrSuber where data is Signer subclass instance .qb64b propery which is a fully qualified serialization and uses the key which is the qb64b of the signer.verfer to get the transferable property of the verfer Automatically serializes and deserializes from qb64b to/from Signer instances

Assumes that last or only element of db key from keys for all entries is the qb64 of a public key for the associated Verfer instance. This allows returned Signer instance to have its .transferable property set correctly.

__init__(*pa, klas: Type[signing.Signer] | None = None, **kwa)[source]

Parameters:

db (LMDBer) – base db
subkey (str) – LMDB sub database key
klas (Type[coring.Matter]) – Class reference to subclass of Matter

get(keys: str | Iterable)[source]

Gets Signer instance at keys

Returns:: transferable determined by key which is verfer None if no entry at keys
Return type:: val (Signer)
Parameters:: keys (Union[str, iterable]) – key strs to be combined in order to form key. Last element of keys is verkey used to determin .transferable for Signer

Usage:

Use walrus operator to catch and raise missing entry if (signer := mydb.get(keys)) is None:

raise ExceptionHere

use signer here

getTopItemIter(keys: str | bytes | memoryview | Iterable = '', *, topive=False)[source]

Iterates over all the items in top branch defined by keys where keys may be truncation of full branch.

Returns:

tuple (key, val) over the all the items in subdb whose key startswith key made from keys. Keys may be keyspace prefix to return branches of key space. When keys is empty then returns all items in subdb

Return type:

iterator (Iteratore

Parameters:

keys (Iterable) – tuple of bytes or strs that may be a truncation of a full keys tuple in in order to get all the items from multiple branches of the key space. If keys is empty then gets all items in database.
topive (bool) – True means treat as partial key tuple from top branch of key space given by partial keys. Resultant key ends in .sep character. False means treat as full branch in key space. Resultant key does not end in .sep character. When last item in keys is empty str then will treat as partial ending in sep regardless of top value

Uses python .startswith to match which always returns True if top is empty string so empty will matches all keys in db.

class keri.db.subing.Suber(db: LMDBer, *, subkey: str = 'docs.', dupsort: bool = False, **kwa)[source]

Subclass of SuberBase with no LMDB duplicates (i.e. multiple values at same key).

__init__(db: LMDBer, *, subkey: str = 'docs.', dupsort: bool = False, **kwa)[source]

Initialze instance

Inherited Parameters:

db (LMDBer): base db subkey (str): LMDB sub database key dupsort (bool): True means enable duplicates at each key

False (default) means do not enable duplicates at each key. Set to False

sep (str): separator to convert keys iterator to key bytes for db key: default is self.Sep == ‘.’
verify (bool): True means reverify when ._des from db when applicable: False means do not reverify. Default False

Parameters:

db (LMDBer) – base db
subkey (str) – LMDB sub database key

cnt()[source]

Counts all the entries in subdb. Should be overidden in subclasses with parameters to count more specifically.

Returns:: count of all entries in sdb
Return type:: cnt (int)

get(keys: str | Iterable)[source]

Gets val at keys

Parameters:: keys (tuple) – of key strs to be combined in order to form key
Returns:: decoded as utf-8 None if no entry at keys
Return type:: data (str)

Usage:

Use walrus operator to catch and raise missing entry if (data := mydb.get(keys)) is None:

raise ExceptionHere

use data here

pin(keys: str | Iterable, val: bytes | str)[source]

Pins (sets) val at key made from keys. Overwrites.

Parameters:

keys (tuple) – of key strs to be combined in order to form key
val (bytes) – value

Returns:

True If successful. False otherwise.

Return type:

result (bool)

put(keys: str | Iterable, val: bytes | str | any)[source]

Puts val at key made from keys. Does not overwrite

Parameters:

keys (tuple) – of key strs to be combined in order to form key
val (bytes) – value

Returns:

True If successful, False otherwise, such as key: already in database.

Return type:

result (bool)

rem(keys: str | Iterable)[source]

Removes entry at keys. This safe if keys empty or missing then

Parameters:

keys (tuple) – of key strs to be combined in order to form key

Returns:

True if key exists so delete successful.: False if key empty or missing from db

Return type:

result (bool)

Raises KeyError if key to big or otherwise bad

class keri.db.subing.SuberBase(db: LMDBer, *, subkey: str = 'docs.', dupsort: bool = False, sep: str = None, verify: bool = False, **kwa)[source]

Base class for Sub DBs of LMDBer Provides common methods for subclasses Do not instantiate but use a subclass

db

base LMDB db

Type:: LMDBer

sdb

instance of lmdb named sub db for this Suber

Type:: lmdb._Database

sep

separator for combining keys tuple of strs into key bytes

Type:: str

verify

True means reverify when ._des from db when applicable False means do not reverify. Default False

Type:: bool

__init__(db: LMDBer, *, subkey: str = 'docs.', dupsort: bool = False, sep: str = None, verify: bool = False, **kwa)[source]

Parameters:

db (LMDBer) – base db
subkey (str) – LMDB sub database key
dupsort (bool) – True means enable duplicates at each key False (default) means do not enable duplicates at each key
sep (str) – separator to convert keys iterator to key bytes for db key default is self.Sep == ‘.’
verify (bool) – True means reverify when ._des from db when applicable False means do not reverify. Default False

cnt()

Counts all the entries in subdb. Should be overidden in subclasses with parameters to count more specifically.

Returns:: count of all entries in sdb
Return type:: cnt (int)

cntAll()[source]

Counts all the entries in subdb. Should be overidden in subclasses with parameters to count more specifically.

Returns:: count of all entries in sdb
Return type:: cnt (int)

cntTop(keys: str | bytes | memoryview | Iterable = '', *, topive=False)[source]

Counts all entries in top branch of db given by keys. When keys is empty then counts all entries in whole db.

Returns:

count of all entries in top branch of sdb

Return type:

cnt (int)

Parameters:

keys (str|bytes|memoryview|Iterable) – of key parts that may be a truncation of a full keys tuple in in order to address all the items from multiple branches of the key space. If keys is empty then gets all items in database. Either append “” to end of keys Iterable to ensure get properly separated top branch key or use topive=True.
topive (bool) – True means treat keys as delimited top of partial branch in key space by forcing resultant key to end in .sep character. False means treat keys as undelimited partial or full branch in key space by not forcing resultant key to ebd in .sep character. When last item in keys is empty str then will treat as delimited partial branch ending in .sep regardless of topive value.

Uses python .startswith() to match keyspace since str.startswith(‘’) always returns True so empty str will match all keys in db.

getFullItemIter(keys: str | bytes | memoryview | Iterable = '', *, topive=False)[source]

Iterator over items in .db that returns full items with subclass specific special hidden parts shown for debugging or testing.

Returns:

(key, val) tuples of each item over the all the items in subdb whose key startswith key made from keys. Keys may be keyspace prefix to return branches of key space. When keys is empty then returns all items in subdb. This is meant to return full parts of items in both keyspace and valuespace which may be useful in debugging or testing.

Return type:

items (Iterator[tuple[key,val]])

Parameters:

keys (str|bytes|memoryview|Iteratable) – of key parts that may be a truncation of a full keys tuple in in order to address all the items from multiple branches of the key space. If keys is empty then gets all items in database. Either append “” to end of keys Iterable to ensure get properly separated top branch key or use topive=True.
topive (bool) – True means treat as partial key tuple from top branch of key space given by partial keys. Resultant key ends in .sep character. False means treat as full branch in key space. Resultant key does not end in .sep character. When last item in keys is empty str then will treat as partial ending in sep regardless of top value

Uses python .startswith() to match keyspace since str.startswith(‘’) always returns True so empty str will match all keys in db.

getTopItemIter(keys: str | bytes | memoryview | Iterable = '', *, topive=False)[source]

Iterates over all the items in top branch defined by keys where keys may be a truncation of a full branch. The truncation format may be modified by the topive parameter.

Iterates over top branch of items in .db subclasses that do special hidden transforms on either the keyspace or valuespace. Should override this method to hide hidden parts from the returned items. For example, adding either a hidden key space suffix or hidden val space proem to ensure insertion order.

Use getFullItemIter instead to return full items with hidden parts shown for debugging or testing.

Returns:

(key, val) tuples of each item over the all the items in subdb whose key startswith key made from keys. Keys may be keyspace prefix to return branches of key space. When keys is empty then returns all items in subdb

Return type:

items (Iterator[tuple[key,val]])

Parameters:

keys (str|bytes|memoryview|Iterable) – of key parts that may be a truncation of a full keys tuple in in order to address all the items from multiple branches of the key space. If keys is empty then gets all items in database. Either append “” to end of keys Iterable to ensure get properly separated top branch key or use topive=True.
topive (bool) – True means treat keys as delimited top of partial branch in key space by forcing resultant key to end in .sep character. False means treat keys as undelimited partial or full branch in key space by not forcing resultant key to ebd in .sep character. When last item in keys is empty str then will treat as delimited partial branch ending in .sep regardless of topive value.

Uses python .startswith() to match keyspace since str.startswith(‘’) always returns True so empty str will match all keys in db.

remTop(keys: str | bytes | memoryview | Iterable = b'', *, topive=False)

Removes all entries in top branch of db given by keys. Enables removal of whole branches of db key space.

Returns:

True if val at key exists so delete successful.: False otherwise

Return type:

result (bool)

Parameters:

keys (str|bytes|memoryview|Iterable) – of key parts that may be a truncation of a full keys tuple in in order to address all the items from multiple branches of the key space. If keys is empty then trims all items in database. Either append “” to end of keys Iterable to ensure get properly separated top branch key or use topive=True.
topive (bool) – True means treat as partial key tuple from top branch of key space given by partial keys. Resultant key ends in .sep character. False means treat as full branch in key space. Resultant key does not end in .sep character. When last item in keys is empty str then will treat as partial ending in sep regardless of top value

Uses python .startswith() to match keyspace since str.startswith(‘’) always returns True so empty str will match all keys in db.

trim(keys: str | bytes | memoryview | Iterable = b'', *, topive=False)[source]

Removes all entries in top branch of db given by keys. Enables removal of whole branches of db key space.

Returns:

True if val at key exists so delete successful.: False otherwise

Return type:

result (bool)

Parameters:

keys (str|bytes|memoryview|Iterable) – of key parts that may be a truncation of a full keys tuple in in order to address all the items from multiple branches of the key space. If keys is empty then trims all items in database. Either append “” to end of keys Iterable to ensure get properly separated top branch key or use topive=True.
topive (bool) – True means treat as partial key tuple from top branch of key space given by partial keys. Resultant key ends in .sep character. False means treat as full branch in key space. Resultant key does not end in .sep character. When last item in keys is empty str then will treat as partial ending in sep regardless of top value

Uses python .startswith() to match keyspace since str.startswith(‘’) always returns True so empty str will match all keys in db.