Internal newtype wrapper around a ledger state l so that we can define a non-blanket Anchorable instance.

Internal state of the ledger DB

The ledger DB looks like

anchor |> snapshots <| current

where anchor records the oldest known snapshot and current the most recent. The anchor is the oldest point we can roll back to.

We take a snapshot after each block is applied and keep in memory a window of the last k snapshots. We have verified empirically (#1936) that the overhead of keeping k snapshots in memory is small, i.e., about 5% compared to keeping a snapshot every 100 blocks. This is thanks to sharing between consecutive snapshots.

As an example, suppose we have k = 6. The ledger DB grows as illustrated below, where we indicate the anchor number of blocks, the stored snapshots, and the current ledger.

anchor |> #   [ snapshots ]                   <| tip
---------------------------------------------------------------------------
G      |> (0) [ ]                             <| G
G      |> (1) [ L1]                           <| L1
G      |> (2) [ L1,  L2]                      <| L2
G      |> (3) [ L1,  L2,  L3]                 <| L3
G      |> (4) [ L1,  L2,  L3,  L4]            <| L4
G      |> (5) [ L1,  L2,  L3,  L4,  L5]       <| L5
G      |> (6) [ L1,  L2,  L3,  L4,  L5,  L6]  <| L6
L1     |> (6) [ L2,  L3,  L4,  L5,  L6,  L7]  <| L7
L2     |> (6) [ L3,  L4,  L5,  L6,  L7,  L8]  <| L8
L3     |> (6) [ L4,  L5,  L6,  L7,  L8,  L9]  <| L9   (*)
L4     |> (6) [ L5,  L6,  L7,  L8,  L9,  L10] <| L10
L5     |> (6) [*L6,  L7,  L8,  L9,  L10, L11] <| L11
L6     |> (6) [ L7,  L8,  L9,  L10, L11, L12] <| L12
L7     |> (6) [ L8,  L9,  L10, L12, L12, L13] <| L13
L8     |> (6) [ L9,  L10, L12, L12, L13, L14] <| L14

The ledger DB must guarantee that at all times we are able to roll back k blocks. For example, if we are on line (*), and roll back 6 blocks, we get

L3 |> []

Initialization log

The initialization log records which snapshots from disk were considered, in which order, and why some snapshots were rejected. It is primarily useful for monitoring purposes.

Which point the replay started from

Initialize the ledger DB from the most recent snapshot on disk

If no such snapshot can be found, use the genesis ledger DB. Returns the initialized DB as well as the block reference corresponding to the snapshot we found on disk (the latter primarily for testing/monitoring purposes).

We do not catch any exceptions thrown during streaming; should any be thrown, it is the responsibility of the ChainDB to catch these and trigger (further) validation. We only discard snapshots if

• We cannot deserialise them, or
• they are ahead of the chain

It is possible that the Ledger DB will not be able to roll back k blocks after initialization if the chain has been truncated (data corruption).

We do not attempt to use multiple ledger states from disk to construct the ledger DB. Instead we load only a single ledger state from disk, and compute all subsequent ones. This is important, because the ledger states obtained in this way will (hopefully) share much of their memory footprint with their predecessors.

Which point the replay is expected to end at

Events traced while replaying blocks against the ledger to bring it up to date w.r.t. the tip of the ImmutableDB during initialisation. As this process takes a while, we trace events to inform higher layers of our progress.

Add the tip of the Immutable DB to the trace event

Between the tip of the immutable DB and the point of the starting block, the node could (if it so desired) easily compute a "percentage complete".

Add the block at which a replay started.

This allows to compute a "percentage complete" when tracing the events.

Information about the state of the ledger at the anchor

The ledger state at the tip of the chain

Have we seen at least k blocks?

How many blocks can we currently roll back?

Get a past ledger state

\( O(\log(\min(i,n-i)) \)

When no ledger state (or anchor) has the given Point, Nothing is returned.

All snapshots currently stored by the ledger DB (new to old)

This also includes the snapshot at the anchor. For each snapshot we also return the distance from the tip.

Reference to the block at the tip of the chain

Ledger DB starting at the specified ledger state

Annotated ledger errors

Ap is used to pass information about blocks to ledger DB updates

The constructors serve two purposes:

• Specify the various parameters a. Are we passing the block by value or by reference? b. Are we applying or reapplying the block?
• Compute the constraint c on the monad m in order to run the query: a. If we are passing a block by reference, we must be able to resolve it. b. If we are applying rather than reapplying, we might have ledger errors.

Exceeded maximum rollback supported by the current ledger DB state

Under normal circumstances this will not arise. It can really only happen in the presence of data corruption (or when switching to a shorter fork, but that is disallowed by all currently known Ouroboros protocols).

Records both the supported and the requested rollback.

Resolve a block

Resolving a block reference to the actual block lives in m because it might need to read the block from disk (and can therefore not be done inside an STM transaction).

NOTE: The ledger DB will only ask the ChainDB for blocks it knows must exist. If the ChainDB is unable to fulfill the request, data corruption must have happened and the ChainDB should trigger validation mode.

Monads in which we can resolve blocks

To guide type inference, we insist that we must be able to infer the type of the block we are resolving from the type of the monad.

Transform the underlying AnchoredSeq using the given functions.

Prune snapshots until at we have at most k snapshots in the LedgerDB, excluding the snapshots stored at the anchor.

Switch to a fork

Name of a disk snapshot.

The snapshot itself might not yet exist on disk.

The snapshots that are periodically created are temporary, they will be deleted when trimming

List on-disk snapshots, highest number first.

Type-safe flag to regulate the checksum policy of the ledger state snapshots.

These patterns are exposed to cardano-node and will be passed as part of DiskPolicy.

Type-safe flag to regulate the checksum policy of the ledger state snapshots.

These patterns are exposed to cardano-node and will be passed as part of DiskPolicy.

Read snapshot from disk.

Fail on data corruption, i.e. when the checksum of the read data differs from the one tracked by DiskSnapshot.

Take a snapshot of the oldest ledger state in the ledger DB

We write the oldest ledger state to disk because the intention is to only write ledger states to disk that we know to be immutable. Primarily for testing purposes, takeSnapshot returns the block reference corresponding to the snapshot that we wrote.

If a snapshot with the same number already exists on disk or if the tip is at genesis, no snapshot is taken.

Note that an EBB can have the same slot number and thus snapshot number as the block after it. This doesn't matter. The one block difference in the ledger state doesn't warrant an additional snapshot. The number in the name of the snapshot is only indicative, we don't rely on it being correct.

NOTE: This is a lower-level API that takes a snapshot independent from whether this snapshot corresponds to a state that is more than k back.

TODO: Should we delete the file if an error occurs during writing?

Trim the number of on disk snapshots so that at most onDiskNumSnapshots snapshots are stored on disk. The oldest snapshots are deleted.

The deleted snapshots are returned.

Write a ledger state snapshot to disk

This function writes two files: * the snapshot file itself, with the name generated by snapshotToPath * the checksum file, with the name generated by snapshotToChecksumPath

To remain backwards compatible with existing snapshots stored on disk, we must accept the old format as well as the new format.

The old format: * The tip: WithOrigin (RealPoint blk) * The chain length: Word64 * The ledger state: l

The new format is described by snapshotEncodingVersion1.

This decoder will accept and ignore them. The encoder (encodeSnapshot) will no longer encode them.

Delete snapshot from disk

Encoder to be used in combination with decodeSnapshotBackwardsCompatible.

On-disk policy

We only write ledger states that are older than k blocks to disk (that is, snapshots that are guaranteed valid). The on-disk policy determines how often we write to disk and how many checkpoints we keep.

The components used by cardano-node to construct a DiskPolicy.

Number of snapshots to be stored on disk. This is either the default value as determined by the DiskPolicy, or it is provided by the user. See the DiskPolicy documentation for more information.

Length of time, requested by the user, that has to pass after which a snapshot is taken. It can be:

1. either explicitly provided by user in seconds
2. or default value can be requested - the specific DiskPolicy determines what that is exactly, see mkDiskPolicy as an example

Default on-disk policy arguments suitable to use with cardano-node