ouroboros-consensus-0.26.0.0: Consensus layer for the Ouroboros blockchain protocol
Safe HaskellNone
LanguageHaskell2010

Ouroboros.Consensus.Storage.LedgerDB.Snapshots

Description

Common logic and types for LedgerDB Snapshots.

Snapshots are saved copies of Ledger states in the chain which can be used to restart the node without having to replay the whole chain. Regardless of the actual LedgerDB implementation chosen, the general management of snapshots is common to all implementations.

Synopsis

Snapshots

data CRCError Source #

Constructors

CRCInvalid 
CRCNoFile 

Instances

Instances details
Show CRCError Source # 
Instance details

Defined in Ouroboros.Consensus.Util.CRC

Methods

showsPrecIntCRCErrorShowS #

showCRCErrorString #

showList ∷ [CRCError] → ShowS #

Eq CRCError Source # 
Instance details

Defined in Ouroboros.Consensus.Util.CRC

Methods

(==)CRCErrorCRCErrorBool #

(/=)CRCErrorCRCErrorBool #

data DiskSnapshot Source #

Name of a disk snapshot.

The snapshot itself might not yet exist on disk.

Constructors

DiskSnapshot 

Fields

  • dsNumberWord64

    Snapshots are numbered. We will try the snapshots with the highest number first.

    When creating a snapshot, we use the slot number of the ledger state it corresponds to as the snapshot number. This gives an indication of how recent the snapshot is.

    Note that the snapshot names are only indicative, we don't rely on the snapshot number matching the slot number of the corresponding ledger state. We only use the snapshots numbers to determine the order in which we try them.

  • dsSuffixMaybe String

    Snapshots can optionally have a suffix, separated by the snapshot number with an underscore, e.g., 4492799_last_Byron. This suffix acts as metadata for the operator of the node. Snapshots with a suffix will not be deleted.

Instances

Instances details
Generic DiskSnapshot Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.Snapshots

Associated Types

type Rep DiskSnapshot 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.Snapshots

type Rep DiskSnapshot = D1 ('MetaData "DiskSnapshot" "Ouroboros.Consensus.Storage.LedgerDB.Snapshots" "ouroboros-consensus-0.26.0.0-inplace" 'False) (C1 ('MetaCons "DiskSnapshot" 'PrefixI 'True) (S1 ('MetaSel ('Just "dsNumber") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedLazy) (Rec0 Word64) :*: S1 ('MetaSel ('Just "dsSuffix") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedLazy) (Rec0 (Maybe String))))
Show DiskSnapshot Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.Snapshots

Eq DiskSnapshot Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.Snapshots

Ord DiskSnapshot Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.Snapshots

type Rep DiskSnapshot Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.Snapshots

type Rep DiskSnapshot = D1 ('MetaData "DiskSnapshot" "Ouroboros.Consensus.Storage.LedgerDB.Snapshots" "ouroboros-consensus-0.26.0.0-inplace" 'False) (C1 ('MetaCons "DiskSnapshot" 'PrefixI 'True) (S1 ('MetaSel ('Just "dsNumber") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedLazy) (Rec0 Word64) :*: S1 ('MetaSel ('Just "dsSuffix") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedLazy) (Rec0 (Maybe String))))

data MetadataErr Source #

Constructors

MetadataFileDoesNotExist

The metadata file does not exist

MetadataInvalid String

The metadata file is invalid and does not deserialize

MetadataBackendMismatch

The metadata file has the incorrect backend

data NumOfDiskSnapshots Source #

Number of snapshots to be stored on disk. This is either the default value as determined by the SnapshotPolicy, or it is provided by the user. See the SnapshotPolicy documentation for more information.

Instances

Instances details
Generic NumOfDiskSnapshots Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.Snapshots

Associated Types

type Rep NumOfDiskSnapshots 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.Snapshots

type Rep NumOfDiskSnapshots = D1 ('MetaData "NumOfDiskSnapshots" "Ouroboros.Consensus.Storage.LedgerDB.Snapshots" "ouroboros-consensus-0.26.0.0-inplace" 'False) (C1 ('MetaCons "DefaultNumOfDiskSnapshots" 'PrefixI 'False) (U1TypeType) :+: C1 ('MetaCons "RequestedNumOfDiskSnapshots" 'PrefixI 'False) (S1 ('MetaSel ('NothingMaybe Symbol) 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedLazy) (Rec0 Word)))
Show NumOfDiskSnapshots Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.Snapshots

Eq NumOfDiskSnapshots Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.Snapshots

type Rep NumOfDiskSnapshots Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.Snapshots

type Rep NumOfDiskSnapshots = D1 ('MetaData "NumOfDiskSnapshots" "Ouroboros.Consensus.Storage.LedgerDB.Snapshots" "ouroboros-consensus-0.26.0.0-inplace" 'False) (C1 ('MetaCons "DefaultNumOfDiskSnapshots" 'PrefixI 'False) (U1TypeType) :+: C1 ('MetaCons "RequestedNumOfDiskSnapshots" 'PrefixI 'False) (S1 ('MetaSel ('NothingMaybe Symbol) 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedLazy) (Rec0 Word)))

data ReadSnapshotErr Source #

Constructors

ReadSnapshotFailed ReadIncrementalErr

Error while de-serialising data

ReadSnapshotDataCorruption

Checksum of read snapshot differs from the one tracked by its corresponding metadata file

ReadMetadataError FsPath MetadataErr

An error occurred while reading the snapshot metadata file

data SnapshotFailure blk Source #

Constructors

InitFailureRead ReadSnapshotErr

We failed to deserialise the snapshot

This can happen due to data corruption in the ledger DB or if the codecs changed.

InitFailureTooRecent (RealPoint blk)

This snapshot is too recent (ahead of the tip of the immutable chain)

InitFailureGenesis

This snapshot was of the ledger state at genesis, even though we never take snapshots at genesis, so this is unexpected.

Instances

Instances details
Generic (SnapshotFailure blk) Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.Snapshots

Associated Types

type Rep (SnapshotFailure blk) 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.Snapshots

type Rep (SnapshotFailure blk) = D1 ('MetaData "SnapshotFailure" "Ouroboros.Consensus.Storage.LedgerDB.Snapshots" "ouroboros-consensus-0.26.0.0-inplace" 'False) (C1 ('MetaCons "InitFailureRead" 'PrefixI 'False) (S1 ('MetaSel ('NothingMaybe Symbol) 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedLazy) (Rec0 ReadSnapshotErr)) :+: (C1 ('MetaCons "InitFailureTooRecent" 'PrefixI 'False) (S1 ('MetaSel ('NothingMaybe Symbol) 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedLazy) (Rec0 (RealPoint blk))) :+: C1 ('MetaCons "InitFailureGenesis" 'PrefixI 'False) (U1TypeType)))

Methods

fromSnapshotFailure blk → Rep (SnapshotFailure blk) x #

toRep (SnapshotFailure blk) x → SnapshotFailure blk #

StandardHash blk ⇒ Show (SnapshotFailure blk) Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.Snapshots

Methods

showsPrecIntSnapshotFailure blk → ShowS #

showSnapshotFailure blk → String #

showList ∷ [SnapshotFailure blk] → ShowS #

StandardHash blk ⇒ Eq (SnapshotFailure blk) Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.Snapshots

Methods

(==)SnapshotFailure blk → SnapshotFailure blk → Bool #

(/=)SnapshotFailure blk → SnapshotFailure blk → Bool #

type Rep (SnapshotFailure blk) Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.Snapshots

type Rep (SnapshotFailure blk) = D1 ('MetaData "SnapshotFailure" "Ouroboros.Consensus.Storage.LedgerDB.Snapshots" "ouroboros-consensus-0.26.0.0-inplace" 'False) (C1 ('MetaCons "InitFailureRead" 'PrefixI 'False) (S1 ('MetaSel ('NothingMaybe Symbol) 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedLazy) (Rec0 ReadSnapshotErr)) :+: (C1 ('MetaCons "InitFailureTooRecent" 'PrefixI 'False) (S1 ('MetaSel ('NothingMaybe Symbol) 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedLazy) (Rec0 (RealPoint blk))) :+: C1 ('MetaCons "InitFailureGenesis" 'PrefixI 'False) (U1TypeType)))

Codec

readExtLedgerState ∷ ∀ (m ∷ TypeType) blk. IOLike m ⇒ SomeHasFS m → (∀ s. Decoder s (ExtLedgerState blk EmptyMK)) → (∀ s. Decoder s (HeaderHash blk)) → FsPathExceptT ReadIncrementalErr m (ExtLedgerState blk EmptyMK, CRC) Source #

Read an extended ledger state from disk

writeExtLedgerStateMonadThrow m ⇒ SomeHasFS m → (ExtLedgerState blk EmptyMKEncoding) → FsPathExtLedgerState blk EmptyMK → m CRC Source #

Write an extended ledger state to disk

Paths

diskSnapshotIsTemporaryDiskSnapshotBool Source #

The snapshots that are periodically created are temporary, they will be deleted when trimming or if they fail to deserialize.

snapshotToDirPathDiskSnapshotFsPath Source #

The path within the LedgerDB's filesystem to the snapshot's directory

Management

deleteSnapshot ∷ (Monad m, HasCallStack) ⇒ SomeHasFS m → DiskSnapshot → m () Source #

Delete snapshot from disk

listSnapshotsMonad m ⇒ SomeHasFS m → m [DiskSnapshot] Source #

List on-disk snapshots, highest number first.

loadSnapshotMetadata ∷ ∀ (m ∷ TypeType). IOLike m ⇒ SomeHasFS m → DiskSnapshotExceptT MetadataErr m SnapshotMetadata Source #

Load a snapshot metadata JSON file.

trimSnapshotsMonad m ⇒ Tracer m (TraceSnapshotEvent r) → SomeHasFS m → SnapshotPolicy → m [DiskSnapshot] Source #

Trim the number of on disk snapshots so that at most onDiskNumSnapshots snapshots are stored on disk. The oldest snapshots are deleted.

The deleted snapshots are returned.

writeSnapshotMetadataMonadThrow m ⇒ SomeHasFS m → DiskSnapshotSnapshotMetadata → m () Source #

Write a snapshot metadata JSON file.

Policy

data SnapshotInterval Source #

Length of time that has to pass after which a snapshot is taken.

Instances

Instances details
Generic SnapshotInterval Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.Snapshots

Associated Types

type Rep SnapshotInterval 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.Snapshots

type Rep SnapshotInterval = D1 ('MetaData "SnapshotInterval" "Ouroboros.Consensus.Storage.LedgerDB.Snapshots" "ouroboros-consensus-0.26.0.0-inplace" 'False) (C1 ('MetaCons "DefaultSnapshotInterval" 'PrefixI 'False) (U1TypeType) :+: (C1 ('MetaCons "RequestedSnapshotInterval" 'PrefixI 'False) (S1 ('MetaSel ('NothingMaybe Symbol) 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedLazy) (Rec0 DiffTime)) :+: C1 ('MetaCons "DisableSnapshots" 'PrefixI 'False) (U1TypeType)))
Show SnapshotInterval Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.Snapshots

Eq SnapshotInterval Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.Snapshots

type Rep SnapshotInterval Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.Snapshots

type Rep SnapshotInterval = D1 ('MetaData "SnapshotInterval" "Ouroboros.Consensus.Storage.LedgerDB.Snapshots" "ouroboros-consensus-0.26.0.0-inplace" 'False) (C1 ('MetaCons "DefaultSnapshotInterval" 'PrefixI 'False) (U1TypeType) :+: (C1 ('MetaCons "RequestedSnapshotInterval" 'PrefixI 'False) (S1 ('MetaSel ('NothingMaybe Symbol) 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedLazy) (Rec0 DiffTime)) :+: C1 ('MetaCons "DisableSnapshots" 'PrefixI 'False) (U1TypeType)))

data SnapshotPolicy Source #

Snapshots policy

We only write ledger states that are older than k blocks to disk (that is, snapshots that are guaranteed valid). The on-disk policy determines how often we write to disk and how many checkpoints we keep.

Constructors

SnapshotPolicy 

Fields

  • onDiskNumSnapshotsWord

    How many snapshots do we want to keep on disk?

    A higher number of on-disk snapshots is primarily a safe-guard against disk corruption: it trades disk space for reliability.

    Examples:

    • 0: Delete the snapshot immediately after writing. Probably not a useful value :-D
    • 1: Delete the previous snapshot immediately after writing the next Dangerous policy: if for some reason the deletion happens before the new snapshot is written entirely to disk (we don't fsync), we have no choice but to start at the genesis snapshot on the next startup.
    • 2: Always keep 2 snapshots around. This means that when we write the next snapshot, we delete the oldest one, leaving the middle one available in case of truncation of the write. This is probably a sane value in most circumstances.
  • onDiskShouldTakeSnapshotMaybe DiffTimeWord64Bool

    Should we write a snapshot of the ledger state to disk?

    This function is passed two bits of information:

    • The time since the last snapshot, or NoSnapshotTakenYet if none was taken yet. Note that NoSnapshotTakenYet merely means no snapshot had been taking yet since the node was started; it does not necessarily mean that none exist on disk.
    • The distance in terms of blocks applied to the oldest ledger snapshot in memory. During normal operation, this is the number of blocks written to the ImmutableDB since the last snapshot. On startup, it is computed by counting how many immutable blocks we had to reapply to get to the chain tip. This is useful, as it allows the policy to decide to take a snapshot on node startup if a lot of blocks had to be replayed.

    See also defaultSnapshotPolicy

defaultSnapshotPolicySecurityParamSnapshotPolicyArgsSnapshotPolicy Source #

Default on-disk policy suitable to use with cardano-node

pattern DoDiskSnapshotChecksumFlag "DoDiskSnapshotChecksum" Source #

Type-safe flag to regulate the checksum policy of the ledger state snapshots.

These patterns are exposed to cardano-node and will be passed as part of SnapshotPolicy.

pattern NoDoDiskSnapshotChecksumFlag "DoDiskSnapshotChecksum" Source #

Type-safe flag to regulate the checksum policy of the ledger state snapshots.

These patterns are exposed to cardano-node and will be passed as part of SnapshotPolicy.

Tracing

data TraceSnapshotEvent blk Source #

Constructors

InvalidSnapshot DiskSnapshot (SnapshotFailure blk)

An on disk snapshot was skipped because it was invalid.

TookSnapshot DiskSnapshot (RealPoint blk) EnclosingTimed

A snapshot was written to disk.

DeletedSnapshot DiskSnapshot

An old or invalid on-disk snapshot was deleted

Instances

Instances details
Generic (TraceSnapshotEvent blk) Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.Snapshots

StandardHash blk ⇒ Show (TraceSnapshotEvent blk) Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.Snapshots

StandardHash blk ⇒ Eq (TraceSnapshotEvent blk) Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.Snapshots

type Rep (TraceSnapshotEvent blk) Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.Snapshots

Re-exports

newtype Flag (name ∷ Symbol) Source #

Type-safe boolean flags with type level tags

It is recommended to create pattern synonyms for the true and false values.

See DiskSnapshotChecksum for an example.

Constructors

Flag 

Fields

Instances

Instances details
Generic (Flag name) Source # 
Instance details

Defined in Ouroboros.Consensus.Util

Associated Types

type Rep (Flag name) 
Instance details

Defined in Ouroboros.Consensus.Util

type Rep (Flag name) = D1 ('MetaData "Flag" "Ouroboros.Consensus.Util" "ouroboros-consensus-0.26.0.0-inplace" 'True) (C1 ('MetaCons "Flag" 'PrefixI 'True) (S1 ('MetaSel ('Just "getFlag") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedLazy) (Rec0 Bool)))

Methods

fromFlag name → Rep (Flag name) x #

toRep (Flag name) x → Flag name #

Show (Flag name) Source # 
Instance details

Defined in Ouroboros.Consensus.Util

Methods

showsPrecIntFlag name → ShowS #

showFlag name → String #

showList ∷ [Flag name] → ShowS #

Eq (Flag name) Source # 
Instance details

Defined in Ouroboros.Consensus.Util

Methods

(==)Flag name → Flag name → Bool #

(/=)Flag name → Flag name → Bool #

type Rep (Flag name) Source # 
Instance details

Defined in Ouroboros.Consensus.Util

type Rep (Flag name) = D1 ('MetaData "Flag" "Ouroboros.Consensus.Util" "ouroboros-consensus-0.26.0.0-inplace" 'True) (C1 ('MetaCons "Flag" 'PrefixI 'True) (S1 ('MetaSel ('Just "getFlag") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedLazy) (Rec0 Bool)))

Testing

decodeLBackwardsCompatible ∷ ∀ l blk. Proxy blk → (∀ s. Decoder s l) → (∀ s. Decoder s (HeaderHash blk)) → ∀ s. Decoder s l Source #

To remain backwards compatible with existing snapshots stored on disk, we must accept the old format as well as the new format.

The old format:

  • The tip: WithOrigin (RealPoint blk)
  • The chain length: Word64
  • The ledger state: l

The new format is described by snapshotEncodingVersion1.

This decoder will accept and ignore them. The encoder (encodeSnapshot) will no longer encode them.

destroySnapshotsMonad m ⇒ SomeHasFS m → m () Source #

Testing only! Destroy all snapshots in the DB.

encodeL ∷ (l → Encoding) → l → Encoding Source #

Encoder to be used in combination with decodeSnapshotBackwardsCompatible.

snapshotsMapM_Monad m ⇒ SomeHasFS m → (FilePath → m a) → m () Source #