Safe Haskell | Safe-Inferred |
---|---|
Language | Haskell2010 |
The mempool API and implementation.
Synopsis
- data Mempool m blk = Mempool {
- addTx ∷ AddTxOnBehalfOf → GenTx blk → m (MempoolAddTxResult blk)
- removeTxs ∷ [GenTxId blk] → m ()
- syncWithLedger ∷ m (MempoolSnapshot blk)
- getSnapshot ∷ STM m (MempoolSnapshot blk)
- getSnapshotFor ∷ ForgeLedgerState blk → STM m (MempoolSnapshot blk)
- getCapacity ∷ STM m MempoolCapacityBytes
- getTxSize ∷ GenTx blk → TxSizeInBytes
- data MempoolAddTxResult blk
- = MempoolTxAdded !(Validated (GenTx blk))
- | MempoolTxRejected !(GenTx blk) !(ApplyTxErr blk)
- addLocalTxs ∷ ∀ m blk t. (MonadSTM m, Traversable t) ⇒ Mempool m blk → t (GenTx blk) → m (t (MempoolAddTxResult blk))
- addTxs ∷ ∀ m blk t. (MonadSTM m, Traversable t) ⇒ Mempool m blk → t (GenTx blk) → m (t (MempoolAddTxResult blk))
- isMempoolTxAdded ∷ MempoolAddTxResult blk → Bool
- isMempoolTxRejected ∷ MempoolAddTxResult blk → Bool
- mempoolTxAddedToMaybe ∷ MempoolAddTxResult blk → Maybe (Validated (GenTx blk))
- data ForgeLedgerState blk
- = ForgeInKnownSlot SlotNo (TickedLedgerState blk)
- | ForgeInUnknownSlot (LedgerState blk)
- data MempoolSnapshot blk = MempoolSnapshot {
- snapshotTxs ∷ [(Validated (GenTx blk), TicketNo)]
- snapshotTxsAfter ∷ TicketNo → [(Validated (GenTx blk), TicketNo)]
- snapshotLookupTx ∷ TicketNo → Maybe (Validated (GenTx blk))
- snapshotHasTx ∷ GenTxId blk → Bool
- snapshotMempoolSize ∷ MempoolSize
- snapshotSlotNo ∷ SlotNo
- snapshotLedgerState ∷ TickedLedgerState blk
- data TicketNo
- type TxSizeInBytes = Word32
- zeroTicketNo ∷ TicketNo
- newtype MempoolCapacityBytes = MempoolCapacityBytes {}
- data MempoolCapacityBytesOverride
- computeMempoolCapacity ∷ LedgerSupportsMempool blk ⇒ TickedLedgerState blk → MempoolCapacityBytesOverride → MempoolCapacityBytes
- data MempoolSize = MempoolSize {
- msNumTxs ∷ !Word32
- msNumBytes ∷ !Word32
- newtype ByteSize = ByteSize {}
- class BoundedMeasure (TxMeasure blk) ⇒ TxLimits blk where
- type TxMeasure blk
- txMeasure ∷ Validated (GenTx blk) → TxMeasure blk
- txsBlockCapacity ∷ Ticked (LedgerState blk) → TxMeasure blk
- data TxOverrides blk
- applyOverrides ∷ TxLimits blk ⇒ TxOverrides blk → TxMeasure blk → TxMeasure blk
- getOverrides ∷ TxOverrides blk → TxMeasure blk
- mkOverrides ∷ TxMeasure blk → TxOverrides blk
- noOverridesMeasure ∷ BoundedMeasure a ⇒ a
- openMempool ∷ (IOLike m, LedgerSupportsMempool blk, HasTxId (GenTx blk), ValidateEnvelope blk) ⇒ ResourceRegistry m → LedgerInterface m blk → LedgerConfig blk → MempoolCapacityBytesOverride → Tracer m (TraceEventMempool blk) → (GenTx blk → TxSizeInBytes) → m (Mempool m blk)
- openMempoolWithoutSyncThread ∷ (IOLike m, LedgerSupportsMempool blk, HasTxId (GenTx blk), ValidateEnvelope blk) ⇒ LedgerInterface m blk → LedgerConfig blk → MempoolCapacityBytesOverride → Tracer m (TraceEventMempool blk) → (GenTx blk → TxSizeInBytes) → m (Mempool m blk)
- data LedgerInterface m blk = LedgerInterface {
- getCurrentLedgerState ∷ STM m (LedgerState blk)
- chainDBLedgerInterface ∷ (IOLike m, IsLedger (LedgerState blk)) ⇒ ChainDB m blk → LedgerInterface m blk
- data TraceEventMempool blk
- = TraceMempoolAddedTx (Validated (GenTx blk)) MempoolSize MempoolSize
- | TraceMempoolRejectedTx (GenTx blk) (ApplyTxErr blk) MempoolSize
- | TraceMempoolRemoveTxs [(Validated (GenTx blk), ApplyTxErr blk)] MempoolSize
- | TraceMempoolManuallyRemovedTxs [GenTxId blk] [Validated (GenTx blk)] MempoolSize
Mempool API
Mempool
Mempool
The mempool is the set of transactions that should be included in the next block. In principle this is a set of all the transactions that we receive from our peers. In order to avoid flooding the network with invalid transactions, however, we only want to keep valid transactions in the mempool. That raises the question: valid with respect to which ledger state?
We opt for a very simple answer to this: the mempool will be interpreted as a list of transactions; which are validated strictly in order, starting from the current ledger state. This has a number of advantages:
- It's simple to implement and it's efficient. In particular, no search for a valid subset is ever required.
- When producing a block, we can simply take the longest possible prefix of transactions that fits in a block.
- It supports wallets that submit dependent transactions (where later transaction depends on outputs from earlier ones).
The mempool provides fairness guarantees for the case of multiple threads
performing addTx
concurrently. Implementations of this interface must
provide this guarantee, and users of this interface may rely on it.
Specifically, multiple threads that continuously use addTx
will, over
time, get a share of the mempool resource (measured by the number of txs
only, not their sizes) roughly proportional to their "weight". The weight
depends on the AddTxOnBehalfOf
: either acting on behalf of remote peers
(AddTxForRemotePeer
) or on behalf of a local client
(AddTxForLocalClient
). The weighting for threads acting on behalf of
remote peers is the same for all remote peers, so all remote peers will get
a roughly equal share of the resource. The weighting for local clients is
the same for all local clients but may be higher than the weighting for
remote peers. The weighting is not unboundedly higher however, so there is
still (weighted) fairness between remote peers and local clients. Thus
local clients will also get a roughly equal share of the resource, but that
share may be strictly greater than the share for each remote peer.
Furthermore, this implies local clients cannot starve remote peers, despite
their higher weighting.
This fairness specification in terms of weighting is deliberately non-specific, which allows multiple strategies. The existing default strategy (for the implementation in Ouroboros.Consensus.Mempool) is as follows. The design uses two FIFOs, to give strictly in-order behaviour. All remote peers get equal weight and all local clients get equal weight. The relative weight between remote and local is that if there are N remote peers and M local clients, each local client gets weight 1/(M+1), while all of the N remote peers together also get total weight 1/(M+1). This means individual remote peers get weight 1/(N * (M+1)). Intuitively: a single local client has the same weight as all the remote peers put together.
Mempool | |
|
Transaction adding
data MempoolAddTxResult blk Source #
The result of attempting to add a transaction to the mempool.
MempoolTxAdded !(Validated (GenTx blk)) | The transaction was added to the mempool. |
MempoolTxRejected !(GenTx blk) !(ApplyTxErr blk) | The transaction was rejected and could not be added to the mempool for the specified reason. |
Instances
(Show (GenTx blk), Show (Validated (GenTx blk)), Show (ApplyTxErr blk)) ⇒ Show (MempoolAddTxResult blk) Source # | |
Defined in Ouroboros.Consensus.Mempool.API showsPrec ∷ Int → MempoolAddTxResult blk → ShowS # show ∷ MempoolAddTxResult blk → String # showList ∷ [MempoolAddTxResult blk] → ShowS # | |
(Eq (GenTx blk), Eq (Validated (GenTx blk)), Eq (ApplyTxErr blk)) ⇒ Eq (MempoolAddTxResult blk) Source # | |
Defined in Ouroboros.Consensus.Mempool.API (==) ∷ MempoolAddTxResult blk → MempoolAddTxResult blk → Bool # (/=) ∷ MempoolAddTxResult blk → MempoolAddTxResult blk → Bool # |
addLocalTxs ∷ ∀ m blk t. (MonadSTM m, Traversable t) ⇒ Mempool m blk → t (GenTx blk) → m (t (MempoolAddTxResult blk)) Source #
addTxs ∷ ∀ m blk t. (MonadSTM m, Traversable t) ⇒ Mempool m blk → t (GenTx blk) → m (t (MempoolAddTxResult blk)) Source #
isMempoolTxAdded ∷ MempoolAddTxResult blk → Bool Source #
isMempoolTxRejected ∷ MempoolAddTxResult blk → Bool Source #
mempoolTxAddedToMaybe ∷ MempoolAddTxResult blk → Maybe (Validated (GenTx blk)) Source #
Ledger state to forge on top of
data ForgeLedgerState blk Source #
The ledger state wrt to which we should produce a block
The transactions in the mempool will be part of the body of a block, but a block consists of a header and a body, and the full validation of a block consists of first processing its header and only then processing the body. This is important, because processing the header may change the state of the ledger: the update system might be updated, scheduled delegations might be applied, etc., and such changes should take effect before we validate any transactions.
ForgeInKnownSlot SlotNo (TickedLedgerState blk) | The slot number of the block is known This will only be the case when we realized that we are the slot leader
and we are actually producing a block. It is the caller's responsibility
to call |
ForgeInUnknownSlot (LedgerState blk) | The slot number of the block is not yet known When we are validating transactions before we know in which block they
will end up, we have to make an assumption about which slot number to use
for |
Mempool Snapshot
data MempoolSnapshot blk Source #
A pure snapshot of the contents of the mempool. It allows fetching information about transactions in the mempool, and fetching individual transactions.
This uses a transaction sequence number type for identifying transactions within the mempool sequence. The sequence number is local to this mempool, unlike the transaction hash. This allows us to ask for all transactions after a known sequence number, to get new transactions. It is also used to look up individual transactions.
Note that it is expected that getTx
will often return Nothing
even for tx sequence numbers returned in previous snapshots. This happens
when the transaction has been removed from the mempool between snapshots.
MempoolSnapshot | |
|
Re-exports
We allocate each transaction a (monotonically increasing) ticket number as it enters the mempool.
Instances
Bounded TicketNo Source # | |
Enum TicketNo Source # | |
Show TicketNo Source # | |
Eq TicketNo Source # | |
Ord TicketNo Source # | |
Defined in Ouroboros.Consensus.Mempool.TxSeq | |
NoThunks TicketNo Source # | |
type TxSizeInBytes = Word32 Source #
Transactions are typically not big, but in principle in future we could have ones over 64k large.
zeroTicketNo ∷ TicketNo Source #
The transaction ticket number from which our counter starts.
Mempool capacity
newtype MempoolCapacityBytes Source #
Represents the maximum number of bytes worth of transactions that a
Mempool
can contain.
Instances
data MempoolCapacityBytesOverride Source #
An override for the default MempoolCapacityBytes
which is 2x the
maximum transaction capacity
NoMempoolCapacityBytesOverride | Use 2x the maximum transaction capacity of a block. This will change dynamically with the protocol parameters adopted in the current ledger. |
MempoolCapacityBytesOverride !MempoolCapacityBytes | Use the following |
computeMempoolCapacity ∷ LedgerSupportsMempool blk ⇒ TickedLedgerState blk → MempoolCapacityBytesOverride → MempoolCapacityBytes Source #
If no override is provided, calculate the default mempool capacity as 2x the current ledger's maximum transaction capacity of a block.
Mempool Size
data MempoolSize Source #
The size of a mempool.
MempoolSize | |
|
Instances
Monoid MempoolSize Source # | |
Defined in Ouroboros.Consensus.Mempool.Capacity mappend ∷ MempoolSize → MempoolSize → MempoolSize # mconcat ∷ [MempoolSize] → MempoolSize # | |
Semigroup MempoolSize Source # | |
Defined in Ouroboros.Consensus.Mempool.Capacity (<>) ∷ MempoolSize → MempoolSize → MempoolSize # sconcat ∷ NonEmpty MempoolSize → MempoolSize # stimes ∷ Integral b ⇒ b → MempoolSize → MempoolSize # | |
Show MempoolSize Source # | |
Defined in Ouroboros.Consensus.Mempool.Capacity showsPrec ∷ Int → MempoolSize → ShowS # show ∷ MempoolSize → String # showList ∷ [MempoolSize] → ShowS # | |
Eq MempoolSize Source # | |
Defined in Ouroboros.Consensus.Mempool.Capacity (==) ∷ MempoolSize → MempoolSize → Bool # (/=) ∷ MempoolSize → MempoolSize → Bool # |
Transaction size
class BoundedMeasure (TxMeasure blk) ⇒ TxLimits blk where Source #
Each block has its limits of how many transactions it can hold. That limit is compared against the sum of measurements taken of each of the transactions in that block.
How we measure the transaction depends of the era that this transaction belongs to (more specifically it depends on the block type to which this transaction will be added). For initial eras (like Byron and initial generations of Shelley based eras) this measure was simply a ByteSize (block could not be bigger then given size - in bytes - specified by the ledger state). In future eras (starting with Alonzo) this measure was a bit more complex as it had to take other factors into account (like execution units). For details please see the individual instances for the TxLimits.
Restricting more strongly than the ledger's limits
data TxOverrides blk Source #
An override that lowers a capacity limit
Specifically, we use this override to let the node operator limit the total
TxMeasure
of transactions in blocks even more severely than would the
ledger state's txsBlockCapacity
. The forge logic will use the min
(ie the lattice's meet
operator) to combine this override with the capacity
given by the ledger state. More concretely, that will typically be a
componentwise minimum operation, along each of the components/dimensions of
.TxMeasure
blk
This newtype wrapper distinguishes the intention of this particular
TxMeasure
as such an override. We use TxMeasure
in different ways in this
code base. The newtype also allows us to distinguish the one most appropriate
monoid among many offered by the TxLimits
superclass constraints: it is the
monoid induced by the bounded meet-semilattice (see BoundedMeasure
) that is
relevant to the notion of overriding the ledger's block capacity.
Instances
TxLimits blk ⇒ Monoid (TxOverrides blk) Source # | |
Defined in Ouroboros.Consensus.Mempool.Capacity mempty ∷ TxOverrides blk # mappend ∷ TxOverrides blk → TxOverrides blk → TxOverrides blk # mconcat ∷ [TxOverrides blk] → TxOverrides blk # | |
TxLimits blk ⇒ Semigroup (TxOverrides blk) Source # | |
Defined in Ouroboros.Consensus.Mempool.Capacity (<>) ∷ TxOverrides blk → TxOverrides blk → TxOverrides blk # sconcat ∷ NonEmpty (TxOverrides blk) → TxOverrides blk # stimes ∷ Integral b ⇒ b → TxOverrides blk → TxOverrides blk # |
applyOverrides ∷ TxLimits blk ⇒ TxOverrides blk → TxMeasure blk → TxMeasure blk Source #
Apply the override
getOverrides ∷ TxOverrides blk → TxMeasure blk Source #
mkOverrides ∷ TxMeasure blk → TxOverrides blk Source #
Smart constructor for Overrides
.
noOverridesMeasure ∷ BoundedMeasure a ⇒ a Source #
applyOverrides
noOverrides
m = m
Mempool initialization
openMempool ∷ (IOLike m, LedgerSupportsMempool blk, HasTxId (GenTx blk), ValidateEnvelope blk) ⇒ ResourceRegistry m → LedgerInterface m blk → LedgerConfig blk → MempoolCapacityBytesOverride → Tracer m (TraceEventMempool blk) → (GenTx blk → TxSizeInBytes) → m (Mempool m blk) Source #
Create a Mempool m blk
in m
to manipulate the mempool. It will also
fork a thread that syncs the mempool and the ledger when the ledger changes.
openMempoolWithoutSyncThread ∷ (IOLike m, LedgerSupportsMempool blk, HasTxId (GenTx blk), ValidateEnvelope blk) ⇒ LedgerInterface m blk → LedgerConfig blk → MempoolCapacityBytesOverride → Tracer m (TraceEventMempool blk) → (GenTx blk → TxSizeInBytes) → m (Mempool m blk) Source #
Unlike openMempool
, this function does not fork a background thread
that synchronises with the ledger state whenever the later changes.
Intended for testing purposes.
ChainDB interface
data LedgerInterface m blk Source #
Abstract interface needed to run a Mempool.
LedgerInterface | |
|
chainDBLedgerInterface ∷ (IOLike m, IsLedger (LedgerState blk)) ⇒ ChainDB m blk → LedgerInterface m blk Source #
Create a LedgerInterface
from a ChainDB
.
Trace
data TraceEventMempool blk Source #
Events traced by the Mempool.
TraceMempoolAddedTx | |
| |
TraceMempoolRejectedTx | |
| |
TraceMempoolRemoveTxs | |
| |
TraceMempoolManuallyRemovedTxs | |
|
Instances
(Show (GenTx blk), Show (Validated (GenTx blk)), Show (GenTxId blk), Show (ApplyTxErr blk)) ⇒ Show (TraceEventMempool blk) Source # | |
Defined in Ouroboros.Consensus.Mempool.Impl.Common showsPrec ∷ Int → TraceEventMempool blk → ShowS # show ∷ TraceEventMempool blk → String # showList ∷ [TraceEventMempool blk] → ShowS # | |
(Eq (GenTx blk), Eq (Validated (GenTx blk)), Eq (GenTxId blk), Eq (ApplyTxErr blk)) ⇒ Eq (TraceEventMempool blk) Source # | |
Defined in Ouroboros.Consensus.Mempool.Impl.Common (==) ∷ TraceEventMempool blk → TraceEventMempool blk → Bool # (/=) ∷ TraceEventMempool blk → TraceEventMempool blk → Bool # |