Documentation

For sqlScanCompleteEbPointsSince

The two conditions are decoupled across rows: the same EB hash can have several (ebSlot, ebHashBytes) rows (one per announcer slot), and missingTxCount is maintained per row on body insert but per hash on tx arrival, so the complete row and the recent row can differ. Requiring both on a single row would wrongly drop a complete EB re-announced recently (its recent row never got a body insert, so its missingTxCount is still NULL), leaving its cert-RB parked forever. Hence: keep a hash that has any complete row and any row at ebSlot >= ?.

Batch-filter txHashes via JSON1. Parameter is a JSON array of hex strings; 'unhex(je.value)' decodes back into a BLOB comparable against the indexed txs.txHashBytes column.

Find EBs that are now complete (missingTxCount reached 0). Volatile rows only: completion is decided within the one coherent volatile set.

Mark complete EBs as notified so they are not found again by sql_find_complete_ebs. Uses -1 as a sentinel for "already notified".

Decrement missingTxCount for every EB still waiting on the given txHash.

Uses ebsMissingTxs rather than ebTxs, which makes this more efficient than a full scan of ebTxs in the average case.

Must be paired with sql_delete_missing_txs in the same transaction.

Parameter 1: txHashBytes

Retire the waiting rows for a tx that has just landed. Parameter 1: txHashBytes

Record which of a freshly-inserted body's txs we do not yet hold.

One anti-join over the EB's own ebTxs range -- the same work sql_init_missing_tx_count used to do to produce a count, now materialised so that the arrival side reads the rows instead of recomputing them. Paying it here rather than on every tx arrival is what earns the index removal: this runs once per body, against ~4.7 times per tx for the old reverse lookup.

Parameter 1: ebHashBytes

Initialize missingTxCount after EB body is inserted, returning the resulting count. Counts ebTxs entries that don't yet have a corresponding tx in the txs table. The RETURNING clause lets the caller detect the special case missingTxCount = 0 (all referenced txs already present) with a PK lookup on the row that was just touched, instead of a full-table scan via sql_find_complete_ebs.

Parameters: 1 = ebHashBytes, 2 = ebHashBytes, 3 = ebSlot

Mark a specific EB as notified (missingTxCount = -1). PK-scoped variant of sql_mark_notified_ebs; used by sqlInsertEbBody when the body's arrival is what completed the closure.

Parameters: 1 = ebSlot, 2 = ebHashBytes

Batch retrieve of tx bytes for a batch of (ebHash, offset) points. ?1 is the ebHash blob (all offsets belong to the same EB); ?2 is a JSON int array of offsets. The join uses ebTxs' PK (ebHashBytes, txOffset), so index lookups still fire.

Every recent announcement, with or without completeness evidence. Companion of sql_scan_complete_ebs_since: the difference of the two sets is probed against the immutable partition (see sqlScanCompleteEbPointsSince).

Pin every announcement row of the EB: certified, awaiting copy, never evicted. The durable record that survives a crash before the copy lands. Also rescues GC-marked rows (status = 3): a promotion arriving between mark and sweep unmarks the hash, and the sweeper's all-marked pick then skips it.

Mark a pinned EB as copied (evictable). A volatile write, so it goes through the writer as MarkCopied -- only ever strictly after the immutable partition committed the EB's closure.

Body row count and closure row count of the EB in the volatile partition, in one probe: the LEFT JOIN's second count skips missing txs, so the EB's closure is complete iff both counts are equal (and non-zero).

Copy the EB's newest announcement row. Only the columns the immutable partition has (sql_schema_imm): completeness and GC status are volatile-only.

OR IGNORE: the mark that retires the pin is a separate volatile write, so a crash in between leaves the EB copied and still pinned, and the copier repeats the copy on the next start.

Copy the EB's body rows. OR IGNORE for the same reason as sql_copy_insert_eb.

Copy the EB's txs. OR IGNORE: a tx shared with an earlier-copied EB is already present.

Which of the given hashes (JSON hex array) the immutable partition holds. Presence is proof of a complete closure: copies land atomically and only complete EBs are copied.

Pinned EBs the copier has not marked as copied yet, for self-heal re-enqueueing. | The next EB waiting to be copied into the immutable partition: pinned (sql_pin_eb) but not yet marked copied (sql_mark_as_copied). Oldest first, and covered by idx_ebs_pinned.

The pin is the copy work list. It is in the database rather than in memory, so a copy interrupted by a crash is simply still pending on the next start.

Whether a GC at slot ?1 could has any work, i.e. if there are any old volatile EBs, already copied EBs or leftover EBs from an interrupted GC pass.

The markability predicate, shared by sql_gc_mark and sql_gc_stage_marked so the marked set and the staged set can never diverge. c is the row under test; it is markable if it is - old enough (its slot is before the GC frontier ?1) and - either volatile (status 0) or already copied (status 2) and - not vetoed by a live row of the same hash (pinned, or announced at or after the frontier).

Add the txs of every EB sql_gc_mark is about to hit as GC candidates. Must run strictly BEFORE sql_gc_mark in the same transaction: the UPDATE changes the status of EBs and orphans the transactions.

Mark for GC (status = 3) every row satisfying sql_gc_markable.

Up to ?1 GC-marked EBs ready to sweep.

Intuition: give me up to N hash values that are currently in status 3 and have never been assigned any status other than 3.

Note: 'FROM ebs cand' and 'FROM ebs live' allows referring to the rows of the ebs table using the alias cand and live.

TODO(geo2a): think how to simplify this query.

Evict the ebTxs rows with the specified ebHashBytes (a JSON array of byte strings).

Evict the ebsMissingTxs rows with the specified ebHashBytes (a JSON array of byte strings).

Evict the ebs rows with the specified ebHashBytes (a JSON array of byte strings).

Whether any GC-marked EBs remain.

Get up to ?1 transactions to be evicted.

Evict transactions with the specified hashes (a JSON array of byte strings), making sure that they are not referenced by any EBs.

Delete GC transaction candidates with the specified hashes (a JSON array of byte strings).

Whether the volatile partition holds any unstaged GC candidates (txs no EB references).

One keyset page of unstaged GC candidates (txs no EB references), for gcReinit: ?1 = cursor (exclusive), ?2 = page size.

Stage one page of GC candidates (JSON hex array ?1).