Documentation

For sqlScanCompleteEbPointsSince

The two conditions are decoupled across rows: the same EB hash can have several (ebSlot, ebHashBytes) rows (one per announcer slot), and missingTxCount is maintained per row on body insert but per hash on tx arrival, so the complete row and the recent row can differ. Requiring both on a single row would wrongly drop a complete EB re-announced recently (its recent row never got a body insert, so its missingTxCount is still NULL), leaving its cert-RB parked forever. Hence: keep a hash that has any complete row and any row at ebSlot >= ?.

Allocate the body's whole ebTxBytes range in one offset-ordered pass: one zeroblob row per body row, physically contiguous and fully packed. OR IGNORE: ignore duplicates (a second delivery of the same EB body).

Parameter 1: ebHashBytes

Fill one pre-allocated row in place. Guarded three ways: the row must exist (a bogus offset updates nothing), must not be filled yet (a duplicate delivery updates nothing), and the payload must have exactly the declared size (the row was allocated as zeroblob of it, and an in-place overwrite must not change the row size). One successful UPDATE is therefore exactly one previously-missing valid tx, which is what makes counting by changes() sound.

Parameters: 1 = ebHashBytes, 2 = txOffset, 3 = txBytes

Decrement missingTxCount on every ebs row with this EB body hash that already has a count, by the number of tx-bytes rows a batch actually filled, returning each touched row so the caller can spot the ones that just completed. Bytes are keyed by (ebHash, txOffset), so a fill can only ever close a hole in this EB.

Parameters: 1 = ebHashBytes, 2 = rows filled

Initialize missingTxCount after an EB body is inserted: the number of the EB's ebTxBytes rows not yet filled. RETURNING lets the caller detect missingTxCount = 0 with a PK lookup on the touched row.

Parameters: 1 = ebHashBytes, 2 = ebHashBytes, 3 = ebSlot

Mark a specific EB as notified (missingTxCount = -1), once a write completed its closure.

Parameters: 1 = ebSlot, 2 = ebHashBytes

Batch retrieve of tx bytes for a batch of (ebHash, offset) points. ?1 is the ebHash blob (all offsets belong to the same EB); ?2 is a JSON int array of offsets. The join uses ebTxs' PK (ebHashBytes, txOffset), so index lookups still fire.

A tx whose bytes have not arrived yet reads as NULL rather than as its zero-filled placeholder. So a request for txs this node does not have, such as a bad MsgLeiosBlockTxsRequest, is detected rather than answered with zeros.

The EB's txs in body order.

A tx whose bytes have not arrived yet reads as NULL rather than as its zero-filled placeholder, which sqlLookupEbClosure reports as an incomplete closure. That is what makes it fall back to the immutable partition, which holds the complete closure of an EB that was copied there and is now being fetched again into the volatile partition.

Every recent announcement, with or without completeness evidence. Companion of sql_scan_complete_ebs_since: the difference of the two sets is probed against the immutable partition (see sqlScanCompleteEbPointsSince).

Pin every announcement row of the EB: certified, awaiting copy, never evicted. The durable record that survives a crash before the copy lands. Also rescues GC-marked rows (status = 3): a promotion arriving between mark and sweep unmarks the hash, and the sweeper's all-marked pick then skips it.

Mark a pinned EB as copied (evictable). A volatile write, so it goes through the writer as MarkCopied -- only ever strictly after the immutable partition committed the EB's closure.

Body row count and closure row count of the EB in the volatile partition, in one probe: the EB's closure is complete iff both counts are equal (and non-zero). Two range COUNTs on the shared PK.

Copy the EB's newest announcement row. Only the columns the immutable partition has (sql_schema_imm): completeness and GC status are volatile-only.

OR IGNORE: the mark that retires the pin is a separate volatile write, so a crash in between leaves the EB copied and still pinned, and the copier repeats the copy on the next start.

Copy the EB's body rows. OR IGNORE for the same reason as sql_copy_insert_eb.

Copy the EB's tx bytes: a range copy on the shared PK. OR IGNORE for the same reason as sql_copy_insert_eb.

Which of the given hashes (JSON hex array) the immutable partition holds. Presence is proof of a complete closure: copies land atomically and only complete EBs are copied.

Pinned EBs the copier has not marked as copied yet, for self-heal re-enqueueing. | The next EB waiting to be copied into the immutable partition: pinned (sql_pin_eb) but not yet marked copied (sql_mark_as_copied). Oldest first, and covered by idx_ebs_pinned.

The pin is the copy work list. It is in the database rather than in memory, so a copy interrupted by a crash is simply still pending on the next start.

Whether a GC at slot ?1 could has any work, i.e. if there are any old volatile EBs, already copied EBs or leftover EBs from an interrupted GC pass.

The markability predicate of sql_gc_mark. c is the row under test; it is markable if it is - old enough (its slot is before the GC frontier ?1) and - either volatile (status 0) or already copied (status 2) and - not vetoed by a live row of the same hash (pinned, or announced at or after the frontier).

Mark for GC (status = 3) every row satisfying sql_gc_markable.

Up to ?1 GC-marked EBs ready to sweep.

Intuition: give me up to N hash values that are currently in status 3 and have never been assigned any status other than 3.

Note: 'FROM ebs cand' and 'FROM ebs live' allows referring to the rows of the ebs table using the alias cand and live.

TODO(geo2a): think how to simplify this query.

Evict the ebTxBytes rows with the specified ebHashBytes (a JSON array of byte strings).

Evict the ebTxs rows with the specified ebHashBytes (a JSON array of byte strings).

Evict the ebs rows with the specified ebHashBytes (a JSON array of byte strings).