* Schedule re-attempts and add a (not filled out) match statement to actually execute them
A comment explains the methodology. To copy it here:
"""
This is because we *always* re-attempt any protocol which had participation. That doesn't
mean we *should* re-attempt this protocol.
The alternatives were:
1) Note on-chain we completed a protocol, halting re-attempts upon 34%.
2) Vote on-chain to re-attempt a protocol.
This schema doesn't have any additional messages upon the success case (whereas
alternative #1 does) and doesn't have overhead (as alternative #2 does, sending votes and
then preprocesses. This only sends preprocesses).
"""
Any signing protocol which reaches sufficient participation will be
re-attempted until it no longer does.
* Have the Substrate scanner track DKG removals/completions for the Tributary code
* Don't keep trying to publish a participant removal if we've already set keys
* Pad out the re-attempt match a bit more
* Have CosignEvaluator reload from the DB
* Correctly schedule cosign re-attempts
* Actuall spawn new DKG removal attempts
* Use u32 for Batch ID in SubstrateSignableId, finish Batch re-attempt routing
The batch ID was an opaque [u8; 5] which also included the network, yet that's
redundant and unhelpful.
* Clarify a pair of TODOs in the coordinator
* Remove old TODO
* Final comment cleanup
* Correct usage of TARGET_BLOCK_TIME in reattempt scheduler
It's in ms and I assumed it was in s.
* Have coordinator tests drop BatchReattempts which aren't relevant yet may exist
* Bug fix and pointless oddity removal
We scheduled a re-attempt upon receiving 2/3rds of preprocesses and upon
receiving 2/3rds of shares, so any signing protocol could cause two re-attempts
(not one more).
The coordinator tests randomly generated the Batch ID since it was prior an
opaque byte array. While that didn't break the test, it was pointless and did
make the already-succeeded check before re-attempting impossible to hit.
* Add log statements, correct dead-lock in coordinator tests
* Increase pessimistic timeout on recv_message to compensate for tighter best-case timeouts
* Further bump timeout by a minute
AFAICT, GH failed by just a few seconds.
This also is worst-case in a single instance, making it fine to be decently long.
* Further further bump timeout due to lack of distinct error
* chore: implement create_db for substrate (fix broken branch)
* Correct rebase artifacts
* chore: remove todo statement
* chore: rename BlockDb to NextBlock
* chore: return empty tuple instead of empty array for event storage
* Finish rebasing
* .Minor tweaks to remove leftover variables
These may be rebase artifacts.
---------
Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
Uses a full-fledged serai-abi to do so.
Removes use of UncheckedExtrinsic as a pointlessly (for us) length-prefixed
block with a more complicated signing algorithm than advantageous.
In the future, we should considering consolidating the various primitives
crates. I'm not convinced we benefit from one primitives crate per pallet.
* Remove subxt
Removes ~20 crates from our Cargo.lock.
Removes downloading the metadata and enables removing the getMetadata RPC route
(relevant to #379).
Moves forward #337.
Done now due to distinctions in the subxt 0.32 API surface which make it
justifiable to not update.
* fmt, update due to deny triggering on a yanked crate
* Correct the handling of substrate_block_notifier now that it's ephemeral, not long-lived
* Correct URL in tests/coordinator from ws to http
* Add a function to deterministically decide which Serai blocks should be co-signed
Has a 5 minute latency between co-signs, also used as the maximal latency
before a co-sign is started.
* Get all active tributaries we're in at a specific block
* Add and route CosignSubstrateBlock, a new provided TX
* Split queued cosigns per network
* Rename BatchSignId to SubstrateSignId
* Add SubstrateSignableId, a meta-type for either Batch or Block, and modularize around it
* Handle the CosignSubstrateBlock provided TX
* Revert substrate_signer.rs to develop (and patch to still work)
Due to SubstrateSigner moving when the prior multisig closes, yet cosigning
occurring with the most recent key, a single SubstrateSigner can be reused.
We could manage multiple SubstrateSigners, yet considering the much lower
specifications for cosigning, I'd rather treat it distinctly.
* Route cosigning through the processor
* Add note to rename SubstrateSigner post-PR
I don't want to do so now in order to preserve the diff's clarity.
* Implement cosign evaluation into the coordinator
* Get tests to compile
* Bug fixes, mark blocks without cosigners available as cosigned
* Correct the ID Batch preprocesses are saved under, add log statements
* Create a dedicated function to handle cosigns
* Correct the flow around Batch verification/queueing
Verifying `Batch`s could stall when a `Batch` was signed before its
predecessors/before the block it's contained in was cosigned (the latter being
inevitable as we can't sign a block containing a signed batch before signing
the batch).
Now, Batch verification happens on a distinct async task in order to not block
the handling of processor messages. This task is the sole caller of verify in
order to ensure last_verified_batch isn't unexpectedly mutated.
When the processor message handler needs to access it, or needs to queue a
Batch, it associates the DB TXN with a lock preventing the other task from
doing so.
This lock, as currently implemented, is a poor and inefficient design. It
should be modified to the pattern used for cosign management. Additionally, a
new primitive of a DB-backed channel may be immensely valuable.
Fixes a standing potential deadlock and a deadlock introduced with the
cosigning protocol.
* Working full-stack tests
After the last commit, this only required extending a timeout.
* Replace "co-sign" with "cosign" to make finding text easier
* Update the coordinator tests to support cosigning
* Inline prior_batch calculation to prevent panic on rotation
Noticed when doing a final review of the branch.
