This isn't necessary. Because we just log the error, we never match off of it,
we don't need any structure beyond String (or now Debug, which still gives us
a way to print the error). This is for the ergonomics of not having to
constantly write `.map_err(|e| format!("{e:?}"))`.
coordinator/tributary was tributary-chain. This crate has been renamed
tributary-sdk and moved to coordinator/tributary-sdk.
coordinator/src/tributary was our instantion of a Tributary, the Transaction
type and scan task. This has been moved to coordinator/tributary.
The main reason for this was due to coordinator/main.rs becoming untidy. There
is now a collection of clean, independent APIs present in the codebase.
coordinator/main.rs is to compose them. Sometimes, these compositions are a bit
silly (reading from a channel just to forward the message to a distinct
channel). That's more than fine as the code is still readable and the value
from the cleanliness of the APIs composed far exceeds the nits from having
these odd compositions.
This breaks down a bit as we now define a global database, and have some APIs
interact with multiple other APIs.
coordinator/src/tributary was a self-contained, clean API. The recently added
task present in coordinator/tributary/mod.rs, which bound it to the rest of the
Coordinator, wasn't.
Now, coordinator/src is solely the API compositions, and all self-contained
APIs are their own crates.
* Upstream GBP, divisor, circuit abstraction, and EC gadgets from FCMP++
* Initial eVRF implementation
Not quite done yet. It needs to communicate the resulting points and proofs to
extract them from the Pedersen Commitments in order to return those, and then
be tested.
* Add the openings of the PCs to the eVRF as necessary
* Add implementation of secq256k1
* Make DKG Encryption a bit more flexible
No longer requires the use of an EncryptionKeyMessage, and allows pre-defined
keys for encryption.
* Make NUM_BITS an argument for the field macro
* Have the eVRF take a Zeroizing private key
* Initial eVRF-based DKG
* Add embedwards25519 curve
* Inline the eVRF into the DKG library
Due to how we're handling share encryption, we'd either need two circuits or to
dedicate this circuit to the DKG. The latter makes sense at this time.
* Add documentation to the eVRF-based DKG
* Add paragraph claiming robustness
* Update to the new eVRF proof
* Finish routing the eVRF functionality
Still needs errors and serialization, along with a few other TODOs.
* Add initial eVRF DKG test
* Improve eVRF DKG
Updates how we calculcate verification shares, improves performance when
extracting multiple sets of keys, and adds more to the test for it.
* Start using a proper error for the eVRF DKG
* Resolve various TODOs
Supports recovering multiple key shares from the eVRF DKG.
Inlines two loops to save 2**16 iterations.
Adds support for creating a constant time representation of scalars < NUM_BITS.
* Ban zero ECDH keys, document non-zero requirements
* Implement eVRF traits, all the way up to the DKG, for secp256k1/ed25519
* Add Ristretto eVRF trait impls
* Support participating multiple times in the eVRF DKG
* Only participate once per key, not once per key share
* Rewrite processor key-gen around the eVRF DKG
Still a WIP.
* Finish routing the new key gen in the processor
Doesn't touch the tests, coordinator, nor Substrate yet.
`cargo +nightly fmt && cargo +nightly-2024-07-01 clippy --all-features -p serai-processor`
does pass.
* Deduplicate and better document in processor key_gen
* Update serai-processor tests to the new key gen
* Correct amount of yx coefficients, get processor key gen test to pass
* Add embedded elliptic curve keys to Substrate
* Update processor key gen tests to the eVRF DKG
* Have set_keys take signature_participants, not removed_participants
Now no one is removed from the DKG. Only `t` people publish the key however.
Uses a BitVec for an efficient encoding of the participants.
* Update the coordinator binary for the new DKG
This does not yet update any tests.
* Add sensible Debug to key_gen::[Processor, Coordinator]Message
* Have the DKG explicitly declare how to interpolate its shares
Removes the hack for MuSig where we multiply keys by the inverse of their
lagrange interpolation factor.
* Replace Interpolation::None with Interpolation::Constant
Allows the MuSig DKG to keep the secret share as the original private key,
enabling deriving FROST nonces consistently regardless of the MuSig context.
* Get coordinator tests to pass
* Update spec to the new DKG
* Get clippy to pass across the repo
* cargo machete
* Add an extra sleep to ensure expected ordering of `Participation`s
* Update orchestration
* Remove bad panic in coordinator
It expected ConfirmationShare to be n-of-n, not t-of-n.
* Improve documentation on functions
* Update TX size limit
We now no longer have to support the ridiculous case of having 49 DKG
participations within a 101-of-150 DKG. It does remain quite high due to
needing to _sign_ so many times. It'd may be optimal for parties with multiple
key shares to independently send their preprocesses/shares (despite the
overhead that'll cause with signatures and the transaction structure).
* Correct error in the Processor spec document
* Update a few comments in the validator-sets pallet
* Send/Recv Participation one at a time
Sending all, then attempting to receive all in an expected order, wasn't working
even with notable delays between sending messages. This points to the mempool
not working as expected...
* Correct ThresholdKeys serialization in modular-frost test
* Updating existing TX size limit test for the new DKG parameters
* Increase time allowed for the DKG on the GH CI
* Correct construction of signature_participants in serai-client tests
Fault identified by akil.
* Further contextualize DkgConfirmer by ValidatorSet
Caught by a safety check we wouldn't reuse preprocesses across messages. That
raises the question of we were prior reusing preprocesses (reusing keys)?
Except that'd have caused a variety of signing failures (suggesting we had some
staggered timing avoiding it in practice but yes, this was possible in theory).
* Add necessary calls to set_embedded_elliptic_curve_key in coordinator set rotation tests
* Correct shimmed setting of a secq256k1 key
* cargo fmt
* Don't use `[0; 32]` for the embedded keys in the coordinator rotation test
The key_gen function expects the random values already decided.
* Big-endian secq256k1 scalars
Also restores the prior, safer, Encryption::register function.
* Clear upons upon round, not block
* Cache the proposal for a round
* Rebase onto develop, which reverted this PR, and re-apply this PR
* Set participation upon participation instead of constantly recalculating
* Cache message instances
* Add missing txn commit
Identified by @akildemir.
* Correct clippy lint identified upon rebase
* Fix tendermint chain sync (#581)
* fix p2p Reqres protocol
* stabilize tributary chain sync
* fix pr comments
---------
Co-authored-by: akildemir <34187742+akildemir@users.noreply.github.com>
* Rewrite tendermint's message handling loop to much more clearly match the paper
No longer checks relevant branches upon messages, yet all branches upon any
state change. This is slower, yet easier to review and likely without one or
two rare edge cases.
When reviewing, please see page 5 of https://arxiv.org/pdf/1807.04938.pdf.
Lines from the specified algorithm can be found in the code by searching for
"// L".
* Sane rebroadcasting of consensus messages
Instead of broadcasting the last n messages on the Tributary side of things, we
now have the machine rebroadcast the message tape for the current block.
* Only rebroadcast messages which didn't error in some way
* Only rebroadcast our own messages for tendermint
Online validators should inherently have them. Offline validators will receive
from the sync protocol.
This does somewhat eliminate the class of nodes who would follow the blockchain
(without validating it), yet that's fine for the performance benefit.
Part of https://github.com/serai-dex/serai/issues/345.
The lack of full DB persistence does mean enough nodes rebooting at the same
time may cause a halt. This will prevent slashes.
* complete various todos
* fix pr comments
* Document bounds on unique hashes in TransactionKind
---------
Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
* add reasons to slash evidence
* fix CI failing
* Remove unnecessary clones
.encode() takes &self
* InvalidVr to InvalidValidRound
* Unrelated to this PR: Clarify reasoning/potentials behind dropping evidence
* Clarify prevotes in SlashEvidence test
* Replace use of read_to_end
* Restore decode_signed_message
---------
Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
* Update the coordinator to give key shares based on weight, not based on existence
Participants are now identified by their starting index. While this compiles,
the following is unimplemented:
1) A conversion for DKG `i` values. It assumes the threshold `i` values used
will be identical for the MuSig signature used to confirm the DKG.
2) Expansion from compressed values to full values before forwarding to the
processor.
* Add a fn to the DkgConfirmer to convert `i` values as needed
Also removes TODOs regarding Serai ensuring validator key uniqueness +
validity. The current infra achieves both.
* Have the Tributary DB track participation by shares, not by count
* Prevent a node from obtaining 34% of the maximum amount of key shares
This is actually mainly intended to set a bound on message sizes in the
coordinator. Message sizes are amplified by the amount of key shares held, so
setting an upper bound on said amount lets it determine constants. While that
upper bound could be 150, that'd be unreasonable and increase the potential for
DoS attacks.
* Correct the mechanism to detect if sufficient accumulation has occured
It used to check if the latest accumulation hit the required threshold. Now,
accumulations may jump past the required threshold. The required mechanism is
to check the threshold wasn't prior met and is now met.
* Finish updating the coordinator to handle a multiple key share per validator environment
* Adjust stategy re: preventing noce reuse in DKG Confirmer
* Add TODOs regarding dropped transactions, add possible TODO fix
* Update tests/coordinator
This doesn't add new multi-key-share tests, it solely updates the existing
single key-share tests to compile and run, with the necessary fixes to the
coordinator.
* Update processor key_gen to handle generating multiple key shares at once
* Update SubstrateSigner
* Update signer, clippy
* Update processor tests
* Update processor docker tests
* fix typos
* remove tributary sleeping
* handle not locally provided txs
* use topic number instead of waiting list
* Clean-up, fixes
1) Uses a single TXN in provided
2) Doesn't continue on non-local provided inside verify_block, skipping further
execution of checks
3) Upon local provision of already on-chain TX, compares
---------
Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
They used &mut self to prevent execution at the same time. This uses a lock
over the channel to achieve the same security, without requiring a lock over
the entire tributary.
This fixes post-provided Provided transactions. sync_block waited for the TX to
be provided, yet it never would as sync_block held a mutable reference over the
entire Tributary, preventing any other read/write operations of any scope.
A timeout increased (bc2f23f72b) due to this bug
not being identified has been decreased back, thankfully.
Also shims in basic support for Completed, which was the WIP before this bug
was identified.
* add slash tx
* ignore unsigned tx replays
* verify that provided evidence is valid
* fix clippy + fmt
* move application tx handling to another module
* partially handle the tendermint txs
* fix pr comments
* support unsigned app txs
* add slash target to the votes
* enforce provided, unsigned, signed tx ordering within a block
* bug fixes
* add unit test for tendermint txs
* bug fixes
* update tests for tendermint txs
* add tx ordering test
* tidy up tx ordering test
* cargo +nightly fmt
* Misc fixes from rebasing
* Finish resolving clippy
* Remove sha3 from tendermint-machine
* Resolve a DoS in SlashEvidence's read
Also moves Evidence from Vec<Message> to (Message, Option<Message>). That
should meet all requirements while being a bit safer.
* Make lazy_static a dev-depend for tributary
* Various small tweaks
One use of sort was inefficient, sorting unsigned || signed when unsigned was
already properly sorted. Given how the unsigned TXs were given a nonce of 0, an
unstable sort may swap places with an unsigned TX and a signed TX with a nonce
of 0 (leading to a faulty block).
The extra protection added here sorts signed, then concats.
* Fix Tributary tests I broke, start review on tendermint/tx.rs
* Finish reviewing everything outside tests and empty_signature
* Remove empty_signature
empty_signature led to corrupted local state histories. Unfortunately, the API
is only sane with a signature.
We now use the actual signature, which risks creating a signature over a
malicious message if we have ever have an invariant producing malicious
messages. Prior, we only signed the message after the local machine confirmed
it was okay per the local view of consensus.
This is tolerated/preferred over a corrupt state history since production of
such messages is already an invariant. TODOs are added to make handling of this
theoretical invariant further robust.
* Remove async_sequential for tokio::test
There was no competition for resources forcing them to be run sequentially.
* Modify block order test to be statistically significant without multiple runs
* Clean tests
---------
Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
It's largely unoptimized, and not yet exclusive to validators, yet has basic
sanity (using message content for ID instead of sender + index).
Fixes bugs as found. Notably, we used a time in milliseconds where the
Tributary expected seconds.
Also has Tributary::new jump to the presumed round number. This reduces slashes
when starting new chains (whose times will be before the current time) and was
the only way I was able to observe successful confirmations given current
surrounding infrastructure.
Reduces lock contention.
Additionally changes block_key to include the genesis. While not technically
needed, the lack of genesis introduced a side effect where any Tributary on the
the database could return the block of any other Tributary. While that wasn't a
security issue, returning it suggested it was on-chain when it wasn't. This may
have been usable to create issues.
We had a race condition where'd we be informed of blocks 1 .. 3, and
immediately add 1 .. 3. Because we immediately tried to add 2 after 1, it'd
fail since the tip was still the genesis, yet 2 needs the tip to be 1.
Adding a channel, while ugly, was the simplest way to accomplish this.
Also has any added block be broadcasted. Else there's a race condition where a
node which syncs up to the most recent block does so, yet fails to add the next
block when it's committed to.
This defines the tart of a very complex series of locks I'm really unhappy
with. At the same time, there's not immediately a better solution. This also
should work without issue.
Impls a LocalP2p for testing.
Moves rebroadcasting into Tendermint, since it's what knows if a message is
fully valid + original.
Removes TributarySpec::validators() HashMap, as its non-determinism caused
different instances to have different round robin schedules. It was already
prior moved to a Vec for this issue, so I'm unsure why this remnant existed.
Also renames the GH no-std workflow from the prior commit.
