* Update `build-dependencies` CI action
* Update `develop` to `patch-polkadot-sdk`
Allows us to finally remove the old `serai-dex/substrate` repository _and_
should have CI pass without issue on `develop` again.
The changes made here should be trivial and maintain all prior
behavior/functionality. The most notable are to `chain_spec.rs`, in order to
still use a SCALE-encoded `GenesisConfig` (avoiding `serde_json`).
* CI fixes
* Add `/usr/local/opt/llvm/lib` to paths on macOS hosts
* Attempt to use `LD_LIBRARY_PATH` in macOS GitHub CI
* Use `libp2p 0.56` in `serai-node`
* Correct Windows build dependencies
* Correct `llvm/lib` path on macOS
* Correct how macOS 13 and 14 have different homebrew paths
* Use `sw_vers` instead of `uname` on macOS
Yields the macOS version instead of the kernel's version.
* Replace hard-coded path with the intended env variable to fix macOS 13
* Add `libclang-dev` as dependency to the Debian Dockerfile
* Set the `CODE` storage slot
* Update to a version of substrate without `wasmtimer`
Turns out `wasmtimer` is WASM only. This should restore the node's functioning
on non-WASM environments.
* Restore `clang` as a dependency due to the Debian Dockerfile as we require a C++ compiler
* Move from Debian bookworm to trixie
* Restore `chain_getBlockBin` to the RPC
* Always generate a new key for the P2P network
* Mention every account on-chain before they publish a transaction
`CheckNonce` required accounts have a provider in order to even have their
nonce considered. This shims that by claiming every account has a provider at
the start of a block, if it signs a transaction.
The actual execution could presumably diverge between block building (which
sets the provider before each transaction) and execution (which sets the
providers at the start of the block). It doesn't diverge in our current
configuration and it won't be propagated to `next` (which doesn't use
`CheckNonce`).
Also uses explicit indexes for the `serai_abi::{Call, Event}` `enum`s.
* Adopt `patch-polkadot-sdk` with fixed peering
* Manually insert the authority discovery key into the keystore
I did try pulling in `pallet-authority-discovery` for this, updating
`SessionKeys`, but that was insufficient for whatever reason.
* Update to latest `substrate-wasm-builder`
* Fix timeline for incrementing providers
e1671dd71b incremented the providers for every
single transaction's sender before execution, noting the solution was fragile
but it worked for us at this time. It did not work for us at this time.
The new solution replaces `inc_providers` with direct access to the `Account`
`StorageMap` to increment the providers, achieving the desired goal, _without_
emitting an event (which is ordered, and the disparate order between building
and execution was causing mismatches of the state root).
This solution is also fragile and may also be insufficient. None of this code
exists anymore on `next` however. It just has to work sufficiently for now.
* clippy
This helps identify where the various functionalities are used, or rather, not
used. The `Ciphersuite` trait present in `patches/ciphersuite`, facilitating
the entire FCMP++ tree, only requires the markers _and_ canonical point
decoding. I've opened a PR to upstream such a trait into `group`
(https://github.com/zkcrypto/group/pull/68).
`WrappedGroup` is still justified for as long as `Group::generator` exists.
Moving `::generator()` to its own trait, on an independent structure (upstream)
would be massively appreciated. @tarcieri also wanted to update from
`fn generator()` to `const GENERATOR`, which would encourage further discussion
on https://github.com/zkcrypto/group/issues/32 and
https://github.com/zkcrypto/group/issues/45, which have been stagnant.
The `Id` trait is occasionally used yet really should be first off the chopping
block.
Finally, `WithPreferredHash` is only actually used around a third of the time,
which more than justifies it being a separate trait.
---
Updates `dalek_ff_group::Scalar` to directly re-export
`curve25519_dalek::Scalar`, as without issue. `dalek_ff_group::RistrettoPoint`
also could be replaced with an export of `curve25519_dalek::RistrettoPoint`,
yet the coordinator relies on how we implemented `Hash` on it for the hell of
it so it isn't worth it at this time. `dalek_ff_group::EdwardsPoint` can't be
replaced for an re-export of `curve25519_dalek::SubgroupPoint` as it doesn't
implement `zeroize`, `subtle` traits within a released, non-yanked version.
Relevance to https://github.com/serai-dex/serai/issues/201 and
https://github.com/dalek-cryptography/curve25519-dalek/issues/811#issuecomment-3247732746.
Also updates the `Ristretto` ciphersuite to prefer `Blake2b-512` over
`SHA2-512`. In order to maintain compliance with FROST's IETF standard,
`modular-frost` defines its own ciphersuite for Ristretto which still uses
`SHA2-512`.
The prior workflow (now deleted) required manually specifying the packages to
check and only checked the package could compile under the stated MSRV. It
didn't verify it was actually the _minimum_ supported Rust version. The new
version finds the MSRV from scratch to check if the stated MSRV aligns.
Updates stated MSRVs accordingly.
Also removes many explicit dependencies from secq256k1 for their re-exports via
k256. Not directly relevant, just part of tidying up all the `toml`s.
This resolves the conflicts and gets the workspace `Cargo.toml`s to not be
invalid. It doesn't actually get clippy to pass again yet.
Does move `crypto/dkg/src/evrf` into a new `crypto/dkg/evrf` crate (which does
not yet compile).
* Upstream GBP, divisor, circuit abstraction, and EC gadgets from FCMP++
* Initial eVRF implementation
Not quite done yet. It needs to communicate the resulting points and proofs to
extract them from the Pedersen Commitments in order to return those, and then
be tested.
* Add the openings of the PCs to the eVRF as necessary
* Add implementation of secq256k1
* Make DKG Encryption a bit more flexible
No longer requires the use of an EncryptionKeyMessage, and allows pre-defined
keys for encryption.
* Make NUM_BITS an argument for the field macro
* Have the eVRF take a Zeroizing private key
* Initial eVRF-based DKG
* Add embedwards25519 curve
* Inline the eVRF into the DKG library
Due to how we're handling share encryption, we'd either need two circuits or to
dedicate this circuit to the DKG. The latter makes sense at this time.
* Add documentation to the eVRF-based DKG
* Add paragraph claiming robustness
* Update to the new eVRF proof
* Finish routing the eVRF functionality
Still needs errors and serialization, along with a few other TODOs.
* Add initial eVRF DKG test
* Improve eVRF DKG
Updates how we calculcate verification shares, improves performance when
extracting multiple sets of keys, and adds more to the test for it.
* Start using a proper error for the eVRF DKG
* Resolve various TODOs
Supports recovering multiple key shares from the eVRF DKG.
Inlines two loops to save 2**16 iterations.
Adds support for creating a constant time representation of scalars < NUM_BITS.
* Ban zero ECDH keys, document non-zero requirements
* Implement eVRF traits, all the way up to the DKG, for secp256k1/ed25519
* Add Ristretto eVRF trait impls
* Support participating multiple times in the eVRF DKG
* Only participate once per key, not once per key share
* Rewrite processor key-gen around the eVRF DKG
Still a WIP.
* Finish routing the new key gen in the processor
Doesn't touch the tests, coordinator, nor Substrate yet.
`cargo +nightly fmt && cargo +nightly-2024-07-01 clippy --all-features -p serai-processor`
does pass.
* Deduplicate and better document in processor key_gen
* Update serai-processor tests to the new key gen
* Correct amount of yx coefficients, get processor key gen test to pass
* Add embedded elliptic curve keys to Substrate
* Update processor key gen tests to the eVRF DKG
* Have set_keys take signature_participants, not removed_participants
Now no one is removed from the DKG. Only `t` people publish the key however.
Uses a BitVec for an efficient encoding of the participants.
* Update the coordinator binary for the new DKG
This does not yet update any tests.
* Add sensible Debug to key_gen::[Processor, Coordinator]Message
* Have the DKG explicitly declare how to interpolate its shares
Removes the hack for MuSig where we multiply keys by the inverse of their
lagrange interpolation factor.
* Replace Interpolation::None with Interpolation::Constant
Allows the MuSig DKG to keep the secret share as the original private key,
enabling deriving FROST nonces consistently regardless of the MuSig context.
* Get coordinator tests to pass
* Update spec to the new DKG
* Get clippy to pass across the repo
* cargo machete
* Add an extra sleep to ensure expected ordering of `Participation`s
* Update orchestration
* Remove bad panic in coordinator
It expected ConfirmationShare to be n-of-n, not t-of-n.
* Improve documentation on functions
* Update TX size limit
We now no longer have to support the ridiculous case of having 49 DKG
participations within a 101-of-150 DKG. It does remain quite high due to
needing to _sign_ so many times. It'd may be optimal for parties with multiple
key shares to independently send their preprocesses/shares (despite the
overhead that'll cause with signatures and the transaction structure).
* Correct error in the Processor spec document
* Update a few comments in the validator-sets pallet
* Send/Recv Participation one at a time
Sending all, then attempting to receive all in an expected order, wasn't working
even with notable delays between sending messages. This points to the mempool
not working as expected...
* Correct ThresholdKeys serialization in modular-frost test
* Updating existing TX size limit test for the new DKG parameters
* Increase time allowed for the DKG on the GH CI
* Correct construction of signature_participants in serai-client tests
Fault identified by akil.
* Further contextualize DkgConfirmer by ValidatorSet
Caught by a safety check we wouldn't reuse preprocesses across messages. That
raises the question of we were prior reusing preprocesses (reusing keys)?
Except that'd have caused a variety of signing failures (suggesting we had some
staggered timing avoiding it in practice but yes, this was possible in theory).
* Add necessary calls to set_embedded_elliptic_curve_key in coordinator set rotation tests
* Correct shimmed setting of a secq256k1 key
* cargo fmt
* Don't use `[0; 32]` for the embedded keys in the coordinator rotation test
The key_gen function expects the random values already decided.
* Big-endian secq256k1 scalars
Also restores the prior, safer, Encryption::register function.
* fix total allocated stake update in wrong time
* Restore mid-set increases
* Correct typo I introduced
---------
Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
* add node side unit test
* complete rotation test for all networks
* set up the fast-epoch docker file
* fix pr comments
* add coordinator side rotation test
* bug fixes
* Remove EPOCH_INTERVAL
* Minor nits
* Add note on origin of publish_tx function in tests/coordinator
* Correct ThresholdParams assert_eq
* fmt
* Correct detection of handover completion
* Restore key gen message match from develop
It was modified in response to the handover completion bug, which has now been
resolved.
* bug fixes
* Correct invalid constant
* Typo fixes
* remove selecting participant to remove at random
---------
Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
We prior required they had the same encoding, yet this ensures they do by
making them one and the same. This does require an large, ugly, From/TryInto
block which is deemed preferable for moving this more and more into syntax
(from semantics).
Further improvements (notably re: Extra) is possible, and this already lets us
strip some members from the Call enum.
* use median price instead of the highest sustained
* add test for lexicographically reversing a byte slice
* fix pr comments
* fix CI fail
* fix dex tests
* Use a fuzz-tested list of prices
* Working median algorithm based on position + lints
---------
Co-authored-by: akildemir <aeg_asd@hotmail.com>
* report_slashes plumbing in Substrate
Notably delays the SetRetired event until it provides a slash report or the set
after it becomes the set to report its slashes.
* Add dedicated AcceptedHandover event
* Add SlashReport TX to Tributary
* Create SlashReport TXs
* Handle SlashReport TXs
* Add logic to generate a SlashReport to the coordinator
* Route SlashReportSigner into the processor
* Finish routing the SlashReport signing/TX publication
* Add serai feature to processor's serai-client
* Route validators for any active set through sc-authority-discovery
Additionally adds an RPC route to retrieve their P2P addresses.
* Have the coordinator get peers from substrate
* Have the RPC return one address, not up to 3
Prevents the coordinator from believing it has 3 peers when it has one.
* Add missing feature to serai-client
* Correct network argument in serai-client for p2p_validators call
* Add a test in serai-client to check DHT population with a much quicker failure than the coordinator tests
* Update to latest Substrate
Removes distinguishing BABE/AuthorityDiscovery keys which causes
sc_authority_discovery to populate as desired.
* Update to a properly tagged substrate commit
* Add all dialed to peers to GossipSub
* cargo fmt
* Reduce common code in serai-coordinator-tests with amore involved new_test
* Use a recursive async function to spawn `n` DockerTests with the necessary networking configuration
* Merge UNIQUE_ID and ONE_AT_A_TIME
* Tidy up the new recursive code in tests/coordinator
* Use a Mutex in CONTEXT to let it be set multiple times
* Make complimentary edits to full-stack tests
* Augment coordinator P2p connection logs
* Drop lock acquisitions before recursing
* Better scope lock acquisitions in full-stack, preventing a deadlock
* Ensure OUTER_OPS is reset across the test boundary
* Add cargo deny allowance for dockertest fork
* implement general design
* add slashing
* bug fixes
* fix pr comments
* misc fixes
* fix grandpa abi call type
* Correct rebase artifacts I introduced
* Cleanups and corrections
1) Uses vec![] for the OpaqueKeyProof as there's no value to passing it around
2) Remove usage of Babe/Grandpa Offences for tracking if an offence is known
for checking if can slash. If can slash, no prior offence must have been
known.
3) Rename DisabledIndices to SeraiDisabledIndices, drop historical data for
current session only.
4) Doesn't remove from the pre-declared upcoming Serai set upon slash due to
breaking light clients.
5) Into/From instead of AsRef for KeyOwnerProofSystem's generic to ensure
safety of the conversion.
* Correct deduction from TotalAllocatedStake on slash
It should only be done if in set and only with allocations contributing to
TotalAllocatedStake (Allocation + latest session's PendingDeallocation).
* Changes meant for prior commit
---------
Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
There's an exploit where the prior set improperly mints coins, the new set
occurs (resetting the oracle), and they immediately deallocate 49.9% of their
coins (which is more than enough to achieve profitability).
Now, anyone in set must wait until after the next set completes to perform any
deallocation, enabling time to halt upon improper mints.
* Update ValidatorSets with a remove_participant call
* Add DkgRemoval, a sign machine for producing the relevant MuSig signatures
* Don't use position-dependent u8s yet Public when removing validators from the DKG
* Add DkgRemovalPreprocess, DkgRemovalShares
Implementation is via a new publish_tributary_tx lambda.
This is code is a copy-pasted mess which will need to be cleaned up.
* Only allow non-removed validators to vote for removals
Otherwise, it's risked that the remaining validators fall below 67% of the
original set.
* Correct publish_serai_tx, which was prior publish_set_keys in practice
They were already banned form publishing `Batch`s, yet the ability to set keys
enabled changing how certain functionality in the coordinator operated.
Removing this malleability ensures operation as expected.
* Move pallet-asset-conversion
* update licensing
* initial integration
* Integrate Currency & Assets types
* integrate liquidity tokens
* fmt
* integrate dex pallet tests
* fmt
* compilation error fixes
* integrate dex benchmarks
* fmt
* cargo clippy
* replace all occurrences of "asset" with "coin"
* add the actual add liq/swap logic to in-instructions
* add client side & tests
* fix deny
* Lint and changes
- Renames InInstruction::AddLiquidity to InInstruction::SwapAndAddLiquidity
- Makes create_pool an internal function
- Makes dex-pallet exclusively create pools against a native coin
- Removes various fees
- Adds new crates to GH workflow
* Fix rebase artifacts
* Correct other rebase artifact
* Correct CI specification for liquidity-tokens
* Correct primitives' test to the standardized pallet account scheme
---------
Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
* Update the coordinator to give key shares based on weight, not based on existence
Participants are now identified by their starting index. While this compiles,
the following is unimplemented:
1) A conversion for DKG `i` values. It assumes the threshold `i` values used
will be identical for the MuSig signature used to confirm the DKG.
2) Expansion from compressed values to full values before forwarding to the
processor.
* Add a fn to the DkgConfirmer to convert `i` values as needed
Also removes TODOs regarding Serai ensuring validator key uniqueness +
validity. The current infra achieves both.
* Have the Tributary DB track participation by shares, not by count
* Prevent a node from obtaining 34% of the maximum amount of key shares
This is actually mainly intended to set a bound on message sizes in the
coordinator. Message sizes are amplified by the amount of key shares held, so
setting an upper bound on said amount lets it determine constants. While that
upper bound could be 150, that'd be unreasonable and increase the potential for
DoS attacks.
* Correct the mechanism to detect if sufficient accumulation has occured
It used to check if the latest accumulation hit the required threshold. Now,
accumulations may jump past the required threshold. The required mechanism is
to check the threshold wasn't prior met and is now met.
* Finish updating the coordinator to handle a multiple key share per validator environment
* Adjust stategy re: preventing noce reuse in DKG Confirmer
* Add TODOs regarding dropped transactions, add possible TODO fix
* Update tests/coordinator
This doesn't add new multi-key-share tests, it solely updates the existing
single key-share tests to compile and run, with the necessary fixes to the
coordinator.
* Update processor key_gen to handle generating multiple key shares at once
* Update SubstrateSigner
* Update signer, clippy
* Update processor tests
* Update processor docker tests
