absolutebi/serai
1
0
Fork 0
mirror of https://github.com/serai-dex/serai.git synced 2025-12-09 20:59:23 +00:00
Code Issues Packages Projects Releases Wiki Activity

Coordinator Cleanup (#481)

Browse Source 
* Move logic for evaluating if a cosign should occur to its own file

Cleans it up and makes it more robust.

* Have expected_next_batch return an error instead of retrying

While convenient to offer an error-free implementation, it potentially caused
very long lived lock acquisitions in handle_processor_message.

* Unify and clean DkgConfirmer and DkgRemoval

Does so via adding a new file for the common code, SigningProtocol.

Modifies from_cache to return the preprocess with the machine, as there's no
reason not to. Also removes an unused Result around the type.

Clarifies the security around deterministic nonces, removing them for
saved-to-disk cached preprocesses. The cached preprocesses are encrypted as the
DB is not a proper secret store.

Moves arguments always present in the protocol from function arguments into the
struct itself.

Removes the horribly ugly code in DkgRemoval, fixing multiple issues present
with it which would cause it to fail on use.

* Set SeraiBlockNumber in cosign.rs as it's used by the cosigning protocol

* Remove unnecessary Clone from lambdas in coordinator

* Remove the EventDb from Tributary scanner

We used per-Transaction DB TXNs so on error, we don't have to rescan the entire
block yet only the rest of it. We prevented scanning multiple transactions by
tracking which we already had.

This is over-engineered and not worth it.

* Implement borsh for HasEvents, removing the manual encoding

* Merge DkgConfirmer and DkgRemoval into signing_protocol.rs

Fixes a bug in DkgConfirmer which would cause it to improperly handle indexes
if any validator had multiple key shares.

* Strictly type DataSpecification's Label

* Correct threshold_i_map_to_keys_and_musig_i_map

It didn't include the participant's own index and accordingly was offset.

* Create TributaryBlockHandler

This struct contains all variables prior passed to handle_block and stops them
from being passed around again and again.

This also ensures fatal_slash is only called while handling a block, as needed
as it expects to operate under perfect consensus.

* Inline accumulate, store confirmation nonces with shares

Inlining accumulate makes sense due to the amount of data accumulate needed to
be passed.

Storing confirmation nonces with shares ensures that both are available or
neither. Prior, one could be yet the other may not have been (requiring an
assert in runtime to ensure we didn't bungle it somehow).

* Create helper functions for handling DkgRemoval/SubstrateSign/Sign Tributary TXs

* Move Label into SignData

All of our transactions which use SignData end up with the same common usage
pattern for Label, justifying this.

Removes 3 transactions, explicitly de-duplicating their handlers.

* Remove CurrentlyCompletingKeyPair for the non-contextual DkgKeyPair

* Remove the manual read/write for TributarySpec for borsh

This struct doesn't have any optimizations booned by the manual impl. Using
borsh reduces our scope.

* Use temporary variables to further minimize LoC in tributary handler

* Remove usage of tuples for non-trivial Tributary transactions

* Remove serde from dkg

serde could be used to deserialize intenrally inconsistent objects which could
lead to panics or faults.

The BorshDeserialize derives have been replaced with a manual implementation
which won't produce inconsistent objects.

* Abstract Future generics using new trait definitions in coordinator

* Move published_signed_transaction to tributary/mod.rs to reduce the size of main.rs

* Split coordinator/src/tributary/mod.rs into spec.rs and transaction.rs
This commit is contained in:
Luke Parker
2023-12-10 20:21:44 -05:00
committed by GitHub
parent 6caf45ea1d
commit 11fdb6da1d
34 changed files with 2531 additions and 2992 deletions
Download Patch File Download Diff File Expand all files Collapse all files

19
coordinator/src/tests/tributary/chain.rs
View File

@@ -13,7 +13,7 @@ use ciphersuite::{
};
use sp_application_crypto::sr25519;
use borsh::BorshDeserialize;
use serai_client::{
primitives::NetworkId,
validator_sets::primitives::{Session, ValidatorSet},
@@ -58,21 +58,26 @@ pub fn new_spec<R: RngCore + CryptoRng>(
.collect::<Vec<_>>();
let res = TributarySpec::new(serai_block, start_time, set, set_participants);
assert_eq!(TributarySpec::read::<&[u8]>(&mut res.serialize().as_ref()).unwrap(), res);
assert_eq!(
TributarySpec::deserialize_reader(&mut borsh::to_vec(&res).unwrap().as_slice()).unwrap(),
res,
);
res
}
pub async fn new_tributaries(
keys: &[Zeroizing<<Ristretto as Ciphersuite>::F>],
spec: &TributarySpec,
) -> Vec<(LocalP2p, Tributary<MemDb, Transaction, LocalP2p>)> {
) -> Vec<(MemDb, LocalP2p, Tributary<MemDb, Transaction, LocalP2p>)> {
let p2p = LocalP2p::new(keys.len());
let mut res = vec![];
for (i, key) in keys.iter().enumerate() {
let db = MemDb::new();
res.push((
db.clone(),
p2p[i].clone(),
Tributary::<_, Transaction, _>::new(
MemDb::new(),
db,
spec.genesis(),
spec.start_time(),
key.clone(),
@@ -152,7 +157,11 @@ async fn tributary_test() {
let keys = new_keys(&mut OsRng);
let spec = new_spec(&mut OsRng, &keys);
let mut tributaries = new_tributaries(&keys, &spec).await;
let mut tributaries = new_tributaries(&keys, &spec)
.await
.into_iter()
.map(|(_, p2p, tributary)| (p2p, tributary))
.collect::<Vec<_>>();
let mut blocks = 0;
let mut last_block = spec.genesis();

120
coordinator/src/tests/tributary/dkg.rs
View File

@@ -8,7 +8,7 @@ use ciphersuite::{group::GroupEncoding, Ciphersuite, Ristretto};
use frost::Participant;
use sp_runtime::traits::Verify;
use serai_client::validator_sets::primitives::KeyPair;
use serai_client::validator_sets::primitives::{ValidatorSet, KeyPair};
use tokio::time::sleep;
@@ -34,10 +34,18 @@ use crate::{
#[tokio::test]
async fn dkg_test() {
env_logger::init();
let keys = new_keys(&mut OsRng);
let spec = new_spec(&mut OsRng, &keys);
let tributaries = new_tributaries(&keys, &spec).await;
let full_tributaries = new_tributaries(&keys, &spec).await;
let mut dbs = vec![];
let mut tributaries = vec![];
for (db, p2p, tributary) in full_tributaries {
dbs.push(db);
tributaries.push((p2p, tributary));
}
// Run the tributaries in the background
tokio::spawn(run_tributaries(tributaries.clone()));
@@ -49,8 +57,11 @@ async fn dkg_test() {
let mut commitments = vec![0; 256];
OsRng.fill_bytes(&mut commitments);
let mut tx =
Transaction::DkgCommitments(attempt, vec![commitments], Transaction::empty_signed());
let mut tx = Transaction::DkgCommitments {
attempt,
commitments: vec![commitments],
signed: Transaction::empty_signed(),
};
tx.sign(&mut OsRng, spec.genesis(), key);
txs.push(tx);
}
@@ -71,7 +82,7 @@ async fn dkg_test() {
.iter()
.enumerate()
.map(|(i, tx)| {
if let Transaction::DkgCommitments(_, commitments, _) = tx {
if let Transaction::DkgCommitments { commitments, .. } = tx {
(Participant::new((i + 1).try_into().unwrap()).unwrap(), commitments[0].clone())
} else {
panic!("txs had non-commitments");
@@ -80,20 +91,20 @@ async fn dkg_test() {
.collect();
async fn new_processors(
db: &mut MemDb,
key: &Zeroizing<<Ristretto as Ciphersuite>::F>,
spec: &TributarySpec,
tributary: &Tributary<MemDb, Transaction, LocalP2p>,
) -> (MemDb, MemProcessors) {
let mut scanner_db = MemDb::new();
) -> MemProcessors {
let processors = MemProcessors::new();
handle_new_blocks::<_, _, _, _, _, _, _, _, LocalP2p>(
&mut scanner_db,
handle_new_blocks::<_, _, _, _, _, LocalP2p>(
db,
key,
|_, _, _, _| async {
&|_, _, _, _| async {
panic!("provided TX caused recognized_id to be called in new_processors")
},
&processors,
|_, _, _| async { panic!("test tried to publish a new Serai TX in new_processors") },
&|_, _, _| async { panic!("test tried to publish a new Serai TX in new_processors") },
&|_| async {
panic!(
"test tried to publish a new Tributary TX from handle_application_tx in new_processors"
@@ -103,11 +114,11 @@ async fn dkg_test() {
&tributary.reader(),
)
.await;
(scanner_db, processors)
processors
}
// Instantiate a scanner and verify it has nothing to report
let (mut scanner_db, processors) = new_processors(&keys[0], &spec, &tributaries[0].1).await;
let processors = new_processors(&mut dbs[0], &keys[0], &spec, &tributaries[0].1).await;
assert!(processors.0.read().await.is_empty());
// Publish the last commitment
@@ -117,14 +128,14 @@ async fn dkg_test() {
sleep(Duration::from_secs(Tributary::<MemDb, Transaction, LocalP2p>::block_time().into())).await;
// Verify the scanner emits a KeyGen::Commitments message
handle_new_blocks::<_, _, _, _, _, _, _, _, LocalP2p>(
&mut scanner_db,
handle_new_blocks::<_, _, _, _, _, LocalP2p>(
&mut dbs[0],
&keys[0],
|_, _, _, _| async {
&|_, _, _, _| async {
panic!("provided TX caused recognized_id to be called after Commitments")
},
&processors,
|_, _, _| async { panic!("test tried to publish a new Serai TX after Commitments") },
&|_, _, _| async { panic!("test tried to publish a new Serai TX after Commitments") },
&|_| async {
panic!(
"test tried to publish a new Tributary TX from handle_application_tx after Commitments"
@@ -151,8 +162,8 @@ async fn dkg_test() {
}
// Verify all keys exhibit this scanner behavior
for (i, key) in keys.iter().enumerate() {
let (_, processors) = new_processors(key, &spec, &tributaries[i].1).await;
for (i, key) in keys.iter().enumerate().skip(1) {
let processors = new_processors(&mut dbs[i], key, &spec, &tributaries[i].1).await;
let mut msgs = processors.0.write().await;
assert_eq!(msgs.len(), 1);
let msgs = msgs.get_mut(&spec.set().network).unwrap();
@@ -182,12 +193,14 @@ async fn dkg_test() {
}
}
let mut txn = dbs[k].txn();
let mut tx = Transaction::DkgShares {
attempt,
shares,
confirmation_nonces: crate::tributary::dkg_confirmation_nonces(key, &spec, 0),
confirmation_nonces: crate::tributary::dkg_confirmation_nonces(key, &spec, &mut txn, 0),
signed: Transaction::empty_signed(),
};
txn.commit();
tx.sign(&mut OsRng, spec.genesis(), key);
txs.push(tx);
}
@@ -201,14 +214,14 @@ async fn dkg_test() {
}
// With just 4 sets of shares, nothing should happen yet
handle_new_blocks::<_, _, _, _, _, _, _, _, LocalP2p>(
&mut scanner_db,
handle_new_blocks::<_, _, _, _, _, LocalP2p>(
&mut dbs[0],
&keys[0],
|_, _, _, _| async {
&|_, _, _, _| async {
panic!("provided TX caused recognized_id to be called after some shares")
},
&processors,
|_, _, _| async { panic!("test tried to publish a new Serai TX after some shares") },
&|_, _, _| async { panic!("test tried to publish a new Serai TX after some shares") },
&|_| async {
panic!(
"test tried to publish a new Tributary TX from handle_application_tx after some shares"
@@ -254,28 +267,30 @@ async fn dkg_test() {
};
// Any scanner which has handled the prior blocks should only emit the new event
handle_new_blocks::<_, _, _, _, _, _, _, _, LocalP2p>(
&mut scanner_db,
&keys[0],
|_, _, _, _| async { panic!("provided TX caused recognized_id to be called after shares") },
&processors,
|_, _, _| async { panic!("test tried to publish a new Serai TX") },
&|_| async { panic!("test tried to publish a new Tributary TX from handle_application_tx") },
&spec,
&tributaries[0].1.reader(),
)
.await;
{
let mut msgs = processors.0.write().await;
assert_eq!(msgs.len(), 1);
let msgs = msgs.get_mut(&spec.set().network).unwrap();
assert_eq!(msgs.pop_front().unwrap(), shares_for(0));
assert!(msgs.is_empty());
for (i, key) in keys.iter().enumerate() {
handle_new_blocks::<_, _, _, _, _, LocalP2p>(
&mut dbs[i],
key,
&|_, _, _, _| async { panic!("provided TX caused recognized_id to be called after shares") },
&processors,
&|_, _, _| async { panic!("test tried to publish a new Serai TX") },
&|_| async { panic!("test tried to publish a new Tributary TX from handle_application_tx") },
&spec,
&tributaries[i].1.reader(),
)
.await;
{
let mut msgs = processors.0.write().await;
assert_eq!(msgs.len(), 1);
let msgs = msgs.get_mut(&spec.set().network).unwrap();
assert_eq!(msgs.pop_front().unwrap(), shares_for(i));
assert!(msgs.is_empty());
}
}
// Yet new scanners should emit all events
for (i, key) in keys.iter().enumerate() {
let (_, processors) = new_processors(key, &spec, &tributaries[i].1).await;
let processors = new_processors(&mut MemDb::new(), key, &spec, &tributaries[i].1).await;
let mut msgs = processors.0.write().await;
assert_eq!(msgs.len(), 1);
let msgs = msgs.get_mut(&spec.set().network).unwrap();
@@ -302,17 +317,16 @@ async fn dkg_test() {
let mut txs = vec![];
for (i, key) in keys.iter().enumerate() {
let attempt = 0;
let mut scanner_db = &mut scanner_db;
let (mut local_scanner_db, _) = new_processors(key, &spec, &tributaries[0].1).await;
if i != 0 {
scanner_db = &mut local_scanner_db;
}
let mut txn = scanner_db.txn();
let mut txn = dbs[i].txn();
let share =
crate::tributary::generated_key_pair::<MemDb>(&mut txn, key, &spec, &key_pair, 0).unwrap();
txn.commit();
let mut tx = Transaction::DkgConfirmed(attempt, share, Transaction::empty_signed());
let mut tx = Transaction::DkgConfirmed {
attempt,
confirmation_share: share,
signed: Transaction::empty_signed(),
};
tx.sign(&mut OsRng, spec.genesis(), key);
txs.push(tx);
}
@@ -325,14 +339,14 @@ async fn dkg_test() {
}
// The scanner should successfully try to publish a transaction with a validly signed signature
handle_new_blocks::<_, _, _, _, _, _, _, _, LocalP2p>(
&mut scanner_db,
handle_new_blocks::<_, _, _, _, _, LocalP2p>(
&mut dbs[0],
&keys[0],
|_, _, _, _| async {
&|_, _, _, _| async {
panic!("provided TX caused recognized_id to be called after DKG confirmation")
},
&processors,
|set, tx_type, tx| {
&|set: ValidatorSet, tx_type, tx: serai_client::Transaction| {
assert_eq!(tx_type, PstTxType::SetKeys);
let spec = spec.clone();

6
coordinator/src/tests/tributary/handle_p2p.rs
View File

@@ -27,7 +27,11 @@ async fn handle_p2p_test() {
let keys = new_keys(&mut OsRng);
let spec = new_spec(&mut OsRng, &keys);
let mut tributaries = new_tributaries(&keys, &spec).await;
let mut tributaries = new_tributaries(&keys, &spec)
.await
.into_iter()
.map(|(_, p2p, tributary)| (p2p, tributary))
.collect::<Vec<_>>();
let mut tributary_senders = vec![];
let mut tributary_arcs = vec![];

53
coordinator/src/tests/tributary/mod.rs
View File

@@ -7,7 +7,7 @@ use processor_messages::coordinator::SubstrateSignableId;
use tributary::{ReadWrite, tests::random_signed_with_nonce};
use crate::tributary::{SignData, Transaction};
use crate::tributary::{Label, SignData, Transaction};
mod chain;
pub use chain::*;
@@ -34,11 +34,12 @@ fn random_vec<R: RngCore>(rng: &mut R, limit: usize) -> Vec<u8> {
fn random_sign_data<R: RngCore, Id: Clone + PartialEq + Eq + Debug + Encode + Decode>(
rng: &mut R,
plan: Id,
nonce: u32,
label: Label,
) -> SignData<Id> {
SignData {
plan,
attempt: random_u32(&mut OsRng),
label,
data: {
let mut res = vec![];
@@ -48,7 +49,7 @@ fn random_sign_data<R: RngCore, Id: Clone + PartialEq + Eq + Debug + Encode + De
res
},
signed: random_signed_with_nonce(&mut OsRng, nonce),
signed: random_signed_with_nonce(&mut OsRng, label.nonce()),
}
}
@@ -87,7 +88,7 @@ fn serialize_sign_data() {
fn test_read_write<Id: Clone + PartialEq + Eq + Debug + Encode + Decode>(value: SignData<Id>) {
let mut buf = vec![];
value.write(&mut buf).unwrap();
assert_eq!(value, SignData::read(&mut buf.as_slice(), value.signed.nonce).unwrap())
assert_eq!(value, SignData::read(&mut buf.as_slice()).unwrap())
}
let mut plan = [0; 3];
@@ -95,28 +96,28 @@ fn serialize_sign_data() {
test_read_write(random_sign_data::<_, _>(
&mut OsRng,
plan,
u32::try_from(OsRng.next_u64() >> 32).unwrap(),
if (OsRng.next_u64() % 2) == 0 { Label::Preprocess } else { Label::Share },
));
let mut plan = [0; 5];
OsRng.fill_bytes(&mut plan);
test_read_write(random_sign_data::<_, _>(
&mut OsRng,
plan,
u32::try_from(OsRng.next_u64() >> 32).unwrap(),
if (OsRng.next_u64() % 2) == 0 { Label::Preprocess } else { Label::Share },
));
let mut plan = [0; 8];
OsRng.fill_bytes(&mut plan);
test_read_write(random_sign_data::<_, _>(
&mut OsRng,
plan,
u32::try_from(OsRng.next_u64() >> 32).unwrap(),
if (OsRng.next_u64() % 2) == 0 { Label::Preprocess } else { Label::Share },
));
let mut plan = [0; 24];
OsRng.fill_bytes(&mut plan);
test_read_write(random_sign_data::<_, _>(
&mut OsRng,
plan,
u32::try_from(OsRng.next_u64() >> 32).unwrap(),
if (OsRng.next_u64() % 2) == 0 { Label::Preprocess } else { Label::Share },
));
}
@@ -134,11 +135,11 @@ fn serialize_transaction() {
OsRng.fill_bytes(&mut temp);
commitments.push(temp);
}
test_read_write(Transaction::DkgCommitments(
random_u32(&mut OsRng),
test_read_write(Transaction::DkgCommitments {
attempt: random_u32(&mut OsRng),
commitments,
random_signed_with_nonce(&mut OsRng, 0),
));
signed: random_signed_with_nonce(&mut OsRng, 0),
});
}
{
@@ -192,25 +193,25 @@ fn serialize_transaction() {
});
}
test_read_write(Transaction::DkgConfirmed(
random_u32(&mut OsRng),
{
test_read_write(Transaction::DkgConfirmed {
attempt: random_u32(&mut OsRng),
confirmation_share: {
let mut share = [0; 32];
OsRng.fill_bytes(&mut share);
share
},
random_signed_with_nonce(&mut OsRng, 2),
));
signed: random_signed_with_nonce(&mut OsRng, 2),
});
{
let mut key = [0; 32];
OsRng.fill_bytes(&mut key);
test_read_write(Transaction::DkgRemovalPreprocess(random_sign_data(&mut OsRng, key, 0)));
test_read_write(Transaction::DkgRemoval(random_sign_data(&mut OsRng, key, Label::Preprocess)));
}
{
let mut key = [0; 32];
OsRng.fill_bytes(&mut key);
test_read_write(Transaction::DkgRemovalShare(random_sign_data(&mut OsRng, key, 1)));
test_read_write(Transaction::DkgRemoval(random_sign_data(&mut OsRng, key, Label::Share)));
}
{
@@ -224,38 +225,38 @@ fn serialize_transaction() {
OsRng.fill_bytes(&mut block);
let mut batch = [0; 5];
OsRng.fill_bytes(&mut batch);
test_read_write(Transaction::Batch(block, batch));
test_read_write(Transaction::Batch { block, batch });
}
test_read_write(Transaction::SubstrateBlock(OsRng.next_u64()));
{
let mut plan = [0; 5];
OsRng.fill_bytes(&mut plan);
test_read_write(Transaction::SubstratePreprocess(random_sign_data(
test_read_write(Transaction::SubstrateSign(random_sign_data(
&mut OsRng,
SubstrateSignableId::Batch(plan),
0,
Label::Preprocess,
)));
}
{
let mut plan = [0; 5];
OsRng.fill_bytes(&mut plan);
test_read_write(Transaction::SubstrateShare(random_sign_data(
test_read_write(Transaction::SubstrateSign(random_sign_data(
&mut OsRng,
SubstrateSignableId::Batch(plan),
1,
Label::Share,
)));
}
{
let mut plan = [0; 32];
OsRng.fill_bytes(&mut plan);
test_read_write(Transaction::SignPreprocess(random_sign_data(&mut OsRng, plan, 0)));
test_read_write(Transaction::Sign(random_sign_data(&mut OsRng, plan, Label::Preprocess)));
}
{
let mut plan = [0; 32];
OsRng.fill_bytes(&mut plan);
test_read_write(Transaction::SignShare(random_sign_data(&mut OsRng, plan, 1)));
test_read_write(Transaction::Sign(random_sign_data(&mut OsRng, plan, Label::Share)));
}
{

6
coordinator/src/tests/tributary/sync.rs
View File

@@ -31,7 +31,11 @@ async fn sync_test() {
// Ensure this can have a node fail
assert!(spec.n() > spec.t());
let mut tributaries = new_tributaries(&keys, &spec).await;
let mut tributaries = new_tributaries(&keys, &spec)
.await
.into_iter()
.map(|(_, p2p, tributary)| (p2p, tributary))
.collect::<Vec<_>>();
// Keep a Tributary back, effectively having it offline
let syncer_key = keys.pop().unwrap();

13
coordinator/src/tests/tributary/tx.rs
View File

@@ -23,7 +23,11 @@ async fn tx_test() {
let keys = new_keys(&mut OsRng);
let spec = new_spec(&mut OsRng, &keys);
let tributaries = new_tributaries(&keys, &spec).await;
let tributaries = new_tributaries(&keys, &spec)
.await
.into_iter()
.map(|(_, p2p, tributary)| (p2p, tributary))
.collect::<Vec<_>>();
// Run the tributaries in the background
tokio::spawn(run_tributaries(tributaries.clone()));
@@ -39,8 +43,11 @@ async fn tx_test() {
// Create the TX with a null signature so we can get its sig hash
let block_before_tx = tributaries[sender].1.tip().await;
let mut tx =
Transaction::DkgCommitments(attempt, vec![commitments.clone()], Transaction::empty_signed());
let mut tx = Transaction::DkgCommitments {
attempt,
commitments: vec![commitments.clone()],
signed: Transaction::empty_signed(),
};
tx.sign(&mut OsRng, spec.genesis(), &key);
assert_eq!(tributaries[sender].1.add_transaction(tx.clone()).await, Ok(true));