mirror of
https://github.com/serai-dex/serai.git
synced 2025-12-08 20:29:23 +00:00
Add support for multiple multisigs to the processor (#377)
* Design and document a multisig rotation flow * Make Scanner::eventualities a HashMap so it's per-key * Don't drop eventualities, always follow through on them Technical improvements made along the way. * Start creating an isolate object to manage multisigs, which doesn't require being a signer Removes key from SubstrateBlock. * Move Scanner/Scheduler under multisigs * Move Batch construction into MultisigManager * Clarify "should" in Multisig Rotation docs * Add block_number to MultisigManager, as it controls the scanner * Move sign_plans into MultisigManager Removes ThresholdKeys from prepare_send. * Make SubstrateMutable an alias for MultisigManager * Rewrite Multisig Rotation The prior scheme had an exploit possible where funds were sent to the old multisig, then burnt on Serai to send from the new multisig, locking liquidity for 6 hours. While a fee could be applied to stragglers, to make this attack unprofitable, the newly described scheme avoids all this. * Add mini mini is a miniature version of Serai, emphasizing Serai's nature as a collection of independent clocks. The intended use is to identify race conditions and prove protocols are comprehensive regarding when certain clocks tick. This uses loom, a prior candidate for evaluating the processor/coordinator as free of race conditions (#361). * Use mini to prove a race condition in the current multisig rotation docs, and prove safety of alternatives Technically, the prior commit had mini prove the race condition. The docs currently say the activation block of the new multisig is the block after the next Batch's. If the two next Batches had already entered the mempool, prior to set_keys being called, the second next Batch would be expected to contain the new key's data yet fail to as the key wasn't public when the Batch was actually created. The naive solution is to create a Batch, publish it, wait until it's included, and only then scan the next block. This sets a bound of `Batch publication time < block time`. Optimistically, we can publish a Batch in 24s while our shortest block time is 2m. Accordingly, we should be fine with the naive solution which doesn't take advantage of throughput. #333 may significantly change latency however and require an algorithm whose throughput exceeds the rate of blocks created. In order to re-introduce parallelization, enabling throughput, we need to define a safe range of blocks to scan without Serai ordering the first one. mini demonstrates safety of scanning n blocks Serai hasn't acknowledged, so long as the first is scanned before block n+1 is (shifting the n-block window). The docs will be updated next, to reflect this. * Fix Multisig Rotation I believe this is finally good enough to be final. 1) Fixes the race condition present in the prior document, as demonstrated by mini. `Batch`s for block `n` and `n+1`, may have been in the mempool when a multisig's activation block was set to `n`. This would cause a potentially distinct `Batch` for `n+1`, despite `n+1` already having a signed `Batch`. 2) Tightens when UIs should use the new multisig to prevent eclipse attacks, and protection against `Batch` publication delays. 3) Removes liquidity fragmentation by tightening flow/handling of latency. 4) Several clarifications and documentation of reasoning. 5) Correction of "prior multisig" to "all prior multisigs" regarding historical verification, with explanation why. * Clarify terminology in mini Synchronizes it from my original thoughts on potential schema to the design actually created. * Remove most of processor's README for a reference to docs/processor This does drop some misc commentary, though none too beneficial. The section on scanning, deemed sufficiently beneficial, has been moved to a document and expanded on. * Update scanner TODOs in line with new docs * Correct documentation on Bitcoin::Block::time, and Block::time * Make the scanner in MultisigManager no longer public * Always send ConfirmKeyPair, regardless of if in-set * Cargo.lock changes from a prior commit * Add a policy document on defining a Canonical Chain I accidentally committed a version of this with a few headers earlier, and this is a proper version. * Competent MultisigManager::new * Update processor's comments * Add mini to copied files * Re-organize Scanner per multisig rotation document * Add RUST_LOG trace targets to e2e tests * Have the scanner wait once it gets too far ahead Also bug fixes. * Add activation blocks to the scanner * Split received outputs into existing/new in MultisigManager * Select the proper scheduler * Schedule multisig activation as detailed in documentation * Have the Coordinator assert if multiple `Batch`s occur within a block While the processor used to have ack_up_to_block, enabling skips in the block acked, support for this was removed while reworking it for multiple multisigs. It should happen extremely infrequently. While it would still be beneficial to have, if multiple `Batch`s could occur within a block (with the complexity here not being worth adding that ban as a policy), multiple `Batch`s were blocked for DoS reasons. * Schedule payments to the proper multisig * Correct >= to < * Use the new multisig's key for change on schedule * Don't report External TXs to prior multisig once deprecated * Forward from the old multisig to the new one at all opportunities * Move unfulfilled payments in queue from prior to new multisig * Create MultisigsDb, splitting it out of MainDb Drops the call to finish_signing from the Signer. While this will cause endless re-attempts, the Signer will still consider them completed and drop them, making this an O(n) cost at boot even if we did nothing from here. The MultisigManager should call finish_signing once the Scanner completes the Eventuality. * Don't check Scanner-emitted completions, trust they are completions Prevents needing to use async code to mark the completion and creates a fault-free model. The current model, on fault, would cause a lack of marked completion in the signer. * Fix a possible panic in the processor A shorter-chain reorg could cause this assert to trip. It's fixed by de-duplicating the data, as the assertion checked consistency. Without the potential for inconsistency, it's unnecessary. * Document why an existing TODO isn't valid * Change when we drop payments for being to the change address The earlier timing prevents creating Plans solely to the branch address, causing the payments to be dropped, and the TX to become an effective aggregation TX. * Extensively document solutions to Eventualities being potentially created after having already scanned their resolutions * When closing, drop External/Branch outputs which don't cause progress * Properly decide if Change outputs should be forward or not when closing This completes all code needed to make the old multisig have a finite lifetime. * Commentary on forwarding schemes * Provide a 1 block window, with liquidity fragmentation risks, due to latency On Bitcoin, this will be 10 minutes for the relevant Batch to be confirmed. On Monero, 2 minutes. On Ethereum, ~6 minutes. Also updates the Multisig Rotation document with the new forwarding plan. * Implement transaction forwarding from old multisig to new multisig Identifies a fault where Branch outputs which shouldn't be dropped may be, if another output fulfills their next step. Locking Branch fulfillment down to only Branch outputs is not done in this commit, but will be in the next. * Only let Branch outputs fulfill branches * Update TODOs * Move the location of handling signer events to avoid a race condition * Avoid a deadlock by using a RwLock on a single txn instead of two txns * Move Batch ID out of the Scanner * Increase from one block of latency on new keys activation to two For Monero, this offered just two minutes when our latency to publish a Batch is around a minute already. This does increase the time our liquidity can be fragmented by up to 20 minutes (Bitcoin), yet it's a stupid attack only possible once a week (when we rotate). Prioritizing normal users' transactions not being subject to forwarding is more important here. Ideally, we'd not do +2 blocks yet plus `time`, such as +10 minutes, making this agnostic of the underlying network's block scheduling. This is a complexity not worth it. * Split MultisigManager::substrate_block into multiple functions * Further tweaks to substrate_block * Acquire a lock on all Scanner operations after calling ack_block Gives time to call register_eventuality and initiate signing. * Merge sign_plans into substrate_block Also ensure the Scanner's lock isn't prematurely released. * Use a HashMap to pass to-be-forwarded instructions, not the DB * Successfully determine in ClosingExisting * Move from 2 blocks of latency when rotating to 10 minutes Superior as noted in 6d07af92ce10cfd74c17eb3400368b0150eb36d7, now trivial to implement thanks to prior commit. * Add note justifying measuring time in blocks when rotating * Implement delaying of outputs received early to the new multisig per specification * Documentation on why Branch outputs don't have the race condition concerns Change do Also ensures 6 hours is at least N::CONFIRMATIONS, for sanity purposes. * Remove TODO re: sanity checking Eventualities We sanity check the Plan the Eventuality is derived from, and the Eventuality is handled moments later (in the same file, with a clear call path). There's no reason to add such APIs to Eventualities for a sanity check given that. * Add TODO(now) for TODOs which must be done in this branch Also deprecates a pair of TODOs to TODO2, and accepts the flow of the Signer having the Eventuality. * Correct errors in potential/future flow descriptions * Accept having a single Plan Vec Per the following code consuming it, there's no benefit to bifurcating it by key. * Only issue sign_transaction on boot for the proper signer * Only set keys when participating in their construction * Misc progress Only send SubstrateBlockAck when we have a signer, as it's only used to tell the Tributary of what Plans are being signed in response to this block. Only immediately sets substrate_signer if session is 0. On boot, doesn't panic if we don't have an active key (as we wouldn't if only joining the next multisig). Continues. * Correctly detect and set retirement block Modifies the retirement block from first block meeting requirements to block CONFIRMATIONS after. Adds an ack flow to the Scanner's Confirmed event and Block event to accomplish this, which may deadlock at this time (will be fixed shortly). Removes an invalid await (after a point declared unsafe to use await) from MultisigsManager::next_event. * Remove deadlock in multisig_completed and document alternative The alternative is simpler, albeit less efficient. There's no reason to adopt it now, yet perhaps if it benefits modeling? * Handle the final step of retirement, dropping the old key and setting new to existing * Remove TODO about emitting a Block on every step If we emit on NewAsChange, we lose the purpose of the NewAsChange period. The only concern is if we reach ClosingExisting, and nothing has happened, then all coins will still be in the old multisig until something finally does. This isn't a problem worth solving, as it's latency under exceptional dead time. * Add TODO about potentially not emitting a Block event for the reitrement block * Restore accidentally deleted CI file * Pair of slight tweaks * Add missing if statement * Disable an assertion when testing One of the test flows currently abuses the Scanner in a way triggering it.
This commit is contained in:
Luke Parker
GitHub
189
processor/src/multisigs/db.rs
Normal file
189
processor/src/multisigs/db.rs
Normal file
@@ -0,0 +1,189 @@
|
||||
use core::marker::PhantomData;
|
||||
|
||||
use ciphersuite::Ciphersuite;
|
||||
|
||||
pub use serai_db::*;
|
||||
|
||||
use scale::{Encode, Decode};
|
||||
use serai_client::in_instructions::primitives::InInstructionWithBalance;
|
||||
|
||||
use crate::{
|
||||
Get, Db, Plan,
|
||||
networks::{Transaction, Network},
|
||||
};
|
||||
|
||||
#[derive(Debug)]
|
||||
pub struct MultisigsDb<N: Network, D: Db>(PhantomData<N>, PhantomData<D>);
|
||||
impl<N: Network, D: Db> MultisigsDb<N, D> {
|
||||
fn multisigs_key(dst: &'static [u8], key: impl AsRef<[u8]>) -> Vec<u8> {
|
||||
D::key(b"MULTISIGS", dst, key)
|
||||
}
|
||||
|
||||
fn next_batch_key() -> Vec<u8> {
|
||||
Self::multisigs_key(b"next_batch", [])
|
||||
}
|
||||
// Set the next batch ID to use
|
||||
pub fn set_next_batch_id(txn: &mut D::Transaction<'_>, batch: u32) {
|
||||
txn.put(Self::next_batch_key(), batch.to_le_bytes());
|
||||
}
|
||||
// Get the next batch ID
|
||||
pub fn next_batch_id<G: Get>(getter: &G) -> u32 {
|
||||
getter.get(Self::next_batch_key()).map_or(0, |v| u32::from_le_bytes(v.try_into().unwrap()))
|
||||
}
|
||||
|
||||
fn plan_key(id: &[u8]) -> Vec<u8> {
|
||||
Self::multisigs_key(b"plan", id)
|
||||
}
|
||||
fn resolved_key(tx: &[u8]) -> Vec<u8> {
|
||||
Self::multisigs_key(b"resolved", tx)
|
||||
}
|
||||
fn signing_key(key: &[u8]) -> Vec<u8> {
|
||||
Self::multisigs_key(b"signing", key)
|
||||
}
|
||||
pub fn save_active_plan(
|
||||
txn: &mut D::Transaction<'_>,
|
||||
key: &[u8],
|
||||
block_number: u64,
|
||||
plan: &Plan<N>,
|
||||
) {
|
||||
let id = plan.id();
|
||||
|
||||
{
|
||||
let mut signing = txn.get(Self::signing_key(key)).unwrap_or(vec![]);
|
||||
|
||||
// If we've already noted we're signing this, return
|
||||
assert_eq!(signing.len() % 32, 0);
|
||||
for i in 0 .. (signing.len() / 32) {
|
||||
if signing[(i * 32) .. ((i + 1) * 32)] == id {
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
signing.extend(&id);
|
||||
txn.put(Self::signing_key(key), id);
|
||||
}
|
||||
|
||||
{
|
||||
let mut buf = block_number.to_le_bytes().to_vec();
|
||||
plan.write(&mut buf).unwrap();
|
||||
txn.put(Self::plan_key(&id), &buf);
|
||||
}
|
||||
}
|
||||
|
||||
pub fn active_plans<G: Get>(getter: &G, key: &[u8]) -> Vec<(u64, Plan<N>)> {
|
||||
let signing = getter.get(Self::signing_key(key)).unwrap_or(vec![]);
|
||||
let mut res = vec![];
|
||||
|
||||
assert_eq!(signing.len() % 32, 0);
|
||||
for i in 0 .. (signing.len() / 32) {
|
||||
let id = &signing[(i * 32) .. ((i + 1) * 32)];
|
||||
let buf = getter.get(Self::plan_key(id)).unwrap();
|
||||
|
||||
let block_number = u64::from_le_bytes(buf[.. 8].try_into().unwrap());
|
||||
let plan = Plan::<N>::read::<&[u8]>(&mut &buf[8 ..]).unwrap();
|
||||
assert_eq!(id, &plan.id());
|
||||
res.push((block_number, plan));
|
||||
}
|
||||
|
||||
res
|
||||
}
|
||||
|
||||
pub fn resolved_plan<G: Get>(
|
||||
getter: &G,
|
||||
tx: <N::Transaction as Transaction<N>>::Id,
|
||||
) -> Option<[u8; 32]> {
|
||||
getter.get(tx.as_ref()).map(|id| id.try_into().unwrap())
|
||||
}
|
||||
pub fn plan_by_key_with_self_change<G: Get>(
|
||||
getter: &G,
|
||||
key: <N::Curve as Ciphersuite>::G,
|
||||
id: [u8; 32],
|
||||
) -> bool {
|
||||
let plan =
|
||||
Plan::<N>::read::<&[u8]>(&mut &getter.get(Self::plan_key(&id)).unwrap()[8 ..]).unwrap();
|
||||
assert_eq!(plan.id(), id);
|
||||
(key == plan.key) && (Some(N::change_address(plan.key)) == plan.change)
|
||||
}
|
||||
pub fn resolve_plan(
|
||||
txn: &mut D::Transaction<'_>,
|
||||
key: &[u8],
|
||||
plan: [u8; 32],
|
||||
resolution: <N::Transaction as Transaction<N>>::Id,
|
||||
) {
|
||||
let mut signing = txn.get(Self::signing_key(key)).unwrap_or(vec![]);
|
||||
assert_eq!(signing.len() % 32, 0);
|
||||
|
||||
let mut found = false;
|
||||
for i in 0 .. (signing.len() / 32) {
|
||||
let start = i * 32;
|
||||
let end = i + 32;
|
||||
if signing[start .. end] == plan {
|
||||
found = true;
|
||||
signing = [&signing[.. start], &signing[end ..]].concat().to_vec();
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if !found {
|
||||
log::warn!("told to finish signing {} yet wasn't actively signing it", hex::encode(plan));
|
||||
}
|
||||
|
||||
txn.put(Self::signing_key(key), signing);
|
||||
|
||||
txn.put(Self::resolved_key(resolution.as_ref()), plan);
|
||||
}
|
||||
|
||||
fn forwarded_output_key(amount: u64) -> Vec<u8> {
|
||||
Self::multisigs_key(b"forwarded_output", amount.to_le_bytes())
|
||||
}
|
||||
pub fn save_forwarded_output(
|
||||
txn: &mut D::Transaction<'_>,
|
||||
instruction: InInstructionWithBalance,
|
||||
) {
|
||||
let key = Self::forwarded_output_key(instruction.balance.amount.0);
|
||||
let mut existing = txn.get(&key).unwrap_or(vec![]);
|
||||
existing.extend(instruction.encode());
|
||||
txn.put(key, existing);
|
||||
}
|
||||
pub fn take_forwarded_output(
|
||||
txn: &mut D::Transaction<'_>,
|
||||
amount: u64,
|
||||
) -> Option<InInstructionWithBalance> {
|
||||
let key = Self::forwarded_output_key(amount);
|
||||
|
||||
let outputs = txn.get(&key)?;
|
||||
let mut outputs_ref = outputs.as_slice();
|
||||
|
||||
let res = InInstructionWithBalance::decode(&mut outputs_ref).unwrap();
|
||||
assert!(outputs_ref.len() < outputs.len());
|
||||
if outputs_ref.is_empty() {
|
||||
txn.del(&key);
|
||||
} else {
|
||||
txn.put(&key, outputs_ref);
|
||||
}
|
||||
Some(res)
|
||||
}
|
||||
|
||||
fn delayed_output_keys() -> Vec<u8> {
|
||||
Self::multisigs_key(b"delayed_outputs", [])
|
||||
}
|
||||
pub fn save_delayed_output(txn: &mut D::Transaction<'_>, instruction: InInstructionWithBalance) {
|
||||
let key = Self::delayed_output_keys();
|
||||
let mut existing = txn.get(&key).unwrap_or(vec![]);
|
||||
existing.extend(instruction.encode());
|
||||
txn.put(key, existing);
|
||||
}
|
||||
pub fn take_delayed_outputs(txn: &mut D::Transaction<'_>) -> Vec<InInstructionWithBalance> {
|
||||
let key = Self::delayed_output_keys();
|
||||
|
||||
let Some(outputs) = txn.get(&key) else { return vec![] };
|
||||
txn.del(key);
|
||||
|
||||
let mut outputs_ref = outputs.as_slice();
|
||||
let mut res = vec![];
|
||||
while !outputs_ref.is_empty() {
|
||||
res.push(InInstructionWithBalance::decode(&mut outputs_ref).unwrap());
|
||||
}
|
||||
res
|
||||
}
|
||||
}
|
||||
927
processor/src/multisigs/mod.rs
Normal file
927
processor/src/multisigs/mod.rs
Normal file
@@ -0,0 +1,927 @@
|
||||
use core::time::Duration;
|
||||
use std::{sync::RwLock, collections::HashMap};
|
||||
|
||||
use ciphersuite::{group::GroupEncoding, Ciphersuite};
|
||||
|
||||
use scale::{Encode, Decode};
|
||||
use messages::SubstrateContext;
|
||||
|
||||
use serai_client::{
|
||||
primitives::{BlockHash, MAX_DATA_LEN},
|
||||
in_instructions::primitives::{
|
||||
InInstructionWithBalance, Batch, RefundableInInstruction, Shorthand, MAX_BATCH_SIZE,
|
||||
},
|
||||
tokens::primitives::{OutInstruction, OutInstructionWithBalance},
|
||||
};
|
||||
|
||||
use log::{info, error};
|
||||
|
||||
use tokio::time::sleep;
|
||||
|
||||
#[cfg(not(test))]
|
||||
mod scanner;
|
||||
#[cfg(test)]
|
||||
pub mod scanner;
|
||||
|
||||
use scanner::{ScannerEvent, ScannerHandle, Scanner};
|
||||
|
||||
mod db;
|
||||
use db::MultisigsDb;
|
||||
|
||||
#[cfg(not(test))]
|
||||
mod scheduler;
|
||||
#[cfg(test)]
|
||||
pub mod scheduler;
|
||||
use scheduler::Scheduler;
|
||||
|
||||
use crate::{
|
||||
Get, Db, Payment, PostFeeBranch, Plan,
|
||||
networks::{OutputType, Output, Transaction, SignableTransaction, Block, Network, get_block},
|
||||
};
|
||||
|
||||
// InInstructionWithBalance from an external output
|
||||
fn instruction_from_output<N: Network>(output: &N::Output) -> Option<InInstructionWithBalance> {
|
||||
assert_eq!(output.kind(), OutputType::External);
|
||||
|
||||
let mut data = output.data();
|
||||
let max_data_len = usize::try_from(MAX_DATA_LEN).unwrap();
|
||||
if data.len() > max_data_len {
|
||||
error!(
|
||||
"data in output {} exceeded MAX_DATA_LEN ({MAX_DATA_LEN}): {}. skipping",
|
||||
hex::encode(output.id()),
|
||||
data.len(),
|
||||
);
|
||||
None?;
|
||||
}
|
||||
|
||||
let Ok(shorthand) = Shorthand::decode(&mut data) else { None? };
|
||||
let Ok(instruction) = RefundableInInstruction::try_from(shorthand) else { None? };
|
||||
|
||||
// TODO2: Set instruction.origin if not set (and handle refunds in general)
|
||||
Some(InInstructionWithBalance { instruction: instruction.instruction, balance: output.balance() })
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
|
||||
enum RotationStep {
|
||||
// Use the existing multisig for all actions (steps 1-3)
|
||||
UseExisting,
|
||||
// Use the new multisig as change (step 4)
|
||||
NewAsChange,
|
||||
// The existing multisig is expected to solely forward transactions at this point (step 5)
|
||||
ForwardFromExisting,
|
||||
// The existing multisig is expected to finish its own transactions and do nothing more
|
||||
// (step 6)
|
||||
ClosingExisting,
|
||||
}
|
||||
|
||||
async fn get_fee<N: Network>(network: &N, block_number: usize) -> N::Fee {
|
||||
// TODO2: Use an fee representative of several blocks
|
||||
get_block(network, block_number).await.median_fee()
|
||||
}
|
||||
|
||||
async fn prepare_send<N: Network>(
|
||||
network: &N,
|
||||
block_number: usize,
|
||||
fee: N::Fee,
|
||||
plan: Plan<N>,
|
||||
) -> (Option<(N::SignableTransaction, N::Eventuality)>, Vec<PostFeeBranch>) {
|
||||
loop {
|
||||
match network.prepare_send(block_number, plan.clone(), fee).await {
|
||||
Ok(prepared) => {
|
||||
return prepared;
|
||||
}
|
||||
Err(e) => {
|
||||
error!("couldn't prepare a send for plan {}: {e}", hex::encode(plan.id()));
|
||||
// The processor is either trying to create an invalid TX (fatal) or the node went
|
||||
// offline
|
||||
// The former requires a patch, the latter is a connection issue
|
||||
// If the latter, this is an appropriate sleep. If the former, we should panic, yet
|
||||
// this won't flood the console ad infinitum
|
||||
sleep(Duration::from_secs(60)).await;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub struct MultisigViewer<N: Network> {
|
||||
activation_block: usize,
|
||||
key: <N::Curve as Ciphersuite>::G,
|
||||
scheduler: Scheduler<N>,
|
||||
}
|
||||
|
||||
#[allow(clippy::type_complexity)]
|
||||
#[derive(Clone, Debug)]
|
||||
pub enum MultisigEvent<N: Network> {
|
||||
// Batches to publish
|
||||
Batches(Option<(<N::Curve as Ciphersuite>::G, <N::Curve as Ciphersuite>::G)>, Vec<Batch>),
|
||||
// Eventuality completion found on-chain
|
||||
Completed(Vec<u8>, [u8; 32], N::Transaction),
|
||||
}
|
||||
|
||||
pub struct MultisigManager<D: Db, N: Network> {
|
||||
scanner: ScannerHandle<N, D>,
|
||||
existing: Option<MultisigViewer<N>>,
|
||||
new: Option<MultisigViewer<N>>,
|
||||
}
|
||||
|
||||
impl<D: Db, N: Network> MultisigManager<D, N> {
|
||||
pub async fn new(
|
||||
raw_db: &D,
|
||||
network: &N,
|
||||
) -> (
|
||||
Self,
|
||||
Vec<<N::Curve as Ciphersuite>::G>,
|
||||
Vec<(Plan<N>, N::SignableTransaction, N::Eventuality)>,
|
||||
) {
|
||||
// The scanner has no long-standing orders to re-issue
|
||||
let (mut scanner, current_keys) = Scanner::new(network.clone(), raw_db.clone());
|
||||
|
||||
let mut schedulers = vec![];
|
||||
|
||||
assert!(current_keys.len() <= 2);
|
||||
let mut actively_signing = vec![];
|
||||
for (_, key) in ¤t_keys {
|
||||
schedulers.push(Scheduler::from_db(raw_db, *key).unwrap());
|
||||
|
||||
// Load any TXs being actively signed
|
||||
let key = key.to_bytes();
|
||||
for (block_number, plan) in MultisigsDb::<N, D>::active_plans(raw_db, key.as_ref()) {
|
||||
let block_number = block_number.try_into().unwrap();
|
||||
|
||||
let fee = get_fee(network, block_number).await;
|
||||
|
||||
let id = plan.id();
|
||||
info!("reloading plan {}: {:?}", hex::encode(id), plan);
|
||||
|
||||
let key_bytes = plan.key.to_bytes();
|
||||
|
||||
let (Some((tx, eventuality)), _) =
|
||||
prepare_send(network, block_number, fee, plan.clone()).await
|
||||
else {
|
||||
panic!("previously created transaction is no longer being created")
|
||||
};
|
||||
|
||||
scanner
|
||||
.register_eventuality(key_bytes.as_ref(), block_number, id, eventuality.clone())
|
||||
.await;
|
||||
actively_signing.push((plan, tx, eventuality));
|
||||
}
|
||||
}
|
||||
|
||||
(
|
||||
MultisigManager {
|
||||
scanner,
|
||||
existing: current_keys.get(0).cloned().map(|(activation_block, key)| MultisigViewer {
|
||||
activation_block,
|
||||
key,
|
||||
scheduler: schedulers.remove(0),
|
||||
}),
|
||||
new: current_keys.get(1).cloned().map(|(activation_block, key)| MultisigViewer {
|
||||
activation_block,
|
||||
key,
|
||||
scheduler: schedulers.remove(0),
|
||||
}),
|
||||
},
|
||||
current_keys.into_iter().map(|(_, key)| key).collect(),
|
||||
actively_signing,
|
||||
)
|
||||
}
|
||||
|
||||
/// Returns the block number for a block hash, if it's known and all keys have scanned the block.
|
||||
// This is guaranteed to atomically increment so long as no new keys are added to the scanner
|
||||
// which activate at a block before the currently highest scanned block. This is prevented by
|
||||
// the processor waiting for `Batch` inclusion before scanning too far ahead, and activation only
|
||||
// happening after the "too far ahead" window.
|
||||
pub async fn block_number<G: Get>(
|
||||
&self,
|
||||
getter: &G,
|
||||
hash: &<N::Block as Block<N>>::Id,
|
||||
) -> Option<usize> {
|
||||
let latest = ScannerHandle::<N, D>::block_number(getter, hash)?;
|
||||
|
||||
// While the scanner has cemented this block, that doesn't mean it's been scanned for all
|
||||
// keys
|
||||
// ram_scanned will return the lowest scanned block number out of all keys
|
||||
if latest > self.scanner.ram_scanned().await {
|
||||
return None;
|
||||
}
|
||||
Some(latest)
|
||||
}
|
||||
|
||||
pub async fn add_key(
|
||||
&mut self,
|
||||
txn: &mut D::Transaction<'_>,
|
||||
activation_block: usize,
|
||||
external_key: <N::Curve as Ciphersuite>::G,
|
||||
) {
|
||||
self.scanner.register_key(txn, activation_block, external_key).await;
|
||||
let viewer = Some(MultisigViewer {
|
||||
activation_block,
|
||||
key: external_key,
|
||||
scheduler: Scheduler::<N>::new::<D>(txn, external_key),
|
||||
});
|
||||
|
||||
if self.existing.is_none() {
|
||||
self.existing = viewer;
|
||||
return;
|
||||
}
|
||||
self.new = viewer;
|
||||
}
|
||||
|
||||
fn current_rotation_step(&self, block_number: usize) -> RotationStep {
|
||||
fn ceil_div(num: usize, denom: usize) -> usize {
|
||||
let res = num / denom;
|
||||
if (res * denom) == num {
|
||||
return res;
|
||||
}
|
||||
res + 1
|
||||
}
|
||||
|
||||
let Some(new) = self.new.as_ref() else { return RotationStep::UseExisting };
|
||||
|
||||
// Period numbering here has no meaning other than these the time values useful here, and the
|
||||
// order they're built in. They have no reference/shared marker with anything else
|
||||
|
||||
// ESTIMATED_BLOCK_TIME_IN_SECONDS is fine to use here. While inaccurate, it shouldn't be
|
||||
// drastically off, and even if it is, it's a hiccup to latency handling only possible when
|
||||
// rotating. The error rate wouldn't be acceptable if it was allowed to accumulate over time,
|
||||
// yet rotation occurs on Serai's clock, disconnecting any errors here from any prior.
|
||||
|
||||
// N::CONFIRMATIONS + 10 minutes
|
||||
let period_1_start = new.activation_block +
|
||||
N::CONFIRMATIONS +
|
||||
ceil_div(10 * 60, N::ESTIMATED_BLOCK_TIME_IN_SECONDS);
|
||||
|
||||
// N::CONFIRMATIONS
|
||||
let period_2_start = period_1_start + N::CONFIRMATIONS;
|
||||
|
||||
// 6 hours after period 2
|
||||
// Also ensure 6 hours is greater than the amount of CONFIRMATIONS, for sanity purposes
|
||||
let period_3_start =
|
||||
period_2_start + ((6 * 60 * 60) / N::ESTIMATED_BLOCK_TIME_IN_SECONDS).max(N::CONFIRMATIONS);
|
||||
|
||||
if block_number < period_1_start {
|
||||
RotationStep::UseExisting
|
||||
} else if block_number < period_2_start {
|
||||
RotationStep::NewAsChange
|
||||
} else if block_number < period_3_start {
|
||||
RotationStep::ForwardFromExisting
|
||||
} else {
|
||||
RotationStep::ClosingExisting
|
||||
}
|
||||
}
|
||||
|
||||
// Convert new Burns to Payments.
|
||||
//
|
||||
// Also moves payments from the old Scheduler to the new multisig if the step calls for it.
|
||||
fn burns_to_payments(
|
||||
&mut self,
|
||||
txn: &mut D::Transaction<'_>,
|
||||
step: RotationStep,
|
||||
burns: Vec<OutInstructionWithBalance>,
|
||||
) -> (Vec<Payment<N>>, Vec<Payment<N>>) {
|
||||
let mut payments = vec![];
|
||||
for out in burns {
|
||||
let OutInstructionWithBalance { instruction: OutInstruction { address, data }, balance } =
|
||||
out;
|
||||
assert_eq!(balance.coin.network(), N::NETWORK);
|
||||
|
||||
if let Ok(address) = N::Address::try_from(address.consume()) {
|
||||
// TODO: Add coin to payment
|
||||
payments.push(Payment {
|
||||
address,
|
||||
data: data.map(|data| data.consume()),
|
||||
amount: balance.amount.0,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
let payments = payments;
|
||||
match step {
|
||||
RotationStep::UseExisting | RotationStep::NewAsChange => (payments, vec![]),
|
||||
RotationStep::ForwardFromExisting | RotationStep::ClosingExisting => {
|
||||
// Consume any payments the prior scheduler was unable to complete
|
||||
// This should only actually matter once
|
||||
let mut new_payments = self.existing.as_mut().unwrap().scheduler.consume_payments::<D>(txn);
|
||||
// Add the new payments
|
||||
new_payments.extend(payments);
|
||||
(vec![], new_payments)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn split_outputs_by_key(&self, outputs: Vec<N::Output>) -> (Vec<N::Output>, Vec<N::Output>) {
|
||||
let mut existing_outputs = Vec::with_capacity(outputs.len());
|
||||
let mut new_outputs = vec![];
|
||||
|
||||
let existing_key = self.existing.as_ref().unwrap().key;
|
||||
let new_key = self.new.as_ref().map(|new| new.key);
|
||||
for output in outputs {
|
||||
if output.key() == existing_key {
|
||||
existing_outputs.push(output);
|
||||
} else {
|
||||
assert_eq!(Some(output.key()), new_key);
|
||||
new_outputs.push(output);
|
||||
}
|
||||
}
|
||||
|
||||
(existing_outputs, new_outputs)
|
||||
}
|
||||
|
||||
// Manually creates Plans for all External outputs needing forwarding/refunding.
|
||||
//
|
||||
// Returns created Plans and a map of forwarded output IDs to their associated InInstructions.
|
||||
fn filter_outputs_due_to_forwarding(
|
||||
&self,
|
||||
existing_outputs: &mut Vec<N::Output>,
|
||||
) -> (Vec<Plan<N>>, HashMap<Vec<u8>, InInstructionWithBalance>) {
|
||||
// Manually create a Plan for all External outputs needing forwarding/refunding
|
||||
|
||||
/*
|
||||
Sending a Plan, with arbitrary data proxying the InInstruction, would require adding
|
||||
a flow for networks which drop their data to still embed arbitrary data. It'd also have
|
||||
edge cases causing failures.
|
||||
|
||||
Instead, we save the InInstruction as we scan this output. Then, when the output is
|
||||
successfully forwarded, we simply read it from the local database. This also saves the
|
||||
costs of embedding arbitrary data.
|
||||
|
||||
Since we can't rely on the Eventuality system to detect if it's a forwarded transaction,
|
||||
due to the asynchonicity of the Eventuality system, we instead interpret an External
|
||||
output with no InInstruction, which has an amount associated with an InInstruction
|
||||
being forwarded, as having been forwarded. This does create a specific edge case where
|
||||
a user who doesn't include an InInstruction may not be refunded however, if they share
|
||||
an exact amount with an expected-to-be-forwarded transaction. This is deemed acceptable.
|
||||
|
||||
TODO: Add a fourth address, forwarded_address, to prevent this.
|
||||
*/
|
||||
|
||||
let mut plans = vec![];
|
||||
let mut forwarding = HashMap::new();
|
||||
existing_outputs.retain(|output| {
|
||||
if output.kind() == OutputType::External {
|
||||
if let Some(instruction) = instruction_from_output::<N>(output) {
|
||||
// Build a dedicated Plan forwarding this
|
||||
plans.push(Plan {
|
||||
key: self.existing.as_ref().unwrap().key,
|
||||
inputs: vec![output.clone()],
|
||||
payments: vec![],
|
||||
change: Some(N::address(self.new.as_ref().unwrap().key)),
|
||||
});
|
||||
|
||||
// Set the instruction for this output to be returned
|
||||
forwarding.insert(output.id().as_ref().to_vec(), instruction);
|
||||
}
|
||||
|
||||
// TODO: Refund here
|
||||
false
|
||||
} else {
|
||||
true
|
||||
}
|
||||
});
|
||||
(plans, forwarding)
|
||||
}
|
||||
|
||||
// Filter newly received outputs due to the step being RotationStep::ClosingExisting.
|
||||
fn filter_outputs_due_to_closing(
|
||||
&mut self,
|
||||
txn: &mut D::Transaction<'_>,
|
||||
existing_outputs: &mut Vec<N::Output>,
|
||||
) -> Vec<Plan<N>> {
|
||||
/*
|
||||
The document says to only handle outputs we created. We don't know what outputs we
|
||||
created. We do have an ordered view of equivalent outputs however, and can assume the
|
||||
first (and likely only) ones are the ones we created.
|
||||
|
||||
Accordingly, only handling outputs we created should be definable as only handling
|
||||
outputs from the resolution of Eventualities.
|
||||
|
||||
This isn't feasible. It requires knowing what Eventualities were completed in this block,
|
||||
when we handle this block, which we don't know without fully serialized scanning + Batch
|
||||
publication.
|
||||
|
||||
Take the following scenario:
|
||||
1) A network uses 10 confirmations. Block x is scanned, meaning x+9a exists.
|
||||
2) 67% of nodes process x, create, sign, and publish a TX, creating an Eventuality.
|
||||
3) A reorganization to a shorter chain occurs, including the published TX in x+1b.
|
||||
4) The 33% of nodes which are latent will be allowed to scan x+1b as soon as x+10b
|
||||
exists. They won't wait for Serai to include the Batch for x until they try to scan
|
||||
x+10b.
|
||||
5) These latent nodes will handle x+1b, post-create an Eventuality, post-learn x+1b
|
||||
contained resolutions, changing how x+1b should've been interpreted.
|
||||
|
||||
We either have to:
|
||||
A) Fully serialize scanning (removing the ability to utilize throughput to allow higher
|
||||
latency, at least while the step is `ClosingExisting`).
|
||||
B) Create Eventualities immediately, which we can't do as then both the external
|
||||
network's clock AND Serai's clock can trigger Eventualities, removing ordering.
|
||||
We'd need to shift entirely to the external network's clock, only handling Burns
|
||||
outside the parallelization window (which would be extremely latent).
|
||||
C) Use a different mechanism to determine if we created an output.
|
||||
D) Re-define which outputs are still to be handled after the 6 hour period expires, such
|
||||
that the multisig's lifetime cannot be further extended yet it does fulfill its
|
||||
responsibility.
|
||||
|
||||
External outputs to the existing multisig will be:
|
||||
- Scanned before the rotation and unused (as used External outputs become Change)
|
||||
- Forwarded immediately upon scanning
|
||||
- Not scanned before the cut off time (and accordingly dropped)
|
||||
|
||||
For the first case, since they're scanned before the rotation and unused, they'll be
|
||||
forwarded with all other available outputs (since they'll be available when scanned).
|
||||
|
||||
Change outputs will be:
|
||||
- Scanned before the rotation and forwarded with all other available outputs
|
||||
- Forwarded immediately upon scanning
|
||||
- Not scanned before the cut off time, requiring an extension exclusive to these outputs
|
||||
|
||||
The important thing to note about honest Change outputs to the existing multisig is that
|
||||
they'll only be created within `CONFIRMATIONS+1` blocks of the activation block. Also
|
||||
important to note is that there's another explicit window of `CONFIRMATIONS` before the
|
||||
6 hour window.
|
||||
|
||||
Eventualities are not guaranteed to be known before we scan the block containing their
|
||||
resolution. They are guaranteed to be known within `CONFIRMATIONS-1` blocks however, due
|
||||
to the limitation on how far we'll scan ahead.
|
||||
|
||||
This means we will know of all Eventualities related to Change outputs we need to forward
|
||||
before the 6 hour period begins (as forwarding outputs will not create any Change outputs
|
||||
to the existing multisig).
|
||||
|
||||
This means a definition of complete can be defined as:
|
||||
1) Handled all Branch outputs
|
||||
2) Forwarded all External outputs received before the end of 6 hour window
|
||||
3) Forwarded the results of all Eventualities with Change, which will have been created
|
||||
before the 6 hour window
|
||||
|
||||
How can we track and ensure this without needing to check if an output is from the
|
||||
resolution of an Eventuality?
|
||||
|
||||
1) We only create Branch outputs before the 6 hour window starts. These are guaranteed
|
||||
to appear within `CONFIRMATIONS` blocks. They will exist with arbitrary depth however,
|
||||
meaning that upon completion they will spawn several more Eventualities. The further
|
||||
created Eventualities re-risk being present after the 6 hour period ends.
|
||||
|
||||
We can:
|
||||
1) Build a queue for Branch outputs, delaying their handling until relevant
|
||||
Eventualities are guaranteed to be present.
|
||||
|
||||
This solution would theoretically work for all outputs and allow collapsing this
|
||||
problem to simply:
|
||||
|
||||
> Accordingly, only handling outputs we created should be definable as only
|
||||
handling outputs from the resolution of Eventualities.
|
||||
|
||||
2) Create all Eventualities under a Branch at time of Branch creation.
|
||||
This idea fails as Plans are tightly bound to outputs.
|
||||
|
||||
3) Don't track Branch outputs by Eventualities, yet by the amount of Branch outputs
|
||||
remaining. Any Branch output received, of a useful amount, is assumed to be our
|
||||
own and handled. All other Branch outputs, even if they're the completion of some
|
||||
Eventuality, are dropped.
|
||||
|
||||
This avoids needing any additional queue, avoiding additional pipelining/latency.
|
||||
|
||||
2) External outputs are self-evident. We simply stop handling them at the cut-off point,
|
||||
and only start checking after `CONFIRMATIONS` blocks if all Eventualities are
|
||||
complete.
|
||||
|
||||
3) Since all Change Eventualities will be known prior to the 6 hour window's beginning,
|
||||
we can safely check if a received Change output is the resolution of an Eventuality.
|
||||
We only need to forward it if so. Forwarding it simply requires only checking if
|
||||
Eventualities are complete after `CONFIRMATIONS` blocks, same as for straggling
|
||||
External outputs.
|
||||
*/
|
||||
|
||||
let mut plans = vec![];
|
||||
existing_outputs.retain(|output| {
|
||||
match output.kind() {
|
||||
OutputType::External => false,
|
||||
OutputType::Branch => {
|
||||
let scheduler = &mut self.existing.as_mut().unwrap().scheduler;
|
||||
// There *would* be a race condition here due to the fact we only mark a `Branch` output
|
||||
// as needed when we process the block (and handle scheduling), yet actual `Branch`
|
||||
// outputs may appear as soon as the next block (and we scan the next block before we
|
||||
// process the prior block)
|
||||
//
|
||||
// Unlike Eventuality checking, which happens on scanning and is therefore asynchronous,
|
||||
// all scheduling (and this check against the scheduler) happens on processing, which is
|
||||
// synchronous
|
||||
//
|
||||
// While we could move Eventuality checking into the block processing, removing its
|
||||
// asynchonicity, we could only check data the Scanner deems important. The Scanner won't
|
||||
// deem important Eventuality resolutions which don't create an output to Serai unless
|
||||
// it knows of the Eventuality. Accordingly, at best we could have a split role (the
|
||||
// Scanner noting completion of Eventualities which don't have relevant outputs, the
|
||||
// processing noting completion of ones which do)
|
||||
//
|
||||
// This is unnecessary, due to the current flow around Eventuality resolutions and the
|
||||
// current bounds naturally found being sufficiently amenable, yet notable for the future
|
||||
if scheduler.can_use_branch(output.amount()) {
|
||||
// We could simply call can_use_branch, yet it'd have an edge case where if we receive
|
||||
// two outputs for 100, and we could use one such output, we'd handle both.
|
||||
//
|
||||
// Individually schedule each output once confirming they're usable in order to avoid
|
||||
// this.
|
||||
let mut plan = scheduler.schedule::<D>(
|
||||
txn,
|
||||
vec![output.clone()],
|
||||
vec![],
|
||||
self.new.as_ref().unwrap().key,
|
||||
false,
|
||||
);
|
||||
assert_eq!(plan.len(), 1);
|
||||
let plan = plan.remove(0);
|
||||
plans.push(plan);
|
||||
}
|
||||
false
|
||||
}
|
||||
OutputType::Change => {
|
||||
// If the TX containing this output resolved an Eventuality...
|
||||
if let Some(plan) = MultisigsDb::<N, D>::resolved_plan(txn, output.tx_id()) {
|
||||
// And the Eventuality had change...
|
||||
// We need this check as Eventualities have a race condition and can't be relied
|
||||
// on, as extensively detailed above. Eventualities explicitly with change do have
|
||||
// a safe timing window however
|
||||
if MultisigsDb::<N, D>::plan_by_key_with_self_change(
|
||||
txn,
|
||||
// Pass the key so the DB checks the Plan's key is this multisig's, preventing a
|
||||
// potential issue where the new multisig creates a Plan with change *and a
|
||||
// payment to the existing multisig's change address*
|
||||
self.existing.as_ref().unwrap().key,
|
||||
plan,
|
||||
) {
|
||||
// Then this is an honest change output we need to forward
|
||||
// (or it's a payment to the change address in the same transaction as an honest
|
||||
// change output, which is fine to let slip in)
|
||||
return true;
|
||||
}
|
||||
}
|
||||
false
|
||||
}
|
||||
}
|
||||
});
|
||||
plans
|
||||
}
|
||||
|
||||
// Returns the Plans caused from a block being acknowledged.
|
||||
//
|
||||
// Will rotate keys if the block acknowledged is the retirement block.
|
||||
async fn plans_from_block(
|
||||
&mut self,
|
||||
txn: &mut D::Transaction<'_>,
|
||||
block_number: usize,
|
||||
block_id: <N::Block as Block<N>>::Id,
|
||||
step: &mut RotationStep,
|
||||
burns: Vec<OutInstructionWithBalance>,
|
||||
) -> (bool, Vec<Plan<N>>, HashMap<Vec<u8>, InInstructionWithBalance>) {
|
||||
let (mut existing_payments, mut new_payments) = self.burns_to_payments(txn, *step, burns);
|
||||
|
||||
// We now have to acknowledge the acknowledged block, if it's new
|
||||
// It won't be if this block's `InInstruction`s were split into multiple `Batch`s
|
||||
let (acquired_lock, (mut existing_outputs, new_outputs)) = {
|
||||
let (acquired_lock, outputs) = if ScannerHandle::<N, D>::db_scanned(txn)
|
||||
.expect("published a Batch despite never scanning a block") <
|
||||
block_number
|
||||
{
|
||||
let (is_retirement_block, outputs) = self.scanner.ack_block(txn, block_id.clone()).await;
|
||||
if is_retirement_block {
|
||||
let existing = self.existing.take().unwrap();
|
||||
assert!(existing.scheduler.empty());
|
||||
self.existing = self.new.take();
|
||||
*step = RotationStep::UseExisting;
|
||||
assert!(existing_payments.is_empty());
|
||||
existing_payments = new_payments;
|
||||
new_payments = vec![];
|
||||
}
|
||||
(true, outputs)
|
||||
} else {
|
||||
(false, vec![])
|
||||
};
|
||||
(acquired_lock, self.split_outputs_by_key(outputs))
|
||||
};
|
||||
|
||||
let (mut plans, forwarded_external_outputs) = match *step {
|
||||
RotationStep::UseExisting | RotationStep::NewAsChange => (vec![], HashMap::new()),
|
||||
RotationStep::ForwardFromExisting => {
|
||||
self.filter_outputs_due_to_forwarding(&mut existing_outputs)
|
||||
}
|
||||
RotationStep::ClosingExisting => {
|
||||
(self.filter_outputs_due_to_closing(txn, &mut existing_outputs), HashMap::new())
|
||||
}
|
||||
};
|
||||
|
||||
plans.extend({
|
||||
let existing = self.existing.as_mut().unwrap();
|
||||
let existing_key = existing.key;
|
||||
self.existing.as_mut().unwrap().scheduler.schedule::<D>(
|
||||
txn,
|
||||
existing_outputs,
|
||||
existing_payments,
|
||||
match *step {
|
||||
RotationStep::UseExisting => existing_key,
|
||||
RotationStep::NewAsChange |
|
||||
RotationStep::ForwardFromExisting |
|
||||
RotationStep::ClosingExisting => self.new.as_ref().unwrap().key,
|
||||
},
|
||||
match *step {
|
||||
RotationStep::UseExisting | RotationStep::NewAsChange => false,
|
||||
RotationStep::ForwardFromExisting | RotationStep::ClosingExisting => true,
|
||||
},
|
||||
)
|
||||
});
|
||||
|
||||
for plan in &plans {
|
||||
assert_eq!(plan.key, self.existing.as_ref().unwrap().key);
|
||||
if plan.change == Some(N::change_address(plan.key)) {
|
||||
// Assert these are only created during the expected step
|
||||
match *step {
|
||||
RotationStep::UseExisting => {}
|
||||
RotationStep::NewAsChange |
|
||||
RotationStep::ForwardFromExisting |
|
||||
RotationStep::ClosingExisting => panic!("change was set to self despite rotating"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if let Some(new) = self.new.as_mut() {
|
||||
plans.extend(new.scheduler.schedule::<D>(txn, new_outputs, new_payments, new.key, false));
|
||||
}
|
||||
|
||||
(acquired_lock, plans, forwarded_external_outputs)
|
||||
}
|
||||
|
||||
/// Handle a SubstrateBlock event, building the relevant Plans.
|
||||
pub async fn substrate_block(
|
||||
&mut self,
|
||||
txn: &mut D::Transaction<'_>,
|
||||
network: &N,
|
||||
context: SubstrateContext,
|
||||
burns: Vec<OutInstructionWithBalance>,
|
||||
) -> (bool, Vec<(<N::Curve as Ciphersuite>::G, [u8; 32], N::SignableTransaction, N::Eventuality)>)
|
||||
{
|
||||
let mut block_id = <N::Block as Block<N>>::Id::default();
|
||||
block_id.as_mut().copy_from_slice(context.network_latest_finalized_block.as_ref());
|
||||
let block_number = ScannerHandle::<N, D>::block_number(txn, &block_id)
|
||||
.expect("SubstrateBlock with context we haven't synced");
|
||||
|
||||
// Determine what step of rotation we're currently in
|
||||
let mut step = self.current_rotation_step(block_number);
|
||||
|
||||
// Get the Plans from this block
|
||||
let (acquired_lock, plans, mut forwarded_external_outputs) =
|
||||
self.plans_from_block(txn, block_number, block_id, &mut step, burns).await;
|
||||
|
||||
let res = {
|
||||
let mut res = Vec::with_capacity(plans.len());
|
||||
let fee = get_fee(network, block_number).await;
|
||||
|
||||
for plan in plans {
|
||||
let id = plan.id();
|
||||
info!("preparing plan {}: {:?}", hex::encode(id), plan);
|
||||
|
||||
let key = plan.key;
|
||||
let key_bytes = key.to_bytes();
|
||||
MultisigsDb::<N, D>::save_active_plan(
|
||||
txn,
|
||||
key_bytes.as_ref(),
|
||||
block_number.try_into().unwrap(),
|
||||
&plan,
|
||||
);
|
||||
|
||||
let to_be_forwarded = forwarded_external_outputs.remove(plan.inputs[0].id().as_ref());
|
||||
if to_be_forwarded.is_some() {
|
||||
assert_eq!(plan.inputs.len(), 1);
|
||||
}
|
||||
let (tx, branches) = prepare_send(network, block_number, fee, plan).await;
|
||||
|
||||
// If this is a Plan for an output we're forwarding, we need to save the InInstruction for
|
||||
// its output under the amount successfully forwarded
|
||||
if let Some(mut instruction) = to_be_forwarded {
|
||||
// If we can't successfully create a forwarding TX, simply drop this
|
||||
if let Some(tx) = &tx {
|
||||
instruction.balance.amount.0 -= tx.0.fee();
|
||||
MultisigsDb::<N, D>::save_forwarded_output(txn, instruction);
|
||||
}
|
||||
}
|
||||
|
||||
for branch in branches {
|
||||
let existing = self.existing.as_mut().unwrap();
|
||||
let to_use = if key == existing.key {
|
||||
existing
|
||||
} else {
|
||||
let new = self
|
||||
.new
|
||||
.as_mut()
|
||||
.expect("plan wasn't for existing multisig yet there wasn't a new multisig");
|
||||
assert_eq!(key, new.key);
|
||||
new
|
||||
};
|
||||
|
||||
to_use.scheduler.created_output::<D>(txn, branch.expected, branch.actual);
|
||||
}
|
||||
|
||||
if let Some((tx, eventuality)) = tx {
|
||||
// The main function we return to will send an event to the coordinator which must be
|
||||
// fired before these registered Eventualities have their Completions fired
|
||||
// Safety is derived from a mutable lock on the Scanner being preserved, preventing
|
||||
// scanning (and detection of Eventuality resolutions) before it's released
|
||||
// It's only released by the main function after it does what it will
|
||||
self
|
||||
.scanner
|
||||
.register_eventuality(key_bytes.as_ref(), block_number, id, eventuality.clone())
|
||||
.await;
|
||||
|
||||
res.push((key, id, tx, eventuality));
|
||||
}
|
||||
|
||||
// TODO: If the TX is None, restore its inputs to the scheduler
|
||||
// Otherwise, if the TX had a change output, dropping its inputs would burn funds
|
||||
// Are there exceptional cases upon rotation?
|
||||
}
|
||||
res
|
||||
};
|
||||
(acquired_lock, res)
|
||||
}
|
||||
|
||||
pub async fn release_scanner_lock(&mut self) {
|
||||
self.scanner.release_lock().await;
|
||||
}
|
||||
|
||||
fn scanner_event_to_multisig_event(
|
||||
&self,
|
||||
txn: &mut D::Transaction<'_>,
|
||||
msg: ScannerEvent<N>,
|
||||
) -> MultisigEvent<N> {
|
||||
let (block_number, event) = match msg {
|
||||
ScannerEvent::Block { is_retirement_block, block, outputs } => {
|
||||
// Since the Scanner is asynchronous, the following is a concern for race conditions
|
||||
// We safely know the step of a block since keys are declared, and the Scanner is safe
|
||||
// with respect to the declaration of keys
|
||||
// Accordingly, the following calls regarding new keys and step should be safe
|
||||
let block_number = ScannerHandle::<N, D>::block_number(txn, &block)
|
||||
.expect("didn't have the block number for a block we just scanned");
|
||||
let step = self.current_rotation_step(block_number);
|
||||
|
||||
let mut instructions = vec![];
|
||||
for output in outputs {
|
||||
// If these aren't externally received funds, don't handle it as an instruction
|
||||
if output.kind() != OutputType::External {
|
||||
continue;
|
||||
}
|
||||
|
||||
// If this is an External transaction to the existing multisig, and we're either solely
|
||||
// forwarding or closing the existing multisig, drop it
|
||||
// In the case of the forwarding case, we'll report it once it hits the new multisig
|
||||
if (match step {
|
||||
RotationStep::UseExisting | RotationStep::NewAsChange => false,
|
||||
RotationStep::ForwardFromExisting | RotationStep::ClosingExisting => true,
|
||||
}) && (output.key() == self.existing.as_ref().unwrap().key)
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
let instruction = if let Some(instruction) = instruction_from_output::<N>(&output) {
|
||||
instruction
|
||||
} else {
|
||||
if !output.data().is_empty() {
|
||||
// TODO2: Refund
|
||||
continue;
|
||||
}
|
||||
|
||||
if let Some(instruction) =
|
||||
MultisigsDb::<N, D>::take_forwarded_output(txn, output.amount())
|
||||
{
|
||||
instruction
|
||||
} else {
|
||||
// TODO2: Refund
|
||||
continue;
|
||||
}
|
||||
};
|
||||
|
||||
// Delay External outputs received to new multisig earlier than expected
|
||||
if Some(output.key()) == self.new.as_ref().map(|new| new.key) {
|
||||
match step {
|
||||
RotationStep::UseExisting => {
|
||||
MultisigsDb::<N, D>::save_delayed_output(txn, instruction);
|
||||
continue;
|
||||
}
|
||||
RotationStep::NewAsChange |
|
||||
RotationStep::ForwardFromExisting |
|
||||
RotationStep::ClosingExisting => {}
|
||||
}
|
||||
}
|
||||
|
||||
instructions.push(instruction);
|
||||
}
|
||||
|
||||
// If any outputs were delayed, append them into this block
|
||||
match step {
|
||||
RotationStep::UseExisting => {}
|
||||
RotationStep::NewAsChange |
|
||||
RotationStep::ForwardFromExisting |
|
||||
RotationStep::ClosingExisting => {
|
||||
instructions.extend(MultisigsDb::<N, D>::take_delayed_outputs(txn));
|
||||
}
|
||||
}
|
||||
|
||||
let mut block_hash = [0; 32];
|
||||
block_hash.copy_from_slice(block.as_ref());
|
||||
let mut batch_id = MultisigsDb::<N, D>::next_batch_id(txn);
|
||||
|
||||
// start with empty batch
|
||||
let mut batches = vec![Batch {
|
||||
network: N::NETWORK,
|
||||
id: batch_id,
|
||||
block: BlockHash(block_hash),
|
||||
instructions: vec![],
|
||||
}];
|
||||
|
||||
for instruction in instructions {
|
||||
let batch = batches.last_mut().unwrap();
|
||||
batch.instructions.push(instruction);
|
||||
|
||||
// check if batch is over-size
|
||||
if batch.encode().len() > MAX_BATCH_SIZE {
|
||||
// pop the last instruction so it's back in size
|
||||
let instruction = batch.instructions.pop().unwrap();
|
||||
|
||||
// bump the id for the new batch
|
||||
batch_id += 1;
|
||||
|
||||
// make a new batch with this instruction included
|
||||
batches.push(Batch {
|
||||
network: N::NETWORK,
|
||||
id: batch_id,
|
||||
block: BlockHash(block_hash),
|
||||
instructions: vec![instruction],
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Save the next batch ID
|
||||
MultisigsDb::<N, D>::set_next_batch_id(txn, batch_id + 1);
|
||||
|
||||
(
|
||||
block_number,
|
||||
MultisigEvent::Batches(
|
||||
if is_retirement_block {
|
||||
Some((self.existing.as_ref().unwrap().key, self.new.as_ref().unwrap().key))
|
||||
} else {
|
||||
None
|
||||
},
|
||||
batches,
|
||||
),
|
||||
)
|
||||
}
|
||||
|
||||
// This must be emitted before ScannerEvent::Block for all completions of known Eventualities
|
||||
// within the block. Unknown Eventualities may have their Completed events emitted after
|
||||
// ScannerEvent::Block however.
|
||||
ScannerEvent::Completed(key, block_number, id, tx) => {
|
||||
MultisigsDb::<N, D>::resolve_plan(txn, &key, id, tx.id());
|
||||
(block_number, MultisigEvent::Completed(key, id, tx))
|
||||
}
|
||||
};
|
||||
|
||||
// If we either received a Block event (which will be the trigger when we have no
|
||||
// Plans/Eventualities leading into ClosingExisting), or we received the last Completed for
|
||||
// this multisig, set its retirement block
|
||||
let existing = self.existing.as_ref().unwrap();
|
||||
|
||||
// This multisig is closing
|
||||
let closing = self.current_rotation_step(block_number) == RotationStep::ClosingExisting;
|
||||
// There's nothing left in its Scheduler. This call is safe as:
|
||||
// 1) When ClosingExisting, all outputs should've been already forwarded, preventing
|
||||
// new UTXOs from accumulating.
|
||||
// 2) No new payments should be issued.
|
||||
// 3) While there may be plans, they'll be dropped to create Eventualities.
|
||||
// If this Eventuality is resolved, the Plan has already been dropped.
|
||||
// 4) If this Eventuality will trigger a Plan, it'll still be in the plans HashMap.
|
||||
let scheduler_is_empty = closing && existing.scheduler.empty();
|
||||
// Nothing is still being signed
|
||||
let no_active_plans = scheduler_is_empty &&
|
||||
MultisigsDb::<N, D>::active_plans(txn, existing.key.to_bytes().as_ref()).is_empty();
|
||||
|
||||
self
|
||||
.scanner
|
||||
.multisig_completed
|
||||
// The above explicitly included their predecessor to ensure short-circuiting, yet their
|
||||
// names aren't defined as an aggregate check. Still including all three here ensures all are
|
||||
// used in the final value
|
||||
.send(closing && scheduler_is_empty && no_active_plans)
|
||||
.unwrap();
|
||||
|
||||
event
|
||||
}
|
||||
|
||||
// async fn where dropping the Future causes no state changes
|
||||
// This property is derived from recv having this property, and recv being the only async call
|
||||
pub async fn next_event(&mut self, txn: &RwLock<D::Transaction<'_>>) -> MultisigEvent<N> {
|
||||
let event = self.scanner.events.recv().await.unwrap();
|
||||
|
||||
// No further code is async
|
||||
|
||||
self.scanner_event_to_multisig_event(&mut *txn.write().unwrap(), event)
|
||||
}
|
||||
}
|
||||
727
processor/src/multisigs/scanner.rs
Normal file
727
processor/src/multisigs/scanner.rs
Normal file
@@ -0,0 +1,727 @@
|
||||
use core::marker::PhantomData;
|
||||
use std::{
|
||||
sync::Arc,
|
||||
io::Read,
|
||||
time::Duration,
|
||||
collections::{VecDeque, HashSet, HashMap},
|
||||
};
|
||||
|
||||
use ciphersuite::group::GroupEncoding;
|
||||
use frost::curve::Ciphersuite;
|
||||
|
||||
use log::{info, debug, warn};
|
||||
use tokio::{
|
||||
sync::{RwLockReadGuard, RwLockWriteGuard, RwLock, mpsc},
|
||||
time::sleep,
|
||||
};
|
||||
|
||||
use crate::{
|
||||
Get, DbTxn, Db,
|
||||
networks::{Output, Transaction, EventualitiesTracker, Block, Network},
|
||||
};
|
||||
|
||||
#[derive(Clone, Debug)]
|
||||
pub enum ScannerEvent<N: Network> {
|
||||
// Block scanned
|
||||
Block { is_retirement_block: bool, block: <N::Block as Block<N>>::Id, outputs: Vec<N::Output> },
|
||||
// Eventuality completion found on-chain
|
||||
Completed(Vec<u8>, usize, [u8; 32], N::Transaction),
|
||||
}
|
||||
|
||||
pub type ScannerEventChannel<N> = mpsc::UnboundedReceiver<ScannerEvent<N>>;
|
||||
|
||||
#[derive(Clone, Debug)]
|
||||
struct ScannerDb<N: Network, D: Db>(PhantomData<N>, PhantomData<D>);
|
||||
impl<N: Network, D: Db> ScannerDb<N, D> {
|
||||
fn scanner_key(dst: &'static [u8], key: impl AsRef<[u8]>) -> Vec<u8> {
|
||||
D::key(b"SCANNER", dst, key)
|
||||
}
|
||||
|
||||
fn block_key(number: usize) -> Vec<u8> {
|
||||
Self::scanner_key(b"block_id", u64::try_from(number).unwrap().to_le_bytes())
|
||||
}
|
||||
fn block_number_key(id: &<N::Block as Block<N>>::Id) -> Vec<u8> {
|
||||
Self::scanner_key(b"block_number", id)
|
||||
}
|
||||
fn save_block(txn: &mut D::Transaction<'_>, number: usize, id: &<N::Block as Block<N>>::Id) {
|
||||
txn.put(Self::block_number_key(id), u64::try_from(number).unwrap().to_le_bytes());
|
||||
txn.put(Self::block_key(number), id);
|
||||
}
|
||||
fn block<G: Get>(getter: &G, number: usize) -> Option<<N::Block as Block<N>>::Id> {
|
||||
getter.get(Self::block_key(number)).map(|id| {
|
||||
let mut res = <N::Block as Block<N>>::Id::default();
|
||||
res.as_mut().copy_from_slice(&id);
|
||||
res
|
||||
})
|
||||
}
|
||||
fn block_number<G: Get>(getter: &G, id: &<N::Block as Block<N>>::Id) -> Option<usize> {
|
||||
getter
|
||||
.get(Self::block_number_key(id))
|
||||
.map(|number| u64::from_le_bytes(number.try_into().unwrap()).try_into().unwrap())
|
||||
}
|
||||
|
||||
fn keys_key() -> Vec<u8> {
|
||||
Self::scanner_key(b"keys", b"")
|
||||
}
|
||||
fn register_key(
|
||||
txn: &mut D::Transaction<'_>,
|
||||
activation_number: usize,
|
||||
key: <N::Curve as Ciphersuite>::G,
|
||||
) {
|
||||
let mut keys = txn.get(Self::keys_key()).unwrap_or(vec![]);
|
||||
|
||||
let key_bytes = key.to_bytes();
|
||||
|
||||
let key_len = key_bytes.as_ref().len();
|
||||
assert_eq!(keys.len() % (8 + key_len), 0);
|
||||
|
||||
// Sanity check this key isn't already present
|
||||
let mut i = 0;
|
||||
while i < keys.len() {
|
||||
if &keys[(i + 8) .. ((i + 8) + key_len)] == key_bytes.as_ref() {
|
||||
panic!("adding {} as a key yet it was already present", hex::encode(key_bytes));
|
||||
}
|
||||
i += 8 + key_len;
|
||||
}
|
||||
|
||||
keys.extend(u64::try_from(activation_number).unwrap().to_le_bytes());
|
||||
keys.extend(key_bytes.as_ref());
|
||||
txn.put(Self::keys_key(), keys);
|
||||
}
|
||||
fn keys<G: Get>(getter: &G) -> Vec<(usize, <N::Curve as Ciphersuite>::G)> {
|
||||
let bytes_vec = getter.get(Self::keys_key()).unwrap_or(vec![]);
|
||||
let mut bytes: &[u8] = bytes_vec.as_ref();
|
||||
|
||||
// Assumes keys will be 32 bytes when calculating the capacity
|
||||
// If keys are larger, this may allocate more memory than needed
|
||||
// If keys are smaller, this may require additional allocations
|
||||
// Either are fine
|
||||
let mut res = Vec::with_capacity(bytes.len() / (8 + 32));
|
||||
while !bytes.is_empty() {
|
||||
let mut activation_number = [0; 8];
|
||||
bytes.read_exact(&mut activation_number).unwrap();
|
||||
let activation_number = u64::from_le_bytes(activation_number).try_into().unwrap();
|
||||
|
||||
res.push((activation_number, N::Curve::read_G(&mut bytes).unwrap()));
|
||||
}
|
||||
res
|
||||
}
|
||||
fn retire_key(txn: &mut D::Transaction<'_>) {
|
||||
let keys = Self::keys(txn);
|
||||
assert_eq!(keys.len(), 2);
|
||||
txn.del(Self::keys_key());
|
||||
Self::register_key(txn, keys[1].0, keys[1].1);
|
||||
}
|
||||
|
||||
fn seen_key(id: &<N::Output as Output<N>>::Id) -> Vec<u8> {
|
||||
Self::scanner_key(b"seen", id)
|
||||
}
|
||||
fn seen<G: Get>(getter: &G, id: &<N::Output as Output<N>>::Id) -> bool {
|
||||
getter.get(Self::seen_key(id)).is_some()
|
||||
}
|
||||
|
||||
fn outputs_key(block: &<N::Block as Block<N>>::Id) -> Vec<u8> {
|
||||
Self::scanner_key(b"outputs", block.as_ref())
|
||||
}
|
||||
fn save_outputs(
|
||||
txn: &mut D::Transaction<'_>,
|
||||
block: &<N::Block as Block<N>>::Id,
|
||||
outputs: &[N::Output],
|
||||
) {
|
||||
let mut bytes = Vec::with_capacity(outputs.len() * 64);
|
||||
for output in outputs {
|
||||
output.write(&mut bytes).unwrap();
|
||||
}
|
||||
txn.put(Self::outputs_key(block), bytes);
|
||||
}
|
||||
fn outputs(
|
||||
txn: &D::Transaction<'_>,
|
||||
block: &<N::Block as Block<N>>::Id,
|
||||
) -> Option<Vec<N::Output>> {
|
||||
let bytes_vec = txn.get(Self::outputs_key(block))?;
|
||||
let mut bytes: &[u8] = bytes_vec.as_ref();
|
||||
|
||||
let mut res = vec![];
|
||||
while !bytes.is_empty() {
|
||||
res.push(N::Output::read(&mut bytes).unwrap());
|
||||
}
|
||||
Some(res)
|
||||
}
|
||||
|
||||
fn scanned_block_key() -> Vec<u8> {
|
||||
Self::scanner_key(b"scanned_block", [])
|
||||
}
|
||||
|
||||
fn save_scanned_block(txn: &mut D::Transaction<'_>, block: usize) -> Vec<N::Output> {
|
||||
let id = Self::block(txn, block); // It may be None for the first key rotated to
|
||||
let outputs =
|
||||
if let Some(id) = id.as_ref() { Self::outputs(txn, id).unwrap_or(vec![]) } else { vec![] };
|
||||
|
||||
// Mark all the outputs from this block as seen
|
||||
for output in &outputs {
|
||||
txn.put(Self::seen_key(&output.id()), b"");
|
||||
}
|
||||
|
||||
txn.put(Self::scanned_block_key(), u64::try_from(block).unwrap().to_le_bytes());
|
||||
|
||||
// Return this block's outputs so they can be pruned from the RAM cache
|
||||
outputs
|
||||
}
|
||||
fn latest_scanned_block<G: Get>(getter: &G) -> Option<usize> {
|
||||
getter
|
||||
.get(Self::scanned_block_key())
|
||||
.map(|bytes| u64::from_le_bytes(bytes.try_into().unwrap()).try_into().unwrap())
|
||||
}
|
||||
|
||||
fn retirement_block_key(key: &<N::Curve as Ciphersuite>::G) -> Vec<u8> {
|
||||
Self::scanner_key(b"retirement_block", key.to_bytes())
|
||||
}
|
||||
fn save_retirement_block(
|
||||
txn: &mut D::Transaction<'_>,
|
||||
key: &<N::Curve as Ciphersuite>::G,
|
||||
block: usize,
|
||||
) {
|
||||
txn.put(Self::retirement_block_key(key), u64::try_from(block).unwrap().to_le_bytes());
|
||||
}
|
||||
fn retirement_block<G: Get>(getter: &G, key: &<N::Curve as Ciphersuite>::G) -> Option<usize> {
|
||||
getter
|
||||
.get(Self::retirement_block_key(key))
|
||||
.map(|bytes| usize::try_from(u64::from_le_bytes(bytes.try_into().unwrap())).unwrap())
|
||||
}
|
||||
}
|
||||
|
||||
/// The Scanner emits events relating to the blockchain, notably received outputs.
|
||||
///
|
||||
/// It WILL NOT fail to emit an event, even if it reboots at selected moments.
|
||||
///
|
||||
/// It MAY fire the same event multiple times.
|
||||
#[derive(Debug)]
|
||||
pub struct Scanner<N: Network, D: Db> {
|
||||
_db: PhantomData<D>,
|
||||
|
||||
keys: Vec<(usize, <N::Curve as Ciphersuite>::G)>,
|
||||
|
||||
eventualities: HashMap<Vec<u8>, EventualitiesTracker<N::Eventuality>>,
|
||||
|
||||
ram_scanned: Option<usize>,
|
||||
ram_outputs: HashSet<Vec<u8>>,
|
||||
|
||||
need_ack: VecDeque<usize>,
|
||||
|
||||
events: mpsc::UnboundedSender<ScannerEvent<N>>,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug)]
|
||||
struct ScannerHold<N: Network, D: Db> {
|
||||
scanner: Arc<RwLock<Option<Scanner<N, D>>>>,
|
||||
}
|
||||
impl<N: Network, D: Db> ScannerHold<N, D> {
|
||||
async fn read(&self) -> RwLockReadGuard<'_, Option<Scanner<N, D>>> {
|
||||
loop {
|
||||
let lock = self.scanner.read().await;
|
||||
if lock.is_none() {
|
||||
drop(lock);
|
||||
tokio::task::yield_now().await;
|
||||
continue;
|
||||
}
|
||||
return lock;
|
||||
}
|
||||
}
|
||||
async fn write(&self) -> RwLockWriteGuard<'_, Option<Scanner<N, D>>> {
|
||||
loop {
|
||||
let lock = self.scanner.write().await;
|
||||
if lock.is_none() {
|
||||
drop(lock);
|
||||
tokio::task::yield_now().await;
|
||||
continue;
|
||||
}
|
||||
return lock;
|
||||
}
|
||||
}
|
||||
// This is safe to not check if something else already acquired the Scanner as the only caller is
|
||||
// sequential.
|
||||
async fn long_term_acquire(&self) -> Scanner<N, D> {
|
||||
self.scanner.write().await.take().unwrap()
|
||||
}
|
||||
async fn restore(&self, scanner: Scanner<N, D>) {
|
||||
let _ = self.scanner.write().await.insert(scanner);
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
pub struct ScannerHandle<N: Network, D: Db> {
|
||||
scanner: ScannerHold<N, D>,
|
||||
held_scanner: Option<Scanner<N, D>>,
|
||||
pub events: ScannerEventChannel<N>,
|
||||
pub multisig_completed: mpsc::UnboundedSender<bool>,
|
||||
}
|
||||
|
||||
impl<N: Network, D: Db> ScannerHandle<N, D> {
|
||||
pub async fn ram_scanned(&self) -> usize {
|
||||
self.scanner.read().await.as_ref().unwrap().ram_scanned.unwrap_or(0)
|
||||
}
|
||||
|
||||
/// Register a key to scan for.
|
||||
pub async fn register_key(
|
||||
&mut self,
|
||||
txn: &mut D::Transaction<'_>,
|
||||
activation_number: usize,
|
||||
key: <N::Curve as Ciphersuite>::G,
|
||||
) {
|
||||
let mut scanner_lock = self.scanner.write().await;
|
||||
let scanner = scanner_lock.as_mut().unwrap();
|
||||
assert!(
|
||||
activation_number > scanner.ram_scanned.unwrap_or(0),
|
||||
"activation block of new keys was already scanned",
|
||||
);
|
||||
|
||||
info!("Registering key {} in scanner at {activation_number}", hex::encode(key.to_bytes()));
|
||||
|
||||
if scanner.keys.is_empty() {
|
||||
assert!(scanner.ram_scanned.is_none());
|
||||
scanner.ram_scanned = Some(activation_number);
|
||||
assert!(ScannerDb::<N, D>::save_scanned_block(txn, activation_number).is_empty());
|
||||
}
|
||||
|
||||
ScannerDb::<N, D>::register_key(txn, activation_number, key);
|
||||
scanner.keys.push((activation_number, key));
|
||||
#[cfg(not(test))] // TODO: A test violates this. Improve the test with a better flow
|
||||
assert!(scanner.keys.len() <= 2);
|
||||
|
||||
scanner.eventualities.insert(key.to_bytes().as_ref().to_vec(), EventualitiesTracker::new());
|
||||
}
|
||||
|
||||
pub fn db_scanned<G: Get>(getter: &G) -> Option<usize> {
|
||||
ScannerDb::<N, D>::latest_scanned_block(getter)
|
||||
}
|
||||
|
||||
// This perform a database read which isn't safe with regards to if the value is set or not
|
||||
// It may be set, when it isn't expected to be set, or not set, when it is expected to be set
|
||||
// Since the value is static, if it's set, it's correctly set
|
||||
pub fn block_number<G: Get>(getter: &G, id: &<N::Block as Block<N>>::Id) -> Option<usize> {
|
||||
ScannerDb::<N, D>::block_number(getter, id)
|
||||
}
|
||||
|
||||
/// Acknowledge having handled a block.
|
||||
///
|
||||
/// Creates a lock over the Scanner, preventing its independent scanning operations until
|
||||
/// released.
|
||||
///
|
||||
/// This must only be called on blocks which have been scanned in-memory.
|
||||
pub async fn ack_block(
|
||||
&mut self,
|
||||
txn: &mut D::Transaction<'_>,
|
||||
id: <N::Block as Block<N>>::Id,
|
||||
) -> (bool, Vec<N::Output>) {
|
||||
debug!("block {} acknowledged", hex::encode(&id));
|
||||
|
||||
let mut scanner = self.scanner.long_term_acquire().await;
|
||||
|
||||
// Get the number for this block
|
||||
let number = ScannerDb::<N, D>::block_number(txn, &id)
|
||||
.expect("main loop trying to operate on data we haven't scanned");
|
||||
log::trace!("block {} was {number}", hex::encode(&id));
|
||||
|
||||
let outputs = ScannerDb::<N, D>::save_scanned_block(txn, number);
|
||||
// This has a race condition if we try to ack a block we scanned on a prior boot, and we have
|
||||
// yet to scan it on this boot
|
||||
assert!(number <= scanner.ram_scanned.unwrap());
|
||||
for output in &outputs {
|
||||
assert!(scanner.ram_outputs.remove(output.id().as_ref()));
|
||||
}
|
||||
|
||||
assert_eq!(scanner.need_ack.pop_front().unwrap(), number);
|
||||
|
||||
self.held_scanner = Some(scanner);
|
||||
|
||||
// Load the key from the DB, as it will have already been removed from RAM if retired
|
||||
let key = ScannerDb::<N, D>::keys(txn)[0].1;
|
||||
let is_retirement_block = ScannerDb::<N, D>::retirement_block(txn, &key) == Some(number);
|
||||
if is_retirement_block {
|
||||
ScannerDb::<N, D>::retire_key(txn);
|
||||
}
|
||||
(is_retirement_block, outputs)
|
||||
}
|
||||
|
||||
pub async fn register_eventuality(
|
||||
&mut self,
|
||||
key: &[u8],
|
||||
block_number: usize,
|
||||
id: [u8; 32],
|
||||
eventuality: N::Eventuality,
|
||||
) {
|
||||
let mut lock;
|
||||
// We won't use held_scanner if we're re-registering on boot
|
||||
(if let Some(scanner) = self.held_scanner.as_mut() {
|
||||
scanner
|
||||
} else {
|
||||
lock = Some(self.scanner.write().await);
|
||||
lock.as_mut().unwrap().as_mut().unwrap()
|
||||
})
|
||||
.eventualities
|
||||
.get_mut(key)
|
||||
.unwrap()
|
||||
.register(block_number, id, eventuality)
|
||||
}
|
||||
|
||||
pub async fn release_lock(&mut self) {
|
||||
self.scanner.restore(self.held_scanner.take().unwrap()).await
|
||||
}
|
||||
}
|
||||
|
||||
impl<N: Network, D: Db> Scanner<N, D> {
|
||||
#[allow(clippy::type_complexity, clippy::new_ret_no_self)]
|
||||
pub fn new(
|
||||
network: N,
|
||||
db: D,
|
||||
) -> (ScannerHandle<N, D>, Vec<(usize, <N::Curve as Ciphersuite>::G)>) {
|
||||
let (events_send, events_recv) = mpsc::unbounded_channel();
|
||||
let (multisig_completed_send, multisig_completed_recv) = mpsc::unbounded_channel();
|
||||
|
||||
let keys = ScannerDb::<N, D>::keys(&db);
|
||||
let mut eventualities = HashMap::new();
|
||||
for key in &keys {
|
||||
eventualities.insert(key.1.to_bytes().as_ref().to_vec(), EventualitiesTracker::new());
|
||||
}
|
||||
|
||||
let ram_scanned = ScannerDb::<N, D>::latest_scanned_block(&db);
|
||||
|
||||
let scanner = ScannerHold {
|
||||
scanner: Arc::new(RwLock::new(Some(Scanner {
|
||||
_db: PhantomData,
|
||||
|
||||
keys: keys.clone(),
|
||||
|
||||
eventualities,
|
||||
|
||||
ram_scanned,
|
||||
ram_outputs: HashSet::new(),
|
||||
|
||||
need_ack: VecDeque::new(),
|
||||
|
||||
events: events_send,
|
||||
}))),
|
||||
};
|
||||
tokio::spawn(Scanner::run(db, network, scanner.clone(), multisig_completed_recv));
|
||||
|
||||
(
|
||||
ScannerHandle {
|
||||
scanner,
|
||||
held_scanner: None,
|
||||
events: events_recv,
|
||||
multisig_completed: multisig_completed_send,
|
||||
},
|
||||
keys,
|
||||
)
|
||||
}
|
||||
|
||||
async fn emit(&mut self, event: ScannerEvent<N>) -> bool {
|
||||
if self.events.send(event).is_err() {
|
||||
info!("Scanner handler was dropped. Shutting down?");
|
||||
return false;
|
||||
}
|
||||
true
|
||||
}
|
||||
|
||||
// An async function, to be spawned on a task, to discover and report outputs
|
||||
async fn run(
|
||||
mut db: D,
|
||||
network: N,
|
||||
scanner_hold: ScannerHold<N, D>,
|
||||
mut multisig_completed: mpsc::UnboundedReceiver<bool>,
|
||||
) {
|
||||
loop {
|
||||
let (ram_scanned, latest_block_to_scan) = {
|
||||
// Sleep 5 seconds to prevent hammering the node/scanner lock
|
||||
sleep(Duration::from_secs(5)).await;
|
||||
|
||||
let ram_scanned = {
|
||||
let scanner_lock = scanner_hold.read().await;
|
||||
let scanner = scanner_lock.as_ref().unwrap();
|
||||
|
||||
// If we're not scanning for keys yet, wait until we are
|
||||
if scanner.keys.is_empty() {
|
||||
continue;
|
||||
}
|
||||
|
||||
let ram_scanned = scanner.ram_scanned.unwrap();
|
||||
// If a Batch has taken too long to be published, start waiting until it is before
|
||||
// continuing scanning
|
||||
// Solves a race condition around multisig rotation, documented in the relevant doc
|
||||
// and demonstrated with mini
|
||||
if let Some(needing_ack) = scanner.need_ack.front() {
|
||||
let next = ram_scanned + 1;
|
||||
let limit = needing_ack + N::CONFIRMATIONS;
|
||||
assert!(next <= limit);
|
||||
if next == limit {
|
||||
continue;
|
||||
}
|
||||
};
|
||||
|
||||
ram_scanned
|
||||
};
|
||||
|
||||
(
|
||||
ram_scanned,
|
||||
loop {
|
||||
break match network.get_latest_block_number().await {
|
||||
// Only scan confirmed blocks, which we consider effectively finalized
|
||||
// CONFIRMATIONS - 1 as whatever's in the latest block already has 1 confirm
|
||||
Ok(latest) => latest.saturating_sub(N::CONFIRMATIONS.saturating_sub(1)),
|
||||
Err(_) => {
|
||||
warn!("couldn't get latest block number");
|
||||
sleep(Duration::from_secs(60)).await;
|
||||
continue;
|
||||
}
|
||||
};
|
||||
},
|
||||
)
|
||||
};
|
||||
|
||||
for block_being_scanned in (ram_scanned + 1) ..= latest_block_to_scan {
|
||||
// Redo the checks for if we're too far ahead
|
||||
{
|
||||
let needing_ack = {
|
||||
let scanner_lock = scanner_hold.read().await;
|
||||
let scanner = scanner_lock.as_ref().unwrap();
|
||||
scanner.need_ack.front().cloned()
|
||||
};
|
||||
|
||||
if let Some(needing_ack) = needing_ack {
|
||||
let limit = needing_ack + N::CONFIRMATIONS;
|
||||
assert!(block_being_scanned <= limit);
|
||||
if block_being_scanned == limit {
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let block = match network.get_block(block_being_scanned).await {
|
||||
Ok(block) => block,
|
||||
Err(_) => {
|
||||
warn!("couldn't get block {block_being_scanned}");
|
||||
break;
|
||||
}
|
||||
};
|
||||
let block_id = block.id();
|
||||
|
||||
info!("scanning block: {} ({block_being_scanned})", hex::encode(&block_id));
|
||||
|
||||
// These DB calls are safe, despite not having a txn, since they're static values
|
||||
// There's no issue if they're written in advance of expected (such as on reboot)
|
||||
// They're also only expected here
|
||||
if let Some(id) = ScannerDb::<N, D>::block(&db, block_being_scanned) {
|
||||
if id != block_id {
|
||||
panic!("reorg'd from finalized {} to {}", hex::encode(id), hex::encode(block_id));
|
||||
}
|
||||
} else {
|
||||
// TODO: Move this to an unwrap
|
||||
if let Some(id) = ScannerDb::<N, D>::block(&db, block_being_scanned.saturating_sub(1)) {
|
||||
if id != block.parent() {
|
||||
panic!(
|
||||
"block {} doesn't build off expected parent {}",
|
||||
hex::encode(block_id),
|
||||
hex::encode(id),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
let mut txn = db.txn();
|
||||
ScannerDb::<N, D>::save_block(&mut txn, block_being_scanned, &block_id);
|
||||
txn.commit();
|
||||
}
|
||||
|
||||
// Scan new blocks
|
||||
// TODO: This lock acquisition may be long-lived...
|
||||
let mut scanner_lock = scanner_hold.write().await;
|
||||
let scanner = scanner_lock.as_mut().unwrap();
|
||||
|
||||
let mut has_activation = false;
|
||||
let mut outputs = vec![];
|
||||
let mut completion_block_numbers = vec![];
|
||||
for (activation_number, key) in scanner.keys.clone() {
|
||||
if activation_number > block_being_scanned {
|
||||
continue;
|
||||
}
|
||||
|
||||
if activation_number == block_being_scanned {
|
||||
has_activation = true;
|
||||
}
|
||||
|
||||
let key_vec = key.to_bytes().as_ref().to_vec();
|
||||
|
||||
// TODO: These lines are the ones which will cause a really long-lived lock acquisiton
|
||||
for output in network.get_outputs(&block, key).await {
|
||||
assert_eq!(output.key(), key);
|
||||
outputs.push(output);
|
||||
}
|
||||
|
||||
for (id, (block_number, tx)) in network
|
||||
.get_eventuality_completions(scanner.eventualities.get_mut(&key_vec).unwrap(), &block)
|
||||
.await
|
||||
{
|
||||
info!(
|
||||
"eventuality {} resolved by {}, as found on chain",
|
||||
hex::encode(id),
|
||||
hex::encode(&tx.id())
|
||||
);
|
||||
|
||||
completion_block_numbers.push(block_number);
|
||||
// This must be before the mission of ScannerEvent::Block, per commentary in mod.rs
|
||||
if !scanner.emit(ScannerEvent::Completed(key_vec.clone(), block_number, id, tx)).await {
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Panic if we've already seen these outputs
|
||||
for output in &outputs {
|
||||
let id = output.id();
|
||||
info!(
|
||||
"block {} had output {} worth {}",
|
||||
hex::encode(&block_id),
|
||||
hex::encode(&id),
|
||||
output.amount(),
|
||||
);
|
||||
|
||||
// On Bitcoin, the output ID should be unique for a given chain
|
||||
// On Monero, it's trivial to make an output sharing an ID with another
|
||||
// We should only scan outputs with valid IDs however, which will be unique
|
||||
|
||||
/*
|
||||
The safety of this code must satisfy the following conditions:
|
||||
1) seen is not set for the first occurrence
|
||||
2) seen is set for any future occurrence
|
||||
|
||||
seen is only written to after this code completes. Accordingly, it cannot be set
|
||||
before the first occurrence UNLESSS it's set, yet the last scanned block isn't.
|
||||
They are both written in the same database transaction, preventing this.
|
||||
|
||||
As for future occurrences, the RAM entry ensures they're handled properly even if
|
||||
the database has yet to be set.
|
||||
|
||||
On reboot, which will clear the RAM, if seen wasn't set, neither was latest scanned
|
||||
block. Accordingly, this will scan from some prior block, re-populating the RAM.
|
||||
|
||||
If seen was set, then this will be successfully read.
|
||||
|
||||
There's also no concern ram_outputs was pruned, yet seen wasn't set, as pruning
|
||||
from ram_outputs will acquire a write lock (preventing this code from acquiring
|
||||
its own write lock and running), and during its holding of the write lock, it
|
||||
commits the transaction setting seen and the latest scanned block.
|
||||
|
||||
This last case isn't true. Committing seen/latest_scanned_block happens after
|
||||
relinquishing the write lock.
|
||||
|
||||
TODO2: Only update ram_outputs after committing the TXN in question.
|
||||
*/
|
||||
let seen = ScannerDb::<N, D>::seen(&db, &id);
|
||||
let id = id.as_ref().to_vec();
|
||||
if seen || scanner.ram_outputs.contains(&id) {
|
||||
panic!("scanned an output multiple times");
|
||||
}
|
||||
scanner.ram_outputs.insert(id);
|
||||
}
|
||||
|
||||
// We could remove this, if instead of doing the first block which passed
|
||||
// requirements + CONFIRMATIONS, we simply emitted an event for every block where
|
||||
// `number % CONFIRMATIONS == 0` (once at the final stage for the existing multisig)
|
||||
// There's no need at this point, yet the latter may be more suitable for modeling...
|
||||
async fn check_multisig_completed<N: Network, D: Db>(
|
||||
db: &mut D,
|
||||
multisig_completed: &mut mpsc::UnboundedReceiver<bool>,
|
||||
block_number: usize,
|
||||
) -> bool {
|
||||
match multisig_completed.recv().await {
|
||||
None => {
|
||||
info!("Scanner handler was dropped. Shutting down?");
|
||||
false
|
||||
}
|
||||
Some(completed) => {
|
||||
// Set the retirement block as block_number + CONFIRMATIONS
|
||||
if completed {
|
||||
let mut txn = db.txn();
|
||||
// The retiring key is the earliest one still around
|
||||
let retiring_key = ScannerDb::<N, D>::keys(&txn)[0].1;
|
||||
// This value is static w.r.t. the key
|
||||
ScannerDb::<N, D>::save_retirement_block(
|
||||
&mut txn,
|
||||
&retiring_key,
|
||||
block_number + N::CONFIRMATIONS,
|
||||
);
|
||||
txn.commit();
|
||||
}
|
||||
true
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
drop(scanner_lock);
|
||||
// Now that we've dropped the Scanner lock, we need to handle the multisig_completed
|
||||
// channel before we decide if this block should be fired or not
|
||||
// (holding the Scanner risks a deadlock)
|
||||
for block_number in completion_block_numbers {
|
||||
if !check_multisig_completed::<N, _>(&mut db, &mut multisig_completed, block_number).await
|
||||
{
|
||||
return;
|
||||
};
|
||||
}
|
||||
|
||||
// Reacquire the scanner
|
||||
let mut scanner_lock = scanner_hold.write().await;
|
||||
let scanner = scanner_lock.as_mut().unwrap();
|
||||
|
||||
// Only emit an event if any of the following is true:
|
||||
// - This is an activation block
|
||||
// - This is a retirement block
|
||||
// - There's outputs
|
||||
// as only those are blocks are meaningful and warrant obtaining synchrony over
|
||||
// TODO: Consider not obtaining synchrony over the retirement block depending on how the
|
||||
// hand-off is implemented on the Substrate side of things
|
||||
let is_retirement_block =
|
||||
ScannerDb::<N, D>::retirement_block(&db, &scanner.keys[0].1) == Some(block_being_scanned);
|
||||
let sent_block = if has_activation || is_retirement_block || (!outputs.is_empty()) {
|
||||
// Save the outputs to disk
|
||||
let mut txn = db.txn();
|
||||
ScannerDb::<N, D>::save_outputs(&mut txn, &block_id, &outputs);
|
||||
txn.commit();
|
||||
|
||||
// Send all outputs
|
||||
if !scanner
|
||||
.emit(ScannerEvent::Block { is_retirement_block, block: block_id, outputs })
|
||||
.await
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
scanner.need_ack.push_back(block_being_scanned);
|
||||
true
|
||||
} else {
|
||||
false
|
||||
};
|
||||
|
||||
// Remove it from memory
|
||||
if is_retirement_block {
|
||||
let retired = scanner.keys.remove(0).1;
|
||||
scanner.eventualities.remove(retired.to_bytes().as_ref());
|
||||
}
|
||||
|
||||
// Update ram_scanned/need_ack
|
||||
scanner.ram_scanned = Some(block_being_scanned);
|
||||
|
||||
drop(scanner_lock);
|
||||
// If we sent a Block event, once again check multisig_completed
|
||||
if sent_block &&
|
||||
(!check_multisig_completed::<N, _>(
|
||||
&mut db,
|
||||
&mut multisig_completed,
|
||||
block_being_scanned,
|
||||
)
|
||||
.await)
|
||||
{
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
460
processor/src/multisigs/scheduler.rs
Normal file
460
processor/src/multisigs/scheduler.rs
Normal file
@@ -0,0 +1,460 @@
|
||||
use std::{
|
||||
io::{self, Read},
|
||||
collections::{VecDeque, HashMap},
|
||||
};
|
||||
|
||||
use ciphersuite::{group::GroupEncoding, Ciphersuite};
|
||||
|
||||
use crate::{
|
||||
networks::{OutputType, Output, Network},
|
||||
DbTxn, Db, Payment, Plan,
|
||||
};
|
||||
|
||||
/// Stateless, deterministic output/payment manager.
|
||||
#[derive(PartialEq, Eq, Debug)]
|
||||
pub struct Scheduler<N: Network> {
|
||||
key: <N::Curve as Ciphersuite>::G,
|
||||
|
||||
// Serai, when it has more outputs expected than it can handle in a single tranaction, will
|
||||
// schedule the outputs to be handled later. Immediately, it just creates additional outputs
|
||||
// which will eventually handle those outputs
|
||||
//
|
||||
// These maps map output amounts, which we'll receive in the future, to the payments they should
|
||||
// be used on
|
||||
//
|
||||
// When those output amounts appear, their payments should be scheduled
|
||||
// The Vec<Payment> is for all payments that should be done per output instance
|
||||
// The VecDeque allows multiple sets of payments with the same sum amount to properly co-exist
|
||||
//
|
||||
// queued_plans are for outputs which we will create, yet when created, will have their amount
|
||||
// reduced by the fee it cost to be created. The Scheduler will then be told how what amount the
|
||||
// output actually has, and it'll be moved into plans
|
||||
queued_plans: HashMap<u64, VecDeque<Vec<Payment<N>>>>,
|
||||
plans: HashMap<u64, VecDeque<Vec<Payment<N>>>>,
|
||||
|
||||
// UTXOs available
|
||||
utxos: Vec<N::Output>,
|
||||
|
||||
// Payments awaiting scheduling due to the output availability problem
|
||||
payments: VecDeque<Payment<N>>,
|
||||
}
|
||||
|
||||
fn scheduler_key<D: Db, G: GroupEncoding>(key: &G) -> Vec<u8> {
|
||||
D::key(b"SCHEDULER", b"scheduler", key.to_bytes())
|
||||
}
|
||||
|
||||
impl<N: Network> Scheduler<N> {
|
||||
pub fn empty(&self) -> bool {
|
||||
self.queued_plans.is_empty() &&
|
||||
self.plans.is_empty() &&
|
||||
self.utxos.is_empty() &&
|
||||
self.payments.is_empty()
|
||||
}
|
||||
|
||||
fn read<R: Read>(key: <N::Curve as Ciphersuite>::G, reader: &mut R) -> io::Result<Self> {
|
||||
let mut read_plans = || -> io::Result<_> {
|
||||
let mut all_plans = HashMap::new();
|
||||
let mut all_plans_len = [0; 4];
|
||||
reader.read_exact(&mut all_plans_len)?;
|
||||
for _ in 0 .. u32::from_le_bytes(all_plans_len) {
|
||||
let mut amount = [0; 8];
|
||||
reader.read_exact(&mut amount)?;
|
||||
let amount = u64::from_le_bytes(amount);
|
||||
|
||||
let mut plans = VecDeque::new();
|
||||
let mut plans_len = [0; 4];
|
||||
reader.read_exact(&mut plans_len)?;
|
||||
for _ in 0 .. u32::from_le_bytes(plans_len) {
|
||||
let mut payments = vec![];
|
||||
let mut payments_len = [0; 4];
|
||||
reader.read_exact(&mut payments_len)?;
|
||||
|
||||
for _ in 0 .. u32::from_le_bytes(payments_len) {
|
||||
payments.push(Payment::read(reader)?);
|
||||
}
|
||||
plans.push_back(payments);
|
||||
}
|
||||
all_plans.insert(amount, plans);
|
||||
}
|
||||
Ok(all_plans)
|
||||
};
|
||||
let queued_plans = read_plans()?;
|
||||
let plans = read_plans()?;
|
||||
|
||||
let mut utxos = vec![];
|
||||
let mut utxos_len = [0; 4];
|
||||
reader.read_exact(&mut utxos_len)?;
|
||||
for _ in 0 .. u32::from_le_bytes(utxos_len) {
|
||||
utxos.push(N::Output::read(reader)?);
|
||||
}
|
||||
|
||||
let mut payments = VecDeque::new();
|
||||
let mut payments_len = [0; 4];
|
||||
reader.read_exact(&mut payments_len)?;
|
||||
for _ in 0 .. u32::from_le_bytes(payments_len) {
|
||||
payments.push_back(Payment::read(reader)?);
|
||||
}
|
||||
|
||||
Ok(Scheduler { key, queued_plans, plans, utxos, payments })
|
||||
}
|
||||
|
||||
// TODO2: Get rid of this
|
||||
// We reserialize the entire scheduler on any mutation to save it to the DB which is horrible
|
||||
// We should have an incremental solution
|
||||
fn serialize(&self) -> Vec<u8> {
|
||||
let mut res = Vec::with_capacity(4096);
|
||||
|
||||
let mut write_plans = |plans: &HashMap<u64, VecDeque<Vec<Payment<N>>>>| {
|
||||
res.extend(u32::try_from(plans.len()).unwrap().to_le_bytes());
|
||||
for (amount, list_of_plans) in plans {
|
||||
res.extend(amount.to_le_bytes());
|
||||
res.extend(u32::try_from(list_of_plans.len()).unwrap().to_le_bytes());
|
||||
for plan in list_of_plans {
|
||||
res.extend(u32::try_from(plan.len()).unwrap().to_le_bytes());
|
||||
for payment in plan {
|
||||
payment.write(&mut res).unwrap();
|
||||
}
|
||||
}
|
||||
}
|
||||
};
|
||||
write_plans(&self.queued_plans);
|
||||
write_plans(&self.plans);
|
||||
|
||||
res.extend(u32::try_from(self.utxos.len()).unwrap().to_le_bytes());
|
||||
for utxo in &self.utxos {
|
||||
utxo.write(&mut res).unwrap();
|
||||
}
|
||||
|
||||
res.extend(u32::try_from(self.payments.len()).unwrap().to_le_bytes());
|
||||
for payment in &self.payments {
|
||||
payment.write(&mut res).unwrap();
|
||||
}
|
||||
|
||||
debug_assert_eq!(&Self::read(self.key, &mut res.as_slice()).unwrap(), self);
|
||||
res
|
||||
}
|
||||
|
||||
pub fn new<D: Db>(txn: &mut D::Transaction<'_>, key: <N::Curve as Ciphersuite>::G) -> Self {
|
||||
let res = Scheduler {
|
||||
key,
|
||||
queued_plans: HashMap::new(),
|
||||
plans: HashMap::new(),
|
||||
utxos: vec![],
|
||||
payments: VecDeque::new(),
|
||||
};
|
||||
// Save it to disk so from_db won't panic if we don't mutate it before rebooting
|
||||
txn.put(scheduler_key::<D, _>(&res.key), res.serialize());
|
||||
res
|
||||
}
|
||||
|
||||
pub fn from_db<D: Db>(db: &D, key: <N::Curve as Ciphersuite>::G) -> io::Result<Self> {
|
||||
let scheduler = db.get(scheduler_key::<D, _>(&key)).unwrap_or_else(|| {
|
||||
panic!("loading scheduler from DB without scheduler for {}", hex::encode(key.to_bytes()))
|
||||
});
|
||||
let mut reader_slice = scheduler.as_slice();
|
||||
let reader = &mut reader_slice;
|
||||
|
||||
Self::read(key, reader)
|
||||
}
|
||||
|
||||
pub fn can_use_branch(&self, amount: u64) -> bool {
|
||||
self.plans.contains_key(&amount)
|
||||
}
|
||||
|
||||
fn execute(
|
||||
&mut self,
|
||||
inputs: Vec<N::Output>,
|
||||
mut payments: Vec<Payment<N>>,
|
||||
key_for_any_change: <N::Curve as Ciphersuite>::G,
|
||||
) -> Plan<N> {
|
||||
let mut change = false;
|
||||
let mut max = N::MAX_OUTPUTS;
|
||||
|
||||
let payment_amounts =
|
||||
|payments: &Vec<Payment<N>>| payments.iter().map(|payment| payment.amount).sum::<u64>();
|
||||
|
||||
// Requires a change output
|
||||
if inputs.iter().map(Output::amount).sum::<u64>() != payment_amounts(&payments) {
|
||||
change = true;
|
||||
max -= 1;
|
||||
}
|
||||
|
||||
let mut add_plan = |payments| {
|
||||
let amount = payment_amounts(&payments);
|
||||
#[allow(clippy::unwrap_or_default)]
|
||||
self.queued_plans.entry(amount).or_insert(VecDeque::new()).push_back(payments);
|
||||
amount
|
||||
};
|
||||
|
||||
let branch_address = N::branch_address(self.key);
|
||||
|
||||
// If we have more payments than we can handle in a single TX, create plans for them
|
||||
// TODO2: This isn't perfect. For 258 outputs, and a MAX_OUTPUTS of 16, this will create:
|
||||
// 15 branches of 16 leaves
|
||||
// 1 branch of:
|
||||
// - 1 branch of 16 leaves
|
||||
// - 2 leaves
|
||||
// If this was perfect, the heaviest branch would have 1 branch of 3 leaves and 15 leaves
|
||||
while payments.len() > max {
|
||||
// The resulting TX will have the remaining payments and a new branch payment
|
||||
let to_remove = (payments.len() + 1) - N::MAX_OUTPUTS;
|
||||
// Don't remove more than possible
|
||||
let to_remove = to_remove.min(N::MAX_OUTPUTS);
|
||||
|
||||
// Create the plan
|
||||
let removed = payments.drain((payments.len() - to_remove) ..).collect::<Vec<_>>();
|
||||
assert_eq!(removed.len(), to_remove);
|
||||
let amount = add_plan(removed);
|
||||
|
||||
// Create the payment for the plan
|
||||
// Push it to the front so it's not moved into a branch until all lower-depth items are
|
||||
payments.insert(0, Payment { address: branch_address.clone(), data: None, amount });
|
||||
}
|
||||
|
||||
Plan {
|
||||
key: self.key,
|
||||
inputs,
|
||||
payments,
|
||||
change: Some(N::change_address(key_for_any_change)).filter(|_| change),
|
||||
}
|
||||
}
|
||||
|
||||
fn add_outputs(
|
||||
&mut self,
|
||||
mut utxos: Vec<N::Output>,
|
||||
key_for_any_change: <N::Curve as Ciphersuite>::G,
|
||||
) -> Vec<Plan<N>> {
|
||||
log::info!("adding {} outputs", utxos.len());
|
||||
|
||||
let mut txs = vec![];
|
||||
|
||||
for utxo in utxos.drain(..) {
|
||||
if utxo.kind() == OutputType::Branch {
|
||||
let amount = utxo.amount();
|
||||
if let Some(plans) = self.plans.get_mut(&amount) {
|
||||
// Execute the first set of payments possible with an output of this amount
|
||||
let payments = plans.pop_front().unwrap();
|
||||
// They won't be equal if we dropped payments due to being dust
|
||||
assert!(amount >= payments.iter().map(|payment| payment.amount).sum::<u64>());
|
||||
|
||||
// If we've grabbed the last plan for this output amount, remove it from the map
|
||||
if plans.is_empty() {
|
||||
self.plans.remove(&amount);
|
||||
}
|
||||
|
||||
// Create a TX for these payments
|
||||
txs.push(self.execute(vec![utxo], payments, key_for_any_change));
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
self.utxos.push(utxo);
|
||||
}
|
||||
|
||||
log::info!("{} planned TXs have had their required inputs confirmed", txs.len());
|
||||
txs
|
||||
}
|
||||
|
||||
// Schedule a series of outputs/payments.
|
||||
pub fn schedule<D: Db>(
|
||||
&mut self,
|
||||
txn: &mut D::Transaction<'_>,
|
||||
utxos: Vec<N::Output>,
|
||||
mut payments: Vec<Payment<N>>,
|
||||
key_for_any_change: <N::Curve as Ciphersuite>::G,
|
||||
force_spend: bool,
|
||||
) -> Vec<Plan<N>> {
|
||||
// Drop payments to our own branch address
|
||||
/*
|
||||
created_output will be called any time we send to a branch address. If it's called, and it
|
||||
wasn't expecting to be called, that's almost certainly an error. The only way to guarantee
|
||||
this however is to only have us send to a branch address when creating a branch, hence the
|
||||
dropping of pointless payments.
|
||||
|
||||
This is not comprehensive as a payment may still be made to another active multisig's branch
|
||||
address, depending on timing. This is safe as the issue only occurs when a multisig sends to
|
||||
its *own* branch address, since created_output is called on the signer's Scheduler.
|
||||
*/
|
||||
{
|
||||
let branch_address = N::branch_address(self.key);
|
||||
payments =
|
||||
payments.drain(..).filter(|payment| payment.address != branch_address).collect::<Vec<_>>();
|
||||
}
|
||||
|
||||
let mut plans = self.add_outputs(utxos, key_for_any_change);
|
||||
|
||||
log::info!("scheduling {} new payments", payments.len());
|
||||
|
||||
// Add all new payments to the list of pending payments
|
||||
self.payments.extend(payments);
|
||||
let payments_at_start = self.payments.len();
|
||||
log::info!("{} payments are now scheduled", payments_at_start);
|
||||
|
||||
// If we don't have UTXOs available, don't try to continue
|
||||
if self.utxos.is_empty() {
|
||||
log::info!("no utxos currently avilable");
|
||||
return plans;
|
||||
}
|
||||
|
||||
// Sort UTXOs so the highest valued ones are first
|
||||
self.utxos.sort_by(|a, b| a.amount().cmp(&b.amount()).reverse());
|
||||
|
||||
// We always want to aggregate our UTXOs into a single UTXO in the name of simplicity
|
||||
// We may have more UTXOs than will fit into a TX though
|
||||
// We use the most valuable UTXOs to handle our current payments, and we return aggregation TXs
|
||||
// for the rest of the inputs
|
||||
// Since we do multiple aggregation TXs at once, this will execute in logarithmic time
|
||||
let utxos = self.utxos.drain(..).collect::<Vec<_>>();
|
||||
let mut utxo_chunks =
|
||||
utxos.chunks(N::MAX_INPUTS).map(|chunk| chunk.to_vec()).collect::<Vec<_>>();
|
||||
|
||||
// Use the first chunk for any scheduled payments, since it has the most value
|
||||
let utxos = utxo_chunks.remove(0);
|
||||
|
||||
// If the last chunk exists and only has one output, don't try aggregating it
|
||||
// Just immediately consider it another output
|
||||
if let Some(mut chunk) = utxo_chunks.pop() {
|
||||
if chunk.len() == 1 {
|
||||
self.utxos.push(chunk.pop().unwrap());
|
||||
} else {
|
||||
utxo_chunks.push(chunk);
|
||||
}
|
||||
}
|
||||
|
||||
for chunk in utxo_chunks.drain(..) {
|
||||
// TODO: While payments have their TXs' fees deducted from themselves, that doesn't hold here
|
||||
// We need the documented, but not yet implemented, virtual amount scheme to solve this
|
||||
log::debug!("aggregating a chunk of {} inputs", N::MAX_INPUTS);
|
||||
plans.push(Plan {
|
||||
key: self.key,
|
||||
inputs: chunk,
|
||||
payments: vec![],
|
||||
change: Some(N::change_address(key_for_any_change)),
|
||||
})
|
||||
}
|
||||
|
||||
// We want to use all possible UTXOs for all possible payments
|
||||
let mut balance = utxos.iter().map(Output::amount).sum::<u64>();
|
||||
|
||||
// If we can't fulfill the next payment, we have encountered an instance of the UTXO
|
||||
// availability problem
|
||||
// This shows up in networks like Monero, where because we spent outputs, our change has yet to
|
||||
// re-appear. Since it has yet to re-appear, we only operate with a balance which is a subset
|
||||
// of our total balance
|
||||
// Despite this, we may be ordered to fulfill a payment which is our total balance
|
||||
// The solution is to wait for the temporarily unavailable change outputs to re-appear,
|
||||
// granting us access to our full balance
|
||||
let mut executing = vec![];
|
||||
while !self.payments.is_empty() {
|
||||
let amount = self.payments[0].amount;
|
||||
if balance.checked_sub(amount).is_some() {
|
||||
balance -= amount;
|
||||
executing.push(self.payments.pop_front().unwrap());
|
||||
} else {
|
||||
// Doesn't check if other payments would fit into the current batch as doing so may never
|
||||
// let enough inputs become simultaneously availabile to enable handling of payments[0]
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
// Now that we have the list of payments we can successfully handle right now, create the TX
|
||||
// for them
|
||||
if !executing.is_empty() {
|
||||
plans.push(self.execute(utxos, executing, key_for_any_change));
|
||||
} else {
|
||||
// If we don't have any payments to execute, save these UTXOs for later
|
||||
self.utxos.extend(utxos);
|
||||
}
|
||||
|
||||
// If we're instructed to force a spend, do so
|
||||
// This is used when an old multisig is retiring and we want to always transfer outputs to the
|
||||
// new one, regardless if we currently have payments
|
||||
if force_spend && (!self.utxos.is_empty()) {
|
||||
assert!(self.utxos.len() <= N::MAX_INPUTS);
|
||||
plans.push(Plan {
|
||||
key: self.key,
|
||||
inputs: self.utxos.drain(..).collect::<Vec<_>>(),
|
||||
payments: vec![],
|
||||
change: Some(N::change_address(key_for_any_change)),
|
||||
});
|
||||
}
|
||||
|
||||
txn.put(scheduler_key::<D, _>(&self.key), self.serialize());
|
||||
|
||||
log::info!(
|
||||
"created {} plans containing {} payments to sign",
|
||||
plans.len(),
|
||||
payments_at_start - self.payments.len(),
|
||||
);
|
||||
plans
|
||||
}
|
||||
|
||||
pub fn consume_payments<D: Db>(&mut self, txn: &mut D::Transaction<'_>) -> Vec<Payment<N>> {
|
||||
let res: Vec<_> = self.payments.drain(..).collect();
|
||||
if !res.is_empty() {
|
||||
txn.put(scheduler_key::<D, _>(&self.key), self.serialize());
|
||||
}
|
||||
res
|
||||
}
|
||||
|
||||
// Note a branch output as having been created, with the amount it was actually created with,
|
||||
// or not having been created due to being too small
|
||||
// This can be called whenever, so long as it's properly ordered
|
||||
// (it's independent to Serai/the chain we're scheduling over, yet still expects outputs to be
|
||||
// created in the same order Plans are returned in)
|
||||
pub fn created_output<D: Db>(
|
||||
&mut self,
|
||||
txn: &mut D::Transaction<'_>,
|
||||
expected: u64,
|
||||
actual: Option<u64>,
|
||||
) {
|
||||
log::debug!("output expected to have {} had {:?} after fees", expected, actual);
|
||||
|
||||
// Get the payments this output is expected to handle
|
||||
let queued = self.queued_plans.get_mut(&expected).unwrap();
|
||||
let mut payments = queued.pop_front().unwrap();
|
||||
assert_eq!(expected, payments.iter().map(|payment| payment.amount).sum::<u64>());
|
||||
// If this was the last set of payments at this amount, remove it
|
||||
if queued.is_empty() {
|
||||
self.queued_plans.remove(&expected);
|
||||
}
|
||||
|
||||
// If we didn't actually create this output, return, dropping the child payments
|
||||
let actual = match actual {
|
||||
Some(actual) => actual,
|
||||
None => return,
|
||||
};
|
||||
|
||||
// Amortize the fee amongst all payments
|
||||
// While some networks, like Ethereum, may have some payments take notably more gas, those
|
||||
// payments will have their own gas deducted when they're created. The difference in output
|
||||
// value present here is solely the cost of the branch, which is used for all of these
|
||||
// payments, regardless of how much they'll end up costing
|
||||
let diff = actual - expected;
|
||||
let payments_len = u64::try_from(payments.len()).unwrap();
|
||||
let per_payment = diff / payments_len;
|
||||
// The above division isn't perfect
|
||||
let mut remainder = diff - (per_payment * payments_len);
|
||||
|
||||
for payment in payments.iter_mut() {
|
||||
payment.amount = payment.amount.saturating_sub(per_payment + remainder);
|
||||
// Only subtract the remainder once
|
||||
remainder = 0;
|
||||
}
|
||||
|
||||
// Drop payments now below the dust threshold
|
||||
let payments =
|
||||
payments.drain(..).filter(|payment| payment.amount >= N::DUST).collect::<Vec<_>>();
|
||||
// Sanity check this was done properly
|
||||
assert!(actual >= payments.iter().map(|payment| payment.amount).sum::<u64>());
|
||||
if payments.is_empty() {
|
||||
return;
|
||||
}
|
||||
|
||||
#[allow(clippy::unwrap_or_default)]
|
||||
self.plans.entry(actual).or_insert(VecDeque::new()).push_back(payments);
|
||||
|
||||
// TODO2: This shows how ridiculous the serialize function is
|
||||
txn.put(scheduler_key::<D, _>(&self.key), self.serialize());
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user