diff --git a/docs/processor/UTXO Management.md b/docs/processor/UTXO Management.md new file mode 100644 index 00000000..130f8ce3 --- /dev/null +++ b/docs/processor/UTXO Management.md @@ -0,0 +1,205 @@ +# UTXO Management + +UTXO-based chains have practical requirements for efficient operation which can +effectively be guaranteed to terminate with a safe end state. This document +attempts to detail such requirements, and the implementations in Serai resolving +them. + +### Fees From Effecting Transactions Out + +When `sriXYZ` is burnt, Serai is expected to create an output for `XYZ` as +instructed. The transaction containing this output will presumably have some fee +necessitating payment. Serai linearly amortizes this fee over all outputs this +transaction intends to create in response to burns. + +While Serai could charge a fee in advance, either static or dynamic to views of +the fee market, it'd risk the fee being inaccurate. If it's too high, users have +paid fees they shouldn't have. If it's too low, Serai is insolvent. This is why +the actual fee is amortized, rather than an estimation being prepaid. + +Serai could report a view, and when burning occurred, that view could be locked +in as the basis for transaction fees as used to fulfill the output in question. +This would require burns specify the most recent fee market view they're aware +of, signifying their agreeance, with Serai erroring is a new view is published +before the burn is included on-chain. Not only would this require more data be +published to Serai (widening data pipeline requirements), it'd prevent any +RBF-based solutions to dynamic fee markets causing transactions to get stuck. + +### Output Frequency + +Outputs can be created on an external network at rate +`max_outputs_per_tx / external_tick_rate`, where `external_tick_rate` is the +external's network limitations on spending outputs. While `external_tick_rate` +is generally writable as zero, due to mempool chaining, some external networks +may not allow spending outputs from transactions which have yet to be ordered. +Monero only allows spending outputs from transactions who have 10 confirmations, +for its own security. + +Serai defines its own tick rate per external network, such that +`serai_tick_rate >= external_tick_rate`. This ensures that Serai never assumes +availability before actual availability. `serai_tick_rate` is also `> 0`. This +is since a zero `external_tick_rate` generally does not truly allow an infinite +output creation rate due to limitations on the amount of transactions allowed +in the mempool. + +Define `output_creation_rate` as `max_outputs_per_tx / serai_tick_rate`. Under a +naive system which greedily accumulates inputs and linearly processes outputs, +this is the highest speed at which outputs which may be processed. + +If the Serai blockchain enables burning sriXYZ at a rate exceeding +`output_creation_rate`, a backlog would form. This backlog could linearly grow +at a rate larger than the outputs could linearly shrink, creating an +ever-growing backlog, performing a DoS against Serai. + +One solution would be to increase the fee associated with burning sriXYZ when +approaching `output_creation_rate`, making such a DoS unsustainable. This would +require the Serai blockchain be aware of each external network's +`output_creation_rate` and implement such a sliding fee. This 'solution' isn't +preferred as it still temporarily has a growing queue, and normal users would +also be affected by the increased fees. + +The solution implemented into Serai is to consume all burns from the start of a +global queue which can be satisfied under currently available inputs. While the +consumed queue may have 256 items, which can't be processed within a single tick +by an external network whose `output_creation_rate` is 16, Serai can immediately +set a finite bound on execution duration. + +For the above example parameters, Serai would create 16 outputs within its tick, +ignoring the necessity of a change output. These 16 outputs would _not_ create +any outputs Serai is expected to create in response to burns, yet instead create +16 "branch" outputs. One tick later, when the branch outputs are available to +spend, each would fund creating of 16 expected outputs. + +For `e` expected outputs, the execution duration is just `log e` ticks _with the +base of the logarithm being `output_creation_rate`_. Since these `e` expected +outputs are consumed from the linearly-implemented global queue into their own +tree structure, execution duration cannot be extended. We can also re-consume +the entire global queue (barring input availability, see next section) after +just one tick, when the change output becomes available again. + +Due to the logarithmic complexity of fulfilling burns, attacks require +exponential growth (which is infeasible to scale). This solution does not +require a sliding fee on Serai's side due to not needing to limit the on-chain +rate of burns, which means it doesn't so adversely affect normal users. While +an increased tree depth will increase the amount of transactions needed to +fulfill an output, increasing the fee amortized over the output and its +siblings, this fee scales linearly with the logarithmically scaling tree depth. +This is considered acceptable. + +### Input Availability + +The following section refers to spending an output, and then spending it again. +Spending it again, which is impossible under the UTXO model, refers to spending +the change output of the transaction it was spent in. The following section +also assumes any published transaction is immediately ordered on-chain, ignoring +the potential for latency from mempool to blockchain (as it is assumed to have a +negligible effect in practice). + +When a burn for amount `a` is issued, the sum amount of immediately available +inputs may be `< a`. This is because despite each output being considered usable +on a tick basis, there is no global tick. Each output may or may not be +spendable at some moment, and spending it will prevent its availability for one +tick of a clock newly started. + +This means all outputs will become available by simply waiting a single tick, +without spending any outputs during the waited tick. Any outputs unlocked at the +start of the tick will carry, and within the tick the rest of the outputs will +become unlocked. + +This means that within a tick of operations, the full balance of Serai can be +considered unlocked and used to consume the entire global queue. While Serai +could wait for all its outputs to be available before popping from the front of +the global queue, eager execution as enough inputs become available provides +lower latency. Considering the tick may be an hour (as in the case of Bitcoin), +this is very appreciated. + +If a full tick is waited for, due to the front of the global queue having a +notably large burn, then the entire global queue will be consumed as full input +availability means the ability to satisfy all potential burns in a solvent +system. + +### Fees Incurred During Operations + +While fees incurred when satisfying burn were covered above, with documentation +on how solvency is maintained, two other operating costs exists. + +1) Input accumulation +2) Multisig rotations + +Input accumulation refers to transactions which exist to merge inputs. Just as +there is a `max_outputs_per_tx`, there is a `max_inputs_per_tx`. When the amount +of inputs belonging to Serai exceeds `max_inputs_per_tx`, a TX merging them is +created. This TX incurs fees yet has no outputs mapping to burns to amortize +them over, creating an insolvency. + +Please note that this merging occurs in parallel to create a logarithmic +execution, similar to how outputs are also processed in parallel. + +As for multisig rotation, multisig rotation occurs when a new multisig for an +external network is created and the old multisig must transfer its inputs in +order for Serai to continue its operations. This operation also incurs fees +without having outputs immediately available to amortize over. + +Serai could charge fees on received outputs, deducting from the amount of +`sriXYZ` minted in order to cover these operating fees. An overt amount would be +deducted to practically ensure solvency, forming a buffer. Once the buffer is +filled, fees would be reduced. As the buffer drains, fees would go back up. + +This would keep charged fees in line with actual fees, once the buffer is +initially filled, yet requires: + +1) Creating and tracking a buffer +2) Overcharging some users on fees + +while still risking insolvency, if the actual fees keep increasing in a way +preventing successful estimation. + +The solution Serai implements is to accrue insolvency, tracking each output with +a virtual amount (the amount it represents on Serai) and the actual amount. When +the output, or a descendant of it, is used to handle burns, the discrepancy +between the virtual amount and the amount is amortized over outputs. This +restores solvency while solely charging the actual fees, making Serai a +generally insolvent, always eventually solvent system. + +There is the concern that a significant amount of outputs could be created, +which when merged as inputs, create a significant amount of fees as an +insolvency. This would then be forced onto random users, while the party who +created the insolvency would then be able to burn their own `sriXYZ` without +the notable insolvency. + +To describe this attack in its optimal form, assume a sole malicious block +producer for an external network where `max_inputs_per_tx` is 16. The malicious +miner adds 256 outputs to Serai, not paying any fees as the block producer. +Serai must create 16 transactions to produce a set of 16 inputs, paying for 16 +transaction fees in the process (the fees of which go to the malicious miner). + +When Serai users burn `sriXYZ`, they are hit with the 16 transaction fees plus +the normally amortized fee. Then, the malicious miner burns their `sriXYZ`, +having the fee they capture be amortized over their output. In this process, +they remain net except for the 16 transaction fees they gain from other users, +which they profit. + +A miner only has to have 7% of the external network's hash power to execute this +attack profitably. By only minting `sriXYZ` during their blocks, they pay no +fees. Then, _a miner_, which has a 7% chance of being themselves, collects the +16 transaction fees. Finally, they burn, with a 7% chance of collecting their +own fee, or a 93% chance of losing a single transaction fee. + +16 attempts, costing 16 transaction fees if they always lose their single +transaction fee, will cause a slight edge they gain the 16 transaction fees at +least once, offsetting their costs. + +To limit this attack vector, a flat fee of +`2 * (the estimation of an input-merging transaction fee) / max_inputs_per_tx` +is applied to each input. This means, assuming an inability to manipulate +Serai's fee estimations, creating 16 outputs to force a merge transaction (and +the associated fee) costs the attacker twice as much as the associated fee. + +Even without the above flat fee, Serai remains solvent. With the above flat fee, +malicious miners on external networks can only steal from other users if they +can manipulate Serai's fee estimations so that the merge transaction fee used is +twice as high as the fees charged for causing a merge transaction. This is +assumed infeasible to perform at scale, yet even if demonstrated feasible, it +would not be a critical vulnerability against Serai. Solely a low/medium/high +vulnerability against the users (though one it would still be our responsibility +to rectify). diff --git a/processor/src/scheduler.rs b/processor/src/scheduler.rs index b333a14c..db089043 100644 --- a/processor/src/scheduler.rs +++ b/processor/src/scheduler.rs @@ -307,7 +307,7 @@ impl Scheduler { // This shows up in networks like Monero, where because we spent outputs, our change has yet to // re-appear. Since it has yet to re-appear, we only operate with a balance which is a subset // of our total balance - // Despite this, we may be order to fulfill a payment which is our total balance + // Despite this, we may be ordered to fulfill a payment which is our total balance // The solution is to wait for the temporarily unavailable change outputs to re-appear, // granting us access to our full balance let mut executing = vec![]; @@ -317,7 +317,8 @@ impl Scheduler { balance -= amount; executing.push(self.payments.pop_front().unwrap()); } else { - // TODO: We could continue checking other payments which aren't [0] + // Doesn't check if other payments would fit into the current batch as doing so may never + // let enough inputs become simultaneously availabile to enable handling of payments[0] break; } }