Clean the Monero lib for auditing (#577)

* Remove unsafe creation of dalek_ff_group::EdwardsPoint in BP+

* Rename Bulletproofs to Bulletproof, since they are a single Bulletproof

Also bifurcates prove with prove_plus, and adds a few documentation items.

* Make CLSAG signing private

Also adds a bit more documentation and does a bit more tidying.

* Remove the distribution cache

It's a notable bandwidth/performance improvement, yet it's not ready. We need a
dedicated Distribution struct which is managed by the wallet and passed in.
While we can do that now, it's not currently worth the effort.

* Tidy Borromean/MLSAG a tad

* Remove experimental feature from monero-serai

* Move amount_decryption into EncryptedAmount::decrypt

* Various RingCT doc comments

* Begin crate smashing

* Further documentation, start shoring up API boundaries of existing crates

* Document and clean clsag

* Add a dedicated send/recv CLSAG mask struct

Abstracts the types used internally.

Also moves the tests from monero-serai to monero-clsag.

* Smash out monero-bulletproofs

Removes usage of dalek-ff-group/multiexp for curve25519-dalek.

Makes compiling in the generators an optional feature.

Adds a structured batch verifier which should be notably more performant.

Documentation and clean up still necessary.

* Correct no-std builds for monero-clsag and monero-bulletproofs

* Tidy and document monero-bulletproofs

I still don't like the impl of the original Bulletproofs...

* Error if missing documentation

* Smash out MLSAG

* Smash out Borromean

* Tidy up monero-serai as a meta crate

* Smash out RPC, wallet

* Document the RPC

* Improve docs a bit

* Move Protocol to monero-wallet

* Incomplete work on using Option to remove panic cases

* Finish documenting monero-serai

* Remove TODO on reading pseudo_outs for AggregateMlsagBorromean

* Only read transactions with one Input::Gen or all Input::ToKey

Also adds a helper to fetch a transaction's prefix.

* Smash out polyseed

* Smash out seed

* Get the repo to compile again

* Smash out Monero addresses

* Document cargo features

Credit to @hinto-janai for adding such sections to their work on documenting
monero-serai in #568.

* Fix deserializing v2 miner transactions

* Rewrite monero-wallet's send code

I have yet to redo the multisig code and the builder. This should be much
cleaner, albeit slower due to redoing work.

This compiles with clippy --all-features. I have to finish the multisig/builder
for --all-targets to work (and start updating the rest of Serai).

* Add SignableTransaction Read/Write

* Restore Monero multisig TX code

* Correct invalid RPC type def in monero-rpc

* Update monero-wallet tests to compile

Some are _consistently_ failing due to the inputs we attempt to spend being too
young. I'm unsure what's up with that. Most seem to pass _consistently_,
implying it's not a random issue yet some configuration/env aspect.

* Clean and document monero-address

* Sync rest of repo with monero-serai changes

* Represent height/block number as a u32

* Diversify ViewPair/Scanner into ViewPair/GuaranteedViewPair and Scanner/GuaranteedScanner

Also cleans the Scanner impl.

* Remove non-small-order view key bound

Guaranteed addresses are in fact guaranteed even with this due to prefixing key
images causing zeroing the ECDH to not zero the shared key.

* Finish documenting monero-serai

* Correct imports for no-std

* Remove possible panic in monero-serai on systems < 32 bits

This was done by requiring the system's usize can represent a certain number.

* Restore the reserialize chain binary

* fmt, machete, GH CI

* Correct misc TODOs in monero-serai

* Have Monero test runner evaluate an Eventuality for all signed TXs

* Fix a pair of bugs in the decoy tests

Unfortunately, this test is still failing.

* Fix remaining bugs in monero-wallet tests

* Reject torsioned spend keys to ensure we can spend the outputs we scan

* Tidy inlined epee code in the RPC

* Correct the accidental swap of stagenet/testnet address bytes

* Remove unused dep from processor

* Handle Monero fee logic properly in the processor

* Document v2 TX/RCT output relation assumed when scanning

* Adjust how we mine the initial blocks due to some CI test failures

* Fix weight estimation for RctType::ClsagBulletproof TXs

* Again increase the amount of blocks we mine prior to running tests

* Correct the if check about when to mine blocks on start

Finally fixes the lack of decoy candidates failures in CI.

* Run Monero on Debian, even for internal testnets

Change made due to a segfault incurred when locally testing.

https://github.com/monero-project/monero/issues/9141 for the upstream.

* Don't attempt running tests on the verify-chain binary

Adds a minimum XMR fee to the processor and runs fmt.

* Increase minimum Monero fee in processor

I'm truly unsure why this is required right now.

* Distinguish fee from necessary_fee in monero-wallet

If there's no change, the fee is difference of the inputs to the outputs. The
prior code wouldn't check that amount is greater than or equal to the necessary
fee, and returning the would-be change amount as the fee isn't necessarily
helpful.

Now the fee is validated in such cases and the necessary fee is returned,
enabling operating off of that.

* Restore minimum Monero fee from develop
This commit is contained in:
Luke Parker
2024-07-07 03:57:18 -07:00
committed by GitHub
parent 703c6a2358
commit a2c3aba82b
191 changed files with 11037 additions and 8253 deletions

View File

@@ -0,0 +1,44 @@
[package]
name = "monero-primitives"
version = "0.1.0"
description = "Primitives for the Monero protocol"
license = "MIT"
repository = "https://github.com/serai-dex/serai/tree/develop/coins/monero/primitives"
authors = ["Luke Parker <lukeparker5132@gmail.com>"]
edition = "2021"
rust-version = "1.79"
[package.metadata.docs.rs]
all-features = true
rustdoc-args = ["--cfg", "docsrs"]
[lints]
workspace = true
[dependencies]
std-shims = { path = "../../../common/std-shims", version = "^0.1.1", default-features = false }
zeroize = { version = "^1.5", default-features = false, features = ["zeroize_derive"] }
# Cryptographic dependencies
sha3 = { version = "0.10", default-features = false }
curve25519-dalek = { version = "4", default-features = false, features = ["alloc", "zeroize"] }
# Other Monero dependencies
monero-io = { path = "../io", version = "0.1", default-features = false }
monero-generators = { path = "../generators", version = "0.4", default-features = false }
[dev-dependencies]
hex = { version = "0.4", default-features = false, features = ["alloc"] }
[features]
std = [
"std-shims/std",
"zeroize/std",
"sha3/std",
"monero-generators/std",
]
default = ["std"]

View File

@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2022-2024 Luke Parker
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

View File

@@ -0,0 +1,11 @@
# Monero Primitives
Primitive structures and functions for the Monero protocol.
This library is usable under no-std when the `std` feature (on by default) is
disabled.
### Cargo Features
- `std` (on by default): Enables `std` (and with it, more efficient internal
implementations).

View File

@@ -0,0 +1,238 @@
#![cfg_attr(docsrs, feature(doc_auto_cfg))]
#![doc = include_str!("../README.md")]
#![deny(missing_docs)]
#![cfg_attr(not(feature = "std"), no_std)]
use std_shims::{io, vec::Vec};
#[cfg(feature = "std")]
use std_shims::sync::OnceLock;
use zeroize::{Zeroize, ZeroizeOnDrop};
use sha3::{Digest, Keccak256};
use curve25519_dalek::{
constants::ED25519_BASEPOINT_POINT,
traits::VartimePrecomputedMultiscalarMul,
scalar::Scalar,
edwards::{EdwardsPoint, VartimeEdwardsPrecomputation},
};
use monero_io::*;
use monero_generators::H;
mod unreduced_scalar;
pub use unreduced_scalar::UnreducedScalar;
#[cfg(test)]
mod tests;
// On std, we cache some variables in statics.
#[cfg(feature = "std")]
static INV_EIGHT_CELL: OnceLock<Scalar> = OnceLock::new();
/// The inverse of 8 over l.
#[cfg(feature = "std")]
#[allow(non_snake_case)]
pub fn INV_EIGHT() -> Scalar {
*INV_EIGHT_CELL.get_or_init(|| Scalar::from(8u8).invert())
}
// In no-std environments, we prefer the reduced memory use and calculate it ad-hoc.
/// The inverse of 8 over l.
#[cfg(not(feature = "std"))]
#[allow(non_snake_case)]
pub fn INV_EIGHT() -> Scalar {
Scalar::from(8u8).invert()
}
#[cfg(feature = "std")]
static G_PRECOMP_CELL: OnceLock<VartimeEdwardsPrecomputation> = OnceLock::new();
/// A cached (if std) pre-computation of the Ed25519 generator, G.
#[cfg(feature = "std")]
#[allow(non_snake_case)]
pub fn G_PRECOMP() -> &'static VartimeEdwardsPrecomputation {
G_PRECOMP_CELL.get_or_init(|| VartimeEdwardsPrecomputation::new([ED25519_BASEPOINT_POINT]))
}
/// A cached (if std) pre-computation of the Ed25519 generator, G.
#[cfg(not(feature = "std"))]
#[allow(non_snake_case)]
pub fn G_PRECOMP() -> VartimeEdwardsPrecomputation {
VartimeEdwardsPrecomputation::new([ED25519_BASEPOINT_POINT])
}
/// The Keccak-256 hash function.
pub fn keccak256(data: impl AsRef<[u8]>) -> [u8; 32] {
Keccak256::digest(data.as_ref()).into()
}
/// Hash the provided data to a scalar via keccak256(data) % l.
///
/// This function panics if it finds the Keccak-256 preimage for [0; 32].
pub fn keccak256_to_scalar(data: impl AsRef<[u8]>) -> Scalar {
let scalar = Scalar::from_bytes_mod_order(keccak256(data.as_ref()));
// Monero will explicitly error in this case
// This library acknowledges its practical impossibility of it occurring, and doesn't bother to
// code in logic to handle it. That said, if it ever occurs, something must happen in order to
// not generate/verify a proof we believe to be valid when it isn't
assert!(scalar != Scalar::ZERO, "ZERO HASH: {:?}", data.as_ref());
scalar
}
/// Transparent structure representing a Pedersen commitment's contents.
#[allow(non_snake_case)]
#[derive(Clone, PartialEq, Eq, Zeroize, ZeroizeOnDrop)]
pub struct Commitment {
/// The mask for this commitment.
pub mask: Scalar,
/// The amount committed to by this commitment.
pub amount: u64,
}
impl core::fmt::Debug for Commitment {
fn fmt(&self, fmt: &mut core::fmt::Formatter<'_>) -> Result<(), core::fmt::Error> {
fmt.debug_struct("Commitment").field("amount", &self.amount).finish_non_exhaustive()
}
}
impl Commitment {
/// A commitment to zero, defined with a mask of 1 (as to not be the identity).
pub fn zero() -> Commitment {
Commitment { mask: Scalar::ONE, amount: 0 }
}
/// Create a new Commitment.
pub fn new(mask: Scalar, amount: u64) -> Commitment {
Commitment { mask, amount }
}
/// Calculate the Pedersen commitment, as a point, from this transparent structure.
pub fn calculate(&self) -> EdwardsPoint {
EdwardsPoint::vartime_double_scalar_mul_basepoint(&Scalar::from(self.amount), &H(), &self.mask)
}
/// Write the Commitment.
///
/// This is not a Monero protocol defined struct, and this is accordingly not a Monero protocol
/// defined serialization.
pub fn write<W: io::Write>(&self, w: &mut W) -> io::Result<()> {
w.write_all(&self.mask.to_bytes())?;
w.write_all(&self.amount.to_le_bytes())
}
/// Serialize the Commitment to a `Vec<u8>`.
///
/// This is not a Monero protocol defined struct, and this is accordingly not a Monero protocol
/// defined serialization.
pub fn serialize(&self) -> Vec<u8> {
let mut res = Vec::with_capacity(32 + 8);
self.write(&mut res).unwrap();
res
}
/// Read a Commitment.
///
/// This is not a Monero protocol defined struct, and this is accordingly not a Monero protocol
/// defined serialization.
pub fn read<R: io::Read>(r: &mut R) -> io::Result<Commitment> {
Ok(Commitment::new(read_scalar(r)?, read_u64(r)?))
}
}
/// Decoy data, as used for producing Monero's ring signatures.
#[derive(Clone, PartialEq, Eq, Debug, Zeroize, ZeroizeOnDrop)]
pub struct Decoys {
offsets: Vec<u64>,
signer_index: u8,
ring: Vec<[EdwardsPoint; 2]>,
}
#[allow(clippy::len_without_is_empty)]
impl Decoys {
/// Create a new instance of decoy data.
///
/// `offsets` are the positions of each ring member within the Monero blockchain, offset from the
/// prior member's position (with the initial ring member offset from 0).
pub fn new(offsets: Vec<u64>, signer_index: u8, ring: Vec<[EdwardsPoint; 2]>) -> Option<Self> {
if (offsets.len() != ring.len()) || (usize::from(signer_index) >= ring.len()) {
None?;
}
Some(Decoys { offsets, signer_index, ring })
}
/// The length of the ring.
pub fn len(&self) -> usize {
self.offsets.len()
}
/// The positions of the ring members within the Monero blockchain, as their offsets.
///
/// The list is formatted as the position of the first ring member, then the offset from each
/// ring member to its prior.
pub fn offsets(&self) -> &[u64] {
&self.offsets
}
/// The positions of the ring members within the Monero blockchain.
pub fn positions(&self) -> Vec<u64> {
let mut res = Vec::with_capacity(self.len());
res.push(self.offsets[0]);
for m in 1 .. self.len() {
res.push(res[m - 1] + self.offsets[m]);
}
res
}
/// The index of the signer within the ring.
pub fn signer_index(&self) -> u8 {
self.signer_index
}
/// The ring.
pub fn ring(&self) -> &[[EdwardsPoint; 2]] {
&self.ring
}
/// The [key, commitment] pair of the signer.
pub fn signer_ring_members(&self) -> [EdwardsPoint; 2] {
self.ring[usize::from(self.signer_index)]
}
/// Write the Decoys.
///
/// This is not a Monero protocol defined struct, and this is accordingly not a Monero protocol
/// defined serialization.
pub fn write(&self, w: &mut impl io::Write) -> io::Result<()> {
write_vec(write_varint, &self.offsets, w)?;
w.write_all(&[self.signer_index])?;
write_vec(
|pair, w| {
write_point(&pair[0], w)?;
write_point(&pair[1], w)
},
&self.ring,
w,
)
}
/// Serialize the Decoys to a `Vec<u8>`.
///
/// This is not a Monero protocol defined struct, and this is accordingly not a Monero protocol
/// defined serialization.
pub fn serialize(&self) -> Vec<u8> {
let mut res =
Vec::with_capacity((1 + (2 * self.offsets.len())) + 1 + 1 + (self.ring.len() * 64));
self.write(&mut res).unwrap();
res
}
/// Read a set of Decoys.
///
/// This is not a Monero protocol defined struct, and this is accordingly not a Monero protocol
/// defined serialization.
pub fn read(r: &mut impl io::Read) -> io::Result<Decoys> {
Decoys::new(
read_vec(read_varint, r)?,
read_byte(r)?,
read_vec(|r| Ok([read_point(r)?, read_point(r)?]), r)?,
)
.ok_or_else(|| io::Error::other("invalid Decoys"))
}
}

View File

@@ -0,0 +1,32 @@
use curve25519_dalek::scalar::Scalar;
use crate::UnreducedScalar;
#[test]
fn recover_scalars() {
let test_recover = |stored: &str, recovered: &str| {
let stored = UnreducedScalar(hex::decode(stored).unwrap().try_into().unwrap());
let recovered =
Scalar::from_canonical_bytes(hex::decode(recovered).unwrap().try_into().unwrap()).unwrap();
assert_eq!(stored.recover_monero_slide_scalar(), recovered);
};
// https://www.moneroinflation.com/static/data_py/report_scalars_df.pdf
// Table 4.
test_recover(
"cb2be144948166d0a9edb831ea586da0c376efa217871505ad77f6ff80f203f8",
"b8ffd6a1aee47828808ab0d4c8524cb5c376efa217871505ad77f6ff80f20308",
);
test_recover(
"343d3df8a1051c15a400649c423dc4ed58bef49c50caef6ca4a618b80dee22f4",
"21113355bc682e6d7a9d5b3f2137a30259bef49c50caef6ca4a618b80dee2204",
);
test_recover(
"c14f75d612800ca2c1dcfa387a42c9cc086c005bc94b18d204dd61342418eba7",
"4f473804b1d27ab2c789c80ab21d034a096c005bc94b18d204dd61342418eb07",
);
test_recover(
"000102030405060708090a0b0c0d0e0f826c4f6e2329a31bc5bc320af0b2bcbb",
"a124cfd387f461bf3719e03965ee6877826c4f6e2329a31bc5bc320af0b2bc0b",
);
}

View File

@@ -0,0 +1,143 @@
use core::cmp::Ordering;
use std_shims::{
sync::OnceLock,
io::{self, *},
};
use zeroize::Zeroize;
use curve25519_dalek::scalar::Scalar;
use monero_io::*;
static PRECOMPUTED_SCALARS_CELL: OnceLock<[Scalar; 8]> = OnceLock::new();
// Precomputed scalars used to recover an incorrectly reduced scalar.
#[allow(non_snake_case)]
fn PRECOMPUTED_SCALARS() -> [Scalar; 8] {
*PRECOMPUTED_SCALARS_CELL.get_or_init(|| {
let mut precomputed_scalars = [Scalar::ONE; 8];
for (i, scalar) in precomputed_scalars.iter_mut().enumerate().skip(1) {
*scalar = Scalar::from(u8::try_from((i * 2) + 1).unwrap());
}
precomputed_scalars
})
}
/// An unreduced scalar.
///
/// While most of modern Monero enforces scalars be reduced, certain legacy parts of the code did
/// not. These section can generally simply be read as a scalar/reduced into a scalar when the time
/// comes, yet a couple have non-standard reductions performed.
///
/// This struct delays scalar conversions and offers the non-standard reduction.
#[derive(Clone, PartialEq, Eq, Debug, Zeroize)]
pub struct UnreducedScalar(pub [u8; 32]);
impl UnreducedScalar {
/// Write an UnreducedScalar.
pub fn write<W: Write>(&self, w: &mut W) -> io::Result<()> {
w.write_all(&self.0)
}
/// Read an UnreducedScalar.
pub fn read<R: Read>(r: &mut R) -> io::Result<UnreducedScalar> {
Ok(UnreducedScalar(read_bytes(r)?))
}
fn as_bits(&self) -> [u8; 256] {
let mut bits = [0; 256];
for (i, bit) in bits.iter_mut().enumerate() {
*bit = core::hint::black_box(1 & (self.0[i / 8] >> (i % 8)))
}
bits
}
// Computes the non-adjacent form of this scalar with width 5.
//
// This matches Monero's `slide` function and intentionally gives incorrect outputs under
// certain conditions in order to match Monero.
//
// This function does not execute in constant time.
fn non_adjacent_form(&self) -> [i8; 256] {
let bits = self.as_bits();
let mut naf = [0i8; 256];
for (b, bit) in bits.into_iter().enumerate() {
naf[b] = i8::try_from(bit).unwrap();
}
for i in 0 .. 256 {
if naf[i] != 0 {
// if the bit is a one, work our way up through the window
// combining the bits with this bit.
for b in 1 .. 6 {
if (i + b) >= 256 {
// if we are at the length of the array then break out
// the loop.
break;
}
// potential_carry - the value of the bit at i+b compared to the bit at i
let potential_carry = naf[i + b] << b;
if potential_carry != 0 {
if (naf[i] + potential_carry) <= 15 {
// if our current "bit" plus the potential carry is less than 16
// add it to our current "bit" and set the potential carry bit to 0.
naf[i] += potential_carry;
naf[i + b] = 0;
} else if (naf[i] - potential_carry) >= -15 {
// else if our current "bit" minus the potential carry is more than -16
// take it away from our current "bit".
// we then work our way up through the bits setting ones to zero, when
// we hit the first zero we change it to one then stop, this is to factor
// in the minus.
naf[i] -= potential_carry;
#[allow(clippy::needless_range_loop)]
for k in (i + b) .. 256 {
if naf[k] == 0 {
naf[k] = 1;
break;
}
naf[k] = 0;
}
} else {
break;
}
}
}
}
}
naf
}
/// Recover the scalar that an array of bytes was incorrectly interpreted as by Monero's `slide`
/// function.
///
/// In Borromean range proofs, Monero was not checking that the scalars used were
/// reduced. This lead to the scalar stored being interpreted as a different scalar.
/// This function recovers that scalar.
///
/// See <https://github.com/monero-project/monero/issues/8438> for more info.
pub fn recover_monero_slide_scalar(&self) -> Scalar {
if self.0[31] & 128 == 0 {
// Computing the w-NAF of a number can only give an output with 1 more bit than
// the number, so even if the number isn't reduced, the `slide` function will be
// correct when the last bit isn't set.
return Scalar::from_bytes_mod_order(self.0);
}
let precomputed_scalars = PRECOMPUTED_SCALARS();
let mut recovered = Scalar::ZERO;
for &numb in self.non_adjacent_form().iter().rev() {
recovered += recovered;
match numb.cmp(&0) {
Ordering::Greater => recovered += precomputed_scalars[usize::try_from(numb).unwrap() / 2],
Ordering::Less => recovered -= precomputed_scalars[usize::try_from(-numb).unwrap() / 2],
Ordering::Equal => (),
}
}
recovered
}
}