Initial In Instructions pallet and Serai client lib (#233)

* Initial work on an In Inherents pallet * Add an event for when a batch is executed * Add a dummy provider for InInstructions * Add in-instructions to the node * Add the Serai runtime API to the processor * Move processor tests around * Build a subxt Client around Serai * Successfully get Batch events from Serai Renamed processor/substrate to processor/serai. * Much more robust InInstruction pallet * Implement the workaround from https://github.com/paritytech/subxt/issues/602 * Initial prototype of processor generated InInstructions * Correct PendingCoins data flow for InInstructions * Minor lint to in-instructions * Remove the global Serai connection for a partial re-impl * Correct ID handling of the processor test * Workaround the delay in the subscription * Make an unwrap an if let Some, remove old comments * Lint the processor toml * Rebase and update * Move substrate/in-instructions to substrate/in-instructions/pallet * Start an in-instructions primitives lib * Properly update processor to subxt 0.24 Also corrects failures from the rebase. * in-instructions cargo update * Implement IsFatalError * is_inherent -> true * Rename in-instructions crates and misc cleanup * Update documentation * cargo update * Misc update fixes * Replace height with block_number * Update processor src to latest subxt * Correct pipeline for InInstructions testing * Remove runtime::AccountId for serai_primitives::NativeAddress * Rewrite the in-instructions pallet Complete with respect to the currently written docs. Drops the custom serializer for just using SCALE. Makes slight tweaks as relevant. * Move instructions' InherentDataProvider to a client crate * Correct doc gen * Add serde to in-instructions-primitives * Add in-instructions-primitives to pallet * Heights -> BlockNumbers * Get batch pub test loop working * Update in instructions pallet terminology Removes the ambiguous Coin for Update. Removes pending/artificial latency for furture client work. Also moves to using serai_primitives::Coin. * Add a BlockNumber primitive * Belated cargo fmt * Further document why DifferentBatch isn't fatal * Correct processor sleeps * Remove metadata at compile time, add test framework for Serai nodes * Remove manual RPC client * Simplify update test * Improve re-exporting behavior of serai-runtime It now re-exports all pallets underneath it. * Add a function to get storage values to the Serai RPC * Update substrate/ to latest substrate * Create a dedicated crate for the Serai RPC * Remove unused dependencies in substrate/ * Remove unused dependencies in coins/ Out of scope for this branch, just minor and path of least resistance. * Use substrate/serai/client for the Serai RPC lib It's a bit out of place, since these client folders are intended for the node to access pallets and so on. This is for end-users to access Serai as a whole. In that sense, it made more sense as a top level folder, yet that also felt out of place. * Move InInstructions test to serai-client for now * Final cleanup * Update deny.toml * Cargo.lock update from merging develop * Update nightly Attempt to work around the current CI failure, which is a Rust ICE. We previously didn't upgrade due to clippy 10134, yet that's been reverted. * clippy * clippy * fmt * NativeAddress -> SeraiAddress * Sec fix on non-provided updates and doc fixes * Add Serai as a Coin Necessary in order to swap to Serai. * Add a BlockHash type, used for batch IDs * Remove origin from InInstruction Makes InInstructionTarget. Adds RefundableInInstruction with origin. * Document storage items in in-instructions * Rename serai/client/tests/serai.rs to updates.rs It only tested publishing updates and their successful acceptance.
2025-12-08 20:29:23 +00:00 · 2023-01-20 11:00:18 -05:00
parent e13cf52c49
commit 8ca90e7905
53 changed files with 1613 additions and 403 deletions
--- a/substrate/in-instructions/pallet/Cargo.toml
+++ b/substrate/in-instructions/pallet/Cargo.toml
@@ -0,0 +1,51 @@
+[package]
+name = "in-instructions-pallet"
+version = "0.1.0"
+description = "Execute calls via In Instructions from inherent transactions"
+license = "AGPL-3.0-only"
+authors = ["Luke Parker <lukeparker5132@gmail.com>"]
+edition = "2021"
+publish = false
+
+[package.metadata.docs.rs]
+all-features = true
+rustdoc-args = ["--cfg", "docsrs"]
+
+[dependencies]
+thiserror = { version = "1", optional = true }
+
+scale = { package = "parity-scale-codec", version = "3", default-features = false, features = ["derive", "max-encoded-len"] }
+scale-info = { version = "2", default-features = false, features = ["derive"] }
+
+serde = { version = "1", optional = true }
+
+sp-std = { git = "https://github.com/serai-dex/substrate", default-features = false }
+sp-inherents = { git = "https://github.com/serai-dex/substrate", default-features = false }
+sp-runtime = { git = "https://github.com/serai-dex/substrate", default-features = false }
+
+frame-system = { git = "https://github.com/serai-dex/substrate", default-features = false }
+frame-support = { git = "https://github.com/serai-dex/substrate", default-features = false }
+
+serai-primitives = { path = "../../serai/primitives", default-features = false }
+in-instructions-primitives = { path = "../primitives", default-features = false }
+
+[features]
+std = [
+  "thiserror",
+
+  "scale/std",
+  "scale-info/std",
+
+  "serde",
+
+  "sp-std/std",
+  "sp-inherents/std",
+  "sp-runtime/std",
+
+  "frame-system/std",
+  "frame-support/std",
+
+  "serai-primitives/std",
+  "in-instructions-primitives/std",
+]
+default = ["std"]
--- a/substrate/in-instructions/pallet/LICENSE
+++ b/substrate/in-instructions/pallet/LICENSE
@@ -0,0 +1,15 @@
+AGPL-3.0-only license
+
+Copyright (c) 2022-2023 Luke Parker
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU Affero General Public License Version 3 as
+published by the Free Software Foundation.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU Affero General Public License for more details.
+
+You should have received a copy of the GNU Affero General Public License
+along with this program. If not, see <http://www.gnu.org/licenses/>.
--- a/substrate/in-instructions/pallet/src/lib.rs
+++ b/substrate/in-instructions/pallet/src/lib.rs
@@ -0,0 +1,240 @@
+#![cfg_attr(docsrs, feature(doc_cfg))]
+#![cfg_attr(docsrs, feature(doc_auto_cfg))]
+#![cfg_attr(not(feature = "std"), no_std)]
+
+use scale::{Encode, Decode};
+use scale_info::TypeInfo;
+
+#[cfg(feature = "std")]
+use serde::{Serialize, Deserialize};
+
+use sp_std::vec::Vec;
+use sp_inherents::{InherentIdentifier, IsFatalError};
+
+use sp_runtime::RuntimeDebug;
+
+use serai_primitives::{BlockNumber, BlockHash, Coin};
+
+pub use in_instructions_primitives as primitives;
+use primitives::InInstruction;
+
+pub const INHERENT_IDENTIFIER: InherentIdentifier = *b"ininstrs";
+
+#[derive(Clone, PartialEq, Eq, Encode, Decode, TypeInfo, RuntimeDebug)]
+#[cfg_attr(feature = "std", derive(Serialize, Deserialize))]
+pub struct Batch {
+  pub id: BlockHash,
+  pub instructions: Vec<InInstruction>,
+}
+
+#[derive(Clone, PartialEq, Eq, Encode, Decode, TypeInfo, RuntimeDebug)]
+#[cfg_attr(feature = "std", derive(Serialize, Deserialize))]
+pub struct Update {
+  // Coin's latest block number
+  pub block_number: BlockNumber,
+  pub batches: Vec<Batch>,
+}
+
+// None if the current block producer isn't operating over this coin or otherwise failed to get
+// data
+pub type Updates = Vec<Option<Update>>;
+
+#[derive(Clone, Copy, Encode, RuntimeDebug)]
+#[cfg_attr(feature = "std", derive(Decode, thiserror::Error))]
+pub enum InherentError {
+  #[cfg_attr(feature = "std", error("invalid call"))]
+  InvalidCall,
+  #[cfg_attr(feature = "std", error("inherent has {0} updates despite us having {1} coins"))]
+  InvalidUpdateQuantity(u32, u32),
+  #[cfg_attr(
+    feature = "std",
+    error("inherent for coin {0:?} has block number {1:?} despite us having {2:?}")
+  )]
+  UnrecognizedBlockNumber(Coin, BlockNumber, BlockNumber),
+  #[cfg_attr(
+    feature = "std",
+    error("inherent for coin {0:?} has block number {1:?} which doesn't succeed {2:?}")
+  )]
+  InvalidBlockNumber(Coin, BlockNumber, BlockNumber),
+  #[cfg_attr(feature = "std", error("coin {0:?} has {1} more batches than we do"))]
+  UnrecognizedBatches(Coin, u32),
+  #[cfg_attr(feature = "std", error("coin {0:?} has a different batch (ID {1:?})"))]
+  DifferentBatch(Coin, BlockHash),
+}
+
+impl IsFatalError for InherentError {
+  fn is_fatal_error(&self) -> bool {
+    match self {
+      InherentError::InvalidCall | InherentError::InvalidUpdateQuantity(..) => true,
+      InherentError::UnrecognizedBlockNumber(..) => false,
+      InherentError::InvalidBlockNumber(..) => true,
+      InherentError::UnrecognizedBatches(..) => false,
+      // One of our nodes is definitively wrong. If it's ours (signified by it passing consensus),
+      // we should panic. If it's theirs, they should be slashed
+      // Unfortunately, we can't return fatal here to trigger a slash as fatal should only be used
+      // for undeniable, technical invalidity
+      // TODO: Code a way in which this still triggers a slash vote
+      InherentError::DifferentBatch(..) => false,
+    }
+  }
+}
+
+fn coin_from_index(index: usize) -> Coin {
+  // Offset by 1 since Serai is the first coin, yet Serai doesn't have updates
+  Coin::from(1 + u32::try_from(index).unwrap())
+}
+
+#[frame_support::pallet]
+pub mod pallet {
+  use frame_support::pallet_prelude::*;
+  use frame_system::pallet_prelude::*;
+
+  use super::*;
+
+  #[pallet::config]
+  pub trait Config: frame_system::Config<BlockNumber = u32> {
+    type RuntimeEvent: From<Event<Self>> + IsType<<Self as frame_system::Config>::RuntimeEvent>;
+  }
+
+  #[pallet::event]
+  #[pallet::generate_deposit(fn deposit_event)]
+  pub enum Event<T: Config> {
+    Batch { coin: Coin, id: BlockHash },
+  }
+
+  #[pallet::pallet]
+  #[pallet::generate_store(pub(crate) trait Store)]
+  pub struct Pallet<T>(PhantomData<T>);
+
+  // Used to only allow one set of updates per block, preventing double updating
+  #[pallet::storage]
+  pub(crate) type Once<T: Config> = StorageValue<_, bool, ValueQuery>;
+  // Latest block number agreed upon for a coin
+  #[pallet::storage]
+  #[pallet::getter(fn block_number)]
+  pub(crate) type BlockNumbers<T: Config> =
+    StorageMap<_, Blake2_256, Coin, BlockNumber, ValueQuery>;
+
+  #[pallet::hooks]
+  impl<T: Config> Hooks<BlockNumberFor<T>> for Pallet<T> {
+    fn on_finalize(_: BlockNumberFor<T>) {
+      Once::<T>::take();
+    }
+  }
+
+  #[pallet::call]
+  impl<T: Config> Pallet<T> {
+    #[pallet::call_index(0)]
+    #[pallet::weight((0, DispatchClass::Mandatory))] // TODO
+    pub fn execute(origin: OriginFor<T>, updates: Updates) -> DispatchResult {
+      ensure_none(origin)?;
+      assert!(!Once::<T>::exists());
+      Once::<T>::put(true);
+
+      for (coin, update) in updates.iter().enumerate() {
+        if let Some(update) = update {
+          let coin = coin_from_index(coin);
+          BlockNumbers::<T>::insert(coin, update.block_number);
+
+          for batch in &update.batches {
+            // TODO: EXECUTE
+            Self::deposit_event(Event::Batch { coin, id: batch.id });
+          }
+        }
+      }
+
+      Ok(())
+    }
+  }
+
+  #[pallet::inherent]
+  impl<T: Config> ProvideInherent for Pallet<T> {
+    type Call = Call<T>;
+    type Error = InherentError;
+    const INHERENT_IDENTIFIER: InherentIdentifier = INHERENT_IDENTIFIER;
+
+    fn create_inherent(data: &InherentData) -> Option<Self::Call> {
+      data
+        .get_data::<Updates>(&INHERENT_IDENTIFIER)
+        .unwrap()
+        .map(|updates| Call::execute { updates })
+    }
+
+    // Assumes that only not yet handled batches are provided as inherent data
+    fn check_inherent(call: &Self::Call, data: &InherentData) -> Result<(), Self::Error> {
+      // First unwrap is for the Result of fetching/decoding the Updates
+      // Second unwrap is for the Option of if they exist
+      let expected = data.get_data::<Updates>(&INHERENT_IDENTIFIER).unwrap().unwrap();
+      // Match to be exhaustive
+      let updates = match call {
+        Call::execute { ref updates } => updates,
+        _ => Err(InherentError::InvalidCall)?,
+      };
+
+      // The block producer should've provided one update per coin
+      // We, an honest node, did provide one update per coin
+      // Accordingly, we should have the same amount of updates
+      if updates.len() != expected.len() {
+        Err(InherentError::InvalidUpdateQuantity(
+          updates.len().try_into().unwrap(),
+          expected.len().try_into().unwrap(),
+        ))?;
+      }
+
+      // This zip is safe since we verified they're equally sized
+      // This should be written as coins.zip(updates.iter().zip(&expected)), where coins is the
+      // validator set's coins
+      // That'd require having context on the validator set right now which isn't worth pulling in
+      // right now, when we only have one validator set
+      for (coin, both) in updates.iter().zip(&expected).enumerate() {
+        let coin = coin_from_index(coin);
+        match both {
+          // Block producer claims there's an update for this coin, as do we
+          (Some(update), Some(expected)) => {
+            if update.block_number.0 > expected.block_number.0 {
+              Err(InherentError::UnrecognizedBlockNumber(
+                coin,
+                update.block_number,
+                expected.block_number,
+              ))?;
+            }
+
+            let prev = BlockNumbers::<T>::get(coin);
+            if update.block_number.0 <= prev.0 {
+              Err(InherentError::InvalidBlockNumber(coin, update.block_number, prev))?;
+            }
+
+            if update.batches.len() > expected.batches.len() {
+              Err(InherentError::UnrecognizedBatches(
+                coin,
+                (update.batches.len() - expected.batches.len()).try_into().unwrap(),
+              ))?;
+            }
+
+            for (batch, expected) in update.batches.iter().zip(&expected.batches) {
+              if batch != expected {
+                Err(InherentError::DifferentBatch(coin, batch.id))?;
+              }
+            }
+          }
+
+          // Block producer claims there's an update for this coin, yet we don't
+          (Some(update), None) => {
+            Err(InherentError::UnrecognizedBatches(coin, update.batches.len().try_into().unwrap()))?
+          }
+
+          // Block producer didn't include update for this coin
+          (None, _) => (),
+        };
+      }
+
+      Ok(())
+    }
+
+    fn is_inherent(_: &Self::Call) -> bool {
+      true
+    }
+  }
+}
+
+pub use pallet::*;