Initial In Instructions pallet and Serai client lib (#233)

* Initial work on an In Inherents pallet

* Add an event for when a batch is executed

* Add a dummy provider for InInstructions

* Add in-instructions to the node

* Add the Serai runtime API to the processor

* Move processor tests around

* Build a subxt Client around Serai

* Successfully get Batch events from Serai

Renamed processor/substrate to processor/serai.

* Much more robust InInstruction pallet

* Implement the workaround from https://github.com/paritytech/subxt/issues/602

* Initial prototype of processor generated InInstructions

* Correct PendingCoins data flow for InInstructions

* Minor lint to in-instructions

* Remove the global Serai connection for a partial re-impl

* Correct ID handling of the processor test

* Workaround the delay in the subscription

* Make an unwrap an if let Some, remove old comments

* Lint the processor toml

* Rebase and update

* Move substrate/in-instructions to substrate/in-instructions/pallet

* Start an in-instructions primitives lib

* Properly update processor to subxt 0.24

Also corrects failures from the rebase.

* in-instructions cargo update

* Implement IsFatalError

* is_inherent -> true

* Rename in-instructions crates and misc cleanup

* Update documentation

* cargo update

* Misc update fixes

* Replace height with block_number

* Update processor src to latest subxt

* Correct pipeline for InInstructions testing

* Remove runtime::AccountId for serai_primitives::NativeAddress

* Rewrite the in-instructions pallet

Complete with respect to the currently written docs.

Drops the custom serializer for just using SCALE.

Makes slight tweaks as relevant.

* Move instructions' InherentDataProvider to a client crate

* Correct doc gen

* Add serde to in-instructions-primitives

* Add in-instructions-primitives to pallet

* Heights -> BlockNumbers

* Get batch pub test loop working

* Update in instructions pallet terminology

Removes the ambiguous Coin for Update.

Removes pending/artificial latency for furture client work.

Also moves to using serai_primitives::Coin.

* Add a BlockNumber primitive

* Belated cargo fmt

* Further document why DifferentBatch isn't fatal

* Correct processor sleeps

* Remove metadata at compile time, add test framework for Serai nodes

* Remove manual RPC client

* Simplify update test

* Improve re-exporting behavior of serai-runtime

It now re-exports all pallets underneath it.

* Add a function to get storage values to the Serai RPC

* Update substrate/ to latest substrate

* Create a dedicated crate for the Serai RPC

* Remove unused dependencies in substrate/

* Remove unused dependencies in coins/

Out of scope for this branch, just minor and path of least resistance.

* Use substrate/serai/client for the Serai RPC lib

It's a bit out of place, since these client folders are intended for the node to
access pallets and so on. This is for end-users to access Serai as a whole.

In that sense, it made more sense as a top level folder, yet that also felt
out of place.

* Move InInstructions test to serai-client for now

* Final cleanup

* Update deny.toml

* Cargo.lock update from merging develop

* Update nightly

Attempt to work around the current CI failure, which is a Rust ICE.

We previously didn't upgrade due to clippy 10134, yet that's been reverted.

* clippy

* clippy

* fmt

* NativeAddress -> SeraiAddress

* Sec fix on non-provided updates and doc fixes

* Add Serai as a Coin

Necessary in order to swap to Serai.

* Add a BlockHash type, used for batch IDs

* Remove origin from InInstruction

Makes InInstructionTarget. Adds RefundableInInstruction with origin.

* Document storage items in in-instructions

* Rename serai/client/tests/serai.rs to updates.rs

It only tested publishing updates and their successful acceptance.

substrate/in-instructions/primitives/src/incoming.rs

@@ -0,0 +1,38 @@
+use scale::{Encode, Decode, MaxEncodedLen};
+use scale_info::TypeInfo;
+
+#[cfg(feature = "std")]
+use serde::{Serialize, Deserialize};
+
+use sp_core::{ConstU32, bounded::BoundedVec};
+
+use serai_primitives::SeraiAddress;
+
+use crate::{MAX_DATA_LEN, ExternalAddress};
+
+#[derive(Clone, Copy, PartialEq, Eq, Debug, Encode, Decode, MaxEncodedLen, TypeInfo)]
+#[cfg_attr(feature = "std", derive(Serialize, Deserialize))]
+pub enum Application {
+  DEX,
+}
+
+#[derive(Clone, PartialEq, Eq, Debug, Encode, Decode, MaxEncodedLen, TypeInfo)]
+#[cfg_attr(feature = "std", derive(Serialize, Deserialize))]
+pub struct ApplicationCall {
+  application: Application,
+  data: BoundedVec<u8, ConstU32<{ MAX_DATA_LEN }>>,
+}
+
+#[derive(Clone, PartialEq, Eq, Debug, Encode, Decode, MaxEncodedLen, TypeInfo)]
+#[cfg_attr(feature = "std", derive(Serialize, Deserialize))]
+pub enum InInstruction {
+  Transfer(SeraiAddress),
+  Call(ApplicationCall),
+}
+
+#[derive(Clone, PartialEq, Eq, Debug, Encode, Decode, MaxEncodedLen, TypeInfo)]
+#[cfg_attr(feature = "std", derive(Serialize, Deserialize))]
+pub struct RefundableInInstruction {
+  pub origin: Option<ExternalAddress>,
+  pub instruction: InInstruction,
+}

substrate/in-instructions/primitives/src/lib.rs

@@ -0,0 +1,47 @@
+#![cfg_attr(docsrs, feature(doc_cfg))]
+#![cfg_attr(docsrs, feature(doc_auto_cfg))]
+#![cfg_attr(not(feature = "std"), no_std)]
+
+use scale::{Encode, Decode, MaxEncodedLen};
+use scale_info::TypeInfo;
+
+#[cfg(feature = "std")]
+use serde::{Serialize, Deserialize};
+
+use sp_core::{ConstU32, bounded::BoundedVec};
+
+// Monero, our current longest address candidate, has a longest address of featured with payment ID
+// 1 (enum) + 1 (flags) + 64 (two keys) + 8 (payment ID) = 74
+pub const MAX_ADDRESS_LEN: u32 = 74;
+// Should be enough for a Uniswap v3 call
+pub const MAX_DATA_LEN: u32 = 512;
+
+#[derive(Clone, PartialEq, Eq, Debug, Encode, Decode, MaxEncodedLen, TypeInfo)]
+#[cfg_attr(feature = "std", derive(Serialize, Deserialize))]
+pub struct ExternalAddress(BoundedVec<u8, ConstU32<{ MAX_ADDRESS_LEN }>>);
+impl ExternalAddress {
+  #[cfg(feature = "std")]
+  pub fn new(address: Vec<u8>) -> Result<ExternalAddress, &'static str> {
+    Ok(ExternalAddress(address.try_into().map_err(|_| "address length exceeds {MAX_ADDRESS_LEN}")?))
+  }
+
+  pub fn address(&self) -> &[u8] {
+    self.0.as_ref()
+  }
+
+  #[cfg(feature = "std")]
+  pub fn consume(self) -> Vec<u8> {
+    self.0.into_inner()
+  }
+}
+
+// Not "in" as "in" is a keyword
+mod incoming;
+pub use incoming::*;
+
+// Not "out" to match in
+mod outgoing;
+pub use outgoing::*;
+
+mod shorthand;
+pub use shorthand::*;

substrate/in-instructions/primitives/src/outgoing.rs

@@ -0,0 +1,25 @@
+use scale::{Encode, Decode, MaxEncodedLen};
+use scale_info::TypeInfo;
+
+#[cfg(feature = "std")]
+use serde::{Serialize, Deserialize};
+
+use sp_core::{ConstU32, bounded::BoundedVec};
+
+use serai_primitives::SeraiAddress;
+
+use crate::{MAX_DATA_LEN, ExternalAddress};
+
+#[derive(Clone, PartialEq, Eq, Debug, Encode, Decode, MaxEncodedLen, TypeInfo)]
+#[cfg_attr(feature = "std", derive(Serialize, Deserialize))]
+pub enum Destination {
+  Native(SeraiAddress),
+  External(ExternalAddress),
+}
+
+#[derive(Clone, PartialEq, Eq, Debug, Encode, Decode, MaxEncodedLen, TypeInfo)]
+#[cfg_attr(feature = "std", derive(Serialize, Deserialize))]
+pub struct OutInstruction {
+  destination: Destination,
+  data: Option<BoundedVec<u8, ConstU32<{ MAX_DATA_LEN }>>>,
+}

substrate/in-instructions/primitives/src/shorthand.rs

@@ -0,0 +1,54 @@
+use scale::{Encode, Decode, MaxEncodedLen};
+use scale_info::TypeInfo;
+
+#[cfg(feature = "std")]
+use serde::{Serialize, Deserialize};
+
+use sp_core::{ConstU32, bounded::BoundedVec};
+
+use serai_primitives::{SeraiAddress, Coin, Amount};
+
+use crate::{MAX_DATA_LEN, ExternalAddress, RefundableInInstruction, InInstruction, OutInstruction};
+
+#[derive(Clone, PartialEq, Eq, Debug, Encode, Decode, MaxEncodedLen, TypeInfo)]
+#[cfg_attr(feature = "std", derive(Serialize, Deserialize))]
+pub enum Shorthand {
+  Raw(BoundedVec<u8, ConstU32<{ MAX_DATA_LEN }>>),
+  Swap {
+    origin: Option<ExternalAddress>,
+    coin: Coin,
+    minimum: Amount,
+    out: OutInstruction,
+  },
+  AddLiquidity {
+    origin: Option<ExternalAddress>,
+    minimum: Amount,
+    gas: Amount,
+    address: SeraiAddress,
+  },
+}
+
+impl Shorthand {
+  pub fn transfer(origin: Option<ExternalAddress>, address: SeraiAddress) -> Option<Self> {
+    Some(Self::Raw(
+      BoundedVec::try_from(
+        (RefundableInInstruction { origin, instruction: InInstruction::Transfer(address) })
+          .encode(),
+      )
+      .ok()?,
+    ))
+  }
+}
+
+impl TryFrom<Shorthand> for RefundableInInstruction {
+  type Error = &'static str;
+  fn try_from(shorthand: Shorthand) -> Result<RefundableInInstruction, &'static str> {
+    Ok(match shorthand {
+      Shorthand::Raw(raw) => {
+        RefundableInInstruction::decode(&mut raw.as_ref()).map_err(|_| "invalid raw instruction")?
+      }
+      Shorthand::Swap { .. } => todo!(),
+      Shorthand::AddLiquidity { .. } => todo!(),
+    })
+  }
+}