Commit Graph

89 Commits

Author SHA1 Message Date
Luke Parker
aab8a417db Have the Coordinator scan the Substrate genesis block
Also adds a workflow for running tests/coordinator.
2023-08-02 12:18:50 -04:00
Luke Parker
d5c787fea2 Add initial coordinator e2e tests 2023-08-01 19:00:48 -04:00
Luke Parker
3c38a0ec11 cargo +nightly fmt 2023-08-01 00:47:36 -04:00
Luke Parker
9f143a9742 Replace "coin" with "network"
The Processor's coins folder referred to the networks it could process, as did
its Coin trait. This, and other similar cases throughout the codebase, have now
been corrected.

Also corrects dated documentation for a key pair is confirmed under the
validator-sets pallet.
2023-07-30 16:11:30 -04:00
Luke Parker
32435d8a4c Consolidate RockDB code
Moves explicitly to zstd. RockDB recommends zstd, or at least lz4 over snappy,
and this minimizes which dependencies we pull in.
2023-07-25 21:43:27 -04:00
Luke Parker
624fb2781d Update how RPCs are handled
The processor now takes three vars and joins them itself. message-queue uses a
single argument, with defaults, as it's a service we control.
2023-07-21 14:01:42 -04:00
Luke Parker
a7c9c1ef55 Integrate coordinator with MessageQueue and RocksDB
Also resolves a couple TODOs.
2023-07-18 01:53:51 -04:00
Luke Parker
807ec30762 Update the flow for completed signing processes
Now, an on-chain transaction exists. This resolves some ambiguities and
provides greater coordination.
2023-07-14 14:05:12 -04:00
Luke Parker
40b2920412 Remove signed Substrate TXs from Coordinator 2023-05-13 22:43:13 -04:00
Luke Parker
f55e9b40e6 Have coordinator publish batches to Substrate 2023-05-10 01:46:20 -04:00
Luke Parker
a70df6a449 Remove TODO about code de-duplication
It's infeasible to write a macro/function there. Does add a type alias which
makes things cleaner.
2023-05-10 01:19:01 -04:00
Luke Parker
168f2899f0 Create a vote transaction upon GeneratedKeyPair 2023-05-10 00:46:51 -04:00
Luke Parker
c95bdb6752 Properly get genesis for a Processor message 2023-05-09 23:51:05 -04:00
Luke Parker
7b7ddbdd97 Move the coordinator to a n-processor design 2023-05-09 23:44:41 -04:00
Luke Parker
9175383e89 Spawn a new async task for each block message
This probably should be done with n-long lived tasks, one per Tributary. While
this may not be suitably performant long-term (potential DoS vector), this at
least resolves the halting concerns.
2023-05-09 17:05:33 -04:00
Luke Parker
964fdee175 Publish ExternablBlock/SubstrateBlock, delay *Preprocess until ID acknowledged
Adds a channel for the Tributary scanner to communicate when an ID has been
acknowledged.
2023-05-08 22:20:51 -04:00
Luke Parker
6032af6692 Have Coordinator MainDb take a mutable borrow 2023-04-26 00:10:06 -04:00
Luke Parker
7824b6cb8b Document the processor/tributary/coordinator/serai flow 2023-04-25 15:05:58 -04:00
Luke Parker
78d5372fb7 Initial code to handle messages from processors 2023-04-25 03:14:42 -04:00
Luke Parker
e74b4ab94f Add a TributaryReader which doesn't require a borrow to operate
Reduces lock contention.

Additionally changes block_key to include the genesis. While not technically
needed, the lack of genesis introduced a side effect where any Tributary on the
the database could return the block of any other Tributary. While that wasn't a
security issue, returning it suggested it was on-chain when it wasn't. This may
have been usable to create issues.
2023-04-24 07:02:00 -04:00
Luke Parker
2feebe536e Test handle_p2p and Tributary syncing
Includes bug fixes.
2023-04-24 03:30:19 -04:00
Luke Parker
14388e746c Implement Tributary syncing
Also adds a forwards-lookup to the Tributary blockchain.
2023-04-24 00:53:18 -04:00
Luke Parker
215155f84b Remove reliance on a blockchain read lock from block/commit 2023-04-23 23:51:10 -04:00
Luke Parker
c476f9b640 Break coordinator main into multiple functions
Also moves from std::sync::RwLock to tokio::sync::RwLock to prevent wasting
cycles on spinning.
2023-04-23 23:15:15 -04:00
Luke Parker
05b1fc5f05 Send a heartbeat message when a Tributary falls behind 2023-04-23 18:55:43 -04:00
Luke Parker
72633d6421 Clarify Arc RwLocks and sleeps in coordinator 2023-04-23 18:29:50 -04:00
Luke Parker
ad5522d854 Start handling P2P messages
This defines the tart of a very complex series of locks I'm really unhappy
with. At the same time, there's not immediately a better solution. This also
should work without issue.
2023-04-23 17:01:30 -04:00
Luke Parker
f2d9d70068 Reload Tributaries
add_active_tributary writes the spec to disk before it returns, so even if the
VecDeque it pushes to isn't popped, the tributary will still be loaded on boot.
2023-04-23 04:31:00 -04:00
Luke Parker
2b09309adc Handle adding new Tributaries
Removes last_block as an argument from Tendermint. It now loads from the DB as
needed. While slightly less performant, it's easiest and should be fine.
2023-04-23 03:51:26 -04:00
Luke Parker
af84b7f707 Add a test for Tributary
Further fleshes out the Tributary testing code.
2023-04-22 22:28:20 -04:00
Luke Parker
8c74576cf0 Add a test to the coordinator for running a Tributary
Impls a LocalP2p for testing.

Moves rebroadcasting into Tendermint, since it's what knows if a message is
fully valid + original.

Removes TributarySpec::validators() HashMap, as its non-determinism caused
different instances to have different round robin schedules. It was already
prior moved to a Vec for this issue, so I'm unsure why this remnant existed.

Also renames the GH no-std workflow from the prior commit.
2023-04-22 10:49:52 -04:00
Luke Parker
8041a0d845 Initial Tributary handling 2023-04-20 05:05:17 -04:00
Luke Parker
92a868e574 Add a processor API to the coordinator 2023-04-17 02:10:33 -04:00
Luke Parker
4d43c04916 Clean up the Substrate block processing code 2023-04-17 00:50:56 -04:00
Luke Parker
2604746586 Fill out code for the rest of the Substrate events 2023-04-16 03:18:52 -04:00
Luke Parker
79655672ef Make progres on handling NewSet events
Further bones out the coordinator.
2023-04-16 00:51:56 -04:00
Luke Parker
eafd054296 Start defining the coordinator 2023-04-15 17:38:47 -04:00
Luke Parker
2cfee536f6 Define all coordinator transaction types 2023-04-11 19:04:53 -04:00
Luke Parker
de52c4db7f Add empty coordinator 2023-04-11 09:21:35 -04:00