Redo Dockerfile generation (#530)

Moves from concatted Dockerfiles to pseudo-templated Dockerfiles via a dedicated Rust program.

Removes the unmaintained kubernetes, not because we shouldn't have/use it, but because it's unmaintained and needs to be reworked before it's present again.

Replaces the compose with the work in the new orchestrator binary which spawns everything as expected. While this arguably re-invents the wheel, it correctly manages secrets and handles the variadic Dockerfiles.

Also adds an unrelated patch for zstd and simplifies running services a bit by greater utilizing the existing infrastructure.

---

* Delete all Dockerfile fragments, add new orchestator to generate Dockerfiles

Enables greater templating.

Also delete the unmaintained kubernetes folder *for now*. This should be
restored in the future.

* Use Dockerfiles from the orchestator

* Ignore Dockerfiles in the git repo

* Remove CI job to check Dockerfiles are as expected now that they're no longer committed

* Remove old Dockerfiles from repo

* Use Debian for monero-wallet-rpc

* Remove replace_cmds for proper usage of entry-dev

Consolidates ports a bit.

Updates serai-docker-tests from "compose" to "build".

* Only write a new dockerfile if it's distinct

Preserves the updated time metadata.

* Update serai-docker-tests

* Correct the path Dockerfiles are built from

* Correct inclusion of orchestration folder in Docker builds

* Correct debug/release flagging in the cargo command

Apparently, --debug isn't an effective NOP yet an error.

* Correct path used to run the Serai node within a Dockerfile

* Correct path in Monero Dockerfile

* Attempt storing monerod in /usr/bin

* Use sudo to move into /usr/bin in CI

* Correct 18.3.0 to 18.3.1

* Escape * with quotes

* Update deny.toml, ADD orchestration in runtime Dockerfile

* Add --detach to the Monero GH CI

* Diversify dockerfiles by network

* Fixes to network-diversified orchestration

* Bitcoin and Monero testnet scripts

* Permissions and tweaks

* Flatten scripts folders

* Add missing folder specification to Monero Dockerfile

* Have monero-wallet-rpc specify the monerod login

* Have the Docker CMD specify env variables inserted at time of Dockerfile generation

They're overrideable with the global enviornment as for tests. This enables
variable generation in orchestrator and output to productionized Docker files
without creating a life-long file within the Docker container.

* Don't add Dockerfiles into Docker containers now that they have secrets

Solely add the source code for them as needed to satisfy the workspace bounds.

* Download arm64 Monero on arm64

* Ensure constant host architecture when reproducibly building the wasm

Host architecture, for some reason, can effect the generated code despite the
target architecture always being foreign to the host architecture.

* Randomly generate infrastructure keys

* Have orchestrator generate a key, be able to create/start containers

* Ensure bash is used over sh

* Clean dated docs

* Change how quoting occurs

* Standardize to sh

* Have Docker test build the dev Dockerfiles

* Only key_gen once

* cargo update

Adds a patch for zstd and reconciles the breaking nightly change which just
occurred.

* Use a dedicated network for Serai

Also fixes SERAI_HOSTNAME passed to coordinator.

* Support providing a key over the env for the Serai node

* Enable and document running daemons for tests via serai-orchestrator

Has running containers under the dev network port forward the RPC ports.

* Use volumes for bitcoin/monero

* Use bitcoin's run.sh in GH CI

* Only use the volume for testnet (not dev)

orchestration/README.md

@@ -1,65 +1,12 @@
-# Deploy
+# Orchestration
 
-## Run with Docker Compose
+This folder contains the tool which generates various dockerfiles and manage
+deployments of Serai.
 
-Running the Serai infrastructure is easy with Docker.
+To start, run:
 
-We utilize compose profiles to easily orchestrate various pieces of the
-infrastructure.
+```sh
+cargo run -p serai-orchestrator
+```
 
-**Example:** `docker compose --profile cluster-coins-sm up`
-
-All commands are assumed to be ran from `/deploy`, not the root folder.
-
-### Profiles:
-
-* `bitcoin`  - Bitcoin node
-* `monero`   - Monero node
-* `ethereum` - Ethereum node
-* `coins`    - Nodes for all external networks (BTC, ETH, XMR)
-
-* `message-queue` - The message queue service.
-* `processor`     - Serai processor for one external network.
-* `coordinator`   - Serai coordinator for the entire Serai stack.
-
-* `serai`      - Serai node
-* `cluster-sm` - "Alice", "Bob", "Charlie", and "Dave" Serai nodes, all as
-                 validators (enough to achieve BFT with one faulty node)
-* `cluster-lg` - `cluster-sm` with non-validators "Eve" and "Ferdie"
-
-You can supply one or more profiles to the docker compose command to orchestrate
-the desired components.
-
-**Example:** `docker compose --profile coins --profile serai up`
-
-## Orchestration Approach
-
-### Builds
-
-The Serai infrastructure is locally compiled. This may take several minutes.
-
-Images for external networks download binaries, before verifying their checksums
-and signatures.
-
-**Stage 1 -- Builder**
-* Configure environment.
-* Get the binary.
-* Verify binary using GPG.
-* Decompress binary to prepare image.
-
-**Stage 2 -- Image**
-* Copy needed files from builder.
-* Move executables to bin folder.
-* Copy scripts folder.
-* Expose necessary ports.
-* Map necessary volumes.
-
-### Entrypoint
-
-The Serai node and external networks' nodes are each started from an entrypoint
-script inside the `/scripts `folder.
-
-To update the scripts on the image you must rebuild the updated images using the
-`--build` flag after `up` in `docker compose`.
-
-**Example:** `docker compose --profile bitcoin up --build`
+to generate all of the dockerfiles needed for development.