* Normalize naming for the stack size CI file
* Extend stack size CI with `posh` and `lksh`
`posh` is a derivative of `pdksh` explicitly intended for ensuring
Debian-policy-compliance.
`lksh` is more-POSIX-esque, legacy shell included along-side `mksh`.
* Ensure a signed long overflow won't occur
Also fixes `write_bytes` when the written bytes have alphabetical digits when
encoded as hexadecimal.
* Improve sh semantics in stack-size workflow
This runs whenever the script is modified, or weekly to ensure the CI doesn't
inadvertently decay (due to using the latest packages for a variety of shells).
This runs with `sh` (presumably `dash`), `ksh`, `bash`, `dash` (explicitly),
`zsh`, `ash` (Busybox), `hush` (Busybox), `mksh`, `yash`, and `brush`. While
none of these guarantee this script is POSIX-compliant, as a fully and
explicitly-only POSIX-compliant environment is not constructed, this does
reasonably test the script itself to be POSIX-compliant. The tools called have
been reviewed for being used to the POSIX standard (although not audited to
that degree).
The script itself is modified with the following changes for compliance with
POSIX:
1) `hexdump` is replaced with `od` (`od` suggested by @PlasmaPower)
2) `printf \xFF` replaced with octal escapes, as `\x` is not part of POSIX
3) `head -c` is replaced with `cut`, as the `-c` option is not standardized
under POSIX (despite it being present for `tail`). This was identified by
@PlasmaPower. As we used `head -c-2` to truncate the last two characters of
a string, we now use `wc -c` for a `strlen` to enable the necessary
arithmetic to calculate what two bytes in from the end of the string is.
This entire effort can be argued pointless, as we could simply run `monerod` on
Debian. This script is useful, the journey down the rabbithole of POSIX
compliance fascinating, and the methodology applicable to other potential
futures though (whether running binaries on Alpine or testing other `sh`
scripts for their portability). As part of this effort overall, our CI was
extended with `shellcheck` for all `sh` scripts in-tree, including all of our
existing `sh` scripts. That there is an actual, direct benefit past this
specific effort.
Updates our scripts to pass. Achieves POSIX compliance for
`increase_default_stack_size.sh` via replacing `hexdump` with `od` and `tr`.
Replaces the non-POSIX `dd status=none` with the POSIX `dd 2> /dev/null`.
While prior this didn't work well, presumably due to stack size limitations,
a shell script is included to raise the default stack size limit. This should
be tried again.
We already had this behavior on `std`. It was omitted when no-`std` due to
deferring to `spin::Mutex`, which does not track poisoning at all. This
increases the parity of the two.
Part of https://github.com/serai-dex/serai/issues/698.
By defining our own build script, we gain complete clarity and control over how
the WASM is built. This also removes the need to patch the upstream due to it
allowing pollution of the environment variables from the host.
Notable appreciation is given to
https://github.com/rust-lang/rust/issues/145491 for identifying an issue
encountered here, with the associated PR clarifying the necessary flags for the
linker to fix this.
Additionally, optimizes the build process a bit via leaving only the runtime
(and `busybox`) in the final image, and additionally building the runtime
without `std` (as we solely need the WASM blob from this process).
`panic = "unwind"` was originally a requirement of Substrate, notably due to
its [native runtime](https://github.com/paritytech/substrate/issues/10874).
This does not mean all of Serai should use this setting however.
As the native runtime has been removed, we do no longer need this for the
Substrate node. With a review of our derivative, a panic guard is only used
when fetching the version from the runtime, causing an error on boot if a
panic occurs. Accordingly, we shouldn't have a need for `panic = "unwind"`
within the node, and the runtime itself should be fine.
The rest of Serai's services already registered bespoke hooks to ensure any
panic caused the process to exit. Those are left as-is, even though they're
now unnecessary.
Ensures the block's size limit is respected.
Defines a policy for weights. While I'm unsure I want to commit to this
forever, I do want to acknowledge it's valid and well-defined.
Cleans up the `serai-runtime` crate a bit with further modules in the `wasm`
folder.
Notably removes the `kvdb-rocksdb` patch via updating the Substrate version
used to one which disables the `jemalloc` feature itself.
Simplifies the path of the built WASM file within the Dockerfile to consumers.
This also ensures if the image is built, the path of the WASM file is as
expected (prior unasserted).
For hash-pinned dependencies, adds comments documenting the associated
versions.
Adds a pin to `slither-analyzer` which was prior missing.
Updates to Monero 0.18.4.4.
`mimalloc` now has the correct option set when building for `musl`. A C++
compiler is no longer required in its Docker image.
The runtime's `Dockerfile` now symlinks a `libc.so` already present on the
image instead of creating one itself. It also builds the runtime within the
image to ensure it only happens once. The test to ensure the methodology is
reproducible has been updated to not simply create containers from the image,
yet rebuild the image entirely, accordingly. This also is more robust and
arguably should have already been done.
The pin to the exact hash of the `patch-polkadot-sdk` repo in every
`Cargo.toml` has been removed. The lockfile already serves that role,
simplifying updating in the future.
The latest Rust nightly is adopted as well (superseding
https://github.com/serai-dex/serai/pull/697).
The `librocksdb-sys` patch is replaced with a `kvdb-rocksdb` patch, removing a
git dependency, thanks to https://github.com/paritytech/parity-common/pull/950.
It wasn't pinned with a hash yet with a version tag. This ensures we are
deterministic to the image (specified by hash), `Cargo.lock`, and source code
alone.
Unfortunately, this was incredibly annoying to do, the exact process uncovering
a SIGSEGV in stable Rust. The extensive documentation details the solution.
Thankfully, it works now.
The use of a dedicated test module ensures the API doesn't hide anything which
needs to be public. There's also now explicit tests for when the median is the
popped value.
