Because we used require() to read build-matrix.json, the file could be
replaced with build-matrix.json.js, allowing code injection into our CI
pipelines. This fixes this vulnerability by reading the JSON text with
the fs module, then explicitly parsing it, rather than relying on
require().
This also changes the location of the file, to match its location in
other projects.
Note that this workflow is not currently giving any elevated permissions
to users, so it is not currently possible to damage the repo through a
PR. But this might have been possible in the past, due to
organization-wide defaults for token permissions (recently fixed). No
evidence has been found of past exploit.
See also https://github.com/shaka-project/shaka-streamer/issues/216 and
https://github.com/shaka-project/static-ffmpeg-binaries/issues/57
The release workflows did not run checkout with `fetch-tags: true`, so
the builds were unable to compute the correct release version number. I
audited all instances of `actions/checkout` to add `fetch-tags` where
needed and clean up unneeded options.
I also had to fix options to `docker/build-push-action`, which by
default ignores `actions/checkout` and tries to pull from git itself.
This led to the Docker build running in a context without the new tag.
Finally, to make verification easier and provide version info in the
build logs, this adds debugging info to the version-generation script
via stderr.
Closes#1366
Support for this came out in January, 2024. Explicit macos versions seem
to be necessary for now, until GitHub offers "latest" labels targeting
specific architectures.
This adds the option FULLY_STATIC to create fully-static executables.
To create portable, fully-static release executables on Linux, we need
to use musl instead of glibc. Static executables from glibc are not
portable.
The popular musl-gcc wrapper does not support C++, so instead we use a
full musl cross-compiler toolchain in the build workflow.
To build FULLY_STATIC, the user must point to the appropriate
cross-compiler, as we do in the workflow. On systems where musl is the
native libc (such as Alpine Linux), this is not necessary.
I have also read that musl's allocator is not very fast in
multi-threaded applications. So when FULLY_STATIC is enabled, we will
also enable mimalloc, a replacement allocator that is very fast.
I tested a very basic packaging command to compare speeds of dynamic
glibc, static musl, and static musl+mimalloc:
dynamic glibc:
runs: 2.527, 2.798, 2.703, 2.756, 2.959
avg = 2.749, std dev = 0.156s
static musl:
runs: 2.813, 2.920, 3.129, 3.003, 2.738
avg = 2.921s, std dev = 0.154s
static musl+mimalloc:
runs: 2.291, 2.034, 2.415, 2.303, 2.265
avg = 2.262s, std dev = 0.140s
The mimalloc build is 82% faster than musl default allocator, 77% faster
than glibc, and has more consistent runtime characteristics (lower
standard deviation).
According to a comment in
packager/third_party/abseil-cpp/source/absl/log/CMakeLists.txt, many
linkers will strip the contents of absl::log_flags because its symbols
symbols are only used in a global constructor, and that for now, clients
should link using
$<LINK_LIBRARY:WHOLE_ARCHIVE,absl::log_flags>.
Closes#1325
This work was done over ~80 individual commits in the `cmake` branch,
which are now being merged back into `main`. As a roll-up commit, it is
too big to be reviewable, but each change was reviewed individually in
context of the `cmake` branch. After this, the `cmake` branch will be
renamed `cmake-porting-history` and preserved.
---------
Co-authored-by: Geoff Jukes <geoffjukes@users.noreply.github.com>
Co-authored-by: Bartek Zdanowski <bartek.zdanowski@gmail.com>
Co-authored-by: Carlos Bentzen <cadubentzen@gmail.com>
Co-authored-by: Dennis E. Mungai <2356871+Brainiarc7@users.noreply.github.com>
Co-authored-by: Cosmin Stejerean <cstejerean@gmail.com>
Co-authored-by: Carlos Bentzen <carlos.bentzen@bitmovin.com>
Co-authored-by: Cosmin Stejerean <cstejerean@meta.com>
Co-authored-by: Cosmin Stejerean <cosmin@offbytwo.com>
Joey Parrish