This adds the option FULLY_STATIC to create fully-static executables.
To create portable, fully-static release executables on Linux, we need
to use musl instead of glibc. Static executables from glibc are not
portable.
The popular musl-gcc wrapper does not support C++, so instead we use a
full musl cross-compiler toolchain in the build workflow.
To build FULLY_STATIC, the user must point to the appropriate
cross-compiler, as we do in the workflow. On systems where musl is the
native libc (such as Alpine Linux), this is not necessary.
I have also read that musl's allocator is not very fast in
multi-threaded applications. So when FULLY_STATIC is enabled, we will
also enable mimalloc, a replacement allocator that is very fast.
I tested a very basic packaging command to compare speeds of dynamic
glibc, static musl, and static musl+mimalloc:
dynamic glibc:
runs: 2.527, 2.798, 2.703, 2.756, 2.959
avg = 2.749, std dev = 0.156s
static musl:
runs: 2.813, 2.920, 3.129, 3.003, 2.738
avg = 2.921s, std dev = 0.154s
static musl+mimalloc:
runs: 2.291, 2.034, 2.415, 2.303, 2.265
avg = 2.262s, std dev = 0.140s
The mimalloc build is 82% faster than musl default allocator, 77% faster
than glibc, and has more consistent runtime characteristics (lower
standard deviation).
According to a comment in
packager/third_party/abseil-cpp/source/absl/log/CMakeLists.txt, many
linkers will strip the contents of absl::log_flags because its symbols
symbols are only used in a global constructor, and that for now, clients
should link using
$<LINK_LIBRARY:WHOLE_ARCHIVE,absl::log_flags>.
Closes#1325
This work was done over ~80 individual commits in the `cmake` branch,
which are now being merged back into `main`. As a roll-up commit, it is
too big to be reviewable, but each change was reviewed individually in
context of the `cmake` branch. After this, the `cmake` branch will be
renamed `cmake-porting-history` and preserved.
---------
Co-authored-by: Geoff Jukes <geoffjukes@users.noreply.github.com>
Co-authored-by: Bartek Zdanowski <bartek.zdanowski@gmail.com>
Co-authored-by: Carlos Bentzen <cadubentzen@gmail.com>
Co-authored-by: Dennis E. Mungai <2356871+Brainiarc7@users.noreply.github.com>
Co-authored-by: Cosmin Stejerean <cstejerean@gmail.com>
Co-authored-by: Carlos Bentzen <carlos.bentzen@bitmovin.com>
Co-authored-by: Cosmin Stejerean <cstejerean@meta.com>
Co-authored-by: Cosmin Stejerean <cosmin@offbytwo.com>
Using the latest depot_tools no longer works. depot_tools also wants
to auto-update itself, which must now be disabled.
We also need to disable the copy of python (vpython) included in
depot_tools, since for some distros, it has dependencies on system
libraries that no longer exist.
Finally, we need to force some distros to use python 2, because our
build system is ancient and needs to be ripped out and replaced some
day soon.
This fixes build issues in our CI, our Dockerfiles, and in general on
certain platforms or distros.
Closes#1023
The official, static-linked linux builds were crashing in their use of
getaddrinfo, which libcurl was configured to use. Both getaddrinfo
and all of its alternatives available in glibc fail with static
linking.
We can fix this by configuring libcurl to use libc-ares on Linux
instead. This allows us to keep the benefits of a statically-linked
Linux binary.
Closes#996
Change-Id: Ib4a9eb939813fd165727788726459ef4adf3fc4d
This updates the main Dockerfile and all the docker-based
distro-specific tests. The base OS versions have been updated to
versions that have not reached end-of-life status yet, and the list of
dependencies required has been updated and pruned.
Change-Id: Ibcff2f60e739fd5d999af100af76c40aa91a75bc
This brings our default build config more in line with what is
necessary for some platforms anyway: using the system-installed
toolchain and sysroot to build everything.
We will no longer fetch source or binaries for any specific build
tools, such as libc++, clang, gold, binutils, or valgrind.
The main part of this change is the changing of default gyp settings
in gyp_packager.py. For this, a bug in gyp_packager.py had to be
fixed, in which similar GYP_DEFINE key names (such as clang and
host_clang) would conflict, causing some defaults not to be installed
properly.
In order to enable clang=0 by default, some changes had to be made in
common.gypi:
- compiler macros added to fix a compatibility issue between
Chromium's base/mac/ folder and the actual OSX SDK
- replaced clang_warning_flags variables with standard cflags
settings, plus xcode_settings for OSX
- turned off warnings-as-errors for non-shaka code, rather than
allow-listing specific warning types, since we can't actually fix
those warnings on any platform
- disabled two specific warnings in shaka code, both of which are
caused by headers from our non-shaka dependencies
Also, one warning (missing "override" keyword) has been fixed in
vod_media_info_dump_muxer_listener.h.
Although these changes were done to make building simpler on a wider
array of platforms (arm64, for example), it seems to make the build a
bit faster, too. For me, at least, on my main Linux workstation:
- "gclient sync" now runs 20-30% faster
- "ninja -C out/Release" now runs 5-13% faster
The following environment variables are no longer required:
- DEPOT_TOOLS_WIN_TOOLCHAIN
- MACOSX_DEPLOYMENT_TARGET
Documentation, Dockerfiles, and GitHub Actions workflows have been
updated to reflect this.
The following GYP_DEFINES are no longer required for anyone:
- clang=0
- host_clang=0
- clang_xcode=1
- use_allocator=none
- use_experimental_allocator_shim=0
Documentation, Dockerfiles, and GitHub Actions workflows have been
updated to reflect this.
The following repos are no longer dependencies in gclient:
- binutils
- clang
- gold
- libc++
- libc++abi
- valgrind
The following gclient hooks have been removed:
- clang
- mac_toolchain
- sysroot
Change-Id: Ie94ccbeec722ab73c291cb7df897d20761a09a70
For windows running `GYP_DEFINES='target_arch=X64'` shows an error GYP_DEFINES is
not recognized as an internal or external command.
The actual command is to set the value of GYP_DEFINES. so it should be
`SET GYP_DEFINES='target_arch=X64'`.
Keep bundled binutil scripts but not downloading actual binaries by default.
Automatic downloading of binutils has been causing problems for some users:
see #164, #412, #440.
Using bundled binutils helps reduce linking time, but packager codebase is
relatively small, so the gain is not significant.
Local testing shows that the full build time increases from 1m42s to 1m50s, i.e.
an increase of about 8s, or ~8%.
User can still enable the usage of bundled binutils by running
'python src/packager/third_party/binutils/download.py' and set
'linux_use_bundled_binutils' and 'linux_use_bundled_gold' to 1 in GYP_DEFINES.
Change-Id: I7ecae2333861f41054f957d0a6081dae948167bd
Note that STYLE and REGION are not supported in mp4 container due to
spec limitation as 14496-30:2014 does not specify a way to signal
styles/regions inside mp4.
Closes#344.
Change-Id: I05c14df916f7b2c7ca4364ee9407e0eda4dc7a3f
- Also fixed compilations in Alpine Linux and other flavors of Linux.
- Added container versions in docker files to always use a verified
version.
Closes#164.
Change-Id: I949a8709e4d70c49129c9c2e8608dd78193d964c
Moved build instructions and docker instructions to separate docs.
Removed command samples which are no longer needed as we have better
tutorials now.
Change-Id: I340c5653a6553158325970dd86d76fa11e69bcbd
Joey Parrish
Dennis E. Mungai
Joey Parrish
KongQun Yang
Suraj kumar (sk)
Alen Vrečko
Dennis E. Mungai