This embeds the full python protobuf library into the output in pssh-box-protos, which is used by installation targets and Docker images. Without this, the installed pssh-box.py tool would depend on a separate installation of the python protobuf library.
Closes#1436
This adds the option FULLY_STATIC to create fully-static executables.
To create portable, fully-static release executables on Linux, we need
to use musl instead of glibc. Static executables from glibc are not
portable.
The popular musl-gcc wrapper does not support C++, so instead we use a
full musl cross-compiler toolchain in the build workflow.
To build FULLY_STATIC, the user must point to the appropriate
cross-compiler, as we do in the workflow. On systems where musl is the
native libc (such as Alpine Linux), this is not necessary.
I have also read that musl's allocator is not very fast in
multi-threaded applications. So when FULLY_STATIC is enabled, we will
also enable mimalloc, a replacement allocator that is very fast.
I tested a very basic packaging command to compare speeds of dynamic
glibc, static musl, and static musl+mimalloc:
dynamic glibc:
runs: 2.527, 2.798, 2.703, 2.756, 2.959
avg = 2.749, std dev = 0.156s
static musl:
runs: 2.813, 2.920, 3.129, 3.003, 2.738
avg = 2.921s, std dev = 0.154s
static musl+mimalloc:
runs: 2.291, 2.034, 2.415, 2.303, 2.265
avg = 2.262s, std dev = 0.140s
The mimalloc build is 82% faster than musl default allocator, 77% faster
than glibc, and has more consistent runtime characteristics (lower
standard deviation).
We already disabled warnings-as-errors in third-party code, but this
goes futher to completely disable third-party warnings. They are mostly
noise, and distracting. If they aren't errors, we shouldn't have our
build logs flooded with them.
protobuf depends on absl, so both needed an update.
Updating absl fixes issues on Alpine 3.19 (see PR #1327), and also
removes the need for hacks around vlog flags.
This work was done over ~80 individual commits in the `cmake` branch,
which are now being merged back into `main`. As a roll-up commit, it is
too big to be reviewable, but each change was reviewed individually in
context of the `cmake` branch. After this, the `cmake` branch will be
renamed `cmake-porting-history` and preserved.
---------
Co-authored-by: Geoff Jukes <geoffjukes@users.noreply.github.com>
Co-authored-by: Bartek Zdanowski <bartek.zdanowski@gmail.com>
Co-authored-by: Carlos Bentzen <cadubentzen@gmail.com>
Co-authored-by: Dennis E. Mungai <2356871+Brainiarc7@users.noreply.github.com>
Co-authored-by: Cosmin Stejerean <cstejerean@gmail.com>
Co-authored-by: Carlos Bentzen <carlos.bentzen@bitmovin.com>
Co-authored-by: Cosmin Stejerean <cstejerean@meta.com>
Co-authored-by: Cosmin Stejerean <cosmin@offbytwo.com>
This issue is observed on Python 3.10+ and above.
This workaround addresses a major backwards compatibility break with a
major Python release version where collections.abc isn't available.
Fixes#1192
The official, static-linked linux builds were crashing in their use of
getaddrinfo, which libcurl was configured to use. Both getaddrinfo
and all of its alternatives available in glibc fail with static
linking.
We can fix this by configuring libcurl to use libc-ares on Linux
instead. This allows us to keep the benefits of a statically-linked
Linux binary.
Closes#996
Change-Id: Ib4a9eb939813fd165727788726459ef4adf3fc4d
The newest pylint release complained about several issues that the
older release did not. This resolves those issues:
- removes unneeded "u" prefix from strings
- adds "encoding" parameter for all open() calls
- because "encoding" is a python3-only parameter, use python3 in all
the scripts that we control
Unfortunately, python2 is required for any scripts that import modules
from the ancient Chromium build system we're using (referenced by
DEPS), as well as kokoro scripts.
Change-Id: I2e9f97af508efe58b5a71de21740e59b1528affd
This fixes the Debug build of libpng on arm64 by avoiding CPU-specific
optimizations that are not in our sources list. The Release build
appears to have been unaffected, possibly due to link-time
optimizations or dead code stripping.
Change-Id: I900e00fe30b9f3748f2587cfea89a636b3a19811
This brings our default build config more in line with what is
necessary for some platforms anyway: using the system-installed
toolchain and sysroot to build everything.
We will no longer fetch source or binaries for any specific build
tools, such as libc++, clang, gold, binutils, or valgrind.
The main part of this change is the changing of default gyp settings
in gyp_packager.py. For this, a bug in gyp_packager.py had to be
fixed, in which similar GYP_DEFINE key names (such as clang and
host_clang) would conflict, causing some defaults not to be installed
properly.
In order to enable clang=0 by default, some changes had to be made in
common.gypi:
- compiler macros added to fix a compatibility issue between
Chromium's base/mac/ folder and the actual OSX SDK
- replaced clang_warning_flags variables with standard cflags
settings, plus xcode_settings for OSX
- turned off warnings-as-errors for non-shaka code, rather than
allow-listing specific warning types, since we can't actually fix
those warnings on any platform
- disabled two specific warnings in shaka code, both of which are
caused by headers from our non-shaka dependencies
Also, one warning (missing "override" keyword) has been fixed in
vod_media_info_dump_muxer_listener.h.
Although these changes were done to make building simpler on a wider
array of platforms (arm64, for example), it seems to make the build a
bit faster, too. For me, at least, on my main Linux workstation:
- "gclient sync" now runs 20-30% faster
- "ninja -C out/Release" now runs 5-13% faster
The following environment variables are no longer required:
- DEPOT_TOOLS_WIN_TOOLCHAIN
- MACOSX_DEPLOYMENT_TARGET
Documentation, Dockerfiles, and GitHub Actions workflows have been
updated to reflect this.
The following GYP_DEFINES are no longer required for anyone:
- clang=0
- host_clang=0
- clang_xcode=1
- use_allocator=none
- use_experimental_allocator_shim=0
Documentation, Dockerfiles, and GitHub Actions workflows have been
updated to reflect this.
The following repos are no longer dependencies in gclient:
- binutils
- clang
- gold
- libc++
- libc++abi
- valgrind
The following gclient hooks have been removed:
- clang
- mac_toolchain
- sysroot
Change-Id: Ie94ccbeec722ab73c291cb7df897d20761a09a70
Also roll_boringssl.py script is also adjusted to work with latest
boringssl.
Disable ASM compilation on Windows x64 which fails to compile. It also
means there are no HW AES on Windows x64.
Issue #198.
Change-Id: Ib7e8ff506f014c8c733f1882eeeddbe34fa28511
With the update, the script needs to be launched from
out/{Debug,Release} directory.
Also updated docs to make it clear how to use the tool.
Fixes#538.
Change-Id: I37b71afc33e3b73ff5232e43d79c52ffa5859d57
- Update curl to 7.57.0
- Roll clang, which is needed due to MacOS / XCode update
- Fixes and suppress compilation errors due to clang update
Fixes#285
Change-Id: Ibac3288c641861605c3c0500d34d27373e6eecfe
Seeing some failures on some platforms when compiled with clang
disabled:
GYP_DEFINES="clang=0" gclient runhooks
Several changes to make it work:
1. Mark packager code with packager_code=1 in GYP definitions.
2. Disable a few checks in non-packager code, which we do not have
direct control: dangling-else, deprecated-declarations,
unused-function
3. Fix the relevant errors in packager code.
4. Revert HAVE_STROPTS_H in curl config which is not available in
all linux distributions.
Fixes#286Fixes#293
Change-Id: I729b41f99403c5ad9487c6cc4a7dc06f6323cef8
Cues used to be generated in the end of the file; if http range
request is not supported, clients have to download the whole
file to get to the cues.
This CL updated TwoPassSingleSegmentSegmenter to write cues
after writing webm header.
This CL also updates libwebm dependency to latest.
Closes#159
Change-Id: Ic73548e1b872e6b13a37627707e7d0ff3e556877
WebRTC portion (gyp build) now checked in because separate repository is no longer available due to transition to git.
Change-Id: I56436cbbc3f02a5f77112917d6229aea69f21d97
Create a new MacOSX configuration for curl:
- Add a curl_config.h created on MacOSX
- Add a find_curl_ca_bundle.sh that searches default folder from Homebrew and MacPorts for a ca bundle.
To find the files either install OpenSSL from Homebrew or MacPorts, or Curl from MacPorts.
When generating ninja files we end up with two libwebm targets.
packager/media/formats/webm/webm.gyp:webm -> becomes target libwebm
packager/third_party/libwebm/libwebm.gyp:libwebm -> becomes target libwebm
Solve this conflict by renaming libwebm.gyp:libwebm to mkvmuxer.
Find curl_ca_bundle script is used by linux only. Move it out of
gyp and make sure it runs only in linux.
Also make zlib deps generic for all systems, not for linux only.
This is the first step of getting edash_packager builds on Windows
and Mac.
Change-Id: I82d3e5872fd0dd6f57127890bc062490332a1b69
Fix "Problem with the SSL CA cert" issue @
https://github.com/google/edash-packager/issues/6. On Ubuntu, CURL_CA_BUNDLE
is located at "/etc/ssl/certs/ca-certificates.crt" while on CentOs, it is
located at "/etc/ssl/certs/ca-bundle.crt".
With this change, GYP scans the common appearing locations for SSL CA cert.
Change-Id: I838bae3954ec51f1430c900d1e82596d45a4db84
Joey Parrish
Dennis E. Mungai
Joey Parrish
Jacob Trimble
KongQun Yang
Haoming Chen
Thomas Inskip
Anders Hasselqvist