Mirror-Tools
/
shaka-packager
mirror of https://github.com/shaka-project/shaka-packager.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
shaka-packager/Dockerfile

30 lines
1.1 KiB
Docker
Raw Normal View History

feat: First phase of CMake build system implementation (#1072) There are a lot of changes in this first phase, because there was a lot of infrastructure required to get some meaningful amount of porting done. Future PRs should be simpler. <b>Summary of changes:</b><details> - Remove old deps: - boringssl (replaced with mbedtls, lighter, easier to build) - gflags (replaced with absl::flags) - Chromium build tools - New deps to replace parts of Chromium base: - abseil-cpp - glog - nlohmann::json (for tests only) - Submodules, updates, and CMake build rules for third-party libraries: - curl - gmock/gtest - Ported internal libraries and their tests by removing Chromium deps and adding CMake build rules: - file (now using C++17 filesystem APIs) - license_notice - status - version - Test improvements - Removed file tests that can never be re-enabled - Re-enabled all other disabled file tests - Debug JSON values when HTTP tests fail - Fixed chunked-encoding issues in HTTP tests - Updated and refactored Dockerfiles testing - All docker files working, with OS versions updated to meet the new tool requirements - Local docker builds no longer write files to your working directory as root - Local docker builds can now be run in parallel without clobbering each others' build outputs - DEBUG=1 can drop you into an interactive shell when a docker build fails - Updated and heavily refactored workflows and Dockerfiles - All docker files now tested in parallel on GitHub, speeding up CI - All common workflow components broken out and using workflow_call instead of custom actions - Self-hosted runners now optional, to make testing easier on forks - CMake porting works-in-process can now be fully tested on GitHub - Building ported libraries and passing ported tests on all three platforms! - CI hacks for macOS removed, now testing on macos-latest! - Python2 no longer required! (Only Python3) - Using strict build flags, treating all warnings as errors. </details> <b>Required to build:</b> - CMake >= 3.16 - Python 3 - A compiler supporting C++ >= 17 - g++ >= 9 if using GCC (Clang also fine) - MSVC for Windows <b>Still needs work:</b><details> - Moving other dependencies into submodules (if we keep them): - apple_apsl - icu - libevent - libpng - libwebm - libxml - modp_b64 - protobuf - zlib - Port remaining internal libraries: - app - hls - media/base - media/chunking - media/codecs - media/crypto - media/demuxer - media/event - media/formats/dvb - media/formats/mp2t - media/formats/mp4 - media/formats/packed_audio - media/formats/ttml - media/formats/webm - media/formats/webvtt - media/formats/wvm - media/origin - media/public - media/replicator - media/trick_play - mpd - Port main application - Add logging flags in absl and connect them to glog (which expects gflags) - Port pssh-box.py - Port main test targets (packager_test.py and packager_app.py) - Updating all requirement and build documentation - Remove any remaining refs to gclient, depot_tools, ninja - Update and complete release workflows using release-please </details> Issue #346 (Switch to abseil) Issue #1047 (New build system)
2022-08-16 18:34:51 +00:00
FROM alpine:3.12 as builder
introduces docker for testing or development
2015-07-23 01:49:26 +00:00
test: Update Dockerfiles This updates the main Dockerfile and all the docker-based distro-specific tests. The base OS versions have been updated to versions that have not reached end-of-life status yet, and the list of dependencies required has been updated and pruned. Change-Id: Ibcff2f60e739fd5d999af100af76c40aa91a75bc
2021-10-13 18:00:37 +00:00
# Install utilities, libraries, and dev tools.
RUN apk add --no-cache \
bash curl \
feat: Build c-ares locally (#1285) c-ares (used on Linux only) was an exception to the rule of only linking against internally-built libraries. This fixes that, to support a truly static build of packager.
2023-10-18 23:44:43 +00:00
bsd-compat-headers linux-headers \
ci: Enable parallel builds (#1241) Our workflows and Dockerfiles now use Ninja on Linux & Mac, which enables safe parallel builds. This significantly speeds up our workflows. GitHub Actions typical compilation times (build step only): - Linux 19m => 9m - macOS 23m => 8m - Windows 12m => 10m - Linux arm64 (self-hosted) 72m => 29m - Docker build 25m => 14m Overall workflow time: 84m => 33m Compilation time on my workstation (12 CPUs @3.3GHz): 15m => 3m This also adds a new environment variable "PACKAGER_LOW_MEMORY_BUILD". If defined when CMake is first run, this will configure the build to disable parallel linking to reduce memory usage. This helps us avoid failures on our self-hosted arm64 machines, where 6 CPUs share 4GB of RAM. NOTE: Parallel builds are **NOT** recommended with Unix Makefiles due to the use of excessive RAM during parallel linking. Unix Makefiles, unlike Ninja, cannot be configured to restrict parallel linking during a parallel build. Anecdotally, parallel builds with Makefiles have exhausted a system with 32GB RAM. (My workstation.) In a follow-up, I will update the build documentation to refer to CMake and recommend all of the flags now used in our workflows.
2023-07-17 20:16:03 +00:00
build-base cmake git ninja python3
test: Update Dockerfiles This updates the main Dockerfile and all the docker-based distro-specific tests. The base OS versions have been updated to versions that have not reached end-of-life status yet, and the list of dependencies required has been updated and pruned. Change-Id: Ibcff2f60e739fd5d999af100af76c40aa91a75bc
2021-10-13 18:00:37 +00:00
# Build shaka-packager from the current directory, rather than what has been
# merged.
feat: First phase of CMake build system implementation (#1072) There are a lot of changes in this first phase, because there was a lot of infrastructure required to get some meaningful amount of porting done. Future PRs should be simpler. <b>Summary of changes:</b><details> - Remove old deps: - boringssl (replaced with mbedtls, lighter, easier to build) - gflags (replaced with absl::flags) - Chromium build tools - New deps to replace parts of Chromium base: - abseil-cpp - glog - nlohmann::json (for tests only) - Submodules, updates, and CMake build rules for third-party libraries: - curl - gmock/gtest - Ported internal libraries and their tests by removing Chromium deps and adding CMake build rules: - file (now using C++17 filesystem APIs) - license_notice - status - version - Test improvements - Removed file tests that can never be re-enabled - Re-enabled all other disabled file tests - Debug JSON values when HTTP tests fail - Fixed chunked-encoding issues in HTTP tests - Updated and refactored Dockerfiles testing - All docker files working, with OS versions updated to meet the new tool requirements - Local docker builds no longer write files to your working directory as root - Local docker builds can now be run in parallel without clobbering each others' build outputs - DEBUG=1 can drop you into an interactive shell when a docker build fails - Updated and heavily refactored workflows and Dockerfiles - All docker files now tested in parallel on GitHub, speeding up CI - All common workflow components broken out and using workflow_call instead of custom actions - Self-hosted runners now optional, to make testing easier on forks - CMake porting works-in-process can now be fully tested on GitHub - Building ported libraries and passing ported tests on all three platforms! - CI hacks for macOS removed, now testing on macos-latest! - Python2 no longer required! (Only Python3) - Using strict build flags, treating all warnings as errors. </details> <b>Required to build:</b> - CMake >= 3.16 - Python 3 - A compiler supporting C++ >= 17 - g++ >= 9 if using GCC (Clang also fine) - MSVC for Windows <b>Still needs work:</b><details> - Moving other dependencies into submodules (if we keep them): - apple_apsl - icu - libevent - libpng - libwebm - libxml - modp_b64 - protobuf - zlib - Port remaining internal libraries: - app - hls - media/base - media/chunking - media/codecs - media/crypto - media/demuxer - media/event - media/formats/dvb - media/formats/mp2t - media/formats/mp4 - media/formats/packed_audio - media/formats/ttml - media/formats/webm - media/formats/webvtt - media/formats/wvm - media/origin - media/public - media/replicator - media/trick_play - mpd - Port main application - Add logging flags in absl and connect them to glog (which expects gflags) - Port pssh-box.py - Port main test targets (packager_test.py and packager_app.py) - Updating all requirement and build documentation - Remove any remaining refs to gclient, depot_tools, ninja - Update and complete release workflows using release-please </details> Issue #346 (Switch to abseil) Issue #1047 (New build system)
2022-08-16 18:34:51 +00:00
WORKDIR shaka-packager
COPY . /shaka-packager/
ci: Refactor release workflows (#1309) This adopts release-please to manage releases and changelogs, similar to other shaka-project repos. All release workflows can be run by forks by configuring repo secrets. See docs in .github/workflows/ for details. - Use release-please for releases, changelogs - Convert publication jobs (docs, docker, npm) into reusable workflows - Update workflow documentation - Modernize docker commands - Fix doc permissions for publication - Update artifact handling in build workflow - Fix paths in Dockerfile - Fix paths and arm64 support in NPM package - Fix install paths for PSSH tools - Fix warnings in NPM & Docker actions - Delete custom changelog tooling
2023-11-30 21:17:01 +00:00
RUN rm -rf build
ci: Enable parallel builds (#1241) Our workflows and Dockerfiles now use Ninja on Linux & Mac, which enables safe parallel builds. This significantly speeds up our workflows. GitHub Actions typical compilation times (build step only): - Linux 19m => 9m - macOS 23m => 8m - Windows 12m => 10m - Linux arm64 (self-hosted) 72m => 29m - Docker build 25m => 14m Overall workflow time: 84m => 33m Compilation time on my workstation (12 CPUs @3.3GHz): 15m => 3m This also adds a new environment variable "PACKAGER_LOW_MEMORY_BUILD". If defined when CMake is first run, this will configure the build to disable parallel linking to reduce memory usage. This helps us avoid failures on our self-hosted arm64 machines, where 6 CPUs share 4GB of RAM. NOTE: Parallel builds are **NOT** recommended with Unix Makefiles due to the use of excessive RAM during parallel linking. Unix Makefiles, unlike Ninja, cannot be configured to restrict parallel linking during a parallel build. Anecdotally, parallel builds with Makefiles have exhausted a system with 32GB RAM. (My workstation.) In a follow-up, I will update the build documentation to refer to CMake and recommend all of the flags now used in our workflows.
2023-07-17 20:16:03 +00:00
RUN cmake -S . -B build -DCMAKE_BUILD_TYPE=Debug -G Ninja
RUN cmake --build build/ --config Debug --parallel
Reduce Docker image size This is achieved with these techniques: - Use Alpine as the base image, which is only ~5MB. - Use docker's multi-stage build to keep only the result binaries in the final image. The new image is ~15MB after this change. Also updated Dockerfile to use the current checkout code instead of always sync from the latest. Also added a .dockerignore file to ignore temporary build artifacts. Closes #535. Change-Id: I3c90805ebba40295e69241214ed6d7adbde465b8
2018-12-19 19:45:40 +00:00
# Copy only result binaries to our final image.
feat: First phase of CMake build system implementation (#1072) There are a lot of changes in this first phase, because there was a lot of infrastructure required to get some meaningful amount of porting done. Future PRs should be simpler. <b>Summary of changes:</b><details> - Remove old deps: - boringssl (replaced with mbedtls, lighter, easier to build) - gflags (replaced with absl::flags) - Chromium build tools - New deps to replace parts of Chromium base: - abseil-cpp - glog - nlohmann::json (for tests only) - Submodules, updates, and CMake build rules for third-party libraries: - curl - gmock/gtest - Ported internal libraries and their tests by removing Chromium deps and adding CMake build rules: - file (now using C++17 filesystem APIs) - license_notice - status - version - Test improvements - Removed file tests that can never be re-enabled - Re-enabled all other disabled file tests - Debug JSON values when HTTP tests fail - Fixed chunked-encoding issues in HTTP tests - Updated and refactored Dockerfiles testing - All docker files working, with OS versions updated to meet the new tool requirements - Local docker builds no longer write files to your working directory as root - Local docker builds can now be run in parallel without clobbering each others' build outputs - DEBUG=1 can drop you into an interactive shell when a docker build fails - Updated and heavily refactored workflows and Dockerfiles - All docker files now tested in parallel on GitHub, speeding up CI - All common workflow components broken out and using workflow_call instead of custom actions - Self-hosted runners now optional, to make testing easier on forks - CMake porting works-in-process can now be fully tested on GitHub - Building ported libraries and passing ported tests on all three platforms! - CI hacks for macOS removed, now testing on macos-latest! - Python2 no longer required! (Only Python3) - Using strict build flags, treating all warnings as errors. </details> <b>Required to build:</b> - CMake >= 3.16 - Python 3 - A compiler supporting C++ >= 17 - g++ >= 9 if using GCC (Clang also fine) - MSVC for Windows <b>Still needs work:</b><details> - Moving other dependencies into submodules (if we keep them): - apple_apsl - icu - libevent - libpng - libwebm - libxml - modp_b64 - protobuf - zlib - Port remaining internal libraries: - app - hls - media/base - media/chunking - media/codecs - media/crypto - media/demuxer - media/event - media/formats/dvb - media/formats/mp2t - media/formats/mp4 - media/formats/packed_audio - media/formats/ttml - media/formats/webm - media/formats/webvtt - media/formats/wvm - media/origin - media/public - media/replicator - media/trick_play - mpd - Port main application - Add logging flags in absl and connect them to glog (which expects gflags) - Port pssh-box.py - Port main test targets (packager_test.py and packager_app.py) - Updating all requirement and build documentation - Remove any remaining refs to gclient, depot_tools, ninja - Update and complete release workflows using release-please </details> Issue #346 (Switch to abseil) Issue #1047 (New build system)
2022-08-16 18:34:51 +00:00
FROM alpine:3.12
RUN apk add --no-cache libstdc++ python3
ci: Refactor release workflows (#1309) This adopts release-please to manage releases and changelogs, similar to other shaka-project repos. All release workflows can be run by forks by configuring repo secrets. See docs in .github/workflows/ for details. - Use release-please for releases, changelogs - Convert publication jobs (docs, docker, npm) into reusable workflows - Update workflow documentation - Modernize docker commands - Fix doc permissions for publication - Update artifact handling in build workflow - Fix paths in Dockerfile - Fix paths and arm64 support in NPM package - Fix install paths for PSSH tools - Fix warnings in NPM & Docker actions - Delete custom changelog tooling
2023-11-30 21:17:01 +00:00
COPY --from=builder /shaka-packager/build/packager/packager \
/shaka-packager/build/packager/mpd_generator \
/shaka-packager/build/packager/pssh-box.py \
/usr/bin/
feat: First phase of CMake build system implementation (#1072) There are a lot of changes in this first phase, because there was a lot of infrastructure required to get some meaningful amount of porting done. Future PRs should be simpler. <b>Summary of changes:</b><details> - Remove old deps: - boringssl (replaced with mbedtls, lighter, easier to build) - gflags (replaced with absl::flags) - Chromium build tools - New deps to replace parts of Chromium base: - abseil-cpp - glog - nlohmann::json (for tests only) - Submodules, updates, and CMake build rules for third-party libraries: - curl - gmock/gtest - Ported internal libraries and their tests by removing Chromium deps and adding CMake build rules: - file (now using C++17 filesystem APIs) - license_notice - status - version - Test improvements - Removed file tests that can never be re-enabled - Re-enabled all other disabled file tests - Debug JSON values when HTTP tests fail - Fixed chunked-encoding issues in HTTP tests - Updated and refactored Dockerfiles testing - All docker files working, with OS versions updated to meet the new tool requirements - Local docker builds no longer write files to your working directory as root - Local docker builds can now be run in parallel without clobbering each others' build outputs - DEBUG=1 can drop you into an interactive shell when a docker build fails - Updated and heavily refactored workflows and Dockerfiles - All docker files now tested in parallel on GitHub, speeding up CI - All common workflow components broken out and using workflow_call instead of custom actions - Self-hosted runners now optional, to make testing easier on forks - CMake porting works-in-process can now be fully tested on GitHub - Building ported libraries and passing ported tests on all three platforms! - CI hacks for macOS removed, now testing on macos-latest! - Python2 no longer required! (Only Python3) - Using strict build flags, treating all warnings as errors. </details> <b>Required to build:</b> - CMake >= 3.16 - Python 3 - A compiler supporting C++ >= 17 - g++ >= 9 if using GCC (Clang also fine) - MSVC for Windows <b>Still needs work:</b><details> - Moving other dependencies into submodules (if we keep them): - apple_apsl - icu - libevent - libpng - libwebm - libxml - modp_b64 - protobuf - zlib - Port remaining internal libraries: - app - hls - media/base - media/chunking - media/codecs - media/crypto - media/demuxer - media/event - media/formats/dvb - media/formats/mp2t - media/formats/mp4 - media/formats/packed_audio - media/formats/ttml - media/formats/webm - media/formats/webvtt - media/formats/wvm - media/origin - media/public - media/replicator - media/trick_play - mpd - Port main application - Add logging flags in absl and connect them to glog (which expects gflags) - Port pssh-box.py - Port main test targets (packager_test.py and packager_app.py) - Updating all requirement and build documentation - Remove any remaining refs to gclient, depot_tools, ninja - Update and complete release workflows using release-please </details> Issue #346 (Switch to abseil) Issue #1047 (New build system)
2022-08-16 18:34:51 +00:00
Add pssh tool in docker image Issue #550. Change-Id: Id3fc6317789609b5d31cc671b2a361f8a8739f77
2018-12-20 21:26:10 +00:00
# Copy pyproto directory, which is needed by pssh-box.py script. This line
# cannot be combined with the line above as Docker's copy command skips the
# directory itself. See https://github.com/moby/moby/issues/15858 for details.
ci: Refactor release workflows (#1309) This adopts release-please to manage releases and changelogs, similar to other shaka-project repos. All release workflows can be run by forks by configuring repo secrets. See docs in .github/workflows/ for details. - Use release-please for releases, changelogs - Convert publication jobs (docs, docker, npm) into reusable workflows - Update workflow documentation - Modernize docker commands - Fix doc permissions for publication - Update artifact handling in build workflow - Fix paths in Dockerfile - Fix paths and arm64 support in NPM package - Fix install paths for PSSH tools - Fix warnings in NPM & Docker actions - Delete custom changelog tooling
2023-11-30 21:17:01 +00:00
COPY --from=builder /shaka-packager/build/packager/pssh-box-protos \
/usr/bin/pssh-box-protos