e6b57f72a8
There are a lot of changes in this first phase, because there was a
lot of infrastructure required to get some meaningful amount of
porting done. Future PRs should be simpler.
<b>Summary of changes:</b><details>
- Remove old deps:
- boringssl (replaced with mbedtls, lighter, easier to build)
- gflags (replaced with absl::flags)
- Chromium build tools
- New deps to replace parts of Chromium base:
- abseil-cpp
- glog
- nlohmann::json (for tests only)
- Submodules, updates, and CMake build rules for third-party
libraries:
- curl
- gmock/gtest
- Ported internal libraries and their tests by removing Chromium deps
and adding CMake build rules:
- file (now using C++17 filesystem APIs)
- license_notice
- status
- version
- Test improvements
- Removed file tests that can never be re-enabled
- Re-enabled all other disabled file tests
- Debug JSON values when HTTP tests fail
- Fixed chunked-encoding issues in HTTP tests
- Updated and refactored Dockerfiles testing
- All docker files working, with OS versions updated to meet the
new tool requirements
- Local docker builds no longer write files to your working
directory as root
- Local docker builds can now be run in parallel without clobbering
each others' build outputs
- DEBUG=1 can drop you into an interactive shell when a docker
build fails
- Updated and heavily refactored workflows and Dockerfiles
- All docker files now tested in parallel on GitHub, speeding up CI
- All common workflow components broken out and using workflow_call
instead of custom actions
- Self-hosted runners now optional, to make testing easier on forks
- CMake porting works-in-process can now be fully tested on GitHub
- Building ported libraries and passing ported tests on all three
platforms!
- CI hacks for macOS removed, now testing on macos-latest!
- Python2 no longer required! (Only Python3)
- Using strict build flags, treating all warnings as errors.
</details>
<b>Required to build:</b>
- CMake >= 3.16
- Python 3
- A compiler supporting C++ >= 17
- g++ >= 9 if using GCC (Clang also fine)
- MSVC for Windows
<b>Still needs work:</b><details>
- Moving other dependencies into submodules (if we keep them):
- apple_apsl
- icu
- libevent
- libpng
- libwebm
- libxml
- modp_b64
- protobuf
- zlib
- Port remaining internal libraries:
- app
- hls
- media/base
- media/chunking
- media/codecs
- media/crypto
- media/demuxer
- media/event
- media/formats/dvb
- media/formats/mp2t
- media/formats/mp4
- media/formats/packed_audio
- media/formats/ttml
- media/formats/webm
- media/formats/webvtt
- media/formats/wvm
- media/origin
- media/public
- media/replicator
- media/trick_play
- mpd
- Port main application
- Add logging flags in absl and connect them to glog (which expects
gflags)
- Port pssh-box.py
- Port main test targets (packager_test.py and packager_app.py)
- Updating all requirement and build documentation
- Remove any remaining refs to gclient, depot_tools, ninja
- Update and complete release workflows using release-please
</details>
Issue #346 (Switch to abseil)
Issue #1047 (New build system)
|
||
|---|---|---|
| .. | ||
| README.md | ||
| build-docs.yaml | ||
| build-matrix.json | ||
| build.yaml | ||
| docker-hub-release.yaml | ||
| docker-image.yaml | ||
| github-release.yaml | ||
| lint.yaml | ||
| npm-release.yaml | ||
| pr.yaml | ||
| sync-labels.yaml | ||
| test-linux-distros.yaml | ||
| update-docs.yaml | ||
| update-issues.yaml | ||
| validate-pr-title.yaml | ||
README.md
GitHub Actions CI
Reusable workflows
-
build.yaml: Build and test all combinations of OS & build settings. Also builds docs on Linux. -
build-docs.yaml: Build Packager docs. Runs only on Linux. -
docker-image.yaml: Build the official Docker image. -
lint.yaml: Lint Shaka Packager. -
test-linux-distros.yaml: Test the build on all Linux distros via docker.
Composed workflows
-
On PR (
pr.yaml), invoke:lint.yamlbuild.yamlbuild-docs.yamldocker-image.yamltest-linux-distros.yaml
-
On release tag (
github-release.yaml):- Create a draft release
- Invoke:
lint.yamlbuild.yamltest-linux-distros.yaml
- Publish the release with binaries from
build.yamlattached
-
On release published:
docker-hub-release.yaml, publishes the official Docker imagenpm-release.yaml, publishes the official NPM packageupdate-docs.yaml:- Invoke
build-docs.yaml - Push the output to the
gh-pagesbranch
- Invoke
Common workflows from shaka-project
sync-labels.yamlupdate-issues.yamlvalidate-pr-title.yaml
Required Repo Secrets
DOCKERHUB_CI_USERNAME: The username of the Docker Hub CI accountDOCKERHUB_CI_TOKEN: An access token for Docker Hub- To generate, visit https://hub.docker.com/settings/security
DOCKERHUB_PACKAGE_NAME: Not a true "secret", but stored here to avoid someone pushing bogus packages to Docker Hub during CI testing from a fork- In a fork, set to a private name which differs from the production one
NPM_CI_TOKEN: An "Automation"-type access token for NPM for theshaka-botaccount- To generate, visit https://www.npmjs.com/settings/shaka-bot/tokens and select the "Automation" type
NPM_PACKAGE_NAME: Not a true "secret", but stored here to avoid someone pushing bogus packages to NPM during CI testing from a fork- In a fork, set to a private name which differs from the production one
Optional Repo Secrets
ENABLE_DEBUG: Set to non-empty to enable debugging via SSH after a failureENABLE_SELF_HOSTED: Set to non-empty to enable self-hosted runners in the build matrix