Related to issue #1047
After this it should be possible to build a working `packager` application from the `cmake` branch. Some further logging improvements may be needed to get full parity with the `main` branch in terms of ability to do verbose debug logging, but other than that everything is expected to work.
As per the AV1 spec, the codec string may contain optional color values.
This extracts the missing color information from the mp4 `colr` atom, if
present, and generates the full AV1 codec string.
Closes#1007
This fixes our use of std::filesystem to interpret all paths names as
UTF8. Before this, UTF8 paths did not work correctly in all cases.
This also adds a new unit test to cover this case.
On Windows, it is critical that a UTF8 locale be set at runtime.
Applications linking with Packager as a library should call setlocale(),
and the Packager frontends now do this automatically after converting
wide character arguments into narrow strings.
Closes#652
PR #1242 introduced some bad assertions that went unnoticed because we
are not yet building those modules in the cmake branch. This fixes those
so that ASSERT_ macros are not used in non-void functions, and so that
assumptions about a successful parse are always after an ASSERT_ that
the data was loaded properly.
All ASSERT_ macros from #1242 were audited, and the others were all
fine.
All gtest-based tests should now use add_gtest. Results from these tests
will be surfaced more prominently in GitHub Actions. Before this, it was
hard to find test failures in the log.
LOG(FATAL) aborts the executable, which means gtest can't write any
reports it might be configured to write when tests are complete. This
will interfere with reporting.
This converts the only use of LOG(FATAL), which was in
ReadTestDataFile(), to LOG(ERROR). This also updates test cases to avoid
crashing when ReadTestDataFile() returns an empty buffer.
The `__FILE__` macro does not always get you an absolute path, so our
assumptions about it in test_data_util.cc were flawed.
If `foo.c` references `__FILE__`, something like `gcc -c foo.c`, will
define `__FILE__` as `"foo.c"`. If you use `gcc -c /path/to/foo.c`, then
`__FILE__` will be `"/path/to/foo.c"`.
The Ninja generator for CMake only generates absolute paths for source
files in certain CMake versions. (Exact range unknown.) Rather than
require newer CMake versions and depend on CMake's Ninja generator
maintaining the latest behavior forever, set the macro TEST_DATA_DIR to
point to the test data folder. This is consistent and never depends on
unspecified behavior.
This change will allow the use of the Ninja generator with older
versions of CMake, as found in most of our Docker builds.
- Update protobuf to v23.4, to fix the cstdint error on Arch Linux, and
make some related changes:
- Silence additional compiler warnings for the new protobuf
- Update absl to 20230125.3, to support the protobuf update, and make
some related changes:
- Silence additional compiler warnings for the new protobuf
- Replace GOOGLE_CHECK_OK with ABSL_CHECK_OK
- Replace GUARDED_BY with ABSL_GUARDED_BY
- Update other instances of cstdint missing in our own code
- Always pull the latest docker images, to avoid stale results when your
workstation has pulled an older image at the same label
Rebasing #1148 on top of current cmake branch
Related to #1047
---------
Co-authored-by: Bartek Zdanowski <bartek.zdanowski@gmail.com>
Co-authored-by: Joey Parrish <joeyparrish@google.com>
All HTTP-based tests now use an embedded test server instead of
httpbin.org, which makes them much faster and more reliable.
These more reliable tests also exposed some issues that began recently
with PR #1201. HttpFile's Flush() semantics were different than those
documented for files in general. Flush() used to close the file for
uploading, so that no further writes were allowed, but the documentation
stated that it would only flush data to its destination. PR #1201
brought HttpFile's Flush() in line with the docs, but gave us no way to
terminate a chunked upload.
This adds a new method to File called CloseForWriting(), which
terminates a chunked upload for HttpFile. The only other implementation
that does anything is UdpFile, which uses the socket library function
shutdown() to terminate writes while allowing reads.
This also tweaks HttpFile::CloseWithStatus() so that it will not
generate an error if the file is closed before the HTTP response is
written to the download cache.
This modifies the test HttpFileTest.MultipleWrites so that the file is
Flushed after each chunk. This adds test coverage for the changes
introduced in PR #1201.
Fixes#1224 (missing test coverage for HttpFile::Flush)
This adds Mongoose as a third-party library, and builds on top of that
an embedded HTTP server for our unit tests.
We are using a fork of Mongoose pending the merging of this PR:
https://github.com/cesanta/mongoose/pull/2301
The embedded web server will make our HTTP-based tests independent of
httpbin.org, which will make them quick and reliable.
Rebasing #1155 and applying some fixes. Had to comment out the
`mpd_notify_muxer_listener_unittest` because it depends on
`MockMpdNotifier` from `mpd/base` which has not been ported yet. Can
bring this test back once that has been ported.
Related to #1047
---------
Co-authored-by: Carlos Bentzen <carlos.bentzen@bitmovin.com>
Co-authored-by: Joey Parrish <joeyparrish@google.com>
This reverts commit 2e349845c6 from PR
#1203.
We are moving away from httpbin.org for testing, and this revert will
make subsequent changes to the tests easier to read.
Fix a bug that if the webvtt file is very short, e.g. only contains one
block
WEBVTT
00:00:00.500 --> 00:00:02.000
The Web is always changing
shaka packager will report error: "Packaging Error: 6 (END_OF_STREAM)".
Fixes#1217
This removes all chromium dependencies from media/base/ and completes
the build system in CMake.
The ClosureThread class and its classes were removed, as they were
specific to chromium base. ClosureThread has been replaced by
std::thread.
The byte-swapping utilities in network_util.cc have been removed and
replaced with absl.
generate_unique_temp_path() was split out of file_unittest.cc into
file_test_util.cc, where other test suites could make use of it.
WARN_UNUSED_RESULT was replaced with the C++ standard attribute
[[nodiscard]].
The base::Clock interface was replaced with a typedef for a function
pointer that returns the current time.
This re-enables the tests in http_key_fetcher_unittest.cc by using
httpbin.org.
Issue #1047 (CMake porting)
Issue #346 (absl porting)
This does not depend on absl, which frees macros.h from any library deps
that would make the CMake dependency tree more complicated. This also
fixes build errors in some environments.
mbedtls works very differently from BoringSSL, and many changes had to
be made in the details of AES decryption to accomodate this.
Beyond the basic changes required for mbedtls, part of the CTS padding
implementation had to be rewritten. I believe this is because of an
assumption that held for BoringSSL, but not for mbedtls. I was unable to
determine what it was, so I rewrote the CTS decryption using reference
materials. After this, tests passed.
The deterministc PRNG I used with mbedtls in the RSA tests differs
somewhat from the old one, so the expected vectors had to be
regenerated. The old determinstic tests were also disabled, and are now
re-enabled.
Since cryptography is sensitive code, and because there were far more
changes needed here than just updating some headers and utility function
calls, this has been split into its own PR for separate review from the
rest of the media/base porting work.
Issue #1047 (CMake porting)
Issue #346 (absl porting)
Rewrite test_data_util.cc to locate files relative to the source file
itself, rather than using a service from chromium `base::`.
Issue #1047 (CMake porting)
Issue #346 (absl porting)
For some reason, the status util unittest was in media/base/ instead of
status/
The broken headers in the status library were not obviously broken until
media/base/ porting started, and they were used for the first time.
Issue #1047 (CMake porting)
Note:
* An xHE-AAC capable encoder will auto adjust the user-specified SAP/RAP
value to the allowed grid where SAP/RAPs can occur.
e.g.: `-rapInterval 5000` (5s) may result in actual SAPs/RAPs every
4.984s.
* To ensure SAP/RAP starts a new segment, Shaka needs to executed with a
"--segment_duration" is less than or equal to that adjusted value.
* If every SAP/RAP should trigger a new segment, just set the segment
length to a very low value e.g.: `--segment_duration 0.1`
Note:
* An xHE-AAC capable encoder will auto adjust the user-specified SAP/RAP
value to the allowed grid where SAP/RAPs can occur.
e.g.: `-rapInterval 5000` (5s) may result in actual SAPs/RAPs every
4.984s.
* To ensure SAP/RAP starts a new segment, Shaka needs to executed with a
"--segment_duration" is less than or equal to that adjusted value.
* If every SAP/RAP should trigger a new segment, just set the segment
length to a very low value e.g.: `--segment_duration 0.1`
Cosmin Stejerean
Cosmin Stejerean
Caitlin O'Callaghan
Prakash Duggaraju
Joey Parrish
sr90
Joey Parrish
Cosmin Stejerean
xyb
Carlos Bentzen
Marcus Spangenberg
Bartek Zdanowski
Geoff Jukes
Vishal Shah