Commit Graph

1515 Commits

Author SHA1 Message Date
Joey Parrish 15b27549a8
chore: Upgrade absl and protobuf (#1337)
protobuf depends on absl, so both needed an update.

Updating absl fixes issues on Alpine 3.19 (see PR #1327), and also
removes the need for hacks around vlog flags.
2024-02-09 20:57:12 -08:00
Joey Parrish 9b384828c6
build: Do not treat third-party warnings as errors (#1338) 2024-02-09 19:10:08 -08:00
Joey Parrish 7ef51671f1
fix: Fix uninitialized value found by Valgrind (#1336) 2024-02-08 12:50:12 -08:00
modernletter c09eb831b8
feat: Parse MPEG-TS PMT ES language and maximum bitrate descriptors (#369) (#1311)
Part of https://github.com/shaka-project/shaka-packager/issues/369

This adds read support for some MPEG-TS PMT elementary stream
descriptors:
- ISO639 Language Descriptor providing language code and audio type
- Maximum Bitrate Descriptor providing peak stream bandwidth

Those metadata are propagated to StreamInfo structures:
- StreamInfo.language field
- AudioStreamMetadata.max_bitrate field for audio streams
- audio type is currently not propagated - corresponding field has to be
added to AudioStreamMetadata

Test vector file containing those descriptors is provided.
2024-02-08 11:58:26 -08:00
SteveR-PMP 2ba67bc24c
feat: default text zero bias (#1330)
A positive value, in milliseconds. It is the threshold used to determine
if we should assume that the text stream actually starts at time zero.
If the first sample comes before default_text_zero_bias_ms, then the
start will be padded as the stream is assumed to start at zero. If the
first sample comes after default_text_zero_bias_ms then the start of the
stream will not be padded as we cannot assume the start time of the
stream.
2024-02-08 10:39:50 -08:00
Cosmin Stejerean db59ad582a
fix: remove extra block assumptions in mbedtls integration (#1323)
The current mbedtls integration was not working for some modes. See for
example #1316 and also lots of failing integration tests.

For example in pattern encryptor it works on one block at a time so it
cannot assume it's going to always get a buffer with a padding for an
extra block.

From what I can tell when the padding mode is correctly set to
`MBEDTLS_PADDING_NONE` there is no extra block being written to or
required.

This passes all crypto unit tests and integration tests.

Closes #1316
2024-02-08 10:16:52 -08:00
Cosmin Stejerean 9b9adf38ff
test: fix fake clock for muxer for integration tests (#1322)
The fix in #1289 was not complete and left the fake clock as null which
didn't have any effect. This was revealed by integration tests showing
mismatches in the timestamps in MP4.
2024-02-08 09:49:15 -08:00
Cosmin Stejerean 5d44368478
fix: preserve case for stream descriptors (#1321)
The accidental tolower in `SplitAndTrimSkipEmpty` was causing stream
descriptors to not preserve case for certain things like
accessibilities.
2024-02-08 09:48:46 -08:00
Cosmin Stejerean 56bd823339
fix: duration formatting and update mpd testdata to reflect new format (#1320)
As part of the CMake port we updated the duration formatting to contain
maximum of 6 decimal places but without trailing 0s. There was a bug
however where it used 6 significant digits rather than 6 decimal places
(`%g` rather than `%f`).

This fixes the bug and also updates the MPD sample files for the
integration tests to contain maximum of 6 decimal places.
2024-02-08 09:48:14 -08:00
Cosmin Stejerean e21519bb28
test: update webm samples to use 0.3.0 version number (#1319)
The current libwebm integration test samples contain `libwebm-0.2.1`
however we have updated to a newer version of libwebm so we need to
update the samples.

As of `libwebm-0.3.0` this signature has been frozen so we won't have to
do this again.
2024-02-08 09:47:36 -08:00
Joey Parrish 3e71302ba4
feat!: Rewrite build system and third-party dependencies (#1310)
This work was done over ~80 individual commits in the `cmake` branch,
which are now being merged back into `main`. As a roll-up commit, it is
too big to be reviewable, but each change was reviewed individually in
context of the `cmake` branch. After this, the `cmake` branch will be
renamed `cmake-porting-history` and preserved.

---------

Co-authored-by: Geoff Jukes <geoffjukes@users.noreply.github.com>
Co-authored-by: Bartek Zdanowski <bartek.zdanowski@gmail.com>
Co-authored-by: Carlos Bentzen <cadubentzen@gmail.com>
Co-authored-by: Dennis E. Mungai <2356871+Brainiarc7@users.noreply.github.com>
Co-authored-by: Cosmin Stejerean <cstejerean@gmail.com>
Co-authored-by: Carlos Bentzen <carlos.bentzen@bitmovin.com>
Co-authored-by: Cosmin Stejerean <cstejerean@meta.com>
Co-authored-by: Cosmin Stejerean <cosmin@offbytwo.com>
2023-12-01 09:32:19 -08:00
modernletter ba5c77155a
fix: Add missing <cstdint> includes (#1306)
CC version 13 needs `<cstdint>` to be explicitly included to
provide fixed bits integer types.

Some files using it inludes `<stdint.h>`, some are missing direct or
undirect inclusion. This PR adds `<cstdint>` inclusion to the
minimal set of files, allowing compilation on GCC 13.

Closes #1305
2023-11-29 22:51:37 -08:00
wjywbs ac59b9ebc9
fix: Update to use official FairPlay UUID. (#1281)
Reference: https://github.com/Dash-Industry-Forum/Identifiers/issues/119 and https://dashif.org/identifiers/content_protection/
2023-11-29 22:33:13 -08:00
sr90 8465f5f020
feat(DASH): Add video transfer characteristics. (#1210)
This PR is related to https://github.com/shaka-project/shaka-packager/issues/1035
2023-09-08 14:41:41 -07:00
Shaka Bot b7660590a6
chore: Sync common workflows (#1262)
This is an automated sync of common workflows for this organization.
The upstream source is:

4245160a0b

Co-authored-by: Shaka Bot <shaka-bot@users.noreply.github.com>
2023-09-01 19:26:57 -07:00
Caitlin O'Callaghan f264befe86
feat: Write colr atom to muxed mp4 (#1261)
This PR is an extension of the full AV1 codec string feature: [PR
1205](https://github.com/shaka-project/shaka-packager/pull/1205) and
relates to [Issue
1007](https://github.com/shaka-project/shaka-packager/issues/1007) and
[Issue
1202](https://github.com/shaka-project/shaka-packager/issues/1202).

As per the AV1 spec, the codec string may contain optional color values.
These color values are critical for detecting HDR video streams - see
[Issue
1007](https://github.com/shaka-project/shaka-packager/issues/1007).
Color information is extracted from the input mp4's `colr` atom and used
to generate the full AV1 codec string. This PR preserves the color
information by writing the `colr` atom to the muxed mp4.

**References**:
- [AV1 Codec ISO Media File Format
Binding](https://aomediacodec.github.io/av1-isobmff/#codecsparam)
- [AV1 Bitstream & Decoding Process
Specification - Section 6.4.2 Color config semantics (page
117)](https://aomediacodec.github.io/av1-spec/av1-spec.pdf)
- [QuickTime File Format
Specification](https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/QTFFChap3/qtff3.html#//apple_ref/doc/uid/TP40000939-CH205-125526)
2023-08-29 18:46:19 -07:00
Prakash Duggaraju dcf32258ff
fix: Fix handling of non-interleaved multi track FMP4 files (#1214)
Do not assume that each fragment contains all tracks. 
Use track id instead of index to pick the correct timestamp.

Fixes #1213
2023-08-21 16:34:32 -07:00
Caitlin O'Callaghan cc9a691aef
feat: Generate the entire AV1 codec string when the colr atom is present (#1205)
As per the AV1 spec, the codec string may contain optional color values.

This extracts the missing color information from the mp4 `colr` atom, if
present, and generates the full AV1 codec string. 

Closes #1007
2023-08-04 09:00:59 -07:00
Shaka Bot 6d745460b6
chore: Sync common workflows (#1250)
This is an automated sync of common workflows for this organization.
The upstream source is:
208556c5ea

Co-authored-by: Shaka Bot <shaka-bot@users.noreply.github.com>
2023-07-24 17:17:54 -07:00
sr90 520926c27a
fix(MP4): Add compatible brand dby1 for Dolby content. (#1211)
This PR adds dby1 compatible brand to dolby content as per
https://professional.dolby.com/siteassets/content-creation/dolby-vision-for-content-creators/dolby_vision_bitstreams_within_the_iso_base_media_file_format_dec2017.pdf
2023-07-18 19:50:33 -07:00
Sergio Garcia Murillo d6f28d456c
fix: Prevent crash in GetEarliestTimestamp() if periods are empty (#1173)
While I have not yet found why the periods are empty, this will prevent shaka from seg faulting

Fixes #1172
2023-07-12 14:51:00 -07:00
xyb dab165d3e5
fix: Fix failure on very short WebVTT files (#1216)
Fix a bug that if the webvtt file is very short, e.g. only contains one
block

    WEBVTT

    00:00:00.500 --> 00:00:02.000
    The Web is always changing

shaka packager will report error: "Packaging Error: 6 (END_OF_STREAM)".

Fixes #1217
2023-07-12 09:07:07 -07:00
Allan Lei d9d3c7f8be
fix: Indexing `bytes` produces `int` on python3 for `pssh-box.py` (#1228)
Fixes #1227
2023-07-12 09:03:43 -07:00
sr90 ac47e529ad
fix: Update golden files for ttml tests and failing hls unit tests. (#1226)
Updating golden files for failing ttml tests in packager_test.py related
to https://github.com/shaka-project/shaka-packager/pull/1179 and
failing HLS unit tests related to
https://github.com/shaka-project/shaka-packager/pull/1170.
2023-07-12 09:01:55 -07:00
Peter Zebühr 53d91cd0f1
fix: Don't close upstream on HttpFile::Flush (#1201)
Closing the upstream on flush will effectively terminate the ongoing
curl connection. This means that we would need re-establish the
connection in order to resume writing, this is not what we want. In the
spirit of the documentation of File::Flush

```c++
/// Flush the file so that recently written data will survive an 
/// application crash (but not necessarily an OS crash). For 
/// instance, in LocalFile the data is flushed into the OS but not 
/// necessarily to disk.
```

We will instead wait for the curl thread to finish consuming what ever
might be in the upload cache, but leave the connection open for
subsequent writes.

Fixes #1196
2023-07-05 14:38:01 -07:00
Caitlin O'Callaghan d687ad1ed0
fix: Low Latency DASH: include the "availabilityTimeComplete=false" attribute (#1198)
# Low Latency DASH - `availabilityTimeComplete=false`

Low Latency DASH manifests generated by Packager were missing the
attribute `availabilityTimeComplete`. As per the [DASH
specs](https://dashif.org/docs/CR-Low-Latency-Live-r8.pdf):

**_the AdaptationSet@availabilityTimeCompleteshould be present and be
set to 'FALSE'_**

## The Issue
The missing attribute caused ULL streams from Shaka Packager to no
longer be compatible with DASH.js. Previous versions of DASH.js allowed
users to specify ULL mode when initializing the player. However, the
most recent releases of DASH.js automatically detect ULL by scanning the
manifest for ULL specific attributes. Although there are many attributes
only associated with ULL, [DASH.js only greps for
`availabilityTimeComplete` in its detection
logic](https://github.com/Dash-Industry-Forum/dash.js/blob/development/src/streaming/controllers/PlaybackController.js#L792-L805).
Because of the missing attribute in Packager and the limited ULL
verification criteria by DASH.js, Packager streams were not being
treated as low latency streams by DASH.js.

## Testing
### Unit Testing
`./mpd_unittest
--gtest_filter="SegmentTemplateTest.OneSegmentLowLatency"`
` ./mpd_unittest
--gtest_filter="LowLatencySegmentTest.LowLatencySegmentTemplate"`

### Manual Testing
- Created a low latency stream with Shaka Packager
- Observed the expected `availabilityTimeComplete=false` attribute in
the generated DASH manifest.
2023-07-05 14:33:51 -07:00
Dennis E. Mungai ef17cf81f2
docs: Update CONTRIBUTORS (#1190) 2023-07-05 14:32:24 -07:00
Dennis E. Mungai 998b9cf65e
docs: Update AUTHORS (#1189) 2023-07-05 14:32:03 -07:00
Dennis E. Mungai e407ec7bea
docs: Update build_instructions.md to point to the `main` branch. (#1186)
The default branch is `main`, not `master`. Indicate this on the
documentation as its' not obvious.

Fixes #1060
2023-07-05 14:31:39 -07:00
Marcus Spangenberg 494769ca86
fix: TTML generator timestamp millisecond formatting (#1179)
Fix bug where milliseconds were formatted with two digits instead of
three, resulting in incorrect timestamps in TTML cues.

Fixes #1180
2023-07-05 14:28:57 -07:00
Mark Sargent 1ab6818832
fix: hls, set the DEFAULT explicitly to NO. Supports native HLS players. (#1170)
It appears that not all Apple implementations follow the HLS guidelines.

While the DEFAULT=NO for an audio track should be optional and default
to NO, in practice native HLS players Safari and iOS devices treat the
missing DEFAULT as a MAYBE.

Fixes #1169
2023-07-05 14:18:41 -07:00
Dennis E. Mungai 80e024013d
fix: Fix issues with `collections.abc` in Python 3.10+ (#1188)
This issue is observed on Python 3.10+ and above.
This workaround addresses a major backwards compatibility break with a
major Python release version where collections.abc isn't available.

Fixes #1192
2023-05-01 09:07:08 -07:00
Dennis E. Mungai 161947f53e
fix: Fix type error in pssh-box.py with Python 3.10+ (#1187)
A single-line change on #L170 to `wv.protection_scheme =
struct.unpack('>L', bytes(protection_scheme, encoding='utf-8'))[0]`,
needed to work around this issue on Ubuntu 22.04LTS+ running Python
3.10+:

```sh
TypeError: a bytes-like object is required, not 'str'
```
On line 170.
2023-05-01 09:06:56 -07:00
Joey Parrish d5ca6e84e6
ci: Fix main workflows (#1122)
This brings some workflow improvements and fixes from the `cmake` branch
to `main`, as well as some unique fixes to keep gclient working, so that
we can continue to accept contributions in `main` until the `cmake`
merge is ready.

 - Fix docs build in GitHub Actions (from `cmake` branch)
 - Cancel workflow when a PR is updated (from `cmake` branch)
 - Fix docker failures caused by running as root (from `cmake` branch)
 - Work around exception in depot_tools on Windows
 - Use Windows 2019 images in GitHub Actions for compatibility with gyp
- Remove Docker build on ArchLinux, which no longer supports python2 at
all
- (NOTE: The `cmake` branch is still building on ArchLinux. Docker
builds for Arch will be restored to the `main` branch when the `cmake`
branch is finally merged to `main`.)
2022-10-28 15:46:33 -07:00
Bartek Zdanowski b221aa9caf
fix: Parse one frame mpeg-ts video (#1015)
Closes #1013

Co-authored-by: Joey Parrish <joeyparrish@users.noreply.github.com>
2022-10-27 20:22:17 -07:00
Bartek Zdanowski ab8ab12d09
fix: PTS diverge DTS when DTS close to 2pow33 and PTS more than 0 (#1050)
Fixes #1049

Co-authored-by: Joey Parrish <joeyparrish@users.noreply.github.com>
2022-10-27 14:21:03 -07:00
Shaka Bot 3d60105738
chore: Sync common workflows (#1120)
This is an automated sync of common workflows for this organization.
The upstream source is:

8bfe75f0d2

Co-authored-by: Shaka Bot <shaka-bot@users.noreply.github.com>
2022-10-27 11:20:43 -07:00
Geoff Jukes 5d998fca7f
feat: Add xHE-AAC support (#1092)
Note:
* An xHE-AAC capable encoder will auto adjust the user-specified SAP/RAP
  value to the allowed grid where SAP/RAPs can occur.
e.g.: `-rapInterval 5000` (5s) may result in actual SAPs/RAPs every
4.984s.
* To ensure SAP/RAP starts a new segment, Shaka needs to executed with a
  "--segment_duration" is less than or equal to that adjusted value.
* If every SAP/RAP should trigger a new segment, just set the segment
  length to a very low value e.g.: `--segment_duration 0.1`
2022-10-18 10:14:31 -07:00
Joey Parrish 31129eed64
docs: Deprecate the mailing list for releases (#1071)
Releases will no longer be announced on the mailing list. Instead,
users can subscribe directly through GitHub.
2022-06-28 09:58:17 -07:00
Marcus Wichelmann 9996c736ae
fix: Fix build errors related to std::numeric_limits (#972)
This PR fixes some build errors on Fedora Linux with `g++ (GCC) 11.2.1 20210728 (Red Hat 11.2.1-1)`.
2022-06-24 11:30:44 -07:00
Vishal Shah dc0395291a
fix: dash_roles add role=description for DVS audio per DASH-IF-IOP-v4.3 (#1054)
Fixes support for description role for audio DVS tracks as per spec.
2022-06-02 09:40:34 -07:00
Vishal Shah b9d477b969
fix: webvtt single cue do not fail on EOS (#1061)
While Parsing cue body check for the block size. 
If it's the last block do not error if it doesn't have a newline.

Fixes #1018
2022-06-02 09:27:47 -07:00
Joey Parrish 3fd538a587
fix: Roll back depot_tools, bypass vpython (#1045)
Using the latest depot_tools no longer works.  depot_tools also wants
to auto-update itself, which must now be disabled.

We also need to disable the copy of python (vpython) included in
depot_tools, since for some distros, it has dependencies on system
libraries that no longer exist.

Finally, we need to force some distros to use python 2, because our
build system is ancient and needs to be ripped out and replaced some
day soon.

This fixes build issues in our CI, our Dockerfiles, and in general on
certain platforms or distros.

Closes #1023
2022-03-08 16:46:18 -08:00
Joey Parrish 2f9065349f
ci: Fix macOS build by downgrading to 10.15 (#1044) 2022-03-08 09:04:49 -08:00
Joey Parrish f577e2a0cf
chore: Update URLs after moving projects (#1042)
Since a project URL is encoded into outputs, this means also updating
the golden output files.

Closes #1043
2022-03-07 11:56:34 -08:00
Shaka Bot 576889eecc
chore: Sync common workflows (#1039)
This is an automated sync of common workflows for this organization.
The upstream source is:
b39597e92d

Co-authored-by: Shaka Bot <shaka-bot@users.noreply.github.com>
2022-02-28 21:14:32 -08:00
Shaka Bot 362a2a5dd1
chore: Sync common workflows (#1037)
This is an automated sync of common workflows for this organization.
The upstream source is:
9517df8f73

Co-authored-by: Shaka Bot <shaka-bot@users.noreply.github.com>
2022-02-25 10:12:44 -08:00
Shaka Bot f68108b1f9
chore: Sync common workflows (#1034)
This is an automated sync of common workflows for this organization.
The upstream source is:
aa5e38eb86

Co-authored-by: Shaka Bot <shaka-bot@users.noreply.github.com>
2022-02-24 14:38:50 -08:00
Joey Parrish 157f8bb060
ci: Add workflow to sync issue labels with central config (#1031) 2022-02-16 08:34:19 -08:00
Joey Parrish 5c6157f67a
ci: Remove kokoro config (#1029)
We are now fully committed to GitHub Actions for CI, and all
contributions will come through public PRs, and not through
Google-internal channels.
2022-02-10 17:52:35 -08:00