Commit Graph

1526 Commits

Author SHA1 Message Date
Marcus Wichelmann 76eb2c1575
feat: Add support for the EXT-X-START tag (#973)
adds an optional `--hls_start_time_offset` parameter, that, when
used, adds `EXT-X-START` tags to the HLS media playlists.

The EXT-X-START tag allows to specify the location where the player
starts playing, either from start (positive value) or from end (negative
value).
This is especially useful in case of livestreams where this tag can be
used to set the target latency of the playback.

Reference: https://datatracker.ietf.org/doc/html/rfc8216#section-4.3.5.2

The RFC says, that the `EXT-X-START` tag could also be added to the
master playlist, but my tests have shown that most players only respect
it when it's in the media playlists.

Fixes #970

---------

Co-authored-by: Joey Parrish <joeyparrish@google.com>
2024-02-15 12:06:06 -08:00
Roy-Funderburk 07f780dae1
feat: This patch adds support for DTS:X Profile 2 audio in MP4 files. (#1303)
feat: Added audio specific configuration udts box to AudioSampleEntry
for MP4 input/output. DASH tags for DTS audio as specified in ETSI TS
103 491 and ETSI TS 102 114.

Closes #1301

---------

Co-authored-by: Cosmin Stejerean <cstejerean@meta.com>
2024-02-14 23:03:03 -08:00
Marcus Wichelmann f7b3986818
fix: misleading log output when HLS target duration updates (fixes #969) (#971)
Minor fix. The parameters were swapped.
2024-02-14 22:46:02 -08:00
Marcus Wichelmann 270888abb1
fix: set array_completeness in HEVCDecoderConfigurationRecord correctly (#975)
ISO/IEC 14496-15 says about the `HEVCDecoderConfigurationRecord`:

> **array_completeness** when equal to 1 indicates that all NAL units of
> the given type are in the following array and none are in the stream; 
> when equal to 0 indicates that additional NAL units of the indicated type 
> may be in the stream; the default and permitted values are constrained 
> by the sample entry name;

This PR sets `array_completeness` to 0 if parameter NAL units may appear
in the stream when they are not stripped by
`--strip_parameter_set_nalus`.

This should increase player-compatibiltity for streams with mid-stream
SAR changes.

---------

Co-authored-by: Cosmin Stejerean <cstejerean@meta.com>
2024-02-14 22:10:23 -08:00
Vishal Shah f73ad0d961
feat: HLS / DASH support forced subtitle (#1020)
Closes #988

---------

Co-authored-by: Cosmin Stejerean <cstejerean@meta.com>
2024-02-14 20:27:57 -08:00
Joey Parrish e19d73321d
build: Only ever build static or shared libraries, but never both (#1343)
Closes #1341
2024-02-14 16:20:59 -08:00
Joey Parrish 1f71fa0a0e
build: Fix absl log flags being stripped (#1340)
According to a comment in
packager/third_party/abseil-cpp/source/absl/log/CMakeLists.txt, many
linkers will strip the contents of absl::log_flags because its symbols
symbols are only used in a global constructor, and that for now, clients
should link using
$<LINK_LIBRARY:WHOLE_ARCHIVE,absl::log_flags>.

Closes #1325
2024-02-14 13:59:18 -08:00
sr90 b1c5a7433e
feat(DASH): Add Label element. (#1175)
Add ability to set `Label` tag in MPD, see page 35 of DASH-IF IOP 4.3
https://dashif.org/docs/DASH-IF-IOP-v4.3.pdf

Implements #881 

---------

Co-authored-by: Cosmin Stejerean <cstejerean@meta.com>
2024-02-14 10:36:08 -08:00
SteveR-PMP aad2a12a9d
feat: order streams in manifest based on command-line order (#1329)
This will force the muxer to order streams in the order given on the
command-line.

Closes #560
Closes #1280
Closes #1313

---------

Co-authored-by: Joey Parrish <joeyparrish@users.noreply.github.com>
Co-authored-by: Cosmin Stejerean <cstejerean@meta.com>
2024-02-14 09:21:11 -08:00
Joey Parrish df04d0ed6e
chore: Unfork zlib (#1339)
The issues we had with zlib and CMake have been resolved upstream.
2024-02-12 14:10:51 -08:00
Michał Idzikowski 9193a90f5a
chore: upgrade to alpine 3.19 (#1327)
Co-authored-by: Joey Parrish <joeyparrish@google.com>
2024-02-10 09:25:37 -08:00
Joey Parrish 15b27549a8
chore: Upgrade absl and protobuf (#1337)
protobuf depends on absl, so both needed an update.

Updating absl fixes issues on Alpine 3.19 (see PR #1327), and also
removes the need for hacks around vlog flags.
2024-02-09 20:57:12 -08:00
Joey Parrish 9b384828c6
build: Do not treat third-party warnings as errors (#1338) 2024-02-09 19:10:08 -08:00
Joey Parrish 7ef51671f1
fix: Fix uninitialized value found by Valgrind (#1336) 2024-02-08 12:50:12 -08:00
modernletter c09eb831b8
feat: Parse MPEG-TS PMT ES language and maximum bitrate descriptors (#369) (#1311)
Part of https://github.com/shaka-project/shaka-packager/issues/369

This adds read support for some MPEG-TS PMT elementary stream
descriptors:
- ISO639 Language Descriptor providing language code and audio type
- Maximum Bitrate Descriptor providing peak stream bandwidth

Those metadata are propagated to StreamInfo structures:
- StreamInfo.language field
- AudioStreamMetadata.max_bitrate field for audio streams
- audio type is currently not propagated - corresponding field has to be
added to AudioStreamMetadata

Test vector file containing those descriptors is provided.
2024-02-08 11:58:26 -08:00
SteveR-PMP 2ba67bc24c
feat: default text zero bias (#1330)
A positive value, in milliseconds. It is the threshold used to determine
if we should assume that the text stream actually starts at time zero.
If the first sample comes before default_text_zero_bias_ms, then the
start will be padded as the stream is assumed to start at zero. If the
first sample comes after default_text_zero_bias_ms then the start of the
stream will not be padded as we cannot assume the start time of the
stream.
2024-02-08 10:39:50 -08:00
Cosmin Stejerean db59ad582a
fix: remove extra block assumptions in mbedtls integration (#1323)
The current mbedtls integration was not working for some modes. See for
example #1316 and also lots of failing integration tests.

For example in pattern encryptor it works on one block at a time so it
cannot assume it's going to always get a buffer with a padding for an
extra block.

From what I can tell when the padding mode is correctly set to
`MBEDTLS_PADDING_NONE` there is no extra block being written to or
required.

This passes all crypto unit tests and integration tests.

Closes #1316
2024-02-08 10:16:52 -08:00
Cosmin Stejerean 9b9adf38ff
test: fix fake clock for muxer for integration tests (#1322)
The fix in #1289 was not complete and left the fake clock as null which
didn't have any effect. This was revealed by integration tests showing
mismatches in the timestamps in MP4.
2024-02-08 09:49:15 -08:00
Cosmin Stejerean 5d44368478
fix: preserve case for stream descriptors (#1321)
The accidental tolower in `SplitAndTrimSkipEmpty` was causing stream
descriptors to not preserve case for certain things like
accessibilities.
2024-02-08 09:48:46 -08:00
Cosmin Stejerean 56bd823339
fix: duration formatting and update mpd testdata to reflect new format (#1320)
As part of the CMake port we updated the duration formatting to contain
maximum of 6 decimal places but without trailing 0s. There was a bug
however where it used 6 significant digits rather than 6 decimal places
(`%g` rather than `%f`).

This fixes the bug and also updates the MPD sample files for the
integration tests to contain maximum of 6 decimal places.
2024-02-08 09:48:14 -08:00
Cosmin Stejerean e21519bb28
test: update webm samples to use 0.3.0 version number (#1319)
The current libwebm integration test samples contain `libwebm-0.2.1`
however we have updated to a newer version of libwebm so we need to
update the samples.

As of `libwebm-0.3.0` this signature has been frozen so we won't have to
do this again.
2024-02-08 09:47:36 -08:00
Joey Parrish 3e71302ba4
feat!: Rewrite build system and third-party dependencies (#1310)
This work was done over ~80 individual commits in the `cmake` branch,
which are now being merged back into `main`. As a roll-up commit, it is
too big to be reviewable, but each change was reviewed individually in
context of the `cmake` branch. After this, the `cmake` branch will be
renamed `cmake-porting-history` and preserved.

---------

Co-authored-by: Geoff Jukes <geoffjukes@users.noreply.github.com>
Co-authored-by: Bartek Zdanowski <bartek.zdanowski@gmail.com>
Co-authored-by: Carlos Bentzen <cadubentzen@gmail.com>
Co-authored-by: Dennis E. Mungai <2356871+Brainiarc7@users.noreply.github.com>
Co-authored-by: Cosmin Stejerean <cstejerean@gmail.com>
Co-authored-by: Carlos Bentzen <carlos.bentzen@bitmovin.com>
Co-authored-by: Cosmin Stejerean <cstejerean@meta.com>
Co-authored-by: Cosmin Stejerean <cosmin@offbytwo.com>
2023-12-01 09:32:19 -08:00
modernletter ba5c77155a
fix: Add missing <cstdint> includes (#1306)
CC version 13 needs `<cstdint>` to be explicitly included to
provide fixed bits integer types.

Some files using it inludes `<stdint.h>`, some are missing direct or
undirect inclusion. This PR adds `<cstdint>` inclusion to the
minimal set of files, allowing compilation on GCC 13.

Closes #1305
2023-11-29 22:51:37 -08:00
wjywbs ac59b9ebc9
fix: Update to use official FairPlay UUID. (#1281)
Reference: https://github.com/Dash-Industry-Forum/Identifiers/issues/119 and https://dashif.org/identifiers/content_protection/
2023-11-29 22:33:13 -08:00
sr90 8465f5f020
feat(DASH): Add video transfer characteristics. (#1210)
This PR is related to https://github.com/shaka-project/shaka-packager/issues/1035
2023-09-08 14:41:41 -07:00
Shaka Bot b7660590a6
chore: Sync common workflows (#1262)
This is an automated sync of common workflows for this organization.
The upstream source is:

4245160a0b

Co-authored-by: Shaka Bot <shaka-bot@users.noreply.github.com>
2023-09-01 19:26:57 -07:00
Caitlin O'Callaghan f264befe86
feat: Write colr atom to muxed mp4 (#1261)
This PR is an extension of the full AV1 codec string feature: [PR
1205](https://github.com/shaka-project/shaka-packager/pull/1205) and
relates to [Issue
1007](https://github.com/shaka-project/shaka-packager/issues/1007) and
[Issue
1202](https://github.com/shaka-project/shaka-packager/issues/1202).

As per the AV1 spec, the codec string may contain optional color values.
These color values are critical for detecting HDR video streams - see
[Issue
1007](https://github.com/shaka-project/shaka-packager/issues/1007).
Color information is extracted from the input mp4's `colr` atom and used
to generate the full AV1 codec string. This PR preserves the color
information by writing the `colr` atom to the muxed mp4.

**References**:
- [AV1 Codec ISO Media File Format
Binding](https://aomediacodec.github.io/av1-isobmff/#codecsparam)
- [AV1 Bitstream & Decoding Process
Specification - Section 6.4.2 Color config semantics (page
117)](https://aomediacodec.github.io/av1-spec/av1-spec.pdf)
- [QuickTime File Format
Specification](https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/QTFFChap3/qtff3.html#//apple_ref/doc/uid/TP40000939-CH205-125526)
2023-08-29 18:46:19 -07:00
Prakash Duggaraju dcf32258ff
fix: Fix handling of non-interleaved multi track FMP4 files (#1214)
Do not assume that each fragment contains all tracks. 
Use track id instead of index to pick the correct timestamp.

Fixes #1213
2023-08-21 16:34:32 -07:00
Caitlin O'Callaghan cc9a691aef
feat: Generate the entire AV1 codec string when the colr atom is present (#1205)
As per the AV1 spec, the codec string may contain optional color values.

This extracts the missing color information from the mp4 `colr` atom, if
present, and generates the full AV1 codec string. 

Closes #1007
2023-08-04 09:00:59 -07:00
Shaka Bot 6d745460b6
chore: Sync common workflows (#1250)
This is an automated sync of common workflows for this organization.
The upstream source is:
208556c5ea

Co-authored-by: Shaka Bot <shaka-bot@users.noreply.github.com>
2023-07-24 17:17:54 -07:00
sr90 520926c27a
fix(MP4): Add compatible brand dby1 for Dolby content. (#1211)
This PR adds dby1 compatible brand to dolby content as per
https://professional.dolby.com/siteassets/content-creation/dolby-vision-for-content-creators/dolby_vision_bitstreams_within_the_iso_base_media_file_format_dec2017.pdf
2023-07-18 19:50:33 -07:00
Sergio Garcia Murillo d6f28d456c
fix: Prevent crash in GetEarliestTimestamp() if periods are empty (#1173)
While I have not yet found why the periods are empty, this will prevent shaka from seg faulting

Fixes #1172
2023-07-12 14:51:00 -07:00
xyb dab165d3e5
fix: Fix failure on very short WebVTT files (#1216)
Fix a bug that if the webvtt file is very short, e.g. only contains one
block

    WEBVTT

    00:00:00.500 --> 00:00:02.000
    The Web is always changing

shaka packager will report error: "Packaging Error: 6 (END_OF_STREAM)".

Fixes #1217
2023-07-12 09:07:07 -07:00
Allan Lei d9d3c7f8be
fix: Indexing `bytes` produces `int` on python3 for `pssh-box.py` (#1228)
Fixes #1227
2023-07-12 09:03:43 -07:00
sr90 ac47e529ad
fix: Update golden files for ttml tests and failing hls unit tests. (#1226)
Updating golden files for failing ttml tests in packager_test.py related
to https://github.com/shaka-project/shaka-packager/pull/1179 and
failing HLS unit tests related to
https://github.com/shaka-project/shaka-packager/pull/1170.
2023-07-12 09:01:55 -07:00
Peter Zebühr 53d91cd0f1
fix: Don't close upstream on HttpFile::Flush (#1201)
Closing the upstream on flush will effectively terminate the ongoing
curl connection. This means that we would need re-establish the
connection in order to resume writing, this is not what we want. In the
spirit of the documentation of File::Flush

```c++
/// Flush the file so that recently written data will survive an 
/// application crash (but not necessarily an OS crash). For 
/// instance, in LocalFile the data is flushed into the OS but not 
/// necessarily to disk.
```

We will instead wait for the curl thread to finish consuming what ever
might be in the upload cache, but leave the connection open for
subsequent writes.

Fixes #1196
2023-07-05 14:38:01 -07:00
Caitlin O'Callaghan d687ad1ed0
fix: Low Latency DASH: include the "availabilityTimeComplete=false" attribute (#1198)
# Low Latency DASH - `availabilityTimeComplete=false`

Low Latency DASH manifests generated by Packager were missing the
attribute `availabilityTimeComplete`. As per the [DASH
specs](https://dashif.org/docs/CR-Low-Latency-Live-r8.pdf):

**_the AdaptationSet@availabilityTimeCompleteshould be present and be
set to 'FALSE'_**

## The Issue
The missing attribute caused ULL streams from Shaka Packager to no
longer be compatible with DASH.js. Previous versions of DASH.js allowed
users to specify ULL mode when initializing the player. However, the
most recent releases of DASH.js automatically detect ULL by scanning the
manifest for ULL specific attributes. Although there are many attributes
only associated with ULL, [DASH.js only greps for
`availabilityTimeComplete` in its detection
logic](https://github.com/Dash-Industry-Forum/dash.js/blob/development/src/streaming/controllers/PlaybackController.js#L792-L805).
Because of the missing attribute in Packager and the limited ULL
verification criteria by DASH.js, Packager streams were not being
treated as low latency streams by DASH.js.

## Testing
### Unit Testing
`./mpd_unittest
--gtest_filter="SegmentTemplateTest.OneSegmentLowLatency"`
` ./mpd_unittest
--gtest_filter="LowLatencySegmentTest.LowLatencySegmentTemplate"`

### Manual Testing
- Created a low latency stream with Shaka Packager
- Observed the expected `availabilityTimeComplete=false` attribute in
the generated DASH manifest.
2023-07-05 14:33:51 -07:00
Dennis E. Mungai ef17cf81f2
docs: Update CONTRIBUTORS (#1190) 2023-07-05 14:32:24 -07:00
Dennis E. Mungai 998b9cf65e
docs: Update AUTHORS (#1189) 2023-07-05 14:32:03 -07:00
Dennis E. Mungai e407ec7bea
docs: Update build_instructions.md to point to the `main` branch. (#1186)
The default branch is `main`, not `master`. Indicate this on the
documentation as its' not obvious.

Fixes #1060
2023-07-05 14:31:39 -07:00
Marcus Spangenberg 494769ca86
fix: TTML generator timestamp millisecond formatting (#1179)
Fix bug where milliseconds were formatted with two digits instead of
three, resulting in incorrect timestamps in TTML cues.

Fixes #1180
2023-07-05 14:28:57 -07:00
Mark Sargent 1ab6818832
fix: hls, set the DEFAULT explicitly to NO. Supports native HLS players. (#1170)
It appears that not all Apple implementations follow the HLS guidelines.

While the DEFAULT=NO for an audio track should be optional and default
to NO, in practice native HLS players Safari and iOS devices treat the
missing DEFAULT as a MAYBE.

Fixes #1169
2023-07-05 14:18:41 -07:00
Dennis E. Mungai 80e024013d
fix: Fix issues with `collections.abc` in Python 3.10+ (#1188)
This issue is observed on Python 3.10+ and above.
This workaround addresses a major backwards compatibility break with a
major Python release version where collections.abc isn't available.

Fixes #1192
2023-05-01 09:07:08 -07:00
Dennis E. Mungai 161947f53e
fix: Fix type error in pssh-box.py with Python 3.10+ (#1187)
A single-line change on #L170 to `wv.protection_scheme =
struct.unpack('>L', bytes(protection_scheme, encoding='utf-8'))[0]`,
needed to work around this issue on Ubuntu 22.04LTS+ running Python
3.10+:

```sh
TypeError: a bytes-like object is required, not 'str'
```
On line 170.
2023-05-01 09:06:56 -07:00
Joey Parrish d5ca6e84e6
ci: Fix main workflows (#1122)
This brings some workflow improvements and fixes from the `cmake` branch
to `main`, as well as some unique fixes to keep gclient working, so that
we can continue to accept contributions in `main` until the `cmake`
merge is ready.

 - Fix docs build in GitHub Actions (from `cmake` branch)
 - Cancel workflow when a PR is updated (from `cmake` branch)
 - Fix docker failures caused by running as root (from `cmake` branch)
 - Work around exception in depot_tools on Windows
 - Use Windows 2019 images in GitHub Actions for compatibility with gyp
- Remove Docker build on ArchLinux, which no longer supports python2 at
all
- (NOTE: The `cmake` branch is still building on ArchLinux. Docker
builds for Arch will be restored to the `main` branch when the `cmake`
branch is finally merged to `main`.)
2022-10-28 15:46:33 -07:00
Bartek Zdanowski b221aa9caf
fix: Parse one frame mpeg-ts video (#1015)
Closes #1013

Co-authored-by: Joey Parrish <joeyparrish@users.noreply.github.com>
2022-10-27 20:22:17 -07:00
Bartek Zdanowski ab8ab12d09
fix: PTS diverge DTS when DTS close to 2pow33 and PTS more than 0 (#1050)
Fixes #1049

Co-authored-by: Joey Parrish <joeyparrish@users.noreply.github.com>
2022-10-27 14:21:03 -07:00
Shaka Bot 3d60105738
chore: Sync common workflows (#1120)
This is an automated sync of common workflows for this organization.
The upstream source is:

8bfe75f0d2

Co-authored-by: Shaka Bot <shaka-bot@users.noreply.github.com>
2022-10-27 11:20:43 -07:00
Geoff Jukes 5d998fca7f
feat: Add xHE-AAC support (#1092)
Note:
* An xHE-AAC capable encoder will auto adjust the user-specified SAP/RAP
  value to the allowed grid where SAP/RAPs can occur.
e.g.: `-rapInterval 5000` (5s) may result in actual SAPs/RAPs every
4.984s.
* To ensure SAP/RAP starts a new segment, Shaka needs to executed with a
  "--segment_duration" is less than or equal to that adjusted value.
* If every SAP/RAP should trigger a new segment, just set the segment
  length to a very low value e.g.: `--segment_duration 0.1`
2022-10-18 10:14:31 -07:00
Joey Parrish 31129eed64
docs: Deprecate the mailing list for releases (#1071)
Releases will no longer be announced on the mailing list. Instead,
users can subscribe directly through GitHub.
2022-06-28 09:58:17 -07:00