Fixes#1356 which was caused by the fix in #1281 which updated this to
use the correct FairPlay system ID. However since old versions
recognized the previous system ID this restores support for it to avoid
breaking clients.
This adds the option FULLY_STATIC to create fully-static executables.
To create portable, fully-static release executables on Linux, we need
to use musl instead of glibc. Static executables from glibc are not
portable.
The popular musl-gcc wrapper does not support C++, so instead we use a
full musl cross-compiler toolchain in the build workflow.
To build FULLY_STATIC, the user must point to the appropriate
cross-compiler, as we do in the workflow. On systems where musl is the
native libc (such as Alpine Linux), this is not necessary.
I have also read that musl's allocator is not very fast in
multi-threaded applications. So when FULLY_STATIC is enabled, we will
also enable mimalloc, a replacement allocator that is very fast.
I tested a very basic packaging command to compare speeds of dynamic
glibc, static musl, and static musl+mimalloc:
dynamic glibc:
runs: 2.527, 2.798, 2.703, 2.756, 2.959
avg = 2.749, std dev = 0.156s
static musl:
runs: 2.813, 2.920, 3.129, 3.003, 2.738
avg = 2.921s, std dev = 0.154s
static musl+mimalloc:
runs: 2.291, 2.034, 2.415, 2.303, 2.265
avg = 2.262s, std dev = 0.140s
The mimalloc build is 82% faster than musl default allocator, 77% faster
than glibc, and has more consistent runtime characteristics (lower
standard deviation).
We already disabled warnings-as-errors in third-party code, but this
goes futher to completely disable third-party warnings. They are mostly
noise, and distracting. If they aren't errors, we shouldn't have our
build logs flooded with them.
This tweaks the default config for stderrthreshold from absl/log so that
we always get logs to stderr by default, as we did in v2. The --quiet
and --v flags that existed in v2 can still be used to modify the log
level, as well as the new --minloglevel from absl/log.
Issue #1325
An updated version of PR #1027
That previous PR was done using 2021 code, and there were many changes
in the codebase from there, so a rebase was needed and also some minor
tweak here and there. But it's the same code, just reimplemented on a
newer codebase.
If you want to take a look at this in action, after building shaka
packager with this PR's code included, try this commands in 3 different
simultaneous bash sessions:
1. Video UDP input: `ffmpeg -f lavfi -re -i
"testsrc=s=320x240:r=30,format=yuv420p" -c:v h264 -sc_threshold 0 -g 30
-keyint_min 30 -r 30 -a53cc 1 -b:v 150k -preset ultrafast -r 30 -f
mpegts "udp://127.0.0.1:10000?pkt_size=1316"`
2. WebVTT UDP input: `for sec in $(seq 0 9999) ; do printf
"%02d:%02d.000 --> %02d:%02d.000\ntest second ${sec}\n\n" "$(( ${sec} /
60 ))" "$(( ${sec} % 60 ))" "$(( (${sec} + 1) / 60 ))" "$(( (${sec} + 1)
% 60 ))" ; sleep 1 ; done > /dev/udp/127.0.0.1/12345`
3. shaka packager command line: `timeout 60
path/to/build/packager/packager
'in=udp://127.0.0.1:10000?timeout=8000000,stream_selector=0,init_segment=240_init.m4s,segment_template=240_$Number%09d$.m4s,bandwidth=150000'
'in=udp://127.0.0.1:12345?timeout=8000000,stream_selector=0,input_format=webvtt,format=webvtt+mp4,init_segment=text_init.m4s,segment_template=text_$Number%09d$.m4s,language=eng,dash_roles=subtitle'
--mpd_output ./manifest.mpd --segment_duration 3.2
--suggested_presentation_delay 3.2 --min_buffer_time 3.2
--minimum_update_period 3.2 --time_shift_buffer_depth 60
--preserved_segments_outside_live_window 1 --default_language=eng
--dump_stream_info 2>&1`
Note the added `input_format=webvtt` to the shaka packager command's
second selector. That's new from this PR. If you don't use that, shaka's
format autodetection will not detect the webvtt format from the input,
as explained in
https://github.com/shaka-project/shaka-packager/issues/685#issuecomment-1029407191.
Try the command without it if you want to.
Fixes#685Fixes#1017
---------
Co-authored-by: Daniel Cantarín <canta@canta.com.ar>
Add startwithSAP/subsegmentstartswithSAP for aac, ac3, ec3 and ac4 audio tracks according to LIVE or VOD profile.
Replaces #1055
Partial solution for #364
---------
Co-authored-by: Xingzhao Yun <xyun@dolby.com>
Co-authored-by: Joey Parrish <joeyparrish@google.com>
Replaces #1181
* Add support for EBU Teletext input following Level 1.5 of the core
specification ETSI EN 300 706 V1.2.1 (2003-04).
* Add support for webvtt in MP4 segments output.
Closes#272
---------
Co-authored-by: Marcus Spangenberg <marcus.spangenberg@eyevinn.se>
This is based on comments at
https://github.com/google/shaka-packager/pull/891. The muxer is deciding
whether to write to a single file or a segment file based on the
configuration.
Example:
```
../packager 'in=TOS.ts,stream=video,output=tos_video.ts,playlist_name=tos_video.m3u8' \
'in=TOS.ts,stream=audio,output=tos_audio.ts,playlist_name=tos_audio.m3u8' \
--hls_master_playlist_output tos.m3u8
```
Tested the content using Exoplayer.
---------
Co-authored-by: Cosmin Stejerean <cstejerean@meta.com>
In some cases it can happen that the http server responds with a
non-successful status code without reading the response body. In this
case curl may decide not to read from the cache since there is really no
point in sending data to the server. In case some other thread of shaka
has already called HttpFile::Flush it may end up deadlocked there
waiting for the cache to either close or become empty. Thus, we close
the cache when leaving the main thread as no data will be read by curl
after it has finished anyways.
Closes#1347
adds an optional `--hls_start_time_offset` parameter, that, when
used, adds `EXT-X-START` tags to the HLS media playlists.
The EXT-X-START tag allows to specify the location where the player
starts playing, either from start (positive value) or from end (negative
value).
This is especially useful in case of livestreams where this tag can be
used to set the target latency of the playback.
Reference: https://datatracker.ietf.org/doc/html/rfc8216#section-4.3.5.2
The RFC says, that the `EXT-X-START` tag could also be added to the
master playlist, but my tests have shown that most players only respect
it when it's in the media playlists.
Fixes#970
---------
Co-authored-by: Joey Parrish <joeyparrish@google.com>
feat: Added audio specific configuration udts box to AudioSampleEntry
for MP4 input/output. DASH tags for DTS audio as specified in ETSI TS
103 491 and ETSI TS 102 114.
Closes#1301
---------
Co-authored-by: Cosmin Stejerean <cstejerean@meta.com>
ISO/IEC 14496-15 says about the `HEVCDecoderConfigurationRecord`:
> **array_completeness** when equal to 1 indicates that all NAL units of
> the given type are in the following array and none are in the stream;
> when equal to 0 indicates that additional NAL units of the indicated type
> may be in the stream; the default and permitted values are constrained
> by the sample entry name;
This PR sets `array_completeness` to 0 if parameter NAL units may appear
in the stream when they are not stripped by
`--strip_parameter_set_nalus`.
This should increase player-compatibiltity for streams with mid-stream
SAR changes.
---------
Co-authored-by: Cosmin Stejerean <cstejerean@meta.com>
According to a comment in
packager/third_party/abseil-cpp/source/absl/log/CMakeLists.txt, many
linkers will strip the contents of absl::log_flags because its symbols
symbols are only used in a global constructor, and that for now, clients
should link using
$<LINK_LIBRARY:WHOLE_ARCHIVE,absl::log_flags>.
Closes#1325
Add ability to set `Label` tag in MPD, see page 35 of DASH-IF IOP 4.3
https://dashif.org/docs/DASH-IF-IOP-v4.3.pdf
Implements #881
---------
Co-authored-by: Cosmin Stejerean <cstejerean@meta.com>
This will force the muxer to order streams in the order given on the
command-line.
Closes#560Closes#1280Closes#1313
---------
Co-authored-by: Joey Parrish <joeyparrish@users.noreply.github.com>
Co-authored-by: Cosmin Stejerean <cstejerean@meta.com>
protobuf depends on absl, so both needed an update.
Updating absl fixes issues on Alpine 3.19 (see PR #1327), and also
removes the need for hacks around vlog flags.
Part of https://github.com/shaka-project/shaka-packager/issues/369
This adds read support for some MPEG-TS PMT elementary stream
descriptors:
- ISO639 Language Descriptor providing language code and audio type
- Maximum Bitrate Descriptor providing peak stream bandwidth
Those metadata are propagated to StreamInfo structures:
- StreamInfo.language field
- AudioStreamMetadata.max_bitrate field for audio streams
- audio type is currently not propagated - corresponding field has to be
added to AudioStreamMetadata
Test vector file containing those descriptors is provided.
A positive value, in milliseconds. It is the threshold used to determine
if we should assume that the text stream actually starts at time zero.
If the first sample comes before default_text_zero_bias_ms, then the
start will be padded as the stream is assumed to start at zero. If the
first sample comes after default_text_zero_bias_ms then the start of the
stream will not be padded as we cannot assume the start time of the
stream.
The current mbedtls integration was not working for some modes. See for
example #1316 and also lots of failing integration tests.
For example in pattern encryptor it works on one block at a time so it
cannot assume it's going to always get a buffer with a padding for an
extra block.
From what I can tell when the padding mode is correctly set to
`MBEDTLS_PADDING_NONE` there is no extra block being written to or
required.
This passes all crypto unit tests and integration tests.
Closes#1316
The fix in #1289 was not complete and left the fake clock as null which
didn't have any effect. This was revealed by integration tests showing
mismatches in the timestamps in MP4.
As part of the CMake port we updated the duration formatting to contain
maximum of 6 decimal places but without trailing 0s. There was a bug
however where it used 6 significant digits rather than 6 decimal places
(`%g` rather than `%f`).
This fixes the bug and also updates the MPD sample files for the
integration tests to contain maximum of 6 decimal places.
The current libwebm integration test samples contain `libwebm-0.2.1`
however we have updated to a newer version of libwebm so we need to
update the samples.
As of `libwebm-0.3.0` this signature has been frozen so we won't have to
do this again.
This work was done over ~80 individual commits in the `cmake` branch,
which are now being merged back into `main`. As a roll-up commit, it is
too big to be reviewable, but each change was reviewed individually in
context of the `cmake` branch. After this, the `cmake` branch will be
renamed `cmake-porting-history` and preserved.
---------
Co-authored-by: Geoff Jukes <geoffjukes@users.noreply.github.com>
Co-authored-by: Bartek Zdanowski <bartek.zdanowski@gmail.com>
Co-authored-by: Carlos Bentzen <cadubentzen@gmail.com>
Co-authored-by: Dennis E. Mungai <2356871+Brainiarc7@users.noreply.github.com>
Co-authored-by: Cosmin Stejerean <cstejerean@gmail.com>
Co-authored-by: Carlos Bentzen <carlos.bentzen@bitmovin.com>
Co-authored-by: Cosmin Stejerean <cstejerean@meta.com>
Co-authored-by: Cosmin Stejerean <cosmin@offbytwo.com>
CC version 13 needs `<cstdint>` to be explicitly included to
provide fixed bits integer types.
Some files using it inludes `<stdint.h>`, some are missing direct or
undirect inclusion. This PR adds `<cstdint>` inclusion to the
minimal set of files, allowing compilation on GCC 13.
Closes#1305
As per the AV1 spec, the codec string may contain optional color values.
This extracts the missing color information from the mp4 `colr` atom, if
present, and generates the full AV1 codec string.
Closes#1007
Fix a bug that if the webvtt file is very short, e.g. only contains one
block
WEBVTT
00:00:00.500 --> 00:00:02.000
The Web is always changing
shaka packager will report error: "Packaging Error: 6 (END_OF_STREAM)".
Fixes#1217
Closing the upstream on flush will effectively terminate the ongoing
curl connection. This means that we would need re-establish the
connection in order to resume writing, this is not what we want. In the
spirit of the documentation of File::Flush
```c++
/// Flush the file so that recently written data will survive an
/// application crash (but not necessarily an OS crash). For
/// instance, in LocalFile the data is flushed into the OS but not
/// necessarily to disk.
```
We will instead wait for the curl thread to finish consuming what ever
might be in the upload cache, but leave the connection open for
subsequent writes.
Fixes#1196
# Low Latency DASH - `availabilityTimeComplete=false`
Low Latency DASH manifests generated by Packager were missing the
attribute `availabilityTimeComplete`. As per the [DASH
specs](https://dashif.org/docs/CR-Low-Latency-Live-r8.pdf):
**_the AdaptationSet@availabilityTimeCompleteshould be present and be
set to 'FALSE'_**
## The Issue
The missing attribute caused ULL streams from Shaka Packager to no
longer be compatible with DASH.js. Previous versions of DASH.js allowed
users to specify ULL mode when initializing the player. However, the
most recent releases of DASH.js automatically detect ULL by scanning the
manifest for ULL specific attributes. Although there are many attributes
only associated with ULL, [DASH.js only greps for
`availabilityTimeComplete` in its detection
logic](https://github.com/Dash-Industry-Forum/dash.js/blob/development/src/streaming/controllers/PlaybackController.js#L792-L805).
Because of the missing attribute in Packager and the limited ULL
verification criteria by DASH.js, Packager streams were not being
treated as low latency streams by DASH.js.
## Testing
### Unit Testing
`./mpd_unittest
--gtest_filter="SegmentTemplateTest.OneSegmentLowLatency"`
` ./mpd_unittest
--gtest_filter="LowLatencySegmentTest.LowLatencySegmentTemplate"`
### Manual Testing
- Created a low latency stream with Shaka Packager
- Observed the expected `availabilityTimeComplete=false` attribute in
the generated DASH manifest.
It appears that not all Apple implementations follow the HLS guidelines.
While the DEFAULT=NO for an audio track should be optional and default
to NO, in practice native HLS players Safari and iOS devices treat the
missing DEFAULT as a MAYBE.
Fixes#1169
This issue is observed on Python 3.10+ and above.
This workaround addresses a major backwards compatibility break with a
major Python release version where collections.abc isn't available.
Fixes#1192
Cosmin Stejerean
Joey Parrish
wjywbs
Daniel Cantarín
sr90
Peter Zebühr
Marcus Wichelmann
Roy-Funderburk
Vishal Shah
SteveR-PMP
Michał Idzikowski
modernletter
Caitlin O'Callaghan
Prakash Duggaraju
Sergio Garcia Murillo
xyb
Allan Lei
Marcus Spangenberg
Mark Sargent
Dennis E. Mungai