Commit Graph

360 Commits

Author SHA1 Message Date
Joey Parrish 249c83f14b build: Fix additional python2/3 issues
The generate_version_string script was only producing correct results
in python 2, not python 3.  The gyp file that references it explicitly
runs it in python3.  The shebang line of the script has been updated
to match.  The script itself has been updated such that it now works
correctly in both python2 and python3.

Scripts that are only used as modules (not executed directly) have had
their shebang lines removed.

This fixes CI failures on GitHub Actions.

Change-Id: I309bafd2fb05e8fb33f5e092ead179c8c42ea5d3
2021-08-26 12:22:20 -07:00
Caitlin O'Callaghan cd018a71c3
Low latency DASH support (#979)
# LL-DASH Support
These changes add support for LL-DASH streaming. 

**NOTE:** LL-HLS support is still in progress, but it's coming. :) 

## Testing
`./chunking_unittest --gtest_filter="ChunkingHandlerTest.LowLatencyDash"`

`./media_event_unittest --gtest_filter="MpdNotifyMuxerListenerTest.LowLatencyDash"`

`./mpd_unittest --gtest_filter="PeriodTest.LowLatencyDashMpdGetXml"`
`./mpd_unittest --gtest_filter="SimpleMpdNotifierTest.NotifyAvailabilityTimeOffset"`
`./mpd_unittest --gtest_filter="SimpleMpdNotifierTest.NotifySegmentDuration"`
`./mpd_unittest --gtest_filter="LowLatencySegmentTest.LowLatencySegmentTemplate"`

Note, packager_test must be run from the main project directory
`./out/Release/packager_test --gtest_filter="PackagerTest.LowLatencyDashEnabledAndUtcTimingNotSet"`
`./out/Release/packager_test --gtest_filter="PackagerTest.LowLatencyDashEnabledAndUtcTimingNotSet"`
2021-08-25 08:38:05 -07:00
Joey Parrish ac125564b9 build: Fix pylint 2.10 issues, use python3 where possible
The newest pylint release complained about several issues that the
older release did not.  This resolves those issues:

 - removes unneeded "u" prefix from strings
 - adds "encoding" parameter for all open() calls
 - because "encoding" is a python3-only parameter, use python3 in all
   the scripts that we control

Unfortunately, python2 is required for any scripts that import modules
from the ancient Chromium build system we're using (referenced by
DEPS), as well as kokoro scripts.

Change-Id: I2e9f97af508efe58b5a71de21740e59b1528affd
2021-08-24 12:27:03 -07:00
Joey Parrish cfbe5c08c2 cleanup: Convert all time parameters to signed
This converts all time parameters to signed, finishing a cleanup that
was started in 2018 in b4256bf0.  This changes the type of:
  - timestamps
    - PTS specifically
  - timestamp offsets
  - timescales
  - durations

This excludes:
  - MP4 box definitions
  - DTS specifically

This is meant to address signed/unsigned conversion issues on arm64
that caused some test cases to fail.

Change-Id: Ic752a20cbc6e31fea6bc0894d1771833171e7cbe
2021-08-05 18:24:15 +00:00
Joey Parrish 56e227267c Fix python linter errors and add linter checks to CI
Internal CI systems and the new GitHub CI system were out of sync,
with the external system not doing any linting.  Further, the internal
system was using an internal-only linter for Python.

This creates a script for Python linting based on the open-source
pylint tool, checks in the Google Style Guide's pylintrc file, creates
a custom action for linting and adds it to the existing workflows,
fixes pre-existing linter errors in Python scripts, and updates pylint
overrides.

b/190743862

Change-Id: Iff1f5d4690b32479af777ded0834c31c2161bd10
2021-06-21 21:46:48 +00:00
Joey Parrish a2e07a901e Refactor actions and workflows
It turns out that workflows were the wrong way to abstract reusable
pieces of work.  This turns common steps into custom actions (build
docs, build packager, test packager) which can be used as encapsulated
steps in multiple workflows.

This is a much more natural way to avoid duplication compared to the
previous approach of triggering one workflow from another.  This also
has the benefit of all of the steps of a release being represented on
GitHub as a single workflow, making it easier to understand what is
happening and what event triggered those steps.

Change-Id: Ife156d60069a39594c7b3bb3bc32080e6453b544
2021-06-17 10:32:24 -07:00
KongQun Yang 72ea5d30f0 Fix gpylint errors
Change-Id: I861cfe98d50867ab9ee9be69cd4ffc3a7157fbd8
2021-06-09 12:17:48 -07:00
Samidh 4686454a73
Add segment_list support for DASH on-demand profile
Configurable under flag --dash_force_segment_list, default to false.

Note that DASH live profile is not supported right now.
2021-05-25 12:08:58 -07:00
Vishal Shah d9124d6aaa
[WEBVTT] Fix missing text alignment tags from output
Legacy players, e.g. older versions of ExoPlayer, do not handle default webvtt text alignment correctly. Need to specify `align:center` explicitly cues without text alignment for backwards compatibility.

Fixes #925.
2021-05-04 22:57:43 -07:00
Joey Parrish d48bf0f48e fix: Explicitly signal the lack of CEA captions in HLS
Closes #922

Change-Id: If097a66642ca40c355a476655ff1394b7b937b17
2021-03-30 18:26:37 +00:00
Jacob Trimble 00af192626 Cleanup HttpFile and related PR.
This implements many of the comments made on the PR and cleans up those
files.

Closes #149

Change-Id: Ice73fe3c04a6f595da6986a4c070e50cb20f9435
2021-03-02 17:43:47 +00:00
Jacob Trimble a0f3f2cd3a Add cc_index to stream descriptor.
This also allows setting the language of different text streams from
the same input.  Multiple streams can use the same input stream
using different cc_index values and can each use a different language.

This also will try to pull the language from the input if not
specified.

Change-Id: I7078710b509b7d77dad8cb4299a82f954af7e9e7
2021-02-17 18:33:53 +00:00
JPeMu 36ef7ec945
[MPEG-TS] Fix PCR reserved bits not being set correctly
Fixes #893.
2021-02-03 12:09:07 -08:00
KongQun Yang 10daa39901 [MP4] Allow not to generate 'sidx' box for single-segment too
I.e. the flag --generate_sidx_in_media_segments,
--nogenerate_sidx_in_media_segments work for both single-segment
and multi-segment mode with this change.

Related to #862.

Change-Id: Icd27fd00e8e036ba0c4709b48650372429cc0351
2020-12-11 19:08:37 +00:00
Jacob Trimble a93eeca5db Add TTML-in-MP4 output support.
This changes the default MP4 output to use TTML and adds a way to
choose which one is used.  This is done with 'format=ttml+mp4' or
'format=vtt+mp4'.

This also fixes the boxes output in WebVTT in MP4.

Change-Id: Ieaa7fc44fbf4dc020a5bb70cfa3578ec10e088ce
2020-11-20 15:10:33 -08:00
Jacob Trimble 4766654b4d Add TTML text output.
This only supports TTML output; meaning the user can convert WebVTT
into TTML, but not the other way around.  This will be useful for
DVB-sub subtitles that would be better supported within TTML.

This only adds text-based output; a follow-up will add MP4 support.

Change-Id: I0944b7df95d7765e55f203fc5e9a644f5c455dd8
2020-11-19 18:03:39 +00:00
Jacob Trimble f4c07b9ce0 Work around non-deterministic tests.
We currently have a bug about non-deterministic output in the MPD
generator.  This works around that bug by optionally doing everything
in a single thread.  This allows us to run manifest comparisons without
making the major changes needed to add that feature.

Issue #177

Change-Id: I10e1084dac77841220161fbd2575cdcb5c13c00e
2020-10-20 22:00:39 +00:00
Jacob Trimble c169c83613 Add integration test for WebVTT in MP4.
Change-Id: Ic8cd785d1fefde4004f27cfce346df2b2a8f643e
2020-10-19 10:55:35 -07:00
Jacob Trimble 56908a83a7 Convert text WebVTT output to muxer.
Now text-based WebVTT also uses the generic media pipeline.  This
converts the WebVttTextOutputHandler to a WebVttMuxer to be more
consistent with the other muxer types.

This also allows choosing between single-segment text and multi-segment.
Before, we would generate both and use single-segment for DASH and
multi-segment for HLS; but now you can choose between either and either
are supported in both DASH and HLS.

Change-Id: I6f7edda09e01b5f40e819290d3fe6e88677018d9
2020-10-19 17:24:38 +00:00
Jacob Trimble 2909ca0c77 Handle segmented text same as audio/video.
Now the same pipeline for handling the audio/videos streams will handle
the segmented text streams too.  This doesn't apply to the text output,
only to the MP4 variants.  This also fixes a bug where we added the
X-TIMESTAMP-MAP tag even when there wasn't TS streams; this doesn't
otherwise change the behavior around that tag.

Change-Id: I03f7cea56efa42e96311c00841330629a14aa053
2020-10-07 17:49:03 +00:00
Jacob Trimble 9babfb883b Fix segmented WebVTT test.
The test added in the previous CL was broken due to a rebase on another
change.  This subtly changed some of the byte offsets that broke the
test.  This wasn't caught since I didn't rebase and re-run the tests
before merging.

Change-Id: Id7e4c7688278eae37da1a14f1648263b4dda98cd
2020-09-16 13:24:16 -07:00
Jacob Trimble e3bc85f12d Make WebVttParser a MediaParser.
This changes it from an OriginHandler to a MediaParser and moves the
handling of it to the Demuxer.  This will allow more generic handling
of text by giving it the same abstractions as video/audio handling.

Change-Id: Ibbde3c84d228ec8e83af1ed266ea97dbc9589c24
2020-09-16 20:12:48 +00:00
koln67 5b9fd409a5
[MP4] Change major brand from isom to mp41
This fixes warnings that 'isom' can only be a compatible instead of major brand.

Issue #755.
2020-09-08 15:45:21 -07:00
koln67 562040e000
[TS] Calculate frame rate using the second sample
Related to #751.
2020-09-02 14:27:02 -07:00
koln67 54bf394c03
[HLS] Ensure playlists with TS streams include FRAME-RATE
It fixes the FRAME-RATE missing issue in #816.
2020-08-21 12:04:24 -07:00
koln67 28074e5c96
[HLS] Add support for independent segments tag
Fixes #564.
2020-08-19 11:17:21 -07:00
koln67 5c91e54b8b
[HLS] Don't include FRAME-RATE in EXT-X-STREAM-INF
Part of #816.
2020-08-06 21:51:35 -07:00
koln67 caa47e374d
Properly handle AVC profiles with SPS extension
Part of #755 to improve DASH spec compliance.
2020-08-05 17:08:53 -07:00
koln67 a8ea7fd085
Always set ES_ID to 0 when writing ES Descriptor in mp4 (#798)
Required by ISO/IEC 14496-14:2018 and ISO/IEC 23000-19:2018.

Issue #755.
2020-07-15 14:14:15 -07:00
Joe Foraci 6b036b9bb1
Support different IVs for each track (#793)
This is supported by allowing IV to be specified in the "key_info_string".

Fixes #543.
2020-06-25 20:37:50 -07:00
Jacob Trimble 1911c1beaa Add flag for extra PlayReady header data.
Fixes #756

Change-Id: I4fa6328480130675a0257fd2c6663d91729cf72c
2020-06-22 22:55:25 +00:00
Jacob Trimble 665e784cbd Move PSSH generation to EncryptionHandler.
The KeySource now only handles fetching the keys and loading any PSSH
info from the license; it will not handle generating new PSSH info
based on the config.

This will allow the PSSH generation to access to the full
EncryptionConfig so we can add additional options to it.

Issue #756

Change-Id: Ia67387aa3d5ec0d723b7f5f21fc517f64c840393
2020-06-22 22:51:44 +00:00
Jacob Trimble 51fe84f9d5 Provide better diffs for MP4 failures.
If the user has the GPAC MP4 tools installed, this will use that to
provide a better, text-based diff of the MP4 files.  This makes looking
at test failures much better than just "binary files differ".

Change-Id: Ifdca54c02226a9f0fe9ddcf9c6b28e960e568111
2020-06-10 17:12:24 +00:00
Jacob Trimble f0107beb9e Make some of the test args better.
Change-Id: I473f3318c6913e78f60a19166a54f7c873078b73
2020-06-04 15:05:40 -07:00
Jacob Trimble 719668aa1c Fix output in Python3 when tests fail.
Change-Id: Icd7552cee1336c10fe42f9f0bae175171347d369
2020-06-04 14:59:40 -07:00
Jacob Trimble 7e41937bb1 Consolidate key system enums.
There were several enum types that all were used for key systems.  This
combines them into one to make it more clear and only needing to update
one.  This also uses a bit field to specify multiple key systems instead
of using a std::vector.

Change-Id: Ia88039835492a5bd47f449ba4b76187046deeec0
2020-06-04 17:19:18 +00:00
KongQun Yang 1b262b5784 Fix failures with latest gpylint
Change-Id: Id80000b530b0582b907730db9311075a7f5307fa
2020-06-03 22:59:22 +00:00
Zajcev Evgeny 98a9d1baf6
Support for MPEG-1 Audio in mpeg2ts I/O and packed-audio output (#778)
Implemented according to https://www.datavoyage.com/mpgscript/mpeghdr.htm.

Closes #779.
2020-06-02 17:32:19 -07:00
sr90 7aefbeb27f
Adding check for creating captions with dash_only and hls_only. (#761)
Fix the problems in the previous CL for #651.
2020-04-29 17:39:12 -07:00
Tim Lansen 8e113af215
[DASH] Include <mspr:pro> alongside to <cenc:pssh> for PlayReady (#749)
Configurable under flag --include_mspr_pro_for_playready. True by default.

Closes #743.
2020-04-17 10:20:03 -07:00
sr90 fbcc8f31e3
Consider DASH stream descriptor field "roles" for WebVTT text streams. (#740)
Fixes #708.
2020-03-26 23:16:07 -07:00
Joey Parrish 9a3ea747a4 Fix python style errors
Change-Id: I78d387329cf4d9b4bead30e4df5d684b31ef104e
2020-03-19 15:50:25 -07:00
sr90 55349aa4c8
Adding --allow_codec_switching (#726)
To allow adaptive switching between different codecs.

Closes #542.
2020-03-17 19:33:44 -07:00
Tomohiro IKEDA c576814d30
Clean up descriptions in mpd command line flags (#730)
Remove the 'Exclusive' statements on --mpd_output and --output_media_info as they
can already work together.
2020-03-11 10:25:33 -07:00
sr90 a1dd82d478
Support dash_only and hls_only parameters (#721)
This allows conditional stream descriptors which apply to DASH or HLS only.

Closes #651.
2020-03-06 10:19:47 -08:00
KongQun Yang 4028bf727b Allow specifying protection pattern for pattern encryption
Added --crypt_byte_block, --skip_byte_block to allow users to specify
protection pattern other than 1:9 for pattern based encryption scheme,
e.g. cbcs and cens.

Closes #710.
b/147307451

Change-Id: I9f64a7639170c737f138572689b28d17286325c7
2020-02-28 00:50:58 +00:00
KongQun Yang b900565a0f Support Dolby Vision backward compatible profiles
Added Dolby Vision backward compatible signalling, i.e. for Dolby Vision
profile 8, both base codec without Dolby Vision and HDR codec with Dolby
Vision are signalled.

This is achieved by using a new MuxerListener implementation
MultiCodecMuxerListener, which wraps multiple child MuxerListeners and
is able to delegate to the child MuxerListeners based on the codecs in
the stream.

Closes #341.

Change-Id: I1967bb1ed503087cdd011c364e5fb5647d516ca4
2020-02-11 05:24:09 +00:00
Daniel Cantarín 7aab7a8b50
Add --hls_media_sequence_number to support custom value in HLS
HLS uses the EXT-X-MEDIA-SEQUENCE tag at the start of a live playlist in
order to specify the first segment sequence number. This is because any
live playlist have a limited number of segments, and they also keep
updating with new segments while removing old ones. When a player refreshes
the playlist, this information is important for keeping track of segments
positions.

When the packager starts, it naturally starts this count from zero. However,
there are many situations where the packager may be restarted, without this
meaning starting this value from zero (but continuing a previous sequence).
The most common situations are problems in the encoder feeding the packager.

With those cases in mind, this parameter allows to set the initial
EXT-X-MEDIA-SEQUENCE value. This way, it's possible to continue the sequence
number from previous packager run.

Closes #691.
2020-01-31 10:25:19 -08:00
Tomohiro IKEDA 0b53b40428 Change --generate_static_mpd to --generate_static_live_mpd (#672) 2019-11-15 11:53:47 -08:00
Vladimir Kazakov 981515e6c8 Generate correct PlayReady PSSH data when AES-CBC encryption is used
Fixes #602.
2019-11-15 11:48:48 -08:00