Commit Graph

136 Commits

Author SHA1 Message Date
Jacob Trimble 51fe84f9d5 Provide better diffs for MP4 failures.
If the user has the GPAC MP4 tools installed, this will use that to
provide a better, text-based diff of the MP4 files.  This makes looking
at test failures much better than just "binary files differ".

Change-Id: Ifdca54c02226a9f0fe9ddcf9c6b28e960e568111
2020-06-10 17:12:24 +00:00
Jacob Trimble 719668aa1c Fix output in Python3 when tests fail.
Change-Id: Icd7552cee1336c10fe42f9f0bae175171347d369
2020-06-04 14:59:40 -07:00
KongQun Yang 1b262b5784 Fix failures with latest gpylint
Change-Id: Id80000b530b0582b907730db9311075a7f5307fa
2020-06-03 22:59:22 +00:00
sr90 7aefbeb27f
Adding check for creating captions with dash_only and hls_only. (#761)
Fix the problems in the previous CL for #651.
2020-04-29 17:39:12 -07:00
sr90 fbcc8f31e3
Consider DASH stream descriptor field "roles" for WebVTT text streams. (#740)
Fixes #708.
2020-03-26 23:16:07 -07:00
Joey Parrish 9a3ea747a4 Fix python style errors
Change-Id: I78d387329cf4d9b4bead30e4df5d684b31ef104e
2020-03-19 15:50:25 -07:00
sr90 55349aa4c8
Adding --allow_codec_switching (#726)
To allow adaptive switching between different codecs.

Closes #542.
2020-03-17 19:33:44 -07:00
sr90 a1dd82d478
Support dash_only and hls_only parameters (#721)
This allows conditional stream descriptors which apply to DASH or HLS only.

Closes #651.
2020-03-06 10:19:47 -08:00
KongQun Yang 4028bf727b Allow specifying protection pattern for pattern encryption
Added --crypt_byte_block, --skip_byte_block to allow users to specify
protection pattern other than 1:9 for pattern based encryption scheme,
e.g. cbcs and cens.

Closes #710.
b/147307451

Change-Id: I9f64a7639170c737f138572689b28d17286325c7
2020-02-28 00:50:58 +00:00
KongQun Yang b900565a0f Support Dolby Vision backward compatible profiles
Added Dolby Vision backward compatible signalling, i.e. for Dolby Vision
profile 8, both base codec without Dolby Vision and HDR codec with Dolby
Vision are signalled.

This is achieved by using a new MuxerListener implementation
MultiCodecMuxerListener, which wraps multiple child MuxerListeners and
is able to delegate to the child MuxerListeners based on the codecs in
the stream.

Closes #341.

Change-Id: I1967bb1ed503087cdd011c364e5fb5647d516ca4
2020-02-11 05:24:09 +00:00
Tomohiro IKEDA 0b53b40428 Change --generate_static_mpd to --generate_static_live_mpd (#672) 2019-11-15 11:53:47 -08:00
KongQun Yang 1bcaf0a835 Add an end to end test for non square pixels
Issue #663.

Change-Id: Iaba52f0fc6c97d413e973f9a3d8c74cf8c83aa03
2019-10-23 18:41:18 +00:00
KongQun Yang 0342adb132 Fix possible packager hangs when reading mp4 from FIFO
For MP4 files, mp4 parser tries to open and read the same file again
to handle trailing MOOV box.

Open FIFO again may result in packager hang.

Update the code to only attempt to load MOOV for local regular files,
as FIFO and remote files are unseekable anyway.

Fixes #664.

Change-Id: Ib286d2876d202cd5a248ffe70b13589b3cc74bc9
2019-10-21 18:49:52 -07:00
KongQun Yang 7973c5396f Replace Dolby Vision test content
Issue #341

Change-Id: I530b0768341e352f6ee0a1326540cd5712f01638
2019-10-09 21:34:53 +00:00
KongQun Yang 3f909fa551 [HLS] Support HDR signaling, i.e. VIDEO-RANGE attribute
- Parse and extract transfer_characteristics from H264/H265 VUI
  parameters.
- Set VIDEO-RANGE attribute in HLS according to HLS specification:
  https://tools.ietf.org/html/draft-pantos-hls-rfc8216bis-02#section-4.4.4.2
- Also added an end to end test.

Fixes #632.

Change-Id: Iadf557d967b42ade321fb0b152e8e7b64fe9ff3e
2019-10-04 22:50:20 +00:00
KongQun Yang 8029004c6b Add HEVC Dolby Vision support
- Add relevant FOURCCs for Dolby Vision.
- Parse DOVIDecoderConfigurationRecord (dvcC, dvvC) to generate
  Dolby Vision codec string.
- Propagate Dolby Vision configs (dvcC, dvvC, hvcE) from Demuxer
  to Muxer.
- Add a Dolby Vision end to end test.

Support for backward compatibility signaling in DASH and HLS will be
added in a later CL.

Issue #341

Change-Id: If1385df5f48e04b59cb7661130bea48e26b453bf
2019-10-03 23:04:15 +00:00
KongQun Yang 796974d2a1 [DASH] Support custom Accessibility and Role elements
Add dash_accessibilities stream descriptor, which is a semi-colon
separated list of accessibility_scheme_id_uri=value. It is optional.

Add dash_roles stream descriptor, which is a semi-colon separated
list of strings. It is optional.

Closes #565.

Change-Id: Idb1c20bb410fdd016db07e11fe507c102a3dd8ea
2019-06-13 15:43:32 -07:00
KongQun Yang 57046be126 Make packager_test.py work with python 3.5
Addresses binary/str conversion problems like
'can't use a string pattern on a bytes-like object'.

With this change, packager_test.py will work with any version of python
after python2.5.

Fixes #553.

Change-Id: I02066942e4bcdddc1db1daf761abab4cc46169a9
2019-02-13 19:07:56 +00:00
KongQun Yang de534f8550 Make packager_test.py work with python 3.0
Use logging instead of print.

Fixes #553.

Change-Id: I46b9a5bb9bb5d4cb03ddd94163905aebc4b4e9dd
2019-02-01 22:52:59 +00:00
KongQun Yang cd7640a5bd Support trick play with Ad cues
Fixes #528.

Change-Id: I558285a2fae3d2048ef7cefabcd4761e35be47a8
2018-12-13 00:27:11 +00:00
KongQun Yang 273ab09f05 Support HLS characteristics
Add hls_characteristics stream descriptor, which is a colon or semi-colon
separated list of strings. It is optional.

Fixes #430.

Change-Id: Ifcf79316e68768ff065891933de565cd0ff32ec4
2018-11-26 20:02:43 +00:00
KongQun Yang 4b19905bc2 Support subsample encryption in AV1
Closes #453.

Change-Id: I68e46fb83cbf7e62a19fa83698cb66bfc0acd98d
2018-10-11 11:35:00 -07:00
KongQun Yang 8d11e5ea64 Support Marlin DRM signaling in DASH
Only Marlin Adaptive Streaming Specification – Simple Profile is
supported.

Two additional updates:
- Remove FairPlay ContentProtection element from DASH mpd as FairPlay
  does not define a signaling in DASH.
- Updated end to end test to include all DRMs we support.

Closes #381.

Change-Id: Id12269b471ea34983b782cbd92f687332292ef59
2018-10-01 20:22:58 +00:00
KongQun Yang 08aa9b6b2b Remove FairPlayPsshGenerator
This was introduced earlier to indicate FairPlay protection system. But
in fact, it is sufficient to just use the system id for the indication.

- Also updated various parts of the pipeline to support empty PSSH.
- Added an additional FairPlay end to end test using fMP4.

Change-Id: Ica48b7b5235e9a2b5a7f722bcd0fc1ef2073ac13
2018-09-21 00:49:35 +00:00
KongQun Yang 42083d205d [DASH] Fix TTML text input passthrough (regression)
Note that TTML in ISO-BMFF is not supported yet.

Also updated packager_test.py:
- Added a test using TTML passthrough.
- Computed output extension from input extension unless output_format
  is specified.

Fixes #478.

Change-Id: Ia917fc4ed3c326782791ed67601fba02ea28b11d
2018-09-19 23:27:48 +00:00
KongQun Yang 31e5f129b5 Fix incorrrect segment name with $Time$ in segment_template
The time for the previous segment was used when generating the segment
name. This resulted in the first segment being overwritten and
mismatching manifest and media files. It led to playback problems.

Issue #472.

Change-Id: Ia8130ce261585e1a2ede83b26de3e32508de087f
2018-09-13 00:56:28 +00:00
KongQun Yang 0709db4bbc Add support for AV1
Implemented per AV1 Codec ISO Media File Format Binding at
  https://aomediacodec.github.io/av1-isobmff/
And AOM AV1 codec mapping in Matroska/WebM at
  https://github.com/Matroska-Org/matroska-specification/blob/av1-mappin/codec/av1.md

Note that AV1 specific boxes are not supported in this CL, i.e.
AV1 Forward Key Frame sample group entry 'av1f', AV1 Multi-Frame
sample group entry 'av1m' etc are not supported. These boxes are optional.
We will add support later if they are useful to the clients / players.

Encryption is not supported yet.

Issue #453.

Change-Id: I630432d0a9bf82d263ffaf40e57f67fc65eee902
2018-08-31 19:16:04 +00:00
KongQun Yang 40a3b42980 Fix pattern signaling in seig for key rotation with cbcs
Closes #460.
Bug: 112769382

Change-Id: Id2277edf9ac1ca637354f3a585666ad139ed8ee2
2018-08-24 23:20:06 +00:00
KongQun Yang 8612798de1 Change --additional_protection_systems to --protection_systems
Issue: #245.

Change-Id: Ib7da3b874207623e3ee0b1a942ecb2509e750669
2018-08-14 22:12:55 +00:00
KongQun Yang db076d6892 Support Fairplay in --additional_protection_systems
Issue: #245.

Change-Id: I15187c1d3463534bf5e11ff97032311bb1d0c3bf
2018-08-09 23:59:40 +00:00
KongQun Yang 51d39d96a1 Add offset to tranport streams (MPEG2-TS, HLS Packed Audio)
Configurable with --transport_stream_offset_ms.

This is needed to compensate for possible negative timestamps in
inputs, which could happen on ISO-BMFF with EditLists.

Issue #112.

Change-Id: I0fce8766c9df2911b9bb859c1e54052a8ed2abfb
2018-07-26 23:20:21 +00:00
KongQun Yang 9a55d4033f Adjust timestamps in mp4 if there is an initial composition offset
In some ISO-BMFF files, there is an initial non-zero composition offset,
but there is no EditList present.

This is against ISO-BMFF spec recommentation [1] and we believe in most
cases it is just missing the EditList.

[1] 14496-12:2015 8.6.6.1
It is recommended that such an edit be used to establish a presentation
time of 0 for the first presented sample, when composition offsets are
used.

Issue: #112.
Fixes: b/110782437.

Change-Id: I23d33810ce536b09a1e22a2644828d824c1314f5
2018-07-26 23:20:21 +00:00
KongQun Yang 26997bb361 Fix flaky test in packager_test.py
MPD cannot be validated right now since we do not generate deterministic
mpd with multiple inputs due to thread racing.

Exclude mpd comparison in this case.

Change-Id: I03b68b4969a39e20927c8a9b1475f72493682696
2018-07-23 22:21:18 +00:00
KongQun Yang 83f29ff964 packager_test.py: clean up output filename logic
Change-Id: I8df03d6ff6af0c621a7863d19d06e98b93fcb372
2018-07-16 23:06:08 +00:00
KongQun Yang 772400e2d5 Cleanup diff comparison in packager_test.py
Change-Id: I30ef75329060e15d54fdfa5fac44d499e3a6925a
2018-07-16 23:06:01 +00:00
KongQun Yang 71072d4e6d Combine some DASH/HLS tests into one
This saves some testing time and also make it easier to compare
DASH and HLS test results.

Change-Id: Ia4bdb39f1f5ef7ee7b3f8f37f83d7490109380c2
2018-07-16 14:59:16 -07:00
KongQun Yang b4ae6561f8 Cleanup packager_test.py
- Support decryption verification in _CheckTestResults
- Allow diff_files in _CheckTestResults
- Update all functional tests to use _CheckTestResults

Change-Id: I3f9c02f35808eba787becf9b1e5c1ce9238f943e
2018-07-16 19:13:02 +00:00
Aaron Vaage b9dee56294 Rebuild Text WebVtt to MP4 WebVtt
The previous text to mp4 webvtt pipeline was incomplete. It
did not insert ad cues and it could only insert a segment
after a sample ended.

Now the pipeline supports ad cue insert and segment insertion
mid text sample. This required the pipeline to use the text
chunker (to split samples and insert segments) and required
a major overhaul of the text to mp4 converter.

Before the converter came before the chunker. This meant that
the converter only expected to see stream info and text samples.
Moving the converter after the cue aligner and chunker means
that the convert had to be aware of segments and cues.

The general approach is the same, however the converter will
convert the samples per-segment as the chunker will introduce
duplicate samples if a sample spans across segments.

Closes #362
Closes #382

Change-Id: I0f54a40524c36a602ad3804a0da26e80851c92fd
2018-06-01 19:49:56 +00:00
KongQun Yang 6e4820eedc [Ad Insertion] Support one file per Representation per Period
Allow the output file to contain template identifiers like $Number$,
$Time$ etc. Unlike the actual template used in SegmentTemplate, the
identifiers in output will be populated before pushing to the DASH
manifest.

Issue: #384

Change-Id: Ife1caadb6fccd32167fa1bc83fe2afcb2d2ad087
2018-05-31 23:59:21 +00:00
KongQun Yang fe944f0656 Update MpdGenerator to allow inject version for testing
- Add command line flag --test_packager_version to inject packager
  version; packager_main.cc is also updated to use the same flag.
- Also add a MpdGenerator test in packager_test.py.

Change-Id: I9a11a0ee5502ba30a8acc4d44ebbfaabbe0f2f6e
2018-05-30 13:16:33 -07:00
KongQun Yang 317425d92f Fix TS mimetype in DASH
Change-Id: I64235d02206bda9a7c18d396b2889bb28091ab5d
2018-05-21 20:31:40 +00:00
KongQun Yang b58753278c Add end to end tests for packed audio
Issue: #342.

Change-Id: Iaa890450615573da84b8934b2ba3ee3602d6fb74
2018-05-21 20:30:26 +00:00
KongQun Yang 8fc17c3653 Remove 'Package' from py.test names
Change-Id: Ifd4400c797eee3573fe732e017e2a05354d35a29
2018-05-18 20:34:18 +00:00
KongQun Yang 7f7843d241 Clean up output format determination
Requiring output format determined from 'output' to be consistent with
output format determined from 'segment_template'.

Change-Id: I32cbd63fcd6e2a4272dd0db531c1d5b385315445
2018-05-17 17:33:47 +00:00
Aaron Vaage 8f3a45c497 Respect Output Format In Packager Test
Make sure to use the output format when given when setting up our
tests. Not doing so results in text always being set to "vtt" when
the output format is "mp4".

Change-Id: I11c5f861091598a67fc76dc19b1b16a9a773a2e0
2018-05-15 23:35:54 +00:00
Aaron Vaage d3fd4e9c05 Create Test For Ad Cues
Create an end-to-end test for ad cues. This test's final result is not
correct but illustrates the problem we have in the cue insertion and will
be fixed by a later CL.

Change-Id: Ia8b43a53848941be52cf9ade018668e6477e8df2
2018-05-15 21:55:40 +00:00
KongQun Yang bbd5e074ee Add a test outputing DASH and HLS at the same time
Change-Id: I38c0ca9ed206536a4fd84e8c9c074609250013cc
2018-05-14 23:55:30 +00:00
KongQun Yang 2d919b5a31 Fix potential partial DASH segments during live packaging
Packager uses ThreadedIO to write media segments and manifest /
playlists. There was a possibility that media segments write being
delayed and scheduled after updating manifest / playlists.

This CL fixes the race condition.

Also added a note on how segments can be synced to cloud storage to
avoid the race condition during file sync.

Also added a live WebM test.

Fixes #386.

Change-Id: Icf9c38cdec715fa3dc2836eab1511131e129fe41
2018-05-03 09:26:23 -07:00
KongQun Yang a7463f60c1 Enable segment auto removal for live packaging
The number of preserved segments outside live window can be
configured using flag --preserved_segments_outside_live_window,
which is default to 50, i.e. 5 minutes for 6s segment.

Note that the segment removal will be disabled if it is set to 0.

Only HLS live playlist and DASH dynamic MPD are affected by this flag.

- Also add end to end tests.

Fixes #223.

Change-Id: I8a566efebe2f1552c7d9509ab017bade5a4a1c98
2018-05-02 00:18:42 +00:00
KongQun Yang b6f0da246f Add support for FLAC codec
Implemented according to spec:
  https://github.com/xiph/flac/blob/master/doc/isoflac.txt

Closes #345.

Change-Id: If2e277d8eac4baff5965faa0e13c44c334f6184d
2018-05-02 00:16:51 +00:00