Commit Graph

1025 Commits

Author SHA1 Message Date
Aaron Vaage 1e9e5d1f58 Integration Test for WebVtt Pipelines
Connected the WebVtt Pipeline together to test that the parser,
segmenter, and output all work together as expected.

Change-Id: I07138f7b5b1318f84c27c5b607d8df207c57ddb3
2018-01-09 22:14:04 +00:00
KongQun Yang ef2c424876 Implement Representation::GetDurationSeconds
Also added Period::GetAdaptationSets and
AdaptationSet::GetRepresentations.

Instead of implementing GetDurationSeconds in all MpdBuilder classes
(MpdBuilder/Period/Adaptation/Representation) like what we used to do
with GetEarliestTimestamp, the two new functions allows MpdBuilder to
iterate through the Representations to get durations.

Also updates GetEarliestTimestamp functions to use the same iteration
method.

Change-Id: I682b70c07c248c0f6511ec3d9019086f986ee10e
2018-01-09 01:03:05 +00:00
Aaron Vaage 509963d60f Create HLS Text Pipeline
Linked together all the handlers to create the HLS text pipeline.
This change does not include tests for the pipeline.

Bug: 36138902
Change-Id: I32e7a7cefae1e7c74ec6d0bdc6335c0d65e5e79c
2018-01-09 00:03:29 +00:00
Aaron Vaage 4ecb567daf Changed muxer listener to use "default"
Changed muxer listener to use "default" instead of {}
when defining its constructor and destructor in its
header file.

Change-Id: I3c80e808aa624293c3d7720ac6a47b470a663312
2018-01-08 17:52:04 +00:00
Aaron Vaage 37b5f401d1 Support MP4 to Mp4 Text
This change adds the missing code for passing mp4 text through
the pipeline.

This introduces the MuxerFactory to simplify creating muxers.

Change-Id: Ic7be1a2e0ff4b720a01061edb43aded946d05a47
2018-01-08 17:41:49 +00:00
KongQun Yang 5f48cbb0c1 Fix build failures in MacOS 10.12.6 with XCode 9.0
- Update curl to 7.57.0
- Roll clang, which is needed due to MacOS / XCode update
- Fixes and suppress compilation errors due to clang update

Fixes #285

Change-Id: Ibac3288c641861605c3c0500d34d27373e6eecfe
2018-01-05 12:21:41 -08:00
Aaron Vaage c88fe54553 WebVTT Output
This change introduces handlers to output WebVtt text files. There is
only one output but there is a common base to support others.

WebVttOutputHandler which handles all communication with other handlers
and WebVttSegmentedOutputHandler is responsible for listening for events
and choosing when and where to write cues and headers.

Bug: 36138902
Change-Id: I2b13a94262554398e66fee8cf024aa21041ddbab
2018-01-05 00:05:48 +00:00
Aaron Vaage 9e7b5c1ca9 Change CharReader to FileReader
Instead of having multiple char readers, one for strings and
one for files, just have one for files and use memory files
when testing.

Change-Id: Id1a2230046ba540ddf69ca10edb3edc74d2419b6
2018-01-04 14:36:52 -08:00
KongQun Yang db74d6756e Propage CueEvent to MpdNotifier/HlsNotifier
Change-Id: I9828af11a28300d20cc8742251bbe1b4ebfdbad1
2018-01-04 18:25:03 +00:00
Aaron Vaage b0edec8c40 Changed Text Segmenter to output segment info last
To better align with the Chunking Handler, the Text Segmenter
now outputs the SegmentInfo at the end of the segment rather
than the start.

Change-Id: If69ab951947d00779b4b63a52c4b6662bbdc4c0d
2018-01-03 23:40:37 +00:00
KongQun Yang 66f713fd23 Merge SimpleMpdNotifier and DashIopMpdNotifier
Also removed MpdNotifier::AddContentProtectionElement which is not
used anywhere.

Change-Id: I69eb596ec377c601bb3e45bbef30c248c1d03e86
2017-12-22 15:56:14 -08:00
KongQun Yang 82842407cb Add content_protection_in_adaptation_set_=false tests
Also make AdaptationSet::AddAdaptationSetSwitching virtual to make it
easier to mock.

Change-Id: I7f66fa0c4824607f77559a61227aaef6cd153f9f
2017-12-22 15:53:35 -08:00
KongQun Yang 718fce068d Refactor DashIopMpdNotifier Part 2
Move AdaptationSet related functions to the new Period class, which
maps to <Period> element and provides methods to add AdaptationSets.

Change-Id: I0fee290769fbe9a6355cc1b8c86baec8fbc4b4fd
2017-12-22 13:24:38 -08:00
KongQun Yang 99469834e8 Refactor DashIopMpdNotifier Part 1
Clean up DashIopMpdNotifier to make it easier to add Period class.
- Consolidate protected AdaptationSet grouping related logic to the
  internal ProtectedAdaptationSetMap class
- Clean up AdaptationSet creation logic and updating logic

Change-Id: Idd5b39bb89d38e490ca4f561a6b840ccc9f1e40a
2017-12-22 21:20:13 +00:00
Aaron Vaage ab19082c20 WebVtt To MP4 Handler
Implemented a MediaHandler that takes text samples and creates
media samples. The data in each media sample is the MP4 box for
non-overlapping cues.

As per WebVtt in Mp4, all cues must be non-overlapping. This handler
takes care of grouping and dividing cues.

Bug: 36138902

Change-Id: I0c1d27964180c14a22cb200591f70e46e04a651f
2017-12-22 21:08:17 +00:00
KongQun Yang fcc334d652 [Doc] Fix a few problems
- Add --time_shift_buffer_depth to HLS doc.
- Fix a problem in HLS tutorial

Partially address #304, fixes #305

Change-Id: I60bc1eb659a30beb0a6ab73433f4cc99a7deab07
2017-12-21 23:07:59 +00:00
Aaron Vaage c03c447e0b Created WebVtt Segmenter
This change creates the webvtt segmenter which will be used for
segmented text webvtt for HLS. It takes a stream of text samples
and injects segment info messages based on the segment duration.

It is possible for samples to extend between segments, if so the
sample will appear twice (once in each segment).

Change-Id: Iae0134ee61cf269948026086520b6d3f8ce3785b
2017-12-21 21:12:34 +00:00
Aaron Vaage 4dcfe413f5 WebVTT Parser
Took the WebVTT Media Parser and created the WebVTT Parser
that will take in a file and output a stream of cues that
will later be passed to another Media Handler that takes in
cues and chunks them.

Bug: 36138902

Change-Id: Ic77813fe19678e85d500269e69f46917510ab7ec
2017-12-21 21:12:18 +00:00
Michael Stillwell a8f46fcd2e Update playlist_name docs (#307)
Make it clear that shaka packager will create the playlist file.

#305
2017-12-20 10:25:58 -08:00
Haoming Chen feeb7361ff Clean up #ifndef in packager/media. (#310) 2017-12-19 16:56:36 -08:00
Haoming Chen f22320bfa7 Remove non-exist class MediaStream. (#308) 2017-12-18 13:28:19 -08:00
KongQun Yang 55edaf0bea Move SegmentTemplateTest and TimeShiftBufferDepthTest too
Moved from mpd_unittest.cc to representation_unittest.cc.

Also removed unused test data files in packager/mpd/test/data.

Change-Id: I23811390b4b1ffbfac3e69c1ffcf92544099a53d
2017-12-15 20:05:32 +00:00
KongQun Yang cacdab2209 Split mpd_unittest.cc into three files
Move AdaptationSet related tests to adaptation_set_unittest.cc and
Representation related tests to representation_unittest.cc.

Change-Id: I360e9aea3ba08a32e5ba3b0e8b34345f2b3f0b20
2017-12-15 20:05:15 +00:00
KongQun Yang 327537df36 Clean up mpd_unittest
Added gMock matchers for matching xmlNodePtr and its attributes.

Clean up mpd_unittests to make it easier to separate adaptation set
tests and representation tests.

Change-Id: I31816b06e9c76f92d4a82656c659f3b9acae8cb5
2017-12-15 20:04:47 +00:00
KongQun Yang 86d960bea6 Move AdaptationSet, Representation out of mpd_builder.h
Change-Id: I61fffb4d956f189b44c7537ddcf183bbf4129840
2017-12-14 21:28:06 +00:00
Aaron Vaage dedbd8b724 Isolate Job Management
Moved all code dealing with jobs into its own class so that the
packager code does not need to worry about initializing, running,
or stopping jobs.

Change-Id: I3e9ef1f22bd93d671f77d59ad15f23d1239078cf
2017-12-13 18:35:51 +00:00
KongQun Yang 2d025dae2d Include shared_library and mpd_generator in ci and releases
Change-Id: Ie9449cbca711324831cb15e83d0efaf7a43cf9ef
2017-12-12 23:11:01 +00:00
KongQun Yang feb7951f33 Add docker files for testing on various linux distros
Change-Id: Ic05e2dc326a2b70c43132118d78f6d496d8b7e18
2017-12-12 22:51:05 +00:00
Kamesh Devarakonda 4e3a24fb41 Determine chunking point
Change-Id: Ic5690f803c1a2c22b80785aefbc1f1cef2f11c38
2017-12-12 14:24:28 -05:00
Kamesh Devarakonda 7e46ad1021 Capture SCTE35 events
Change-Id: Id06239ed4820104fecbca08a4103a47ecbed5eda
2017-12-12 01:20:52 +00:00
KongQun Yang b3194866a6 [HLS] Add CHANNELS attribute to #EXT-X-MEDIA
As specified in HLS spec (https://tools.ietf.org/html/rfc8216):

All audio EXT-X-MEDIA tags SHOULD have a CHANNELS attribute.  If a
Master Playlist contains two Renditions encoded with the same
codec but a different number of channels, then the CHANNELS
attribute is REQUIRED; otherwise, it is OPTIONAL.

Fixes #299

Change-Id: Ic2308c39b170178b11cb0d94c3a8083c8b5f3353
2017-12-12 01:04:21 +00:00
KongQun Yang d089a36ca9 [Doc] Replace basic_usage.md with basic_usage.rst
Change-Id: I0b1284041000174809365659631985bcaf9db3f7
2017-12-12 00:24:36 +00:00
KongQun Yang 8e96dd3b37 Address gcc build failures in some platforms
Seeing some failures on some platforms when compiled with clang
disabled:
  GYP_DEFINES="clang=0" gclient runhooks

Several changes to make it work:
1. Mark packager code with packager_code=1 in GYP definitions.
2. Disable a few checks in non-packager code, which we do not have
   direct control: dangling-else, deprecated-declarations,
                   unused-function
3. Fix the relevant errors in packager code.
4. Revert HAVE_STROPTS_H in curl config which is not available in
   all linux distributions.

Fixes #286
Fixes #293

Change-Id: I729b41f99403c5ad9487c6cc4a7dc06f6323cef8
2017-12-07 11:45:26 -08:00
KongQun Yang e97c89391a Address flakiness in ProducerConsumerQueueTest
1. Expect the time elapsed to be greater than kTimeout instead of
   approximately equal to kTimeout if there is a "wait";
2. Expect the time elapsed to be smaller than kTimeout instead of
   approximately equal to 0 if there is no "wait".

Also rename a variable in packager.cc to better reflect what it is.

Change-Id: I67975a6263b8dbc1124b78feae0f8e0d112bda50
2017-12-05 22:56:22 +00:00
KongQun Yang 9631ec71de Roll curl to 7.56.1
Fixes #285

Change-Id: I3cda80974e8cc66f11763c41c36d832565077374
2017-12-04 19:28:27 +00:00
KongQun Yang 7097270f13 Roll boringssl to fc9c67599d9bdeb2e0467085133b81a8e28f77a4
Change-Id: I8ff75ede6c5b6a51719163d208c96d6ec0be61bc
2017-12-04 19:27:34 +00:00
KongQun Yang 82e0fd2095 Default multicast interface address to "0.0.0.0" if absent
It aligns with what a lot of other apps are doing, e.g. tsplay.

Also added notes in docker to adjust network setting if wants to use
multicast in docker container.

Change-Id: I666a8979cc041c904f7ff8e3e022f800c0830d5d
2017-12-04 19:21:52 +00:00
KongQun Yang 0fdb0d02aa Log an error when seeing unsupported stream type in MPEG2TS
The error is logged only once to avoid log spamming.

Added definitions for the unsupported stream types for easy look up.

Change-Id: I097e2f05759bc84ef03f264cfabd2fb20da7c711
2017-12-01 14:56:22 -08:00
Kamesh Devarakonda e7540bf303 Dispatch Scte35Events.
This patch also fixes a bug which was introduced in
https://widevine-internal-review.googlesource.com/c/packager/+/38040
where it was inconsistently assumed that AdCueGenerator would be MIMO
while connecting with other handlers and at the same time it was assumed
SISO in the AdCueGenerator class. Now we assume SISO everywhere.

Change-Id: Icd8c40b5ccfe0d98f47f09a60ea1635f74dceef9
2017-12-01 02:23:36 +00:00
KongQun Yang 75e1fc3175 Added architecture diagram
Also remove outdated design.md.

Change-Id: I237b374a2fffe348b1daf5f8d1c37c57b4898fde
2017-12-01 00:29:50 +00:00
KongQun Yang 068e220ac7 Handle additional unused 'mdat' properly
The spec allows having more than one 'mdat' boxes even if it is not
used.

This is happening on some mp4 files in the wild:
http://www.sample-videos.com/.

Fixes #298.

Change-Id: I729cc94bee095560b5c4b3ee8511323f25f7ad5a
2017-12-01 00:28:30 +00:00
l-law 22c758ed06 Fix order of VPS, SPS, PPS in hvcC box of HEVC/MP4 (#300)
Fix order of VPS, SPS, PPS in hvcC box of HEVC/MP4

Fixes #297
2017-11-30 14:34:42 -08:00
Kamesh Devarakonda cc778f60ab Defined CueEvent struct.
Change-Id: If4ba55a7ff403248de9d0411c2ba8553b88c491b
2017-11-28 14:26:32 -05:00
Kamesh Devarakonda c4e7d02b80 Connect AdCueGenerator in the pipeline.
Change-Id: Iaae73661bc57ddecd2cc103dbd34d0c3102ffdaa
2017-11-27 21:09:26 +00:00
Kamesh Devarakonda 154e309f3f Remove unused argument.
Change-Id: I1ee3ca6527ec4ecce7dd57e18eee70a8c6a11eb0
2017-11-21 15:04:36 -05:00
Kamesh Devarakonda dac0328b47 Parse ad_cues flag.
Change-Id: I4f5897dcde20e73a01468b80a04d3f3142bc5e42
2017-11-20 16:39:16 -05:00
Kamesh Devarakonda 622ff64595 Add skeleton AdCueGenerator handler
Change-Id: I53ba982eb76c7f61122e226d295e6989fe425ae5
2017-11-17 00:06:27 +00:00
Kamesh Devarakonda dcf2d2fc0b Created a build target for files in media/public/
This new build target is explicitly listed in the packager/packager.gyp
file.

Change-Id: I49e68ea64ae9a9a30204a269a1af17744f9d25d7
2017-11-16 00:24:17 +00:00
Kamesh Devarakonda 8c877379b6 Defined Scte35Event struct and out of band cue marker flags
Change-Id: I5a4db1a691544cbedf2ccfbaf6f71169a8eed6e7
2017-11-16 00:13:29 +00:00
KongQun Yang 5cf2b17ade Add --hls_key_uri to allow setting fairplay/identity key uri
Change-Id: I52e0f56cd10390faf515170f407ea488a6b3c9fc
2017-11-14 15:49:27 +00:00