Commit Graph

613 Commits

Author SHA1 Message Date
Aaron Vaage d5e820c744 Fix Spelling Mistake in Cue Alignment Handler
"bufferring" -> "buffering"

Change-Id: Ib516460295a52f0e5aa09583432868cf0efb3fe3
2018-04-06 13:16:55 -07:00
Aaron Vaage 113a7d123b Don't Output Empty Text Samples
Only output text samples to webvtt text output when they have a payload.

Change-Id: I958d2b4f087209f540fa32cef002cd2cef37c65c
2018-04-03 18:29:41 +00:00
Aaron Vaage e5fe2a76d2 Created Text Padder Handler To Fill Gaps
Created a media handler to come after parsers that will handle filling
in gaps between text samples. The padder takes a min duration, and if
the samples do not cover the min duration when flushed, one last empty
sample will be injected so that the samples will go up to the min duration.

Change-Id: I88605059664d09279676edac418ff3d4990d7556
2018-04-02 23:11:26 +00:00
Aaron Vaage 901d37a2a7 Correct some spelling mistakes in media_handler.h
Corrected some minor spelling mistakes in media_handler.h.

Change-Id: Ia120e96c9d5c29dba14919a505fb86c3dc1c15b2
2018-03-29 22:36:36 +00:00
Aaron Vaage a11cbf93a7 Handle CueEvent in TextChunker
Updated TextChunker to handle incoming CueEvents. Connecting the
text chunker with the cue alignment handler will happen in a later
CL.

Issue: #362

Change-Id: Ib1fa9f457cf4ec0ce413dadcfa7eed5895ecd628
2018-03-29 11:03:15 -07:00
KongQun Yang 14aab56474 Allow SyncPointQueue to be cancelled
Change-Id: Idbf6ee7a5d9721681811189fca4190e7a0286c83
2018-03-27 19:14:37 +00:00
KongQun Yang db45e0868a Generate full segment after cue point
Also added RETURN_IF_ERROR status macro.

Change-Id: I04643b6252ea5623128f9a16fa744a255c91be17
2018-03-27 19:14:26 +00:00
Aaron Vaage 8f565bf388 Change Text Sample to Use int64_t
Changed Text Sample to use int64_t so that it will use the same
type for time as Media Sample.

Change-Id: I4cfbfdc60c37bb511517993976cd1a459bdf6667
2018-03-26 20:46:58 +00:00
Haoming Chen fac16cbf0a Implement multi DRM support. (Part 2)
- Allow including  Widevine and Playready PSSH boxes for RawKeySource.

Partially addresses issue #245

Change-Id: I8a48d6dbbbc21c76cf2c44384dd70286ccf80363
2018-03-26 20:18:22 +00:00
Aaron Vaage 56caecd073 Rename WebVttSegmenter To TextChunker
Renamed the webvtt segmenter to text chunker as there is nothing
webvtt specific in the code.

Change-Id: I0111c50d545fa168d945b39bc8a78c46c62678be
2018-03-26 17:58:56 +00:00
Aaron Vaage 2dd198a9d2 Move WebVtt Segmenter To Chunking Directory
Move the webvtt segmenter to the chunking directory so that it
can be converted to a general purpose text chunker.

Change-Id: I9ecd7ee39cb73070dab07b64f65ef24af1404813
2018-03-26 17:58:38 +00:00
KongQun Yang 4e57259d2a Integrate CueAlignmentHandler
Also changed ChunkingHandler to be one-one handler.

Issue: #355

Change-Id: Ie98a96bcc0ddded347699c9f333f604826976d11
2018-03-26 17:58:30 +00:00
Aaron Vaage 2b337e29f6 Removed WebVTT Pipeline Tests
Now that we have the end-to-end tests, we no longer need the webvtt pipeline
tests to verify that it is working.

Change-Id: I4ebec34e66eda67c40999d8802b447e2551e1fa6
2018-03-26 17:57:47 +00:00
Aaron Vaage a191c25f57 Clean Up WebVtt Parser
There were a couple of bad practices in WebVtt that needed to
be cleaned-up.

Change-Id: I0fe21e26f11141709d8d855077805fc625e6dad5
2018-03-23 14:19:52 -07:00
Aaron Vaage 3b5b2bccca Implements SyncPointQueue and CueAlignmentHandler
SyncPointQueue manages all cue points and returns aligned cue
points to the callers (CueAlignmentHandlers).

CueAlignmentHandler is responsible for aligning cues from different
streams. It uses SyncPointQueue internally to align / synchronize the
cue points.

Issue: #355

Change-Id: I281fecb46a3ca7172d71e7495bdd07b8efdeb283
2018-03-22 18:21:40 +00:00
Aaron Vaage 833b8165f1 Change Scte35 to use double for time
Change-Id: I2588d3c86d48dc2d9f3692aa0aab28b9de63d109
2018-03-15 18:11:00 -07:00
Aaron Vaage 834ceffd18 Change CueEvent to use time in seconds
Change-Id: I03527db76905f25f0a142940f5fcf73e2a92d42f
2018-03-16 01:10:10 +00:00
KongQun Yang b24a95b1aa Rewrite init segment in MultiSegmentSegmenter::DoFinalize
This is called when reaching end of the file of the media. The duration
of the media is updated.

Fixes #340.

Change-Id: I446f2d341b02125d4a7d8c958bda269b5403cb9c
2018-03-05 14:25:08 -08:00
Haoming Chen 76d68de6c1 Implement multi DRM support. (Part 1)
- Refactoring current clearkey DRM support for Widevine key source.
- Add playready pssh box in Widevine key source encrypted content.

Partially addresses issue #245

Change-Id: Id30ef02ff9f8dc0e2c58d62479e07896452919cc
2018-03-03 01:39:16 +00:00
KongQun Yang 62bfc1c6c1 Recognize m4s as a valid extension for mp4
Closes #331.

Change-Id: Ic4aaa02245b31a0aa4fdfe08212fa3b242cd9262
2018-03-02 23:22:45 +00:00
Aaron Vaage 033fa65105 Removed Old WebVtt Code
Now that we have the new webvtt code in place, we don't need to keep the
old solution.

Change-Id: I20540ba3adf93f535f0ed011acb8e2555653522a
2018-03-02 20:59:23 +00:00
Aaron Vaage 7dd80884c3 WebVtt Ignore Zero Duration Cues
We have an assert that ensures that the end time is greater than
the start time for any cue. However we never checked that cues
had a non-zero duration when parsing them.

We will throw away cues with a duration of zero (and print a
warning message) as they are not spec compliant.

Closes: #335
Change-Id: I404e8f3a5a8d43eff75a2554db3e38e8d340f421
2018-03-02 01:25:41 +00:00
KongQun Yang de716a6544 Skip reading meta box as it may not be well formed
In the video captured by Android's default camera app, meta box is written as
as Box instead of FullBox specified in the spec.

Closes #319.

Change-Id: I526492fdd505d5929c5161cb1ed1503b724de7e9
2018-03-01 20:21:37 +00:00
Aaron Vaage 30b5fdf19f Add Text Codec To Codec String
This change has the hls master playlist add the text codec string
to the list of codecs in a variant.

Bug: 72942756

Change-Id: Ib25bb2064a291d10d7b1d261a4307991df62220c
2018-02-28 19:23:36 +00:00
Aaron Vaage 387fe8e33b Isolated OnMediaSample in Chunker
Change-Id: I3af45543a1da1d1cf0b0741dd732dc71b2c30abe
2018-02-23 23:16:35 +00:00
Aaron Vaage 21acb41e63 Isolate OnScte35Event in ChunkerHandler
Change-Id: I9b34f959c22df498a338eb49c87b67003617ead2
2018-02-23 23:12:11 +00:00
Aaron Vaage d9d2cb3e1f Isolate OnStreamInfo in Chunking Handler
Change-Id: If35b9f545f96237095177438a5f95f964cbbfe3d
2018-02-23 23:12:00 +00:00
Aaron Vaage 7096ca8998 HLS WebVtt End-to-End Tests
Created end-to-end tests for the segmented webvtt text in HLS. The
text stream is longer than both audio and video, so there are more
text segments than video/audio.

Change-Id: Ib860ccd3f2ac0048983fdeebe3847a0b9dcbc160
2018-02-23 16:55:50 +00:00
KongQun Yang 594d1f0de4 Copy over parameter sets in the sample if different to sample entry
When converting from NAL unit stream to byte stream.

The packager should not assume the parameter sets in the sample is the
same as the parameter sets in sample entry (decoder configuration).

Fixes #327.

Change-Id: I7e84d28a296f4b33db0523cca5eabd62f623e852
2018-02-09 15:43:00 -08:00
KongQun Yang 11cbbd86fb Fix fMP4 'cbcs' playback issue in Safari
Looks like Safari does not like v0 tenc box.

- Use v1 tenc box for cbcs and cens protection_scheme as required by
  CENCv3 spec.
- Set crypt_byte_block and skip_byte_block to 0 for full sample
  encryption cbcs and cens.

Fixes #326

Change-Id: I5581cd856fffc4ff104d950f3ca19b9337d57a78
2018-02-08 23:40:46 -08:00
KongQun Yang aa4bb49b2f Fix language option not honored on text streams
Change-Id: I9034b779e9b193c39201a5f2d142ec390927e5f7
2018-02-08 03:48:46 +00:00
Aaron Vaage 48cb55c8d4 Correct WEBVTT Header In Sample Test Files
The old vtt sample data has the header as "WEBVTT FILE" which does
not reflect what the spec says it should be.

This changes it to "WEBVTT" as described in the WebVTT spec.

Change-Id: Ib54396a8b937501a835dafe966bf05b95b0b22c4
2018-02-07 00:07:16 +00:00
Aaron Vaage c7eb2b4a7e Output Empty WebVTT Text Segments
Have the text webvtt pipeline output empty segments rather than skipping
them.

Bug: #324
Change-Id: I175cf22b65b241f6bf566aba2440518f3df95277
2018-02-06 13:32:42 -08:00
Aaron Vaage f35410deb1 Output Empty MP4 WebVtt Cues
When a gap is found in the text stream, the WebVtt to Mp4 converter will
now output the special empty vtt cue.

Change-Id: I8be88c6b7589aa120a2215e1e4b8e98031fe326d
Closes: #324
2018-02-06 16:45:04 +00:00
KongQun Yang 8257eea804 Bug fixes and clean-ups for I-Frame playlists
- Add empty lines between different types of renditions to improve
  readability.
- Group variants with the same audio/text group together, as it is
  where the Adaptation occurs.
- Write master playlist after writing media playlists. This makes
  more sense and it is also necessary to have the bandwidth of
  the last iframe playist segment correctly computed.
- For fMP4, I-Frame segment must include the 'moof' header.
- Fix a problem that hls_iframe_playlist_name is not passed to
  MuxerListenerFactory.

Issue: #287

Change-Id: Icf37c5de1dc29f85ae3f419cbc3264d04ca491a4
2018-02-05 17:36:28 -08:00
Aaron Vaage c991490e82 Prepare WebVTT To MP4 to handle gaps
Adding all the code needed to allow the webvtt to mp4 converter to
send empty vtt cues. This change does everything except writing the
mp4 box.

Bug: #324
Change-Id: I16188e6357632b2ed06f7e9bab7844f093266696
2018-02-06 00:55:32 +00:00
Aaron Vaage 9452a1bb1a Make Segments Their Own Struct
Instead of always tracking segments with an index, this change makes a struct
that will act as the segment and track which segments are in it. Each
segment will store all the samples in the order they were given, it will
avoid any sorting.

Closes: 72867775
Change-Id: Ic5829161510fe8f3320d960c3bc4a276c26ff3be
2018-02-06 00:55:00 +00:00
Aaron Vaage 80ed014cf8 Create Preserve Order Test
Create WebVtt test that shows that the current implementation of the
webvtt segmenter does not respect the input order of cues.

Bug: 72867775
Change-Id: I811b3a93c10650e0cf9290c9c0c1680f562deb30
2018-02-06 00:54:15 +00:00
KongQun Yang b647c6731c Support KeyFrame events in TS and MP4
Issue: #287

Change-Id: I7c50853c7cc61c5fadbb620f44f7574a27dc2b68
2018-02-02 14:05:50 -08:00
KongQun Yang 570a2d1a15 Support 'iframe_playlist_name' stream descriptor
Issue: #287

Change-Id: I484761dfacadce05175d16c4c12454f0ed932579
2018-02-02 22:00:18 +00:00
KongQun Yang 82735be58d Support key frame events in MuxerListener and HlsNotifier
Issue: #287

Change-Id: I33b91b73988fc40e113bd2e627d08f549a7e3dc5
2018-02-02 00:00:53 +00:00
Aaron Vaage 44847a0737 Changed Group Name Fallback
If no hls group name is given, we would default to "audio" but as
we are working to support text, this won't make sense to see
GROUP="audio" on a text stream.

Instead default to "default-audio-group" for audio streams. A specific
text default will be added later.

Bug: #205

Change-Id: Iefc8bfd35708286619d0004348294d98f2c38482
2018-02-01 10:24:58 -08:00
Aaron Vaage 771944a3f5 Output Empty WebVtt Segments
According to the HLS+WebVTT spec, if there is no text in a segment
a webvtt file with no cues can be added to the manifest. By outputting
empty segments it allows the accumulated duration in the Master Playlist
to better represent the duration of the text stream.

Spec Reference: //tools.ietf.org/html/draft-pantos-http-live-streaming-23
  Section 3.5. WebVTT

Bug: #205

Change-Id: I5de01200fd9fa99c57949c773e8ee926b0f6ba8a
2018-02-01 17:02:19 +00:00
KongQun Yang 4194f0746e [DASH][WebM] Fix subsegmentAlignment not set problem
MuxerListener::OnNewSegment was not sent for WebM SingleSegmentSegmenter.

Change-Id: Iecae84e7a7c1a8a4d3f577158fa286b9c9959934
2018-01-31 19:58:15 +00:00
KongQun Yang 610e112d16 Cleans up Status class
- Removed SetError, Swap, Matches functions, which are not used
- Added moveable version of Update function
- Added VLOG(1) warning in Status for non-ok status
- Replaced OS_WIN with _WIN32 as OS_WIN may not have been defined.

Change-Id: Ib6b7aab6e6fee270937b150f1e4bf993e914a568
2018-01-31 03:32:07 +00:00
KongQun Yang 9ecee46658 Fix SAMPLE-AES EC3 encryption
The IV was incorrectly updated across samples.

Created a new cryptor SampleAesEc3Cryptor specially for SAMPLE-AES
EC3 encryption / decryption. The new cryptor uses constant-iv, and
makes sure the IV is reset to the initial value at the beginning
of each audio frame and chained across syncframes within the the
audio frame.

Also added E-AC3 end to end test.

Fixes #279

Change-Id: I0aa60c17836daeef5ba433a05e5ff0906191d9ac
2018-01-24 03:02:49 +00:00
KongQun Yang 1455f43c02 Share chunkers from the same input except for WVM
Updated test files and added end to end tests for AdCues.

Change-Id: I28f57c2f0d15f65fa04f599fae446a5c1373bf47
2018-01-22 17:43:25 -08:00
KongQun Yang c9fb2b4a85 Use start_time from CueEvent in NotifyCueEvent
Instead of using next segment start time, as CueEvent time may not
align with segment start time exactly.

Also remove the incorrect DCHECK in ChunkingHandler when processing
kScte35Event.

Change-Id: I4987740c99c8d0d25c9b99bddc5e557e45d308e0
2018-01-20 01:18:38 +00:00
KongQun Yang 53892c7467 Add end to end test for ac3
Also updated bear-640x360.ts test files.

Change-Id: Iff53800b7b4706fb2f6d94682b9446406a94cd6f
2018-01-18 17:23:20 -08:00
KongQun Yang feabf770bf Sync streams coming to ChunkingHandler
Change-Id: I50d1a64f2ec3c50dfb1b7282241b47f3a55e26ce
2018-01-18 16:23:59 +00:00
KongQun Yang 2ba23e1075 Fix incorrect CHANNELS reporting for AC3/EC3
Fixes #313

Change-Id: I0b8d8081cda8aaa4db2ca4916bfc6721bfa3bd22
2018-01-11 03:50:56 +00:00
KongQun Yang 858fcd8698 Fix incorrect HEVC decoder configuration data
When transmuxing from TS to MP4 if there is emulation prevention
bytes in general_profile_tier_level_data.

Fixes #312

Change-Id: I48857046c1ea4bed4f455d03d060551e3ef4bec8
2018-01-10 23:53:35 +00:00
Aaron Vaage c57da2b42d Create Muxer Listener Factory
Moved muxer listener creation into a factory so that it can be
initialized once and then the factory only needs to be passed
around rather than all the parameters.

Change-Id: Ibf7df66c08debddbe6f6ff8995e7910b9ad17f3f
2018-01-10 18:59:11 +00:00
Aaron Vaage c877af9f3b Use Mock Muxer Listener in WebVttOutput Tests
Use a mock muxer listener so that we can verify that the webvtt
output handlers will write to a manifest correctly.

Change-Id: I0294a998bfaf06a6d8f7d4c287fb014839b38f73
2018-01-10 17:51:39 +00:00
Aaron Vaage 1e9e5d1f58 Integration Test for WebVtt Pipelines
Connected the WebVtt Pipeline together to test that the parser,
segmenter, and output all work together as expected.

Change-Id: I07138f7b5b1318f84c27c5b607d8df207c57ddb3
2018-01-09 22:14:04 +00:00
Aaron Vaage 4ecb567daf Changed muxer listener to use "default"
Changed muxer listener to use "default" instead of {}
when defining its constructor and destructor in its
header file.

Change-Id: I3c80e808aa624293c3d7720ac6a47b470a663312
2018-01-08 17:52:04 +00:00
KongQun Yang 5f48cbb0c1 Fix build failures in MacOS 10.12.6 with XCode 9.0
- Update curl to 7.57.0
- Roll clang, which is needed due to MacOS / XCode update
- Fixes and suppress compilation errors due to clang update

Fixes #285

Change-Id: Ibac3288c641861605c3c0500d34d27373e6eecfe
2018-01-05 12:21:41 -08:00
Aaron Vaage c88fe54553 WebVTT Output
This change introduces handlers to output WebVtt text files. There is
only one output but there is a common base to support others.

WebVttOutputHandler which handles all communication with other handlers
and WebVttSegmentedOutputHandler is responsible for listening for events
and choosing when and where to write cues and headers.

Bug: 36138902
Change-Id: I2b13a94262554398e66fee8cf024aa21041ddbab
2018-01-05 00:05:48 +00:00
Aaron Vaage 9e7b5c1ca9 Change CharReader to FileReader
Instead of having multiple char readers, one for strings and
one for files, just have one for files and use memory files
when testing.

Change-Id: Id1a2230046ba540ddf69ca10edb3edc74d2419b6
2018-01-04 14:36:52 -08:00
KongQun Yang db74d6756e Propage CueEvent to MpdNotifier/HlsNotifier
Change-Id: I9828af11a28300d20cc8742251bbe1b4ebfdbad1
2018-01-04 18:25:03 +00:00
Aaron Vaage b0edec8c40 Changed Text Segmenter to output segment info last
To better align with the Chunking Handler, the Text Segmenter
now outputs the SegmentInfo at the end of the segment rather
than the start.

Change-Id: If69ab951947d00779b4b63a52c4b6662bbdc4c0d
2018-01-03 23:40:37 +00:00
Aaron Vaage ab19082c20 WebVtt To MP4 Handler
Implemented a MediaHandler that takes text samples and creates
media samples. The data in each media sample is the MP4 box for
non-overlapping cues.

As per WebVtt in Mp4, all cues must be non-overlapping. This handler
takes care of grouping and dividing cues.

Bug: 36138902

Change-Id: I0c1d27964180c14a22cb200591f70e46e04a651f
2017-12-22 21:08:17 +00:00
Aaron Vaage c03c447e0b Created WebVtt Segmenter
This change creates the webvtt segmenter which will be used for
segmented text webvtt for HLS. It takes a stream of text samples
and injects segment info messages based on the segment duration.

It is possible for samples to extend between segments, if so the
sample will appear twice (once in each segment).

Change-Id: Iae0134ee61cf269948026086520b6d3f8ce3785b
2017-12-21 21:12:34 +00:00
Aaron Vaage 4dcfe413f5 WebVTT Parser
Took the WebVTT Media Parser and created the WebVTT Parser
that will take in a file and output a stream of cues that
will later be passed to another Media Handler that takes in
cues and chunks them.

Bug: 36138902

Change-Id: Ic77813fe19678e85d500269e69f46917510ab7ec
2017-12-21 21:12:18 +00:00
Haoming Chen feeb7361ff Clean up #ifndef in packager/media. (#310) 2017-12-19 16:56:36 -08:00
Haoming Chen f22320bfa7 Remove non-exist class MediaStream. (#308) 2017-12-18 13:28:19 -08:00
Kamesh Devarakonda 4e3a24fb41 Determine chunking point
Change-Id: Ic5690f803c1a2c22b80785aefbc1f1cef2f11c38
2017-12-12 14:24:28 -05:00
Kamesh Devarakonda 7e46ad1021 Capture SCTE35 events
Change-Id: Id06239ed4820104fecbca08a4103a47ecbed5eda
2017-12-12 01:20:52 +00:00
KongQun Yang 8e96dd3b37 Address gcc build failures in some platforms
Seeing some failures on some platforms when compiled with clang
disabled:
  GYP_DEFINES="clang=0" gclient runhooks

Several changes to make it work:
1. Mark packager code with packager_code=1 in GYP definitions.
2. Disable a few checks in non-packager code, which we do not have
   direct control: dangling-else, deprecated-declarations,
                   unused-function
3. Fix the relevant errors in packager code.
4. Revert HAVE_STROPTS_H in curl config which is not available in
   all linux distributions.

Fixes #286
Fixes #293

Change-Id: I729b41f99403c5ad9487c6cc4a7dc06f6323cef8
2017-12-07 11:45:26 -08:00
KongQun Yang e97c89391a Address flakiness in ProducerConsumerQueueTest
1. Expect the time elapsed to be greater than kTimeout instead of
   approximately equal to kTimeout if there is a "wait";
2. Expect the time elapsed to be smaller than kTimeout instead of
   approximately equal to 0 if there is no "wait".

Also rename a variable in packager.cc to better reflect what it is.

Change-Id: I67975a6263b8dbc1124b78feae0f8e0d112bda50
2017-12-05 22:56:22 +00:00
KongQun Yang 0fdb0d02aa Log an error when seeing unsupported stream type in MPEG2TS
The error is logged only once to avoid log spamming.

Added definitions for the unsupported stream types for easy look up.

Change-Id: I097e2f05759bc84ef03f264cfabd2fb20da7c711
2017-12-01 14:56:22 -08:00
Kamesh Devarakonda e7540bf303 Dispatch Scte35Events.
This patch also fixes a bug which was introduced in
https://widevine-internal-review.googlesource.com/c/packager/+/38040
where it was inconsistently assumed that AdCueGenerator would be MIMO
while connecting with other handlers and at the same time it was assumed
SISO in the AdCueGenerator class. Now we assume SISO everywhere.

Change-Id: Icd8c40b5ccfe0d98f47f09a60ea1635f74dceef9
2017-12-01 02:23:36 +00:00
KongQun Yang 068e220ac7 Handle additional unused 'mdat' properly
The spec allows having more than one 'mdat' boxes even if it is not
used.

This is happening on some mp4 files in the wild:
http://www.sample-videos.com/.

Fixes #298.

Change-Id: I729cc94bee095560b5c4b3ee8511323f25f7ad5a
2017-12-01 00:28:30 +00:00
l-law 22c758ed06 Fix order of VPS, SPS, PPS in hvcC box of HEVC/MP4 (#300)
Fix order of VPS, SPS, PPS in hvcC box of HEVC/MP4

Fixes #297
2017-11-30 14:34:42 -08:00
Kamesh Devarakonda cc778f60ab Defined CueEvent struct.
Change-Id: If4ba55a7ff403248de9d0411c2ba8553b88c491b
2017-11-28 14:26:32 -05:00
Kamesh Devarakonda c4e7d02b80 Connect AdCueGenerator in the pipeline.
Change-Id: Iaae73661bc57ddecd2cc103dbd34d0c3102ffdaa
2017-11-27 21:09:26 +00:00
Kamesh Devarakonda 622ff64595 Add skeleton AdCueGenerator handler
Change-Id: I53ba982eb76c7f61122e226d295e6989fe425ae5
2017-11-17 00:06:27 +00:00
Kamesh Devarakonda dcf2d2fc0b Created a build target for files in media/public/
This new build target is explicitly listed in the packager/packager.gyp
file.

Change-Id: I49e68ea64ae9a9a30204a269a1af17744f9d25d7
2017-11-16 00:24:17 +00:00
Kamesh Devarakonda 8c877379b6 Defined Scte35Event struct and out of band cue marker flags
Change-Id: I5a4db1a691544cbedf2ccfbaf6f71169a8eed6e7
2017-11-16 00:13:29 +00:00
KongQun Yang 61e36d7d21 Support encrypted E-AC3 in TS
Implemented E-AC3 MPEG-2 Stream Encryption Format for HTTP Live
Streaming specified in https://goo.gl/1sgcwY

Issue #279

Change-Id: I36c1a05e3d0529ff810eaf52bdca45414baa93eb
2017-11-13 22:54:07 +00:00
Kamesh Devarakonda 09cdb2bbec Removed unused structs PeriodInfo and MediaEvent.
Change-Id: I0504853ec555fd161df21e5680d26aab50648052
2017-11-09 17:28:31 +00:00
KongQun Yang 0a69779f7c Fix a parsing problem with AC3 in TS
Audio samples per frame for AC3 was not specified correctly earlier.

Also the number of channels is not correct if LFE channel is on.

Issue #165

Change-Id: Ibf20aa4c7aec43c07ec7cd394d631c537cb387dd
2017-11-07 18:47:09 -08:00
KongQun Yang 72df5af150 Support encrypted AC3 in TS
Implemented MPEG-2 Stream Encryption Format for HTTP Live Streaming
specified in https://goo.gl/N7Tvqi.

This change also moved ProgramMapTableWriter creation from TsWriter
to TsSegment.

Issue #165

Change-Id: Ia89dd16a5e6405706dd3d85ec6b6de580f7b13a7
2017-10-30 15:22:22 -07:00
KongQun Yang d7f531fe10 Add AC3 support in TS
EC3 and encrypted AC3 in TS are not handled yet.

Partially address #165

Change-Id: If4839ee7801eac902e64e9c677dd37709ec8e88b
2017-10-30 15:11:43 -07:00
KongQun Yang ad836a5cf1 Rename EsParserAdts to EsParserAudio
Change-Id: I8d9fc265a4cf740970256e1a46b172349f1bf794
2017-10-27 23:58:50 +00:00
KongQun Yang 79a54d7748 Cleanup ProgramMapTableWriter
Change-Id: I08093a0fc35dc15a20f6e72166ed46d3ab94b334
2017-10-27 23:58:32 +00:00
KongQun Yang 2a2493ec2e Refactor EsParserAdts to handle other audio
Rename will happen in next CL

Change-Id: I28df29bb264eea7a6225b0a4a2eb38de7968f9d6
2017-10-25 23:21:35 +00:00
KongQun Yang 867244645a Implement BitWriter
Change-Id: I5b7dfc2bc07227c2fe05f6de8857042bcab944c4
2017-10-25 16:19:12 -07:00
KongQun Yang b5a8185543 Refactor ProgramMapTableWriter
To make it easier to support more codecs.

Change-Id: Ifcde0f3d7d6f74015d723a0b74401a86c8e65a72
2017-10-25 20:25:56 +00:00
KongQun Yang d66d307fc2 Rename fixed_key to raw_key
Change-Id: I1ee9398387b54aa685a5bef5b8704849e62ebae3
2017-10-24 21:52:01 +00:00
KongQun Yang 7c38963c29 [doc] Update raw key tutorial
Change-Id: Ife841975c036cf8042c9775583c409e0daa089c7
2017-10-24 21:51:39 +00:00
KongQun Yang 1bf1ec2445 Add multi-key support in fixed/raw keys
The keys can be specified in --keys option, with the form of
  label=<label>:key_id=<32-digit hex string>:key=<32-digit hex string>,label=...

There can be multiple "label=..." block. The DRM label can be one
of "AUDIO, SD, HD, UHD1, UHD2" or a custom label.

Mark --key and --key_id as deprecated. We can achieve the same result
with --keys=label=:key_id=<key_id>:key=<key>.

Also add a new optional field in stream descriptor: drm_label, which
is used to overwrite the internally generated DRM label for the stream.
If not provided, the DRM label is generated automatically based on
audio/video information, e.g. resolution.

Code changes:
- Merged RawKeyEncryptionParams and RawKeyDecryptionParams into
  RawKeyParams.
- Make FixedKeySource accepts RawKeyParams as input.

Change-Id: Ic8c2f071cc71188e13f14bc6396fc2b3ffa5cac6
2017-10-24 21:04:49 +00:00
KongQun Yang c9cc68ba82 [Playready] Allow unencrypted client cert private key
Change-Id: I3244b428f8e3e32787520d9dd0f015e6d9301fe0
2017-10-18 21:37:36 +00:00
KongQun Yang 0861452932 [Test] Rotate key_ids in PSSH as well for key rotation
Note that fixed key / raw key rotation should not be used in production.

Change-Id: I87fca41ebd00f3cadbf157ff129b8e6a9ca04e7c
2017-10-09 16:44:14 -07:00
KongQun Yang 94a64bef03 Fix WebM parsing crash when the source content contains color elements
Also updates outdated comment in packager.h related to multi-manifests.

Fixes #275

Change-Id: Ia2eb235607860f92a3c19bd791d711209da85c08
2017-10-06 22:48:04 +00:00
KongQun Yang 9a60760815 Change MediaSample::CopyFrom to MediaSample::Clone
Change-Id: Ibc8e6ba149496bb62179230e95aa6803a015b12b
2017-09-29 09:58:39 -07:00
KongQun Yang 92e1e39868 Update MediaSample to avoid redundent copying
Use std::shared_ptr<const uint8_t> with a custom deleter to
represent MediaSample::data_ instead of std::vector<uint8_t>.

MediaSample::data_ can be shared by multiple MediaSamples and it is
immutable. A new data instance must be created if the clients want to
modify the underlying data. The new data instance can be transferred
to MediaSample using provided MediaSample::TransferData function.
This avoids unnecessary data copying.

Change-Id: Ib59785a9e19d0abb3283179b12eb6779ee922f79
2017-09-28 14:01:30 -07:00
Aaron Vaage 6c0f2bebef Use TrickPlay Test Structure As Common Test Structure
Took the test structure for trick play and made a common test
structure that can be used for new tests. The next tests to
use this structure will be the new WebVtt pipeline.

Change-Id: I5faca306b8a2bd16b9f945d1ff5c336aed82bc35
2017-09-27 22:10:08 +00:00
Aaron Vaage 23a2893a35 Define Text Sample Matchers and Test Functions
This change create some matchers and text functions that
will be used by the upcoming webvtt work.

Change-Id: I2d7931d1bdabd761f74e8aee7524d5ecfb7cfdc4
2017-09-27 09:44:56 -07:00
Aaron Vaage 8067fd4818 Made Trick Play A 1:1 Handler
Before the trick play handler would have one input and multiple
outputs. The normal media handler API for adding handlers had to
be ignored when setting-up the trick play handler.

This changes the trick play handler to have one input and one output.
It uses the standard AddHandler method calls. When initialized, the
trick play factor must be provided.

This included a run of 'clang-format' over all edited files.

Change-Id: I7b3cdf0a2472e2f89ab194867c4b984e26f18f24
2017-09-26 08:25:04 -07:00
Aaron Vaage 31bf6dab59 Separate MediaHandlerTestBase From FakeMediaHandler
To allow more tests to use the MediaHandlerTestBase and get access to the
helpful creation methods, this change moves the use of FakeMediaHandler
and setting up the graph to a sub class.

Change-Id: I90dd151e3c96d8fbe4bd02a1d6b11e66a279d95b
2017-09-25 16:41:06 -07:00
Aaron Vaage b5800fad77 Created Mock Input and Output Media Handlers
Created media handlers that make testing a handler easier by providing
methods to send messages from upstream and check messages downstream.

Change-Id: I7b3ba3a51bcf0d73bcaacd46b2dcbc16f87d8694
2017-09-20 15:22:12 -07:00
KongQun Yang 327cb8df88 Update WebM error message when segment is too big
To make it clearer.

#271

Change-Id: I6abfe94b417be293320516cfa654d909412e36c0
2017-09-19 22:30:39 +00:00
Aaron Vaage 16eff80497 Made StreamData Have Const Pointers
In prep for changes to Trick Play, we want to make all messages
copy on write so that if the same message is sent to multiple
handlers, it is not possible for one handler to change the data
another handler is using.

Change-Id: I554166ca11c532412e4dfced5603972ca24dc2bb
2017-09-19 21:06:29 +00:00
KongQun Yang 9c861d03f7 Report an error when disk is full
Previously packaging completes successfully without any error
or warning.

With the fix, an error will be reported if write fails. It may
appear as "Cannot close file error" as we use threaded IO, which
could delay the error reporting until Close() call, so the user
of the File API needs to make sure Close() returns successfully.

Also fixed a deadlock in threaded_io_file if internal_file->Write
fails.

Fixes #160

Change-Id: I17f945150fb4021d2dcdbe784e557673f53ca583
2017-09-15 22:22:12 +00:00
Aaron Vaage 28385c0fc0 Change Text Sample set_style to AppendStyle
To ensure that the style data is formatted correctly, changing the
set_style method to AppendStyle so that joining values will be
done in controlled setting.

Change-Id: Ia88e2d260b11339e6777ff018813e8aeb802a501
2017-09-14 18:37:42 +00:00
Aaron Vaage a9d957e7f0 Move Muxer Creation To Own Method
Moved all the code to do with creating a muxer to its own function to
help isolate it.

Change-Id: Ib259da7aaa41d325632d5f9bb3d54ae9df8d0e31
2017-09-11 12:23:07 -07:00
Aaron Vaage 7c5508555c Made Verify Functions Return Status
There are multiple functions we use to verify input from the user
and must of them would return a boolean expect for the top level
function which would return a Status. Each function would log a
message but to the user the error was not clear. This change makes
each verify function return a status so that the actual problem
will be surfaced easier.

Change-Id: I12ab43f8ca3a4d379b4309336644a70eb8cbbbe0
2017-09-06 22:22:55 +00:00
Aaron Vaage ec83d8a73a Implement MsToWebVTTTimestamp
Implemented the other end of the webvtt timestamp parsing as it will
be needed when we write out WebVtt files. Also changed the name of
the parse function to be more inline with the writing function.

Change-Id: I1f36ddbbf80028732b4cb1b15e871dec17767f63
2017-08-30 18:30:43 +00:00
Ivan L. Isaev 1706b0bc3a multi-manifest (e.g. DASH + HLS) output support
Fixes #262
2017-08-29 13:42:33 -07:00
KongQun Yang 4e82ab13bd Consolidate Mp4OutputParams into MuxerOptions
Change-Id: I533e40ef9eb84453c15cbdb7ef42fc85bef5125d
2017-08-18 11:57:34 -07:00
KongQun Yang 9eaf1dcae0 Consolidate ChunkingParams and ChunkingOptions
Change-Id: Ibbb85981d2b424432b61ca693e28334ef00d475e
2017-08-18 11:55:45 -07:00
KongQun Yang c7c201b00c Consolidate EncryptionParam and EncryptionOptions
Change-Id: I8a5b5c329d4731341a3184d50bed061aa00cdeda
2017-08-18 17:37:45 +00:00
Aaron Vaage e605203fa7 Added Origin Handler
Created a handler called OriginHandler that can be used by the Job class
(previously called RemuxJob). Origin handlers represent the start (or
origin) of a pipeline (chain of handlers).

Change-Id: Ibd748ae0a932b6e0ebb879ea292fcb83c548214b
2017-08-18 09:20:10 -07:00
David Cavar 34c5e011a5 Fairplay key system support
Support generation of fairplay key system tag: "com.apple.streamingkeydelivery" when --pssh 
includes fairplay key system id:

// Unofficial fairplay system id extracted from
// https://forums.developer.apple.com/thread/6185.
const uint8_t kFairplaySystemId[] = {0x29, 0x70, 0x1F, 0xE4, 0x3C, 0xC7,
                                     0x4A, 0x34, 0x8C, 0x5B, 0xAE, 0x90,
                                     0xC7, 0x43, 0x9A, 0x47};

Closes #258
2017-08-08 10:31:40 -07:00
KongQun Yang b451d3a7ca Consolidate various playlist_type types
Change-Id: If6fafc0867e7e1f3cd2ed612617e91c7cb144f20
2017-07-21 20:43:14 +00:00
KongQun Yang 61c58724ec Remove size parameter in OnMediaEnd
Derive file size from media ranges instead.

Also fix subsegment range reporting for WebM (internal variable,
no output affected).

Change-Id: I5f8152dff4c2cd5fbae5550992b86a669e278f7b
2017-07-21 20:42:44 +00:00
Yohann Connell 8d2b87c773 Adding the ability to provide the group_id in Widevine Cenc requests
Change-Id: I4903cc9422433f20b76d084578d8fc06d7ed45a8
2017-07-20 09:46:25 -07:00
KongQun Yang eb98a87f7d Move packager/media/file to packager/file
Change-Id: Idc0d75a7ceab7c8ce460ef896fdefdec088124bb
2017-07-17 12:08:28 -07:00
Rintaro Kuroiwa d096f4b485 HLS byte range
- Use byte range format if segment_template is not set in MediaInfo.
- Change HlsNotifier::NotifyNewSegment() to take start byte offset.
- Change HlsNotifyMuxerListener to cache the segment information if
  MuxerOptions::segment_template is set.

Change-Id: Ida1b8b9198acc5c43e3a5d9a641c78c81bc9a6ac
2017-07-12 17:19:56 -07:00
Rintaro Kuroiwa 0dfdace280 Change MuxerListener::OnMediaEnd to take ranges
- Changed OnMediaEnd() to take (sub)segment ranges.
- Define a structure to pass the ranges.
- This requires muxers to "cache" the segment ranges but in most cases,
  they already do it, e.g. 'sidx' for mp4.

Change-Id: I16b974c1f0f54ca658cc6dbe605efff84a3f52eb
2017-07-11 15:59:16 -07:00
KongQun Yang 2db1867951 Write manifests atomically
In the original manifest updating process, the file was truncated first
before was written with new contents. There is a small chance that
the web server may read back empty file or incomplete file.

The new code makes the update operation atomic (by writing to a temporary
file first then replace the old file with the temporary file).

Fixes #186

Change-Id: I2fd564cb12b922b032c0e9f70d2132a5b12ff098
2017-07-07 11:05:19 -07:00
KongQun Yang 11210b400a Workaround Edge problem with non-zero clear lead
Somehow Edge requires sinf box to be put after avcC box:
https://developer.microsoft.com/en-us/microsoft-edge/platform/issues/12658991/

b/37913785

Change-Id: I201c96768328b3910001dba49a9ddda7f06f3c20
2017-07-06 22:48:09 +00:00
KongQun Yang 627af1558e Use VLOG(1) instead for unsupported stream type
Change-Id: I8ec3027edefa07f32bba3d3b39d0804d706d3798
2017-07-05 23:09:58 +00:00
KongQun Yang 3bad4ca700 Move status.h under packager/
Change-Id: I4a7f34c575c1c8b9a9a275f3889cf8641cfdd471
2017-06-30 11:35:18 -07:00
KongQun Yang e56d1faaf0 Make HLS with key rotation work
- Remove the TS key rotation not supported check as there is nothing
  to be done for key rotation in TS for SAMPLE-AES.
- Fix IV updated problem in new segments even if crypto period does
  not change.
- Avoid duplicate EXT-X-KEY tags if it does not change.
- Make EXT-X-DISCONTINUITY-SEQUENCE match with number of removed
  EXT-X-DISCONTINUITY.
- Added end to end test for HLS with key rotation.

Change-Id: I73cb82e9f5575fcdf63ee643228efe78e6766302
2017-06-29 21:03:36 +00:00
KongQun Yang b4aa34b803 Support live and event HLS playlists
Closes #114

Change-Id: I6440383c3dc022fba5a9926839f79b9ed8a51f55
2017-06-24 01:09:32 +00:00
Rintaro Kuroiwa e2401f02ec Make NaluReader skip encrypted portion
Change-Id: Ibb47a1e62cd8ac3057c8f1512a88280991e48b62
2017-06-22 22:09:59 +00:00
KongQun Yang 03889e6465 Move hex string out of packager.h
Also added validation failure tests in packager_test.py.

Change-Id: I6697a3138d57889110250404205536dd5cd53cd0
2017-06-20 22:33:47 +00:00
Haoming Chen 78f4fd9d3f Fix an issue in NalUnitToByteStreamConverter.
- This fixes the issue when one NAL unit has multiple
  subsamples, which is allowed in spec and could happen
  when the clear_bytes exceeds 2^16 bytes.
- The new algorithm updates the subsample entries in
  following steps:
  1) When NaluReader parses a new NAL unit, the
  function splits subsamples and align them with the
  NAL unit. It also finds the range of subsamples
  for that NAL unit.
  2) Drop or update subsamples of that NAL unit,
  based on the type. Other inserted bytes are treated
  as a new all-clear subsample.
  3) Merge those subsamples if possible.
- This addresses #237.

Change-Id: I2cab47cd00d90e29ec52b90d4fde057c632193ac
2017-06-19 20:19:11 +00:00
Aaron Vaage 26d39db0e7 Moved Timestamp Parsing For Easier Testing
Moved all the timestamp parsing code into its own class with its
own unit tests. This is to make testing easier.

Change-Id: I624472baba51dfa1254a9f4bae55ebb79e310855
2017-06-16 18:26:16 +00:00
KongQun Yang 1aeedd102e Use free-form strings for stream labels (track types)
Change-Id: I38489acbdfaf4bb491635fdc7f6b0cab77a53574
2017-06-13 15:04:00 -07:00
KongQun Yang 05a5a41969 Support generation of libpackager.so (shared_library)
Shared libpackager can be built by setting libpackager_type to
shared_library, e.g.

GYP_DEFINES='libpackager_type=shared_library' gclient runhooks
ninja -C out/Debug

will generate libpackager.so in out/Debug/lib directory.

Here is a few other changes to make shared_library builds and
tests pass:

- Add several test parameters to packager.h, which is needed for
  testing.
- Create a protoc.gypi from build/protoc.gypi but depending on
  protobuf_full_do_not_use instead of protobuf_lite, since we need
  protobuf_full_do_not_use for text parsing and generation of media
  info proto. Somehow shared_library build does not allow mixed use
  of protobuf_full_do_not_use and protobuf_lite.
- Remove the use of LazyInstance in version/version.cc and use static
  variable directly. This is because LazyInstance needs AtExitManager
  which may not be easy to setup when calling GetVersion.
- Allow skipping testPackageWvmInputWithoutStrippingParameterSetNalus
  with flag --shared_library, which is needed as shared_library build
  does not support --strip_parameter_set_nalus flag yet.

Fixes #227

Change-Id: Iff05a50baa28134faa7218664c96114cb9e70329
2017-06-13 20:42:32 +00:00
KongQun Yang 2816a7262e Integrate packager top level library
Change-Id: Ic2e2ef79d2f5db5aab6565835aaaf6cb1c7e44bb
2017-06-13 13:40:26 -07:00
Aaron Vaage 2d2825290c Added TextSample
Added a text sample type that can be passed between media handlers.
This will allow text samples to safely be moved between media
handlers before converting to their final format.

Bug: 36138902
Change-Id: Ic4946f774a7d37c43066b9ea46596d5c5f3c05a8
2017-06-09 16:03:34 -07:00
KongQun Yang a251208a75 Report an error on encrypted stream without decryption
Change-Id: I784af57ca79aeb8a12245bd130f413ea6e3d04f1
2017-06-07 21:49:46 +00:00
KongQun Yang 5ad40df810 Fix misleading error messages in WebM output
Issue #239

Change-Id: I4e6e6aafbeb0e9613769e05bd81353409576876d
2017-05-30 16:29:47 -07:00
KongQun Yang 3443048e80 Fix WebM timestamp overflow if longer than 2.5 hours
Fixes #233

Change-Id: I3431f8e68bfb1de222b8ab967c9cac6a821b4f53
2017-05-17 17:00:44 -07:00
Haoming Chen 27abb208aa Remove the trick_play_factor in media_info.proto.
- Use the playback_rate instead.

Change-Id: I341c39de1c28aacebab5b31aa013bcbbcc2fa9d0
2017-05-17 22:10:49 +00:00
Haoming Chen da8877d8a2 Rename trick play rate to trick play factor.
- Add a "tpf" abbreviation in stream descriptor for
  trick_play_factor.

Change-Id: Ia09313bb9778456b53bdb0a8cc46d2ba8caa2ac2
2017-05-17 21:51:34 +00:00
Kongqun Yang 5fc6e540f3 Include protection_scheme in Widevine CENC request
Widevine license server added support for protection_schemes recently;
If CENC request includes protection_scheme in the CENC request, server
will return with a PSSH with protection_scheme filled in.

Change-Id: I6dcac498b5e039503d6ac0f6e057737f7c53efaf
2017-05-16 01:18:39 +00:00
KongQun Yang d9e7e2f1d0 Generate two sample entries only if there is clear lead
This addresses playready broken with clear lead = 0.

clear lead != 0 is still broken with playready on Edge. We think it is
likely an Edge bug, which does not support multiple sample entry boxes,
thus does not support clear lead.

b/37913785

Change-Id: I7adb77a913dccf669153b03b31b4a1e1c98d1cb0
2017-05-15 14:13:52 -07:00
Haoming Chen 4ba5bec660 Integrate trick play.
- Add trick play adaptation in mpd. This addresses #178.

Change-Id: Id6cc08c788eda074c261d0274768051837b84541
2017-05-09 13:33:11 -07:00
KongQun Yang 1e2da22c88 Add support for WebM colour element
Change-Id: I359cde97171118d3b928a9dd6650d11fade8f4a9
2017-05-02 20:22:17 +00:00
KongQun Yang 84ff940e22 Add init_segment name in --help
Closes #228

Change-Id: I596ae7519632f269e8d3e9cfcf0e96564b248d01
2017-05-02 10:25:31 -07:00
Kongqun Yang 4a47268f3d Fix avc3 codec string problem
- Added wvm decryption and demuxing test
- Added avc3 test (not strip parameter set nal units)

Change-Id: I28ff4d7f9ef539eafe45cfadfc0912cd11c72b05
2017-04-28 19:24:11 +00:00
KongQun Yang 59f393779c Support v1 vp9 in iso-bmff
- Implemented according to v1.0 spec @ https://www.webmproject.org/vp9/mp4/
- v0 is no longer supported

Change-Id: I189c813d788400beda797eea7da943a83dfa7d79
2017-04-27 19:37:14 +00:00
Kongqun Yang fb0790e1d5 Refactor KeySource::FetchKeys to use EME init data types
Change-Id: I517ea992a8868d7382988a8837f4f05df9f974d7
2017-04-20 20:21:29 +00:00
Kongqun Yang 75a4b55b34 Fix AAC-HE not correctly signaled in codec string
- Also fixed a bug with dts audio in esds.

Closes #225

Change-Id: I50ca731ac81bedfec37b0e2d577c51d643b7a839
2017-04-13 23:40:23 +00:00
Zajcev Evgeny a24444cc1b [fix] kAdtsNumChannelsTable: Standard specifies 3 channels (#226)
CHANNEL_LAYOUT_SURROUND = Front L + Front R + Front C
2017-04-11 10:23:32 -07:00
Rintaro Kuroiwa f3e91301c2 Widevine HLS new key format media event side
- This change has the muxer listener side changes.
- Also changed SetContentProtectionFields() to take vector for
  default_key_id so that MuxerListener subclasses don't need to convert
  it to string.

Change-Id: I8832848a0c7e34b84e2b4e6baeba364861900ff1
2017-04-05 11:12:57 -07:00
Kongqun Yang 437d48f041 Fix Windows 64 break (#222)
Change-Id: I88bfdee9020a6abb96d6d38397263c08773501df
2017-03-31 14:37:02 -07:00
Kongqun Yang d007ddf20b [MP4] Make outputs CMAF compatible
- Added flag --mp4_include_pssh_in_stream, default to true. If it is
  set to false, the encrypted mp4 stream will not include pssh.
- Align BytesOfProtectedData to multiple of 16 bytes for cenc.
- Set TrackHeader flag to kTrackEnabled | kTrackInMovie |
                          kTrackInPreview
- Move mvex to after trak, required by HLS
- Add cmfc/cmfs compatible brands except for avc3/hev1, where CMAF
  requires single initialization switching set which is not supported.
- Set duration to 0 in tkhd, mdhd, mvhd.

Also updated major_brand and compatible brands:
- Set major_brand to isom (iso-bmff media file format) and made dash
  a compatible brand
- Replaced compatible brand iso6 with iso8 since we use sthd for text
  tracks

Fixes b/36278260

Change-Id: I3cc5dd5aa1621714d517fe02fe3841d19a1a07f6
2017-03-30 15:33:39 -07:00
Kongqun Yang 5d71d8cc9f Move demuxer.cc/h to packager/media/demuxer
- Also fix the misleading error if an invalid input is provided
  The command line app now correctly reports that the file is not
  found instead of stream not available.

Change-Id: I8cfccb62d7a50de57766666f126cf061e3df5ded
2017-03-30 14:35:34 -07:00
Kongqun Yang 1af8f04e40 Rename webm_subsample_encryption flag to vp9_subsample_encryption
This is what webm_subsample_encryption flag actually does.

- Fixes the flag not working problem introduced by previous CLs.
- Also mark flag webm_subsample_encryption as deprecated.

Closes #220

Change-Id: I03ff843786572a91c01b8bd346a3ce50b129118f
2017-03-30 20:36:43 +00:00
Rintaro Kuroiwa 92280bf214 Full vtt in mp4 path
Change-Id: I7f49dbfed188ff7a451b0b4fbd6947590e8935d1
2017-03-29 18:52:25 +00:00
Kongqun Yang ec904f3f79 Add a new test to exercise emulation prevention after encryption
Also added a few more command line flags for end to end test:
- no-remove_temp_files_after_test: do not remove test artifacts after
  test
- encryption_key, encryption_iv: allow injecting encryption key/iv from
  command line

Change-Id: I62084790e10fe6a385b90cb96d9515d8436b2a49
2017-03-28 13:18:33 -07:00
Haoming Chen 6347b29ad7 Fix Windows build for trick play handler. (#221) 2017-03-27 23:04:29 -07:00
Haoming Chen cc04698460 Implement trick play handler.
Change-Id: I3d288dc7c4b77fd3aca1bad2ce96ad733384bcc8
2017-03-24 18:34:42 -07:00
Rintaro Kuroiwa 0a3ef49c11 Change {luma,chrome}_weight_flag to arrays
- The flags were overwritten in a loop and therefore only stored the
  last value of the flag.
- Change the fields into arrays so that the flags are not overwritten.

Change-Id: I59762d7c84bc3d81bf3b0b5e85ffe689260d970a
2017-03-24 21:14:18 +00:00
Kongqun Yang bc4c511ad4 Fix Windows build break (#217)
Change-Id: Id0a00c01077adb97f294c68357b629441321e6c7
2017-03-24 10:21:05 -07:00
Kongqun Yang 277c5b23c0 Allow generating avc3 / hev1 from TS and Wvm files
Added a flag --strip_parameter_set_nalus. When enabled, parameter
set NAL units, SPS/PPS for H264 and SPS/PPS/VPS for H265, are stripped
from frames when converting NAL byte stream (AnnexB stream) to NAL
unit stream, which generates avc1/hvc1; otherwise they are not
stripped, and avc3/hev1 is generated.

Parameter set NAL units should not be stripped if they are varying
in the frames otherwise the frames may fail to be decoded.

The flag is enabled by default as we don't usually see varying
SPS/PPS/VPS and it is more space efficient with them stripped.

Set --strip_parameter_set_nalus=false to disable the flag if there
are varying SPS/PPS/VPS in the frames. This addresses #206.

Change-Id: I34bde6f33069f9f77d51a510b39f58a0f0e141aa
2017-03-23 13:53:14 -07:00
KongQun Yang 3d7635d5de Integrate EncryptionHandler
Change-Id: I639e5707994a6601ee02078cd6f77cb38b481585
2017-03-23 18:25:59 +00:00
KongQun Yang 16615095f0 Update NalUnitToByteStreamConvert to support escaping encrypted NAL only
- Also fixed a bug if nalu_length_size != 4.

Change-Id: Ia8c67c4b1ff08a6b64fa1e309b5ecd8f5c8ea421
2017-03-23 18:21:24 +00:00
sylt d2c9a88ec5 Fix possible integer overflow in WebM duration
The issue could lead to the MPD attribute mediaPresentationDuration being
wrongly generated for WebM streams with a duration longer than INT32_MAX.

The root-cause was that StreamInfo::set_duration() accepted an int instead of a
uint64_t. This seems like a pure typo, since StreamInfo already uses a uint64_t
internally for representing duration.
2017-03-23 09:38:51 -07:00
Kongqun Yang d323d4f091 Fix break in Windows (#213) 2017-03-22 09:57:18 -07:00
KongQun Yang b09c8f6521 Update EncryptionHandler to support key rotation and SampleAes
This CL also removes EncryptionConfig stream data type and merges it
into StreamInfo/SegmentInfo instead.

Change-Id: Idb70ce503e61d3c951225cc78b6b15c084e16dcd
2017-03-21 23:50:53 +00:00
Kongqun Yang b891e0271e Make CENC behavior of VP8/9 in ISO-BMFF the same as in WebM
- Disable subsample encryption for VP8 in ISO-BMFF
- Apply block alignment to all subsamples for VP9 in ISO-BMFF,
  instead of just superframes.

Change-Id: I8dd31cc16e87abc4d538330eaff9acb0509497df
2017-03-21 23:12:57 +00:00
Kongqun Yang 0cb0f79c33 Fix build break in Windows (#211)
Change-Id: I0ed7a712fd02ea4dddd9c20541c2df653dc456cb
2017-03-17 12:45:28 -07:00
Kongqun Yang 0c3fb49eeb [WebM] Fix output truncated if using the same file name for I/O
Open output file in DoFinalize() to ensure that the input file has been
closed already.

Fixes #210

Change-Id: I935941b31c667e49be030c8da9f953d8387c7a9d
2017-03-17 01:07:49 +00:00
Rintaro Kuroiwa 15fd745fa7 Remove redundant flags in H264SliceHeader
- luma_weight_l0_flag, luma_weight_l1_flag,
  chroma_weight_l0_flag, chroma_weight_l1_flag were there but not set.
- The user should use the values in pred_weight_table_l0 and
  pred_weight_table_l1 instead which are set.

Change-Id: Ic9c44fb113717346938a339faf074daa32d4c2d2
2017-03-17 00:05:36 +00:00
Kongqun Yang 8806f78655 Use size_t as the type of stream_index
Change-Id: Ica8ffeccc45c1adf5def6fe9b2edde5708d6b432
2017-03-16 20:54:48 +00:00
Kongqun Yang 160700b452 Integrate ChunkingHandler
Also moved MediaHandler output validation to Initialize instead.

This CL also addresses #122 with consistent chunking.

Change-Id: I60c0da6d1b33421d7828bcb827d18899e71884ce
2017-03-16 20:54:36 +00:00
Rintaro Kuroiwa a3ce51785a Make WebVttMediaParser use WebVttSampleConverter
- WebVttMediaParser uses WebVttSampleConverter to generate non
  overlapping media samples.
- The media samples contains ISO BMFF boxes.
- Add kCodecWebVtt to signal that the media is webvtt and
  the samples will be in ISO BMFF boxes.

Change-Id: I639902cdba7b04af75428bc20622e26b8203cfb2
2017-03-15 20:44:36 +00:00
Joey Parrish 924d6d4693 Add HLS audio language support
Before this, HLS output did not contain language information.
Now, media playlists are properly tagged with a language in the
master playlist.

b/36134267

Change-Id: I172e946dbedd096a44cb2f917b007cc004756228
2017-03-10 16:14:14 -08:00
Kongqun Yang c6cbd73465 Rename FlushStream to OnFlushRequest; add FlushOutput
Change-Id: I2b37030b0c35aa34e9c72e43825f157924034227
2017-03-08 11:21:19 -08:00
KongQun Yang 8f2cd6da91 Implements Demuxer and Muxer media handlers
- Also sets up the packaging and verify it works.

Some of the changes are temporary to get the integration going.

Change-Id: I0cf6c379d185e157808acabb9ef58ff93d4a39ae
2017-03-08 19:11:52 +00:00
Kongqun Yang 24ba12c18e Remove escape parameter in NalUnitToByteStreamConverter
The spec actually requires all NAL units to be escaped, so there
is no need to escape it again.

NAL byte stream is one demarcation method; NAL unit stream is another
demarcation method. Regardless of the demarcation method used, the
requirement for NAL unit is the same. See the last paragraph of
7.4.1 NAL unit semantics on the requirement on emulation prevention bytes.

Change-Id: Icc63fcc5cf965632e331f5af5f673164c7c1663a
2017-03-02 23:01:53 +00:00
Kongqun Yang c223bc9144 Keep slice data partition NALs in clear
CENCv3 recommends only encrypting video data in slice NALs. Slice
data partition NALs should not be encrypted.

In the code, differentiate is_vcl and is_video_slice. They are the
same for H265; for H264, vcl NALs include slice data partition NALs
but video_slice NALs do not.

Change-Id: I91f4bdd76d25f0eac50e39aed350ebce3f667121
2017-02-28 18:10:30 -08:00
Rintaro Kuroiwa 5c07ff6b1c Move and rename webvtt fragmenter to webvtt sample converter
- The class should be used to generate samples from the demuxer.

Change-Id: If7f245e5809de8773c0b1cf08673ae62dfeffe09
2017-02-14 16:33:18 -08:00
Rintaro Kuroiwa 593f513c83 Fix windows build
Change-Id: Ie23ebd9f18bf2151e52550bff62e2b043ac65ac2
2017-02-14 11:21:52 -08:00
Rintaro Kuroiwa 1879bca4bb Fragmenter for WebVTT
- Logic for splitting up WebVTT cues into MP4 samples.
- For any cue intervals that overlap, a new sample must be created.

Change-Id: Icf7478f42d8c5790bf6f404d512c1251dd34c405
2017-02-13 13:33:36 -08:00
Kongqun Yang 50787c605d Fix build break in win64
Also updated appveyor to build x64 first.

Change-Id: I1970e39bad559b3105ea31fb82375406d95f4a8b
2017-02-10 22:57:35 -08:00
Kongqun Yang 5f6e4a1dc1 Using std::shared_ptr instead of std::unique_ptr in stream data entry
Change-Id: Ib8c6f97c2329ba5b9431c301d85c1d61f89fc3d8
2017-02-11 02:41:49 +00:00
Kongqun Yang 9990524f98 Implement ChunkingHandler
This handler is a multi-in multi-out handler. If more than one input is
provided, there should be one and only one video stream; also, all inputs
should come from the same thread and are synchronized.
There can be multiple chunking handler running in different threads or even
different processes, we use the "consistent chunking algorithm" to make sure
the chunks in different streams are aligned without explicit communcating
with each other - which is not efficient and often difficult.

Consistent Chunking Algorithm:
 1. Find the consistent chunkable boundary
 Let the timestamps for video frames be (t1, t2, t3, ...). Then a
 consistent chunkable boundary is simply the first chunkable boundary after
 (tk / N) != (tk-1 / N), where '/' denotes integer division, and N is the
 intended chunk duration.
 2. Chunk only at the consistent chunkable boundary

This algorithm will make sure the chunks from different video streams are
aligned if they have aligned GoPs. However, this algorithm will only work
for video streams. To be able to chunk non video streams at similar
positions as video streams, ChunkingHandler is designed to accept one video
input and multiple non video inputs, the non video inputs are chunked when
the video input is chunked. If the inputs are synchronized - which is true
if the inputs come from the same demuxer, the video and non video chunks
are aligned.

Change-Id: Id3bad51ab14f311efdb8713b6cd36d36cf9e4639
2017-02-11 02:41:34 +00:00
Kongqun Yang 7a90ee70ab Implement EncryptionHandler
Change-Id: Iabedf3b02057d6124d5393ae9618490e5595ad89
2017-02-11 01:21:28 +00:00
Kongqun Yang 6235352bf4 Fix mpd duration not set in static live manifest
Closes #201

Change-Id: Ie9ab58ec58e1b135931a6576fb46145454b0f049
2017-02-11 00:52:32 +00:00
Kongqun Yang d592217761 Add a warning message when seeing unsupported stream type
Issue #202

Change-Id: If16fe7ccec5dfd869070dfbbb711240a89b2295b
2017-02-09 12:38:49 -08:00
Kongqun Yang 64c6821dc8 MediaHandler implementation
MediaHandler is the basic processing unit in shaka-packager 2.0.

Change-Id: I5f8d744e39adfff5cfebe0c50a29ead4d717cca4
2017-01-26 03:34:18 +00:00
Kongqun Yang 97fbaaa567 Replace scoped_refptr with std::shared_ptr
Change-Id: Ie8ea66e78c42c7387a97bb1abc331c636be11aa4
2017-01-26 02:20:47 +00:00
Yohann Connell 24b0e3031a Adding PlayReady support to Shaka packager
PlayReady Header Object Spec (PSSH Data VOD)
http://download.microsoft.com/download/2/3/8/238F67D9-1B8B-48D3-AB83-9C00112268B2/PlayReady%20Header%20Object%202015-08-13-FINAL-CL.PDF

PlayReady Live Streaming Spec (Consult Appendix for package acquisition information)
http://download.microsoft.com/download/2/D/D/2DD6B4E8-CABF-4DE9-8F61-895BE8F1ED33/ProtectingLiveTVServicesWithPlayReady_March2015.pdf

Issue #128
Change-Id: Ic5c0d6c642ceba3bdf9e8e413934f0fbcba242e6
2017-01-24 13:36:47 -08:00
Kongqun Yang 80c54a533a Update cmd flags to support static mpd with live profile
- Deprecated command line flags --profile and --single_segment.
  'segment_template' in stream descriptors implies live profile
  and non-single segment.
- Added flag --generate_static_mpd_for_live_profile to generate
  static mpd for live profile; if not set, dynamic mpd will be
  generated.

Close #142

Change-Id: I78879297ed118f0f246c4753a16ad125bd6b5e4f
2017-01-13 20:00:10 +00:00
Kongqun Yang 5aaae303e8 Part one of supporting live profile with static mpd
- Added two new fields in MpdOptions: dash_profile and mpd_type
- Updated MpdBuilder to support LiveProfile with static mpd
- Command line arguments will be updated in the next CL

Issue #142

Change-Id: Ibd35e5c9e1b1ef98043392e3f04a0af4700135c1
2017-01-13 19:59:49 +00:00
Chun-da Chen ce23fbacc4 Add support for udp_file in Windows. (#185)
Add support for udp_file in Windows
2017-01-10 10:05:28 -08:00
Haoming Chen 15dacf6288 Fix signed/unsigned mismatch on Windows.
Issue #168
2017-01-06 17:29:53 -08:00
Haoming Chen 93fe5457bf Fix x64 build on Windows.
- Fix integer conversion warnings.
- Fix #168.

Change-Id: Ie9aadea86a293b49e0854d6549800083143c79df
2017-01-05 14:19:35 -08:00
Rintaro Kuroiwa 655fe7b699 Add gitignores for msvs files
- Add MSVS related patterns to gitignore.
- Removed encryption_modes.h from media_base.gyp. It doesn't exist.

Change-Id: I44bc86ff7e79daf6f7bc61c21a558937530d8e66
2016-12-19 21:26:26 +00:00
Rintaro Kuroiwa 7a4a40acb8 Change values for slice header attributes
- nalu_data points to the beginning of the nalu, instead of
  beginning of the slice header which excluded the nalu
  header.
- nalu_size is the size of the whole nalu.
- header_bit_size calculation is corrected. The size used to exclude
  emulation prevention bytes.

Change-Id: Ic00f42caeddf7b4701f3948e6ef8a3537fd7f770
2016-12-15 17:07:17 -08:00
Kyle Alexander 0c2ee4c844 Add support for 4K and 8K content.
Adds UHD1 and UHD2 track types to support 4K and 8K content.

Bug: Closes #163
Change-Id: I8fd893725cae88e9944244a48607cbaab591b401
2016-12-13 19:24:22 +00:00
Kongqun Yang eab7910029 Fix crash when seeing unsupported es descriptor data
Addresses #175

Change-Id: Ia9c2f0c7a8c17a6614db0fe478892bd3c5efd5b0
2016-11-29 23:27:22 +00:00