Commit Graph

298 Commits

Author SHA1 Message Date
KongQun Yang 33c4f344e7 Implemented PackedAudioSegmenter
Per HLS specification:
https://tools.ietf.org/html/draft-pantos-http-live-streaming-23#section-3.4

Issue #342.

Change-Id: Iba7fae7a9b8912bcd13ae55d25b22a5803f7f7ea
2018-05-16 21:52:11 +00:00
KongQun Yang 5965b3e136 Move TS encrypted audio setup for HLS to hls_audio_util
Change-Id: Ibbc1fe2a8013abded6df2f2f57701ec328d3e5c6
2018-05-16 01:04:14 +00:00
KongQun Yang 8333908df1 Implemented Id3Tag class to handle ID3 tag
Also switched the original code in mp4 to use the new Id3Tag class.

Change-Id: I1db2c6c6142ed98b72a432980a6a54815f1a8cc4
2018-05-16 01:04:02 +00:00
KongQun Yang b5a73fc1d5 Refactor ProtectionSystemSpecificInfo class to struct
This is in preparation of supporting entitlement license API, where
common encryption server may return concatenated PSSHs directly.

Refactored ProtectionSystemSpecificInfo into a struct containing
concatenated PSSHs. This will make it easier to pass PSSHs around.

Also, most of the time, users of ProtectionSystemSpecificInfo do
not care what is in PSSH; so moved PSSH box parsing and building out
of ProtectionSystemSpecificInfo.

b/78171767

Change-Id: I1c4d5e7e23efd2f7d4b2b9704378323112e47f00
2018-05-11 00:10:07 +00:00
KongQun Yang 2d919b5a31 Fix potential partial DASH segments during live packaging
Packager uses ThreadedIO to write media segments and manifest /
playlists. There was a possibility that media segments write being
delayed and scheduled after updating manifest / playlists.

This CL fixes the race condition.

Also added a note on how segments can be synced to cloud storage to
avoid the race condition during file sync.

Also added a live WebM test.

Fixes #386.

Change-Id: Icf9c38cdec715fa3dc2836eab1511131e129fe41
2018-05-03 09:26:23 -07:00
KongQun Yang b6f0da246f Add support for FLAC codec
Implemented according to spec:
  https://github.com/xiph/flac/blob/master/doc/isoflac.txt

Closes #345.

Change-Id: If2e277d8eac4baff5965faa0e13c44c334f6184d
2018-05-02 00:16:51 +00:00
Aaron Vaage 56c2f227ff Skip Style and Region Blocks
To ensure that we can parse content with style and region blocks,
this change updates the parser to skip those blocks so that we
can still parse the cues from a file.

Full style and region support will be added later this year.

Issue #380

Change-Id: I11b8fd862a108c27a5c67b15d4703532b44a1214
2018-04-25 20:41:10 +00:00
KongQun Yang 772aa7c93f Do not force earliest_presentation_time to 0 for VOD
Instead, the actual earliest presentation time is used except for
the first segment if there is an offset between presentation time
(pts) and decoding time (dts).

Chrome (as of v66) reports dts instead of pts in buffered ranges in
MSE API. To avoid breaking Chrome, the earliest_presentation_time
of the first segment is set to its dts as Chrome does not like negative
values for
  adjusted dts = dts + Period@start (0 for the first period)
                 - presentationTimeOffset (earliest_presentation_time).

Fixes #303.

Change-Id: I5ca80e05d5570961400499436f2bcc01f06e69e0
2018-04-09 18:39:15 +00:00
Aaron Vaage 061777db9b Add CueEvent Support To WebVtt Output
The WebVtt Output Handler did not recognize cue events. This change
allows the handler to accept the events and tell muxer listener
about them.

Issue #362

Change-Id: I7c3318b72e539adc19af587c8e213fdb0af8290b
2018-04-07 02:45:52 +00:00
Aaron Vaage 113a7d123b Don't Output Empty Text Samples
Only output text samples to webvtt text output when they have a payload.

Change-Id: I958d2b4f087209f540fa32cef002cd2cef37c65c
2018-04-03 18:29:41 +00:00
Aaron Vaage e5fe2a76d2 Created Text Padder Handler To Fill Gaps
Created a media handler to come after parsers that will handle filling
in gaps between text samples. The padder takes a min duration, and if
the samples do not cover the min duration when flushed, one last empty
sample will be injected so that the samples will go up to the min duration.

Change-Id: I88605059664d09279676edac418ff3d4990d7556
2018-04-02 23:11:26 +00:00
Aaron Vaage 8f565bf388 Change Text Sample to Use int64_t
Changed Text Sample to use int64_t so that it will use the same
type for time as Media Sample.

Change-Id: I4cfbfdc60c37bb511517993976cd1a459bdf6667
2018-03-26 20:46:58 +00:00
Aaron Vaage 2dd198a9d2 Move WebVtt Segmenter To Chunking Directory
Move the webvtt segmenter to the chunking directory so that it
can be converted to a general purpose text chunker.

Change-Id: I9ecd7ee39cb73070dab07b64f65ef24af1404813
2018-03-26 17:58:38 +00:00
Aaron Vaage 2b337e29f6 Removed WebVTT Pipeline Tests
Now that we have the end-to-end tests, we no longer need the webvtt pipeline
tests to verify that it is working.

Change-Id: I4ebec34e66eda67c40999d8802b447e2551e1fa6
2018-03-26 17:57:47 +00:00
Aaron Vaage a191c25f57 Clean Up WebVtt Parser
There were a couple of bad practices in WebVtt that needed to
be cleaned-up.

Change-Id: I0fe21e26f11141709d8d855077805fc625e6dad5
2018-03-23 14:19:52 -07:00
KongQun Yang b24a95b1aa Rewrite init segment in MultiSegmentSegmenter::DoFinalize
This is called when reaching end of the file of the media. The duration
of the media is updated.

Fixes #340.

Change-Id: I446f2d341b02125d4a7d8c958bda269b5403cb9c
2018-03-05 14:25:08 -08:00
Aaron Vaage 033fa65105 Removed Old WebVtt Code
Now that we have the new webvtt code in place, we don't need to keep the
old solution.

Change-Id: I20540ba3adf93f535f0ed011acb8e2555653522a
2018-03-02 20:59:23 +00:00
Aaron Vaage 7dd80884c3 WebVtt Ignore Zero Duration Cues
We have an assert that ensures that the end time is greater than
the start time for any cue. However we never checked that cues
had a non-zero duration when parsing them.

We will throw away cues with a duration of zero (and print a
warning message) as they are not spec compliant.

Closes: #335
Change-Id: I404e8f3a5a8d43eff75a2554db3e38e8d340f421
2018-03-02 01:25:41 +00:00
KongQun Yang de716a6544 Skip reading meta box as it may not be well formed
In the video captured by Android's default camera app, meta box is written as
as Box instead of FullBox specified in the spec.

Closes #319.

Change-Id: I526492fdd505d5929c5161cb1ed1503b724de7e9
2018-03-01 20:21:37 +00:00
KongQun Yang 11cbbd86fb Fix fMP4 'cbcs' playback issue in Safari
Looks like Safari does not like v0 tenc box.

- Use v1 tenc box for cbcs and cens protection_scheme as required by
  CENCv3 spec.
- Set crypt_byte_block and skip_byte_block to 0 for full sample
  encryption cbcs and cens.

Fixes #326

Change-Id: I5581cd856fffc4ff104d950f3ca19b9337d57a78
2018-02-08 23:40:46 -08:00
KongQun Yang aa4bb49b2f Fix language option not honored on text streams
Change-Id: I9034b779e9b193c39201a5f2d142ec390927e5f7
2018-02-08 03:48:46 +00:00
Aaron Vaage c7eb2b4a7e Output Empty WebVTT Text Segments
Have the text webvtt pipeline output empty segments rather than skipping
them.

Bug: #324
Change-Id: I175cf22b65b241f6bf566aba2440518f3df95277
2018-02-06 13:32:42 -08:00
Aaron Vaage f35410deb1 Output Empty MP4 WebVtt Cues
When a gap is found in the text stream, the WebVtt to Mp4 converter will
now output the special empty vtt cue.

Change-Id: I8be88c6b7589aa120a2215e1e4b8e98031fe326d
Closes: #324
2018-02-06 16:45:04 +00:00
KongQun Yang 8257eea804 Bug fixes and clean-ups for I-Frame playlists
- Add empty lines between different types of renditions to improve
  readability.
- Group variants with the same audio/text group together, as it is
  where the Adaptation occurs.
- Write master playlist after writing media playlists. This makes
  more sense and it is also necessary to have the bandwidth of
  the last iframe playist segment correctly computed.
- For fMP4, I-Frame segment must include the 'moof' header.
- Fix a problem that hls_iframe_playlist_name is not passed to
  MuxerListenerFactory.

Issue: #287

Change-Id: Icf37c5de1dc29f85ae3f419cbc3264d04ca491a4
2018-02-05 17:36:28 -08:00
Aaron Vaage c991490e82 Prepare WebVTT To MP4 to handle gaps
Adding all the code needed to allow the webvtt to mp4 converter to
send empty vtt cues. This change does everything except writing the
mp4 box.

Bug: #324
Change-Id: I16188e6357632b2ed06f7e9bab7844f093266696
2018-02-06 00:55:32 +00:00
Aaron Vaage 9452a1bb1a Make Segments Their Own Struct
Instead of always tracking segments with an index, this change makes a struct
that will act as the segment and track which segments are in it. Each
segment will store all the samples in the order they were given, it will
avoid any sorting.

Closes: 72867775
Change-Id: Ic5829161510fe8f3320d960c3bc4a276c26ff3be
2018-02-06 00:55:00 +00:00
Aaron Vaage 80ed014cf8 Create Preserve Order Test
Create WebVtt test that shows that the current implementation of the
webvtt segmenter does not respect the input order of cues.

Bug: 72867775
Change-Id: I811b3a93c10650e0cf9290c9c0c1680f562deb30
2018-02-06 00:54:15 +00:00
KongQun Yang b647c6731c Support KeyFrame events in TS and MP4
Issue: #287

Change-Id: I7c50853c7cc61c5fadbb620f44f7574a27dc2b68
2018-02-02 14:05:50 -08:00
Aaron Vaage 771944a3f5 Output Empty WebVtt Segments
According to the HLS+WebVTT spec, if there is no text in a segment
a webvtt file with no cues can be added to the manifest. By outputting
empty segments it allows the accumulated duration in the Master Playlist
to better represent the duration of the text stream.

Spec Reference: //tools.ietf.org/html/draft-pantos-http-live-streaming-23
  Section 3.5. WebVTT

Bug: #205

Change-Id: I5de01200fd9fa99c57949c773e8ee926b0f6ba8a
2018-02-01 17:02:19 +00:00
KongQun Yang 4194f0746e [DASH][WebM] Fix subsegmentAlignment not set problem
MuxerListener::OnNewSegment was not sent for WebM SingleSegmentSegmenter.

Change-Id: Iecae84e7a7c1a8a4d3f577158fa286b9c9959934
2018-01-31 19:58:15 +00:00
KongQun Yang 2ba23e1075 Fix incorrect CHANNELS reporting for AC3/EC3
Fixes #313

Change-Id: I0b8d8081cda8aaa4db2ca4916bfc6721bfa3bd22
2018-01-11 03:50:56 +00:00
Aaron Vaage c877af9f3b Use Mock Muxer Listener in WebVttOutput Tests
Use a mock muxer listener so that we can verify that the webvtt
output handlers will write to a manifest correctly.

Change-Id: I0294a998bfaf06a6d8f7d4c287fb014839b38f73
2018-01-10 17:51:39 +00:00
Aaron Vaage 1e9e5d1f58 Integration Test for WebVtt Pipelines
Connected the WebVtt Pipeline together to test that the parser,
segmenter, and output all work together as expected.

Change-Id: I07138f7b5b1318f84c27c5b607d8df207c57ddb3
2018-01-09 22:14:04 +00:00
KongQun Yang 5f48cbb0c1 Fix build failures in MacOS 10.12.6 with XCode 9.0
- Update curl to 7.57.0
- Roll clang, which is needed due to MacOS / XCode update
- Fixes and suppress compilation errors due to clang update

Fixes #285

Change-Id: Ibac3288c641861605c3c0500d34d27373e6eecfe
2018-01-05 12:21:41 -08:00
Aaron Vaage c88fe54553 WebVTT Output
This change introduces handlers to output WebVtt text files. There is
only one output but there is a common base to support others.

WebVttOutputHandler which handles all communication with other handlers
and WebVttSegmentedOutputHandler is responsible for listening for events
and choosing when and where to write cues and headers.

Bug: 36138902
Change-Id: I2b13a94262554398e66fee8cf024aa21041ddbab
2018-01-05 00:05:48 +00:00
Aaron Vaage 9e7b5c1ca9 Change CharReader to FileReader
Instead of having multiple char readers, one for strings and
one for files, just have one for files and use memory files
when testing.

Change-Id: Id1a2230046ba540ddf69ca10edb3edc74d2419b6
2018-01-04 14:36:52 -08:00
Aaron Vaage b0edec8c40 Changed Text Segmenter to output segment info last
To better align with the Chunking Handler, the Text Segmenter
now outputs the SegmentInfo at the end of the segment rather
than the start.

Change-Id: If69ab951947d00779b4b63a52c4b6662bbdc4c0d
2018-01-03 23:40:37 +00:00
Aaron Vaage ab19082c20 WebVtt To MP4 Handler
Implemented a MediaHandler that takes text samples and creates
media samples. The data in each media sample is the MP4 box for
non-overlapping cues.

As per WebVtt in Mp4, all cues must be non-overlapping. This handler
takes care of grouping and dividing cues.

Bug: 36138902

Change-Id: I0c1d27964180c14a22cb200591f70e46e04a651f
2017-12-22 21:08:17 +00:00
Aaron Vaage c03c447e0b Created WebVtt Segmenter
This change creates the webvtt segmenter which will be used for
segmented text webvtt for HLS. It takes a stream of text samples
and injects segment info messages based on the segment duration.

It is possible for samples to extend between segments, if so the
sample will appear twice (once in each segment).

Change-Id: Iae0134ee61cf269948026086520b6d3f8ce3785b
2017-12-21 21:12:34 +00:00
Aaron Vaage 4dcfe413f5 WebVTT Parser
Took the WebVTT Media Parser and created the WebVTT Parser
that will take in a file and output a stream of cues that
will later be passed to another Media Handler that takes in
cues and chunks them.

Bug: 36138902

Change-Id: Ic77813fe19678e85d500269e69f46917510ab7ec
2017-12-21 21:12:18 +00:00
Haoming Chen feeb7361ff Clean up #ifndef in packager/media. (#310) 2017-12-19 16:56:36 -08:00
Haoming Chen f22320bfa7 Remove non-exist class MediaStream. (#308) 2017-12-18 13:28:19 -08:00
KongQun Yang 8e96dd3b37 Address gcc build failures in some platforms
Seeing some failures on some platforms when compiled with clang
disabled:
  GYP_DEFINES="clang=0" gclient runhooks

Several changes to make it work:
1. Mark packager code with packager_code=1 in GYP definitions.
2. Disable a few checks in non-packager code, which we do not have
   direct control: dangling-else, deprecated-declarations,
                   unused-function
3. Fix the relevant errors in packager code.
4. Revert HAVE_STROPTS_H in curl config which is not available in
   all linux distributions.

Fixes #286
Fixes #293

Change-Id: I729b41f99403c5ad9487c6cc4a7dc06f6323cef8
2017-12-07 11:45:26 -08:00
KongQun Yang 0fdb0d02aa Log an error when seeing unsupported stream type in MPEG2TS
The error is logged only once to avoid log spamming.

Added definitions for the unsupported stream types for easy look up.

Change-Id: I097e2f05759bc84ef03f264cfabd2fb20da7c711
2017-12-01 14:56:22 -08:00
KongQun Yang 068e220ac7 Handle additional unused 'mdat' properly
The spec allows having more than one 'mdat' boxes even if it is not
used.

This is happening on some mp4 files in the wild:
http://www.sample-videos.com/.

Fixes #298.

Change-Id: I729cc94bee095560b5c4b3ee8511323f25f7ad5a
2017-12-01 00:28:30 +00:00
KongQun Yang 61e36d7d21 Support encrypted E-AC3 in TS
Implemented E-AC3 MPEG-2 Stream Encryption Format for HTTP Live
Streaming specified in https://goo.gl/1sgcwY

Issue #279

Change-Id: I36c1a05e3d0529ff810eaf52bdca45414baa93eb
2017-11-13 22:54:07 +00:00
KongQun Yang 0a69779f7c Fix a parsing problem with AC3 in TS
Audio samples per frame for AC3 was not specified correctly earlier.

Also the number of channels is not correct if LFE channel is on.

Issue #165

Change-Id: Ibf20aa4c7aec43c07ec7cd394d631c537cb387dd
2017-11-07 18:47:09 -08:00
KongQun Yang 72df5af150 Support encrypted AC3 in TS
Implemented MPEG-2 Stream Encryption Format for HTTP Live Streaming
specified in https://goo.gl/N7Tvqi.

This change also moved ProgramMapTableWriter creation from TsWriter
to TsSegment.

Issue #165

Change-Id: Ia89dd16a5e6405706dd3d85ec6b6de580f7b13a7
2017-10-30 15:22:22 -07:00
KongQun Yang d7f531fe10 Add AC3 support in TS
EC3 and encrypted AC3 in TS are not handled yet.

Partially address #165

Change-Id: If4839ee7801eac902e64e9c677dd37709ec8e88b
2017-10-30 15:11:43 -07:00
KongQun Yang ad836a5cf1 Rename EsParserAdts to EsParserAudio
Change-Id: I8d9fc265a4cf740970256e1a46b172349f1bf794
2017-10-27 23:58:50 +00:00