Added new path to allow webvtt text input to be written as
webvtt mp4. This will happen when the input is a vtt file and
the output path is a mp4 path or the output format is mp4.
Missing test content for this.
Close: 70990714
Change-Id: Idbb4023db4a126a560528e73801f762cc45f4685
Moved muxer listener creation into a factory so that it can be
initialized once and then the factory only needs to be passed
around rather than all the parameters.
Change-Id: Ibf7df66c08debddbe6f6ff8995e7910b9ad17f3f
Use a mock muxer listener so that we can verify that the webvtt
output handlers will write to a manifest correctly.
Change-Id: I0294a998bfaf06a6d8f7d4c287fb014839b38f73
To remove the direct dependencies between packager.cc and the different
muxer implementations, this change remove the muxer factory to its
own file.
Change-Id: I4a1503c493f0f4d10a67e78461f96da3f4238944
Connected the WebVtt Pipeline together to test that the parser,
segmenter, and output all work together as expected.
Change-Id: I07138f7b5b1318f84c27c5b607d8df207c57ddb3
Also added Period::GetAdaptationSets and
AdaptationSet::GetRepresentations.
Instead of implementing GetDurationSeconds in all MpdBuilder classes
(MpdBuilder/Period/Adaptation/Representation) like what we used to do
with GetEarliestTimestamp, the two new functions allows MpdBuilder to
iterate through the Representations to get durations.
Also updates GetEarliestTimestamp functions to use the same iteration
method.
Change-Id: I682b70c07c248c0f6511ec3d9019086f986ee10e
Linked together all the handlers to create the HLS text pipeline.
This change does not include tests for the pipeline.
Bug: 36138902
Change-Id: I32e7a7cefae1e7c74ec6d0bdc6335c0d65e5e79c
Changed muxer listener to use "default" instead of {}
when defining its constructor and destructor in its
header file.
Change-Id: I3c80e808aa624293c3d7720ac6a47b470a663312
This change adds the missing code for passing mp4 text through
the pipeline.
This introduces the MuxerFactory to simplify creating muxers.
Change-Id: Ic7be1a2e0ff4b720a01061edb43aded946d05a47
- Update curl to 7.57.0
- Roll clang, which is needed due to MacOS / XCode update
- Fixes and suppress compilation errors due to clang update
Fixes#285
Change-Id: Ibac3288c641861605c3c0500d34d27373e6eecfe
This change introduces handlers to output WebVtt text files. There is
only one output but there is a common base to support others.
WebVttOutputHandler which handles all communication with other handlers
and WebVttSegmentedOutputHandler is responsible for listening for events
and choosing when and where to write cues and headers.
Bug: 36138902
Change-Id: I2b13a94262554398e66fee8cf024aa21041ddbab
Instead of having multiple char readers, one for strings and
one for files, just have one for files and use memory files
when testing.
Change-Id: Id1a2230046ba540ddf69ca10edb3edc74d2419b6
To better align with the Chunking Handler, the Text Segmenter
now outputs the SegmentInfo at the end of the segment rather
than the start.
Change-Id: If69ab951947d00779b4b63a52c4b6662bbdc4c0d
Move AdaptationSet related functions to the new Period class, which
maps to <Period> element and provides methods to add AdaptationSets.
Change-Id: I0fee290769fbe9a6355cc1b8c86baec8fbc4b4fd
Clean up DashIopMpdNotifier to make it easier to add Period class.
- Consolidate protected AdaptationSet grouping related logic to the
internal ProtectedAdaptationSetMap class
- Clean up AdaptationSet creation logic and updating logic
Change-Id: Idd5b39bb89d38e490ca4f561a6b840ccc9f1e40a
Implemented a MediaHandler that takes text samples and creates
media samples. The data in each media sample is the MP4 box for
non-overlapping cues.
As per WebVtt in Mp4, all cues must be non-overlapping. This handler
takes care of grouping and dividing cues.
Bug: 36138902
Change-Id: I0c1d27964180c14a22cb200591f70e46e04a651f
- Add --time_shift_buffer_depth to HLS doc.
- Fix a problem in HLS tutorial
Partially address #304, fixes#305
Change-Id: I60bc1eb659a30beb0a6ab73433f4cc99a7deab07
This change creates the webvtt segmenter which will be used for
segmented text webvtt for HLS. It takes a stream of text samples
and injects segment info messages based on the segment duration.
It is possible for samples to extend between segments, if so the
sample will appear twice (once in each segment).
Change-Id: Iae0134ee61cf269948026086520b6d3f8ce3785b
Took the WebVTT Media Parser and created the WebVTT Parser
that will take in a file and output a stream of cues that
will later be passed to another Media Handler that takes in
cues and chunks them.
Bug: 36138902
Change-Id: Ic77813fe19678e85d500269e69f46917510ab7ec
Moved from mpd_unittest.cc to representation_unittest.cc.
Also removed unused test data files in packager/mpd/test/data.
Change-Id: I23811390b4b1ffbfac3e69c1ffcf92544099a53d
Move AdaptationSet related tests to adaptation_set_unittest.cc and
Representation related tests to representation_unittest.cc.
Change-Id: I360e9aea3ba08a32e5ba3b0e8b34345f2b3f0b20
Added gMock matchers for matching xmlNodePtr and its attributes.
Clean up mpd_unittests to make it easier to separate adaptation set
tests and representation tests.
Change-Id: I31816b06e9c76f92d4a82656c659f3b9acae8cb5
Moved all code dealing with jobs into its own class so that the
packager code does not need to worry about initializing, running,
or stopping jobs.
Change-Id: I3e9ef1f22bd93d671f77d59ad15f23d1239078cf
As specified in HLS spec (https://tools.ietf.org/html/rfc8216):
All audio EXT-X-MEDIA tags SHOULD have a CHANNELS attribute. If a
Master Playlist contains two Renditions encoded with the same
codec but a different number of channels, then the CHANNELS
attribute is REQUIRED; otherwise, it is OPTIONAL.
Fixes#299
Change-Id: Ic2308c39b170178b11cb0d94c3a8083c8b5f3353
Seeing some failures on some platforms when compiled with clang
disabled:
GYP_DEFINES="clang=0" gclient runhooks
Several changes to make it work:
1. Mark packager code with packager_code=1 in GYP definitions.
2. Disable a few checks in non-packager code, which we do not have
direct control: dangling-else, deprecated-declarations,
unused-function
3. Fix the relevant errors in packager code.
4. Revert HAVE_STROPTS_H in curl config which is not available in
all linux distributions.
Fixes#286Fixes#293
Change-Id: I729b41f99403c5ad9487c6cc4a7dc06f6323cef8
1. Expect the time elapsed to be greater than kTimeout instead of
approximately equal to kTimeout if there is a "wait";
2. Expect the time elapsed to be smaller than kTimeout instead of
approximately equal to 0 if there is no "wait".
Also rename a variable in packager.cc to better reflect what it is.
Change-Id: I67975a6263b8dbc1124b78feae0f8e0d112bda50
It aligns with what a lot of other apps are doing, e.g. tsplay.
Also added notes in docker to adjust network setting if wants to use
multicast in docker container.
Change-Id: I666a8979cc041c904f7ff8e3e022f800c0830d5d
The error is logged only once to avoid log spamming.
Added definitions for the unsupported stream types for easy look up.
Change-Id: I097e2f05759bc84ef03f264cfabd2fb20da7c711
This patch also fixes a bug which was introduced in
https://widevine-internal-review.googlesource.com/c/packager/+/38040
where it was inconsistently assumed that AdCueGenerator would be MIMO
while connecting with other handlers and at the same time it was assumed
SISO in the AdCueGenerator class. Now we assume SISO everywhere.
Change-Id: Icd8c40b5ccfe0d98f47f09a60ea1635f74dceef9
The spec allows having more than one 'mdat' boxes even if it is not
used.
This is happening on some mp4 files in the wild:
http://www.sample-videos.com/.
Fixes#298.
Change-Id: I729cc94bee095560b5c4b3ee8511323f25f7ad5a
Implemented E-AC3 MPEG-2 Stream Encryption Format for HTTP Live
Streaming specified in https://goo.gl/1sgcwY
Issue #279
Change-Id: I36c1a05e3d0529ff810eaf52bdca45414baa93eb
Audio samples per frame for AC3 was not specified correctly earlier.
Also the number of channels is not correct if LFE channel is on.
Issue #165
Change-Id: Ibf20aa4c7aec43c07ec7cd394d631c537cb387dd
Implemented MPEG-2 Stream Encryption Format for HTTP Live Streaming
specified in https://goo.gl/N7Tvqi.
This change also moved ProgramMapTableWriter creation from TsWriter
to TsSegment.
Issue #165
Change-Id: Ia89dd16a5e6405706dd3d85ec6b6de580f7b13a7
The keys can be specified in --keys option, with the form of
label=<label>:key_id=<32-digit hex string>:key=<32-digit hex string>,label=...
There can be multiple "label=..." block. The DRM label can be one
of "AUDIO, SD, HD, UHD1, UHD2" or a custom label.
Mark --key and --key_id as deprecated. We can achieve the same result
with --keys=label=:key_id=<key_id>:key=<key>.
Also add a new optional field in stream descriptor: drm_label, which
is used to overwrite the internally generated DRM label for the stream.
If not provided, the DRM label is generated automatically based on
audio/video information, e.g. resolution.
Code changes:
- Merged RawKeyEncryptionParams and RawKeyDecryptionParams into
RawKeyParams.
- Make FixedKeySource accepts RawKeyParams as input.
Change-Id: Ic8c2f071cc71188e13f14bc6396fc2b3ffa5cac6
Problem occurs when the media playlists are in a sub-directory under
master playlist. If base_url is not set, segment URL should be
relative to the media playlist.
This only affects fMP4 as TS does not have init segment.
Fixes#253
Change-Id: Icddd9ed500d0a705e8b3260bfd4e916ecbba3f28
To help isolate the differences between audio/video and text in
preparation for adding the better text support, this change moves
all audio/video job creation and text job creation into their
own functions.
Change-Id: Ia71cf82ac147672b70c69123eeabdfc6b41c4a3d
This fixes shared_library not able to compile problem.
Also removed StreamDescriptorList and replaced with expicit
comparison to make the code clearer.
Also removed an out-dated DCHECK that mpd_notifier and hls_notifier
does not co-exist.
Change-Id: I3d04b8880741fecf0931a764cc12e50aa5e392b9
Use std::shared_ptr<const uint8_t> with a custom deleter to
represent MediaSample::data_ instead of std::vector<uint8_t>.
MediaSample::data_ can be shared by multiple MediaSamples and it is
immutable. A new data instance must be created if the clients want to
modify the underlying data. The new data instance can be transferred
to MediaSample using provided MediaSample::TransferData function.
This avoids unnecessary data copying.
Change-Id: Ib59785a9e19d0abb3283179b12eb6779ee922f79
Took the test structure for trick play and made a common test
structure that can be used for new tests. The next tests to
use this structure will be the new WebVtt pipeline.
Change-Id: I5faca306b8a2bd16b9f945d1ff5c336aed82bc35
This change create some matchers and text functions that
will be used by the upcoming webvtt work.
Change-Id: I2d7931d1bdabd761f74e8aee7524d5ecfb7cfdc4
Before the trick play handler would have one input and multiple
outputs. The normal media handler API for adding handlers had to
be ignored when setting-up the trick play handler.
This changes the trick play handler to have one input and one output.
It uses the standard AddHandler method calls. When initialized, the
trick play factor must be provided.
This included a run of 'clang-format' over all edited files.
Change-Id: I7b3cdf0a2472e2f89ab194867c4b984e26f18f24
To allow more tests to use the MediaHandlerTestBase and get access to the
helpful creation methods, this change moves the use of FakeMediaHandler
and setting up the graph to a sub class.
Change-Id: I90dd151e3c96d8fbe4bd02a1d6b11e66a279d95b
Created media handlers that make testing a handler easier by providing
methods to send messages from upstream and check messages downstream.
Change-Id: I7b3ba3a51bcf0d73bcaacd46b2dcbc16f87d8694
Moved build instructions and docker instructions to separate docs.
Removed command samples which are no longer needed as we have better
tutorials now.
Change-Id: I340c5653a6553158325970dd86d76fa11e69bcbd
In prep for changes to Trick Play, we want to make all messages
copy on write so that if the same message is sent to multiple
handlers, it is not possible for one handler to change the data
another handler is using.
Change-Id: I554166ca11c532412e4dfced5603972ca24dc2bb
Problem occurs when the media playlists are in a sub-directory under
master playlist. If base_url is not set, media segment URL should be
relative to the media playlist instead of the master playlist.
Also make sure the path separator is "/" in URL instead of "\" on
Windows.
Fixes#253
Change-Id: I8e933750276435d94dd01bfa53ee2bc050dfd193
Previously packaging completes successfully without any error
or warning.
With the fix, an error will be reported if write fails. It may
appear as "Cannot close file error" as we use threaded IO, which
could delay the error reporting until Close() call, so the user
of the File API needs to make sure Close() returns successfully.
Also fixed a deadlock in threaded_io_file if internal_file->Write
fails.
Fixes#160
Change-Id: I17f945150fb4021d2dcdbe784e557673f53ca583
Doxygen should run under src/ directory:
$ doxygen docs/Doxyfile
Outputs are created in out/doxygen.
Change-Id: I9dd1a9aa158851d37ff9a2e37f701757eb3e577c
To ensure that the style data is formatted correctly, changing the
set_style method to AppendStyle so that joining values will be
done in controlled setting.
Change-Id: Ia88e2d260b11339e6777ff018813e8aeb802a501
There are multiple functions we use to verify input from the user
and must of them would return a boolean expect for the top level
function which would return a Status. Each function would log a
message but to the user the error was not clear. This change makes
each verify function return a status so that the actual problem
will be surfaced easier.
Change-Id: I12ab43f8ca3a4d379b4309336644a70eb8cbbbe0
Implemented the other end of the webvtt timestamp parsing as it will
be needed when we write out WebVtt files. Also changed the name of
the parse function to be more inline with the writing function.
Change-Id: I1f36ddbbf80028732b4cb1b15e871dec17767f63
Inside of CreateRemuxJobs, we would take a status value from another call,
log the message, and return a boolean. The caller of CreateRemuxJobs would
then create status from the boolean.
Now we will pass the status value from the other call directly to the caller
of CreateRemuxJobs. So that failures can be better communicate in our code.
This includes a pass of clang-format over packager/packager.cc.
Change-Id: I071e7bad13f916e963641b9e53699573fe1060ed
Rewrote the RunRemuxJobs function to use waitable events so that we never
get stuck waiting for a job to finish before getting to a thread that
has already exited.
This should allow us to end execution when a job failed rather than
waiting for all jobs before the failed job to finish.
Closes#94
Change-Id: I413f62561a7a4cab83b8905e75986230b6c498bc
Created a handler called OriginHandler that can be used by the Job class
(previously called RemuxJob). Origin handlers represent the start (or
origin) of a pipeline (chain of handlers).
Change-Id: Ibd748ae0a932b6e0ebb879ea292fcb83c548214b
Support generation of fairplay key system tag: "com.apple.streamingkeydelivery" when --pssh
includes fairplay key system id:
// Unofficial fairplay system id extracted from
// https://forums.developer.apple.com/thread/6185.
const uint8_t kFairplaySystemId[] = {0x29, 0x70, 0x1F, 0xE4, 0x3C, 0xC7,
0x4A, 0x34, 0x8C, 0x5B, 0xAE, 0x90,
0xC7, 0x43, 0x9A, 0x47};
Closes#258
ImportantFileWriter::WriteFileAtomically uses mkstemp internally,
which set file permission to 0600, which is not what we want.
Update the code to not use mkstemp instead.
Also updated temporary file name logic to include process id in
the name so it can be unique across processes.
Fixes#259
Change-Id: I2d5a375925cf552bc0db5269f409d7522e63fca5
This avoids interference caused when two sockets are bound to the
same port in different multicast groups.
Fixes#241
Change-Id: Iba1b3300e3850f1f268886b7a49f5a3315d95b80
Derive file size from media ranges instead.
Also fix subsegment range reporting for WebM (internal variable,
no output affected).
Change-Id: I5f8152dff4c2cd5fbae5550992b86a669e278f7b
- The master playlist should have the display resolution instead of the
encoded resolution.
- Changed MediaPlaylist to return display resolution.
Change-Id: I162727d0bdeed0302518286e42a22b69a58fc4a3
- Use byte range format if segment_template is not set in MediaInfo.
- Change HlsNotifier::NotifyNewSegment() to take start byte offset.
- Change HlsNotifyMuxerListener to cache the segment information if
MuxerOptions::segment_template is set.
Change-Id: Ida1b8b9198acc5c43e3a5d9a641c78c81bc9a6ac
- Changed OnMediaEnd() to take (sub)segment ranges.
- Define a structure to pass the ranges.
- This requires muxers to "cache" the segment ranges but in most cases,
they already do it, e.g. 'sidx' for mp4.
Change-Id: I16b974c1f0f54ca658cc6dbe605efff84a3f52eb
In the original manifest updating process, the file was truncated first
before was written with new contents. There is a small chance that
the web server may read back empty file or incomplete file.
The new code makes the update operation atomic (by writing to a temporary
file first then replace the old file with the temporary file).
Fixes#186
Change-Id: I2fd564cb12b922b032c0e9f70d2132a5b12ff098
- Remove the TS key rotation not supported check as there is nothing
to be done for key rotation in TS for SAMPLE-AES.
- Fix IV updated problem in new segments even if crypto period does
not change.
- Avoid duplicate EXT-X-KEY tags if it does not change.
- Make EXT-X-DISCONTINUITY-SEQUENCE match with number of removed
EXT-X-DISCONTINUITY.
- Added end to end test for HLS with key rotation.
Change-Id: I73cb82e9f5575fcdf63ee643228efe78e6766302
Resolving TODO in packager/hls/base/master_playlist.cc that asked for
media playlists to be grouped when they are added.
Change-Id: I01c7687400bc74ce45f271d0242d1c3bf3610cf2
- This fixes the issue when one NAL unit has multiple
subsamples, which is allowed in spec and could happen
when the clear_bytes exceeds 2^16 bytes.
- The new algorithm updates the subsample entries in
following steps:
1) When NaluReader parses a new NAL unit, the
function splits subsamples and align them with the
NAL unit. It also finds the range of subsamples
for that NAL unit.
2) Drop or update subsamples of that NAL unit,
based on the type. Other inserted bytes are treated
as a new all-clear subsample.
3) Merge those subsamples if possible.
- This addresses #237.
Change-Id: I2cab47cd00d90e29ec52b90d4fde057c632193ac
Moved all the timestamp parsing code into its own class with its
own unit tests. This is to make testing easier.
Change-Id: I624472baba51dfa1254a9f4bae55ebb79e310855
The HLS manifests output by the packager did not have the RESOLUTION
attribute on EXT-X-STREAM-INF tags. This change adds the resolution
tag when resolution values are present.
Close#235
Change-Id: I15ca6de47cb34793cf6a940123d593261627baa4
Shared libpackager can be built by setting libpackager_type to
shared_library, e.g.
GYP_DEFINES='libpackager_type=shared_library' gclient runhooks
ninja -C out/Debug
will generate libpackager.so in out/Debug/lib directory.
Here is a few other changes to make shared_library builds and
tests pass:
- Add several test parameters to packager.h, which is needed for
testing.
- Create a protoc.gypi from build/protoc.gypi but depending on
protobuf_full_do_not_use instead of protobuf_lite, since we need
protobuf_full_do_not_use for text parsing and generation of media
info proto. Somehow shared_library build does not allow mixed use
of protobuf_full_do_not_use and protobuf_lite.
- Remove the use of LazyInstance in version/version.cc and use static
variable directly. This is because LazyInstance needs AtExitManager
which may not be easy to setup when calling GetVersion.
- Allow skipping testPackageWvmInputWithoutStrippingParameterSetNalus
with flag --shared_library, which is needed as shared_library build
does not support --strip_parameter_set_nalus flag yet.
Fixes#227
Change-Id: Iff05a50baa28134faa7218664c96114cb9e70329
Added a text sample type that can be passed between media handlers.
This will allow text samples to safely be moved between media
handlers before converting to their final format.
Bug: 36138902
Change-Id: Ic4946f774a7d37c43066b9ea46596d5c5f3c05a8
frame_duration is not available when the VideoInfo is created, but
it will become available when a media segment arrives.
Change-Id: I2c24852fca4ff77c2b86ff78dbf0963ad0a0afa6
'availability_time_offset' is not an attribute in MPD specification,
but used to calculate availability_start_time to adjust segment
availability time. A similar effect can be achieved with standard
blessed suggested_presentation_delay flag.
If suggested_presentation_delay is not set, player is expected to
choose a suitable value.
Change-Id: I1d540bd4347b4f04b6cc5ec0778e1a5392009b09
Aaron Vaage
KongQun Yang
Michael Stillwell
Haoming Chen
Kamesh Devarakonda
l-law
Joey Parrish
Widevine Buildbot
Ivan L. Isaev
David Cavar
Yohann Connell
Rintaro Kuroiwa
Haoming Chen