This flag was designed for two purpose:
- Grouping fragments into subsegments, achieving three level hierarchy:
segment < subsegment < fragment.
- Indicate whether to generate 'sidx' box in media segments (when the
value is set to a negative number).
There are no practical use case for the first purpose. Removing it to
simplify the code and reduce the confusion.
Introduce another flag --generate_sidx_in_media_segments for the second
purpose.
Change-Id: I4be7cd42662fb324c1158b978e05768ee49dd048
It was implemented to workaround Chromium's DTS
https://crbug.com/398130, but the workaround does not really work in
all situations.
Remove it now as we already have another workaround available.
Change-Id: I291f559d78120fb743a6679b7d927e5bbc5b6b4e
DTS was used in ChunkingHandler. As a result, SegmentInfo contained
timestamp in DTS. MP4Muxer has a logic to change SegmentInfo to use
PTS but not in other muxers.
Benefits of using PTS in ChunkingHandler:
- De-dup the redundant logic in MP4Muxer
- Ensure consistent behavior in different output containers
- Consistent with other timestamps, e.g. Ad Cue timestamps
Issue #413
Change-Id: Ib671badf144e0c0866d60f4ff0ac0cbbdd33817e
We always assumed that text stream started at zero in the text chunker.
This meant that if the text stream started later than zero (like in a
live stream) we would generate a lot of empty segments.
Instead the text chunker will assume that segments should be adjusted
to align with multiple of segment duration. The text chunker will assume
that an earlier component in the pipeline (i.e. text padder) will ensure
that the first sample has the time that we want to start segments at
(even it needs to add an empty sample).
Issue #416
Change-Id: Ie45844354d6e9448787cae896841b5ab31721ed6
Previously, the text padder media handler would assume that text always
started at time zero. This would work for VOD but would result with a
large pad at the start of LIVE content.
To avoid this, the text padder will use a bias to test whether or not it
thinks the content starts at zero. Right now the bias is set to be 10
minutes, but will later be configurable with a command line flag.
10 minutes was used as LIVE content will have much larger values and VOD
content should have much lower values.
Issue: #416
Change-Id: I07af15a577392fb030e36f052085cd4e667700e8
This is a more faithful implementation of more_rbsp_data().
There could be trailing null bytes in NAL units. This isn't valid per
H264 specification, but the referenced bug includes a sample where the
PPS in the avcC record includes a trailing null byte.
Workaround the problem so packager does not fail.
A similar problem is workarounded in Chrome:
https://codereview.chromium.org/1107593004Closes#418
Change-Id: I28cb8a9371945dc094f766c3e559d7a66859b451
This allows UDP receive buffer size to be increased to avoid potential
packet loss result from receive buffer overrun.
Another related flag is --io_cache_size. buffer_size in UDP options
defines the UDP buffer size of the underlying system while
io_cache_size defines the size of the internal circular buffer managed
by Shaka Packager.
Closes#411
Change-Id: I57c843a88f13da546417dadc2a78df2bee0a00f3
Under VLOG(1).
IO Cache circular buffer full could result in UDP packet loss with UDP
inputs.
Related to #390 and #411.
Change-Id: Ia5636a70827978b5c2f71f6495256d758766632f
When we built the cue alignment media handler, we did not remove
the old solution.
This change removes all the code for the old solution.
Change-Id: I851b284c449c7d25aaabc2f55df5579ba7b5aad1
Originally the create and dispatch of stream data in the cue
alignment tests were separate to improve readability. The idea was
to separate building each stream and dispatching the streams muxed
together would make it easier to understand the streams.
However, coming back to the code, this did not hold up as it took just as
long to understand the code. So merging the two together allows for the
code to be more concise.
Change-Id: I77f6bb67e7fb0ca7af0553c3baf07082770da74c
In the media handler there is an example of what a one-to-many media
handler would be. It used the trick play handler as an example, but that
handler is now a one-to-one media handler. Changed the example to use the
replicator media handler.
Change-Id: I26908c6e27ea4a697a19c0fa0179c60842a449d2
Made ChainHandlers to MediaHandler::Chain so that it can be used in our
test code as well. After all it is a pretty helpful function.
Change-Id: I8d83ee184052cd9fa9b37f2741c96f3223d5ab48
Ran clang-format over media_handler.cc and media_handler.h so that
later changes won't update the formatting as much.
Change-Id: I2db4f9f4e8a66fca1e1418ab99b283a0e4a70e4c
The copy and assign in MediaHandlerGraphTestBase referred to the wrong
types. This changes them to refer to the correct types.
Change-Id: I165c9a1da39b49adf41e7c35ae3e4a0b9ecfa838
Having "wvtt" in the codec string (in the master playlist) causes
errors on some older Apple products. As including it is optional,
we are opted to omit it to ensure support for all Apple products.
Close#402
Change-Id: Ib1072bcc26a3ff66e3a6d3204789c0c8c678d4db
Reduce which variables we check in the cue alignment tests by replacing
the less important variables with "_".
Change-Id: Id8770b5f2045ad5c8ac6eee54d6a291d933a4b6f
We have adopted some new standards for how we are writing the media
unit tests. This change goes through the trick play tests and
updates them to be more inline with how we are doing new media
tests.
This include using "_" for parameters that are not the focus of the
tests (used in the EXPECT calls).
This includes wrapping Dispatch calls so that the test bodies can
be more readable.
Change-Id: Iab4b1ce67666b6c8a4a821db1a8f4369cd366dc3
Added a matcher for Video Streams. This will allow us to use a general
matcher in tests for video specific fields.
Change-Id: I91b9aebd66be37ec6d5f09b762263fafaef854c2
Added a StreamTypeToString function in StreamInfo (where stream type is
defined) so that we can print the type in a more readable way.
Change-Id: Icb1d10f5a72dca314d9bdf07e9cd0c0a185bd791
Ran 'clang-format' over stream_info.h and stream_info.cc so that
later changes can focus on the actual changes and not the formatting.
Change-Id: I8a75fe0f88db533f4a811d65dabd023033f07bda
Added the key frame field to the IsMediaSample mather. All
tests that do not care about a sample being a keyframe (or not)
have been updated to use "_".
Change-Id: I44180687c58c260b6856e683d647f532227b14d5
Updated all the stream data matcher we use in our unit tests
to allow us to use matchers in them. We are now able to use "_"
to ignore specific parameters.
With this we were able to replace the different version of
matchers for each stream data type with a single instance for
each type.
Includes updates to printing strings to the listener. Strings
now go through a "pretty" function to help make it easier to
read them in the output.
Change-Id: I146351b54fccd63ab9ec936877e6c6b30f9aa9fc
In the cue aligner, we assumed that all text will be in milliseconds.
This was the last place with that assumption. This change removes that
assumption and uses the stream info's time scale.
Issue #399
Change-Id: Ie21bf27148e020bd85111dcace0bbdff3419c1ac
Before, the text chunker would assume that all text streams were in
MS, which is not a safe assumption to make.
This changes it to take the time scale from StreamInfo and work
natively with scaled time units.
This required updating the tests. While doing so the tests were
rewritten with the goal to make them easier to read.
Closes: #399
Change-Id: Ib792ad306f40d749763418cde645337913a6046b
Make the text stream info factory method in media_handler_test_base
require the caller to specify the time scale.
Issue: #399
Change-Id: Ibdfb183e0aa3f4ff50edf6b58c4e9b966006c6d2
Configurable under flag --use_legacy_vp9_codec_string, which defaults
to false as all major browsers and platforms support new style vp09
codec string already.
Closes#406.
Change-Id: I22e917777f9d66db815ff9d55eb47b6d55806269
- Also fixed documentation missing --hls_playlist_type LIVE for HLS
live examples.
- Also updated packager.cc to use RETURN_IF_ERROR macro for
consistency.
Fixes#347Fixes#403
Change-Id: Idbccd7137b873170cd54e2c780bd554d25031a0b
Since the WebVtt mp4 to mp4 path did not include all the features
we needed and we know of no-one who is asking for it. We are opted
to remove the path so that there will be an error rather than
incorrect output if they try to use it.
Issue #405
Change-Id: Id2c37bb385c514dd8e31f7d3bd75fb3904b70d78
To ensure that every variable in a box is explicitly set
every variable has been assigned a default in the header.
Change-Id: Iaa806c4058ac6621a64363a00040fbd9903c6710
Text Samples with no payload should be ignored, so this adds a
test to check that samples with no payload get treated the
same as a gap.
As long as this case is true, using gaps in our other tests should
functionally be the same as using samples with no payload.
Change-Id: Ic16b240c43eda2514b537a2d938d4135638adc4e
Before we used the sample payload for each text sample as we were
focusing on the times rather than the contents.
As we look to add tests that rely on specific sample payloads, we
need to change the tests to explicitly set the payload for each
sample.
Change-Id: I24174686f46535cf6c2d59a18308101a3bb51c87
EPT (earliest presentation time) may be adjusted not to be lower than
the decoding timestamp (dts), but the adjustment should only be done
on the first file when there is one file per Representation per Period.
The second file and onwards should not be adjusted otherwise a GAP
would be created.
Closes#384
Closes b/78517422
Change-Id: I56771ad8fbbe6a87b832ec58854cfbf37d5f1817
In our text to mp4 tests, we were only checking if the times on
the samples lined-up with what we were expecting.
We want to check that text sample contents (ids, settings, and
payloads) were correctly merged into a single media sample. To
verify this, we now check if the sample ids appear in the
media sample.
Change-Id: Ica1a85a14e7b116275e3571332b2e90d7bc44c45
In our text to mp4 tests, we used the same sample id for each sample,
this changes it so that each sample (within a single test) has a
unique id.
This is done in preparation to look for the ids in the created
media samples.
Change-Id: I3215a6f09279af8f40e1ce8a959e0a522a811173
The previous text to mp4 webvtt pipeline was incomplete. It
did not insert ad cues and it could only insert a segment
after a sample ended.
Now the pipeline supports ad cue insert and segment insertion
mid text sample. This required the pipeline to use the text
chunker (to split samples and insert segments) and required
a major overhaul of the text to mp4 converter.
Before the converter came before the chunker. This meant that
the converter only expected to see stream info and text samples.
Moving the converter after the cue aligner and chunker means
that the convert had to be aware of segments and cues.
The general approach is the same, however the converter will
convert the samples per-segment as the chunker will introduce
duplicate samples if a sample spans across segments.
Closes#362Closes#382
Change-Id: I0f54a40524c36a602ad3804a0da26e80851c92fd
Allow the output file to contain template identifiers like $Number$,
$Time$ etc. Unlike the actual template used in SegmentTemplate, the
identifiers in output will be populated before pushing to the DASH
manifest.
Issue: #384
Change-Id: Ife1caadb6fccd32167fa1bc83fe2afcb2d2ad087
- Add command line flag --test_packager_version to inject packager
version; packager_main.cc is also updated to use the same flag.
- Also add a MpdGenerator test in packager_test.py.
Change-Id: I9a11a0ee5502ba30a8acc4d44ebbfaabbe0f2f6e
This is needed to support one file per representation per period where
there will be multiple content files generated.
Issue: #384
Change-Id: Ib7af750edf864d99075b8da4f3640217a5a94302
In the text sample box (for mp4) there was a value called
"data_reference_index" that was never initialized. This meant
that it took on various values can caused different results
between runs.
Change-Id: I4b18ac97ec4700f6e651b14898ef250713a4253c
Renamed all the files called "webvtt_output_handler*" to
"webvtt_text_output_handler*" to better reflect the class name
in them.
Change-Id: I977bab362076974a124f263bcefff716ed8b6a0f
The SetUpAndInitialize function returns a status. We were not
asserting that it return an OK status. This change adds the
missing ASSERT_OK calls.
Change-Id: I044f60c0450d46fc0f6495319b1ed7ef8ab6e794
We don't want to allow any handler to be copyable or assigned-over
so this change enforces that for the webvtt output handler.
Change-Id: Ie0d59d6dbfb7a5e00bb4dd1422cd696d1a2d6072
Before, the webvtt output handler was written so that it could
share code between a segmented and non-segmented handler. As
we are not worried about that right now, this change simplifies
the handler to just be about segmented output.
Change-Id: I29dbc4e3a4ffbeb7ea10e23db489ee74b398a6c4
Previously for the last iframe in a segment, we wait for the next
segment to arrive before writing the EXTINF tag. If an Ad Cue comes
in before the next segment, the EXT-X-PLACEMENT_OPPORTUNITY tag would
be inserted before the iframe in previous segment.
Fixes#378, #396.
Change-Id: I1ede72a4d4edca94781c7b05bc25397d67916d1a
Instead of failing immediately, ignore unsupported audio codec when
parsing the source file, as there may be more than one stream in the
source file. This allows the supported streams to be packaged.
Closes#395.
Change-Id: I01005a93a19012c19065251647c9b06dd25c673a
The Cue class was from a previous WebVTT implementation and
is not used in the current implementation. It was missed when the
other classes were removed. This change removes it.
Change-Id: I661ab3fcd80b5e5ef98b5213746b341a4028d1a1
- Added a new --enable_entitlement_license flag, which sets
'enable_entitlement_license' in Widevine CommonEncryptionRequest;
- Support 'boxes' in Widevine CommonEncryptionResponse.
b/78171767
Change-Id: Id399fc7fcb2948c571e12c8af7687cfcfcef41fe
Requiring output format determined from 'output' to be consistent with
output format determined from 'segment_template'.
Change-Id: I32cbd63fcd6e2a4272dd0db531c1d5b385315445
When originally implementing the webvtt parser, there was a
misunderstanding in what the BOM was suppose to be
(https://en.wikipedia.org/wiki/Byte_order_mark). This corrects the
misunderstanding.
Close#397
Change-Id: I250d392db228e5e9b86684614b57adc5d8a4e5fe
Make sure to use the output format when given when setting up our
tests. Not doing so results in text always being set to "vtt" when
the output format is "mp4".
Change-Id: I11c5f861091598a67fc76dc19b1b16a9a773a2e0
Problem : Text samples have variable length and therefore act
more like continuous samples whereas audio and video
act more like discrete samples. Since we use sample
start time, a cue event could be inserted after the
start time of the last text sample and never get
inserted as there are no more samples.
Change : After all streams have requested flushing, we make sure
to collect all remaining cue events from the sync point
queue and insert them into each stream.
Issue #362
Change-Id: Id8f136f7ef53531f7a7f412613eac352324e0130
Create an end-to-end test for ad cues. This test's final result is not
correct but illustrates the problem we have in the cue insertion and will
be fixed by a later CL.
Change-Id: Ia8b43a53848941be52cf9ade018668e6477e8df2
Uses protobuf JSON util instead.
packager/base/json_writer comes from Chromium base. We are going to
replace it with abseil, which does not have a JSON library.
The code is much more cleaner now.
This is unfortunately, at the expense of increased output binary size.
packager binary increases by another 100KB.
Change-Id: I83a9217a484cad3c41147ad9a75311384347c49c
Use protobuf utilities instead.
packager/base/json_writer comes from Chromium base. We are going
to switch to abseil which does not have a JSON library.
This is unfortunately, at the expense of increased output binary size.
packager binary increases by about 300KB due to:
- Protos cannot be compiled with LITE_RUNTIME any more.
- Additional protobuf library needed to convert between JSON and proto.
Change-Id: I45a497376925b42d147ffcaabcfc2fa4dbdeacc1
In the cue alignment handler, instead of storing just one cue, store
a full queue. This will make it easier to handle text streams that
end before audio and text.
Change-Id: Ida97008fa015639350261bd3f76f4cb901747c66
In the cue aligner, instead of storing the cue event, store a
stream data so that we don't need to know the stream index when
sending the cue event downstream.
Change-Id: Ice27da021fad2872e2a23975b959630a9d43b736
This is in preparation of supporting entitlement license API, where
common encryption server may return concatenated PSSHs directly.
Refactored ProtectionSystemSpecificInfo into a struct containing
concatenated PSSHs. This will make it easier to pass PSSHs around.
Also, most of the time, users of ProtectionSystemSpecificInfo do
not care what is in PSSH; so moved PSSH box parsing and building out
of ProtectionSystemSpecificInfo.
b/78171767
Change-Id: I1c4d5e7e23efd2f7d4b2b9704378323112e47f00
To make it easier to understand what a video stream and a non
video streams is doing in the cue aligner, each stream type is
given their own functions.
Change-Id: I8b8ca403721bcb06ca3056004420902667a30f6c
Use a common hint for all stream states as the hint is always
updated when we get a new cue event. Cue events are only gotten
when be pass the hint, so there should only need to be one hint.
Change-Id: I0838110b9b10325a9e99f8fca0b11f0a6b48f8a0
To allow generating Widevine / Playready PSSHs if the corresponding
PSSH generator is specified.
Note that Key Rotation with RawKeySource is designed to be used for
testing only.
Change-Id: Icaf9e74955c082a7b000bd6a08f4561f2e01a2e2
By having 'disable-clang-format' in commit message.
If you have the script setup as git pre-commit, it can be disabled
by '--no-verify' option.
Change-Id: I6fb358e85105255fddde41f950e986c74b7defc9
Problem: Sending samples to the cue alignmenet handler did
not reflect what was more likely to happen. In our
tests we would send all the samples for one stream
then all the samples for another stream. This created
some special cases that would either not happen in
reality or miss cases that would likely happen in
reality.
Changes: Changes all tests to dispatch samples in an interlaced
pattern that better reflects muxed content.
Change-Id: I985092154b62eb12d95499663d195ca6c103bc19
In H265Parser::ParseSliceHeader, the parser does not handle
byte_alignment() from the spec. byte_alignment() reportedly contains
at least one bit, which is not handled right now.
See Section 7.3.2.12 Rec. ITU-T H.265 v3 (04/2015).
Also added a few size sanity checks in H265Parser to make sure the
code does not crash if an invalid input is provided.
Fixes#383.
Change-Id: I33b31396058fc5ba67a0fc119be5fe56ec9443b0
Packager uses ThreadedIO to write media segments and manifest /
playlists. There was a possibility that media segments write being
delayed and scheduled after updating manifest / playlists.
This CL fixes the race condition.
Also added a note on how segments can be synced to cloud storage to
avoid the race condition during file sync.
Also added a live WebM test.
Fixes#386.
Change-Id: Icf9c38cdec715fa3dc2836eab1511131e129fe41
Fixes#387.
Note that the output will not play in Chrome until the Chrome bug
https://crbug.com/837832 is fixed.
Change-Id: Ic3e917161cedfa773c0a18b4a5d7b1254c6f1313
The number of preserved segments outside live window can be
configured using flag --preserved_segments_outside_live_window,
which is default to 50, i.e. 5 minutes for 6s segment.
Note that the segment removal will be disabled if it is set to 0.
Only HLS live playlist and DASH dynamic MPD are affected by this flag.
- Also add end to end tests.
Fixes#223.
Change-Id: I8a566efebe2f1552c7d9509ab017bade5a4a1c98
Problem: The time scale for text was being set to zero in our tests
because it was never needed.
Change: This changes it to use MS as that is what's most common and
requires the least amount of changes.
Change-Id: Ia046ac1994b4cede079d2f801275c7f058d5bdd3
To move us toward no longer need to ensure order when building
our pipeline, use a map to share demuxers between stream descriptors.
This will even allow use to use the same demuxers in the text pipelines
while still building them separately from the audio and video streams.
Change-Id: I4d4dbddbc06adee36cbe7f4aa1f6769f7bb2a3f6
It is not always possible to align segment duration to target duration
exactly. For example, for AAC with sampling rate of 44100, there are
always 1024 audio frames per sample, so the sample duration is
1024/44100. For a target duration of 2 seconds, the closest segment
duration would be 1.984 or 2.00533.
This feature allows MPD generator to treat these segments as having
the same duration, thus allows MPD generator to generate less
SegmentTimeline entries and potentially no SegmentTimeline entries
(replaced with SegmentTemplate@duration instead if
--segment_template_constant_duration flag is enabled).
Under flag --allow_approximate_segment_timeline. Disabled by default.
Fixes#330.
Change-Id: I5044eaa348ebbf45bf792a2af53fc95a115ae21b
- Allow including Widevine and Common SystemID PSSH boxes
for PlayReadyKeySource.
- --playready_key_id and --playready_key flags are deprecated.
- --enable_raw_key_encryption already supports playready PSSH generation.
Addresses issue #245
Change-Id: I072d4f43a3239875959e4c5b1eb6854415d7367e
To ensure that we can parse content with style and region blocks,
this change updates the parser to skip those blocks so that we
can still parse the cues from a file.
Full style and region support will be added later this year.
Issue #380
Change-Id: I11b8fd862a108c27a5c67b15d4703532b44a1214
Removed the logic in MuxerListener to estimate bandwidth from file
size and duration, since it is not compliant to the spec.
MpdBuilder will estimate bandwidth from segment size and duration
if bandwidth is not specified in MediaInfo.
Here is the statement from DASH spec (23009-1:2014):
Consider a hypothetical constant bitrate channel of
bandwidth with the value of this attribute in bits per second
(bps). Then, if the Representation is continuously delivered
at this bitrate, starting at any SAP that is indicated either by
@startwithsap or by any Segment Index box, a client can
be assured of having enough data for continuous playout
providing playout begins after @minbuffertime *
@bandwidth bits have been received (i.e. at time
@minbuffertime after the first bit is received).
For dependent Representations this value specifies the
bandwidth according to the above definition for the
aggregation of this Representation and all complementary
Representations.
Fixes#376.
Change-Id: I0fddce39e709d0cded0a4c9ae59adbbcc97ec5ea
And for HlsNotifier and SimpleHlsNotifier as well.
This will make it easier to add preserve_segments_outside_live_window
param in a later CL.
Change-Id: I86d464fe247e04574158a0a76e39d8a122960ae4
The file_name fields will be used to solely indicate file paths on the
designated file system, and they are used to do normal file operations,
including file creation, file updating and file removal if needed;
added new xxx_url fields, for the URLs that should appear on DASH
manifest or HLS playlists.
xxx_url are the URIs of the media in the manifest. The fields are
converted from file_name fields but adjusted to be relative to DASH
manifest path or HLS playlist path, optionally with base_url prepended.
Previously the file_name fields are converted in place to indicate
URLs when passing to manifest / playlist builders. The original file
names were lost, which made it difficult to remove files outside of
live window.
Now that the input file names are preserved. File system APIs can
operate on the original file names while manifest / playlist generation
functions can operate on URLs.
Issue: #233
Change-Id: I36a64f16e3d1261ce91783a86588f24ad1371662
According to DASH spec (23009-1:2014):
Consider a hypothetical constant bitrate channel of
bandwidth with the value of this attribute in bits per second
(bps). Then, if the Representation is continuously delivered
at this bitrate, starting at any SAP that is indicated either by
@startwithsap or by any Segment Index box, a client can
be assured of having enough data for continuous playout
providing playout begins after @minbuffertime *
@bandwidth bits have been received (i.e. at time
@minbuffertime after the first bit is received).
For dependent Representations this value specifies the
bandwidth according to the above definition for the
aggregation of this Representation and all complementary
Representations.
This suggests that max bitrate should be used instead of average
bitrate.
Also cleaned up BandwidthEstimator code.
Fixes#376.
Change-Id: Ibf5896394c5c6bb820849771a2129c59202d2273
Content-Type for Widevine key request was incorrectly set to text/xml,
but it should be application/json.
Also added VLOGS for curl calls.
Fixes#372.
Change-Id: I4230795a582112c6d9c12883b5e61481b63284aa
Two-character ISO-639 code in --default_language was ignored due to
a bug in language code matching as the language code in stream is
always converted to 3-character code.
Fixes#371.
Change-Id: I8618938af583a417446636ff9efe1c72ce822c33
This flag was introduced to workaround a rounding error in Chrome
(probably in other browsers too).
Also although this flag avoids the first frame of a Period to be
dropped due to rounding error but it could cause the last frame of a
Period to be dropped.
Now that we use a high precision Period@duration, we do not expect to
see rounding errors any more. The player would be a better place for
the workaround even if it is still needed.
Related issue: #368.
Change-Id: I3bd517ecc6d548ff62e0c13394edb49d4bc68e8f
Instead, the actual earliest presentation time is used except for
the first segment if there is an offset between presentation time
(pts) and decoding time (dts).
Chrome (as of v66) reports dts instead of pts in buffered ranges in
MSE API. To avoid breaking Chrome, the earliest_presentation_time
of the first segment is set to its dts as Chrome does not like negative
values for
adjusted dts = dts + Period@start (0 for the first period)
- presentationTimeOffset (earliest_presentation_time).
Fixes#303.
Change-Id: I5ca80e05d5570961400499436f2bcc01f06e69e0
The WebVtt Output Handler did not recognize cue events. This change
allows the handler to accept the events and tell muxer listener
about them.
Issue #362
Change-Id: I7c3318b72e539adc19af587c8e213fdb0af8290b
UTCTiming schemeIdUri and value pairs can be provided to packager using
--utc_timings flag. It should be comma separated list of
schemeIdUri=value pairs.
Note that urn:mpeg:dash:utc:direct:2014 scheme is not supported as it
requires the MPD to be dynamically generated on the fly when MPD is
served to client.
Fixes#311.
Change-Id: Ibc07af8a6d8b2b6261ba3ecd2c02f23809f96614
Chrome internally uses time accurate to microseconds, which is
implemented per MSE spec (https://www.w3.org/TR/media-source/).
Generate Period@duration with better precision to avoid possible
buffered range gaps in Chrome (possibly other browsers too), which
may lead to other problems like playback stall.
b/74238961
Fixes#368.
Change-Id: I357a0f62b67f75c7ca044bb99ea4e3c8bbb6fecd
Created "Chain" function to connect a series of handlers together so
that the connection is easier to read.
Changed other calls to use RETURN_IF_ERROR to help make it easier to
read as we no longer need to track the status variable anymore.
Change-Id: Iffb76ca395b6d62f8feb054c09470c77718e8feb
Created a media handler to come after parsers that will handle filling
in gaps between text samples. The padder takes a min duration, and if
the samples do not cover the min duration when flushed, one last empty
sample will be injected so that the samples will go up to the min duration.
Change-Id: I88605059664d09279676edac418ff3d4990d7556
Updated TextChunker to handle incoming CueEvents. Connecting the
text chunker with the cue alignment handler will happen in a later
CL.
Issue: #362
Change-Id: Ib1fa9f457cf4ec0ce413dadcfa7eed5895ecd628
Move the webvtt segmenter to the chunking directory so that it
can be converted to a general purpose text chunker.
Change-Id: I9ecd7ee39cb73070dab07b64f65ef24af1404813
Now that we have the end-to-end tests, we no longer need the webvtt pipeline
tests to verify that it is working.
Change-Id: I4ebec34e66eda67c40999d8802b447e2551e1fa6
It is now managed in SimpleMpdNotifier.
This avoids unnecessary increment in AdaptationSet id counter.
Also makes sure the AdaptationSet is sorted by id in XML output.
Change-Id: Ibcd0b047a71c19cd30ad7d8af9a2ed0bb05e043e
SyncPointQueue manages all cue points and returns aligned cue
points to the callers (CueAlignmentHandlers).
CueAlignmentHandler is responsible for aligning cues from different
streams. It uses SyncPointQueue internally to align / synchronize the
cue points.
Issue: #355
Change-Id: I281fecb46a3ca7172d71e7495bdd07b8efdeb283
This changes almost all encrypted tests to use DiffDir. The remaining
un-updated tests are tests that won't match every time.
Change-Id: I858bbaa0beea34887cc3100eb784aa2aecd5a0a5
Change MediaInfo media paths in test output to only use the filename so
that they can be compared between test runs.
Change-Id: I26bb3f5ce79a83400b8a162c61cf5b54e41f9c70
iframe playlists will be given a name based on the output name of
the main stream. This is done to ensure unique filenames between
different video output streams.
Change-Id: Id0de9c5834599e9d896243e30569c55f34e7cd68
Instead of using the test file's index in a list to create the output
name, use the original filename and the descriptor.
This caused some problems with file name collisions when some tests
were using the same name. That was fixed by changing the names. This
will go away once they are transitioned to use DiffDir like the other
tests.
Change-Id: I0a4c480406705ca63fcea61c86c67d4a5f739295
This is called when reaching end of the file of the media. The duration
of the media is updated.
Fixes#340.
Change-Id: I446f2d341b02125d4a7d8c958bda269b5403cb9c
Created a method to compare test output for live tests. This wraps
CheckTestResults but first replaces the times that would diff between
manifests.
Change-Id: Iafb51a28a9bcb9f32b210c6d76bf23e2b9a3e0d6
This require a change to the update gold method as it would not
create nested directories. Instead it deletes the old directory
and copies the whole new directory over. This made the code simplier
and far-less error prone.
Change-Id: I1da5d9bda171b0f106d1425b204139dfbcb39b42
To make working with file names easier in packager_test.py _GetStream
the full path is not created until needed so that multiple paths can
use the same root filename.
Change-Id: I23a9231433b22ab5cef9b88bd6359f351da56eda
Changed all output file names (not paths) to follow the pattern that
any qualifier (e.g. trick play) will be join the current name using '-'
but will use '_' within itself (e.g. trick_play_1).
Change-Id: Ib0247bf1ca6d94815fedaaf73d3a400d31c20c40
We have an assert that ensures that the end time is greater than
the start time for any cue. However we never checked that cues
had a non-zero duration when parsing them.
We will throw away cues with a duration of zero (and print a
warning message) as they are not spec compliant.
Closes: #335
Change-Id: I404e8f3a5a8d43eff75a2554db3e38e8d340f421
In the video captured by Android's default camera app, meta box is written as
as Box instead of FullBox specified in the spec.
Closes#319.
Change-Id: I526492fdd505d5929c5161cb1ed1503b724de7e9
Prefer timestamps from Video AdaptationSets if available - this avoids
possible video playback jitters due to gaps.
presentationTimeOffset is not applied to the first period as it may in
negative dts which Chrome does not like: https://crbug.com/398141.
It is safe to apply to subsequent periods as the actual offset applied
takes Period@start into consideration:
offset = Period@start - presentationTimeOffset
The result timestamp with offset applied is close to Period@start, so
it is unlikely to result in a negative dts value.
Closes b/73899306.
Change-Id: If8361f5469610093b3aac6675754536ad7e83c4c
This change has the hls master playlist add the text codec string
to the list of codecs in a variant.
Bug: 72942756
Change-Id: Ib25bb2064a291d10d7b1d261a4307991df62220c
Updated as many tests as we easily could. The tests that were not updated
all use live manifests or encryption that all require some "help", either
by dealing with times differing or with verifying decryption.
Bug: 73830478
Change-Id: I6803c2d960b71b459eb57b7a5e562164bb713e2a
Created GetStream, a function used to get the stream descriptor for a
single stream. This allows for finer control over the stream compared
to the batch GetStreams.
Added "trick_play_factor" to GetStream so that the trick play factor
did not have to be added the stream descriptor value and "injected"
into the stream.
Change-Id: Ifbde64b961a673aafa5f6d21f33a68d02fb46610
Created end-to-end tests for the segmented webvtt text in HLS. The
text stream is longer than both audio and video, so there are more
text segments than video/audio.
Change-Id: Ib860ccd3f2ac0048983fdeebe3847a0b9dcbc160
Created a method to diff the output dir against the golden copy
of the output dir. The goal with this is to simplify our end-to-end
test structure by removing the need to track the output files within
the test runner.
With all the golden output in one directory, this makes it easier to
setup end-to-end tests and even allows the golden files to be playable.
This should not affect the repo size too much as duplicate files will
only be stored in the git object store once.
Tests will migrate to this in separate CLs.
Change-Id: Id1be9f8f60c9e362b9b615445dd8ca7da996af4c
Changed the "live" setting in _GetStreams to be "segmented" so that it
make sense in a more general sense.
Change-Id: I2ceaad67653e1e2237533b68fccbc9fe3fabfc9b
When converting from NAL unit stream to byte stream.
The packager should not assume the parameter sets in the sample is the
same as the parameter sets in sample entry (decoder configuration).
Fixes#327.
Change-Id: I7e84d28a296f4b33db0523cca5eabd62f623e852
Looks like Safari does not like v0 tenc box.
- Use v1 tenc box for cbcs and cens protection_scheme as required by
CENCv3 spec.
- Set crypt_byte_block and skip_byte_block to 0 for full sample
encryption cbcs and cens.
Fixes#326
Change-Id: I5581cd856fffc4ff104d950f3ca19b9337d57a78
Create a StreamDescriptor class that can be used to build the stream
descriptor command line argument.
Change-Id: Ifcba4f37d69a1a36c4add51a2a10c3f0c8b2ccd7
The old vtt sample data has the header as "WEBVTT FILE" which does
not reflect what the spec says it should be.
This changes it to "WEBVTT" as described in the WebVTT spec.
Change-Id: Ib54396a8b937501a835dafe966bf05b95b0b22c4
When a gap is found in the text stream, the WebVtt to Mp4 converter will
now output the special empty vtt cue.
Change-Id: I8be88c6b7589aa120a2215e1e4b8e98031fe326d
Closes: #324
- Add empty lines between different types of renditions to improve
readability.
- Group variants with the same audio/text group together, as it is
where the Adaptation occurs.
- Write master playlist after writing media playlists. This makes
more sense and it is also necessary to have the bandwidth of
the last iframe playist segment correctly computed.
- For fMP4, I-Frame segment must include the 'moof' header.
- Fix a problem that hls_iframe_playlist_name is not passed to
MuxerListenerFactory.
Issue: #287
Change-Id: Icf37c5de1dc29f85ae3f419cbc3264d04ca491a4
Adding all the code needed to allow the webvtt to mp4 converter to
send empty vtt cues. This change does everything except writing the
mp4 box.
Bug: #324
Change-Id: I16188e6357632b2ed06f7e9bab7844f093266696
Instead of always tracking segments with an index, this change makes a struct
that will act as the segment and track which segments are in it. Each
segment will store all the samples in the order they were given, it will
avoid any sorting.
Closes: 72867775
Change-Id: Ic5829161510fe8f3320d960c3bc4a276c26ff3be
Create WebVtt test that shows that the current implementation of the
webvtt segmenter does not respect the input order of cues.
Bug: 72867775
Change-Id: I811b3a93c10650e0cf9290c9c0c1680f562deb30
Took the same logic for DEFAULT and AUTOSELECT used by audio and
applied it to text. Combined the build tag logic for audio and
text as they were the same expect for a couple fields.
Bug: #205
Change-Id: I75ecbf4b25cd559b826982d12a5b132e70b83b69
The master playlist and media playlist did not have implementations
for handling text streams. This change adds support for both.
Bug: #205
Change-Id: I1329b8cc2585f15b89959071db9dd16d35847cba
If no hls group name is given, we would default to "audio" but as
we are working to support text, this won't make sense to see
GROUP="audio" on a text stream.
Instead default to "default-audio-group" for audio streams. A specific
text default will be added later.
Bug: #205
Change-Id: Iefc8bfd35708286619d0004348294d98f2c38482
In the master play list, use a class called variant to build
the video tags.
The variant list will always have one entry. In the case that there
are no audio streams, the variant list will have a null entry.
Bug: #205
Change-Id: I6e4acd83a31cd267c173e4f4c910f93781fe6efd
According to the HLS+WebVTT spec, if there is no text in a segment
a webvtt file with no cues can be added to the manifest. By outputting
empty segments it allows the accumulated duration in the Master Playlist
to better represent the duration of the text stream.
Spec Reference: //tools.ietf.org/html/draft-pantos-http-live-streaming-23
Section 3.5. WebVTT
Bug: #205
Change-Id: I5de01200fd9fa99c57949c773e8ee926b0f6ba8a
It was the same code to make the media playlists for each test, so this
change moves that logic into common functions.
Change-Id: Ifb8680247ad4335332251815a2db5d56d28151aa
- Removed SetError, Swap, Matches functions, which are not used
- Added moveable version of Update function
- Added VLOG(1) warning in Status for non-ok status
- Replaced OS_WIN with _WIN32 as OS_WIN may not have been defined.
Change-Id: Ib6b7aab6e6fee270937b150f1e4bf993e914a568
Moved the file copy out of StreamInfoToTextMediaInfo as the file
copy is only needed by the MPD output but the Media Info is needed
by MPD output and media info dump.
Bug: 36138902
Change-Id: Id233f2041b3e72345b8f709791c6b6070484222f
The IV was incorrectly updated across samples.
Created a new cryptor SampleAesEc3Cryptor specially for SAMPLE-AES
EC3 encryption / decryption. The new cryptor uses constant-iv, and
makes sure the IV is reset to the initial value at the beginning
of each audio frame and chained across syncframes within the the
audio frame.
Also added E-AC3 end to end test.
Fixes#279
Change-Id: I0aa60c17836daeef5ba433a05e5ff0906191d9ac
There could be rounding errors in MSE which could cut the first key
frame of the representation and thus cut all the frames until the next
key frame, which then leads to a big gap in presentation
timeline which stalls playback.
Adjusting presentationTimeOffset (PTO) by -1, i.e. backing off PTO by 1
to compensate for the possible rounding error. It should not cause any
playback issues as it is small enough. The workaround can be removed
once the problem is handled in all players.
The PTO adjustment is configurable with command line flag:
--pto_adjustment, with value set to -1 by default.
b/71808910
Change-Id: I9c4d1e2ee84008b859a3638a8146c910cead1f15
Instead of using next segment start time, as CueEvent time may not
align with segment start time exactly.
Also remove the incorrect DCHECK in ChunkingHandler when processing
kScte35Event.
Change-Id: I4987740c99c8d0d25c9b99bddc5e557e45d308e0
According to HLS spec: https://goo.gl/MiqjNd 4.3.4.1.1. Rendition Groups
- A Group MUST NOT have more than one member with a DEFAULT attribute of
YES.
- Each EXT-X-MEDIA tag with an AUTOSELECT=YES attribute SHOULD have a
combination of LANGUAGE[RFC5646], ASSOC-LANGUAGE, FORCED, and
CHARACTERISTICS attributes that is distinct from those of other
AUTOSELECT=YES members of its Group.
We tag the first rendition with a particular language in an audio group
with 'AUTOSELECT'; it is tagged with 'DEFAULT' too if the language
matches --default_language.
Fixes#315
Change-Id: Iacc0bc8c89ebffce8717fa65e82d6daf5a1f6adc
When transmuxing from TS to MP4 if there is emulation prevention
bytes in general_profile_tier_level_data.
Fixes#312
Change-Id: I48857046c1ea4bed4f455d03d060551e3ef4bec8
Added new path to allow webvtt text input to be written as
webvtt mp4. This will happen when the input is a vtt file and
the output path is a mp4 path or the output format is mp4.
Missing test content for this.
Close: 70990714
Change-Id: Idbb4023db4a126a560528e73801f762cc45f4685
Moved muxer listener creation into a factory so that it can be
initialized once and then the factory only needs to be passed
around rather than all the parameters.
Change-Id: Ibf7df66c08debddbe6f6ff8995e7910b9ad17f3f
Use a mock muxer listener so that we can verify that the webvtt
output handlers will write to a manifest correctly.
Change-Id: I0294a998bfaf06a6d8f7d4c287fb014839b38f73
To remove the direct dependencies between packager.cc and the different
muxer implementations, this change remove the muxer factory to its
own file.
Change-Id: I4a1503c493f0f4d10a67e78461f96da3f4238944
Connected the WebVtt Pipeline together to test that the parser,
segmenter, and output all work together as expected.
Change-Id: I07138f7b5b1318f84c27c5b607d8df207c57ddb3
Also added Period::GetAdaptationSets and
AdaptationSet::GetRepresentations.
Instead of implementing GetDurationSeconds in all MpdBuilder classes
(MpdBuilder/Period/Adaptation/Representation) like what we used to do
with GetEarliestTimestamp, the two new functions allows MpdBuilder to
iterate through the Representations to get durations.
Also updates GetEarliestTimestamp functions to use the same iteration
method.
Change-Id: I682b70c07c248c0f6511ec3d9019086f986ee10e
Linked together all the handlers to create the HLS text pipeline.
This change does not include tests for the pipeline.
Bug: 36138902
Change-Id: I32e7a7cefae1e7c74ec6d0bdc6335c0d65e5e79c
Changed muxer listener to use "default" instead of {}
when defining its constructor and destructor in its
header file.
Change-Id: I3c80e808aa624293c3d7720ac6a47b470a663312
This change adds the missing code for passing mp4 text through
the pipeline.
This introduces the MuxerFactory to simplify creating muxers.
Change-Id: Ic7be1a2e0ff4b720a01061edb43aded946d05a47
- Update curl to 7.57.0
- Roll clang, which is needed due to MacOS / XCode update
- Fixes and suppress compilation errors due to clang update
Fixes#285
Change-Id: Ibac3288c641861605c3c0500d34d27373e6eecfe
This change introduces handlers to output WebVtt text files. There is
only one output but there is a common base to support others.
WebVttOutputHandler which handles all communication with other handlers
and WebVttSegmentedOutputHandler is responsible for listening for events
and choosing when and where to write cues and headers.
Bug: 36138902
Change-Id: I2b13a94262554398e66fee8cf024aa21041ddbab
Instead of having multiple char readers, one for strings and
one for files, just have one for files and use memory files
when testing.
Change-Id: Id1a2230046ba540ddf69ca10edb3edc74d2419b6
To better align with the Chunking Handler, the Text Segmenter
now outputs the SegmentInfo at the end of the segment rather
than the start.
Change-Id: If69ab951947d00779b4b63a52c4b6662bbdc4c0d
Move AdaptationSet related functions to the new Period class, which
maps to <Period> element and provides methods to add AdaptationSets.
Change-Id: I0fee290769fbe9a6355cc1b8c86baec8fbc4b4fd
Clean up DashIopMpdNotifier to make it easier to add Period class.
- Consolidate protected AdaptationSet grouping related logic to the
internal ProtectedAdaptationSetMap class
- Clean up AdaptationSet creation logic and updating logic
Change-Id: Idd5b39bb89d38e490ca4f561a6b840ccc9f1e40a
Implemented a MediaHandler that takes text samples and creates
media samples. The data in each media sample is the MP4 box for
non-overlapping cues.
As per WebVtt in Mp4, all cues must be non-overlapping. This handler
takes care of grouping and dividing cues.
Bug: 36138902
Change-Id: I0c1d27964180c14a22cb200591f70e46e04a651f
- Add --time_shift_buffer_depth to HLS doc.
- Fix a problem in HLS tutorial
Partially address #304, fixes#305
Change-Id: I60bc1eb659a30beb0a6ab73433f4cc99a7deab07
This change creates the webvtt segmenter which will be used for
segmented text webvtt for HLS. It takes a stream of text samples
and injects segment info messages based on the segment duration.
It is possible for samples to extend between segments, if so the
sample will appear twice (once in each segment).
Change-Id: Iae0134ee61cf269948026086520b6d3f8ce3785b
Took the WebVTT Media Parser and created the WebVTT Parser
that will take in a file and output a stream of cues that
will later be passed to another Media Handler that takes in
cues and chunks them.
Bug: 36138902
Change-Id: Ic77813fe19678e85d500269e69f46917510ab7ec
Moved from mpd_unittest.cc to representation_unittest.cc.
Also removed unused test data files in packager/mpd/test/data.
Change-Id: I23811390b4b1ffbfac3e69c1ffcf92544099a53d
Move AdaptationSet related tests to adaptation_set_unittest.cc and
Representation related tests to representation_unittest.cc.
Change-Id: I360e9aea3ba08a32e5ba3b0e8b34345f2b3f0b20
Added gMock matchers for matching xmlNodePtr and its attributes.
Clean up mpd_unittests to make it easier to separate adaptation set
tests and representation tests.
Change-Id: I31816b06e9c76f92d4a82656c659f3b9acae8cb5
Moved all code dealing with jobs into its own class so that the
packager code does not need to worry about initializing, running,
or stopping jobs.
Change-Id: I3e9ef1f22bd93d671f77d59ad15f23d1239078cf
As specified in HLS spec (https://tools.ietf.org/html/rfc8216):
All audio EXT-X-MEDIA tags SHOULD have a CHANNELS attribute. If a
Master Playlist contains two Renditions encoded with the same
codec but a different number of channels, then the CHANNELS
attribute is REQUIRED; otherwise, it is OPTIONAL.
Fixes#299
Change-Id: Ic2308c39b170178b11cb0d94c3a8083c8b5f3353
Seeing some failures on some platforms when compiled with clang
disabled:
GYP_DEFINES="clang=0" gclient runhooks
Several changes to make it work:
1. Mark packager code with packager_code=1 in GYP definitions.
2. Disable a few checks in non-packager code, which we do not have
direct control: dangling-else, deprecated-declarations,
unused-function
3. Fix the relevant errors in packager code.
4. Revert HAVE_STROPTS_H in curl config which is not available in
all linux distributions.
Fixes#286Fixes#293
Change-Id: I729b41f99403c5ad9487c6cc4a7dc06f6323cef8
1. Expect the time elapsed to be greater than kTimeout instead of
approximately equal to kTimeout if there is a "wait";
2. Expect the time elapsed to be smaller than kTimeout instead of
approximately equal to 0 if there is no "wait".
Also rename a variable in packager.cc to better reflect what it is.
Change-Id: I67975a6263b8dbc1124b78feae0f8e0d112bda50
KongQun Yang
Aaron Vaage
Haoming Chen
KongQun Yang
fatpelt
Michael Stillwell
Haoming Chen
Kamesh Devarakonda