- Update curl to 7.57.0
- Roll clang, which is needed due to MacOS / XCode update
- Fixes and suppress compilation errors due to clang update
Fixes#285
Change-Id: Ibac3288c641861605c3c0500d34d27373e6eecfe
This change introduces handlers to output WebVtt text files. There is
only one output but there is a common base to support others.
WebVttOutputHandler which handles all communication with other handlers
and WebVttSegmentedOutputHandler is responsible for listening for events
and choosing when and where to write cues and headers.
Bug: 36138902
Change-Id: I2b13a94262554398e66fee8cf024aa21041ddbab
Instead of having multiple char readers, one for strings and
one for files, just have one for files and use memory files
when testing.
Change-Id: Id1a2230046ba540ddf69ca10edb3edc74d2419b6
To better align with the Chunking Handler, the Text Segmenter
now outputs the SegmentInfo at the end of the segment rather
than the start.
Change-Id: If69ab951947d00779b4b63a52c4b6662bbdc4c0d
Move AdaptationSet related functions to the new Period class, which
maps to <Period> element and provides methods to add AdaptationSets.
Change-Id: I0fee290769fbe9a6355cc1b8c86baec8fbc4b4fd
Clean up DashIopMpdNotifier to make it easier to add Period class.
- Consolidate protected AdaptationSet grouping related logic to the
internal ProtectedAdaptationSetMap class
- Clean up AdaptationSet creation logic and updating logic
Change-Id: Idd5b39bb89d38e490ca4f561a6b840ccc9f1e40a
Implemented a MediaHandler that takes text samples and creates
media samples. The data in each media sample is the MP4 box for
non-overlapping cues.
As per WebVtt in Mp4, all cues must be non-overlapping. This handler
takes care of grouping and dividing cues.
Bug: 36138902
Change-Id: I0c1d27964180c14a22cb200591f70e46e04a651f
- Add --time_shift_buffer_depth to HLS doc.
- Fix a problem in HLS tutorial
Partially address #304, fixes#305
Change-Id: I60bc1eb659a30beb0a6ab73433f4cc99a7deab07
This change creates the webvtt segmenter which will be used for
segmented text webvtt for HLS. It takes a stream of text samples
and injects segment info messages based on the segment duration.
It is possible for samples to extend between segments, if so the
sample will appear twice (once in each segment).
Change-Id: Iae0134ee61cf269948026086520b6d3f8ce3785b
Took the WebVTT Media Parser and created the WebVTT Parser
that will take in a file and output a stream of cues that
will later be passed to another Media Handler that takes in
cues and chunks them.
Bug: 36138902
Change-Id: Ic77813fe19678e85d500269e69f46917510ab7ec
Moved from mpd_unittest.cc to representation_unittest.cc.
Also removed unused test data files in packager/mpd/test/data.
Change-Id: I23811390b4b1ffbfac3e69c1ffcf92544099a53d
Move AdaptationSet related tests to adaptation_set_unittest.cc and
Representation related tests to representation_unittest.cc.
Change-Id: I360e9aea3ba08a32e5ba3b0e8b34345f2b3f0b20
Added gMock matchers for matching xmlNodePtr and its attributes.
Clean up mpd_unittests to make it easier to separate adaptation set
tests and representation tests.
Change-Id: I31816b06e9c76f92d4a82656c659f3b9acae8cb5
Moved all code dealing with jobs into its own class so that the
packager code does not need to worry about initializing, running,
or stopping jobs.
Change-Id: I3e9ef1f22bd93d671f77d59ad15f23d1239078cf
As specified in HLS spec (https://tools.ietf.org/html/rfc8216):
All audio EXT-X-MEDIA tags SHOULD have a CHANNELS attribute. If a
Master Playlist contains two Renditions encoded with the same
codec but a different number of channels, then the CHANNELS
attribute is REQUIRED; otherwise, it is OPTIONAL.
Fixes#299
Change-Id: Ic2308c39b170178b11cb0d94c3a8083c8b5f3353
Seeing some failures on some platforms when compiled with clang
disabled:
GYP_DEFINES="clang=0" gclient runhooks
Several changes to make it work:
1. Mark packager code with packager_code=1 in GYP definitions.
2. Disable a few checks in non-packager code, which we do not have
direct control: dangling-else, deprecated-declarations,
unused-function
3. Fix the relevant errors in packager code.
4. Revert HAVE_STROPTS_H in curl config which is not available in
all linux distributions.
Fixes#286Fixes#293
Change-Id: I729b41f99403c5ad9487c6cc4a7dc06f6323cef8
1. Expect the time elapsed to be greater than kTimeout instead of
approximately equal to kTimeout if there is a "wait";
2. Expect the time elapsed to be smaller than kTimeout instead of
approximately equal to 0 if there is no "wait".
Also rename a variable in packager.cc to better reflect what it is.
Change-Id: I67975a6263b8dbc1124b78feae0f8e0d112bda50
It aligns with what a lot of other apps are doing, e.g. tsplay.
Also added notes in docker to adjust network setting if wants to use
multicast in docker container.
Change-Id: I666a8979cc041c904f7ff8e3e022f800c0830d5d
The error is logged only once to avoid log spamming.
Added definitions for the unsupported stream types for easy look up.
Change-Id: I097e2f05759bc84ef03f264cfabd2fb20da7c711
This patch also fixes a bug which was introduced in
https://widevine-internal-review.googlesource.com/c/packager/+/38040
where it was inconsistently assumed that AdCueGenerator would be MIMO
while connecting with other handlers and at the same time it was assumed
SISO in the AdCueGenerator class. Now we assume SISO everywhere.
Change-Id: Icd8c40b5ccfe0d98f47f09a60ea1635f74dceef9
The spec allows having more than one 'mdat' boxes even if it is not
used.
This is happening on some mp4 files in the wild:
http://www.sample-videos.com/.
Fixes#298.
Change-Id: I729cc94bee095560b5c4b3ee8511323f25f7ad5a
Implemented E-AC3 MPEG-2 Stream Encryption Format for HTTP Live
Streaming specified in https://goo.gl/1sgcwY
Issue #279
Change-Id: I36c1a05e3d0529ff810eaf52bdca45414baa93eb
Audio samples per frame for AC3 was not specified correctly earlier.
Also the number of channels is not correct if LFE channel is on.
Issue #165
Change-Id: Ibf20aa4c7aec43c07ec7cd394d631c537cb387dd
Implemented MPEG-2 Stream Encryption Format for HTTP Live Streaming
specified in https://goo.gl/N7Tvqi.
This change also moved ProgramMapTableWriter creation from TsWriter
to TsSegment.
Issue #165
Change-Id: Ia89dd16a5e6405706dd3d85ec6b6de580f7b13a7
The keys can be specified in --keys option, with the form of
label=<label>:key_id=<32-digit hex string>:key=<32-digit hex string>,label=...
There can be multiple "label=..." block. The DRM label can be one
of "AUDIO, SD, HD, UHD1, UHD2" or a custom label.
Mark --key and --key_id as deprecated. We can achieve the same result
with --keys=label=:key_id=<key_id>:key=<key>.
Also add a new optional field in stream descriptor: drm_label, which
is used to overwrite the internally generated DRM label for the stream.
If not provided, the DRM label is generated automatically based on
audio/video information, e.g. resolution.
Code changes:
- Merged RawKeyEncryptionParams and RawKeyDecryptionParams into
RawKeyParams.
- Make FixedKeySource accepts RawKeyParams as input.
Change-Id: Ic8c2f071cc71188e13f14bc6396fc2b3ffa5cac6
Problem occurs when the media playlists are in a sub-directory under
master playlist. If base_url is not set, segment URL should be
relative to the media playlist.
This only affects fMP4 as TS does not have init segment.
Fixes#253
Change-Id: Icddd9ed500d0a705e8b3260bfd4e916ecbba3f28
To help isolate the differences between audio/video and text in
preparation for adding the better text support, this change moves
all audio/video job creation and text job creation into their
own functions.
Change-Id: Ia71cf82ac147672b70c69123eeabdfc6b41c4a3d
This fixes shared_library not able to compile problem.
Also removed StreamDescriptorList and replaced with expicit
comparison to make the code clearer.
Also removed an out-dated DCHECK that mpd_notifier and hls_notifier
does not co-exist.
Change-Id: I3d04b8880741fecf0931a764cc12e50aa5e392b9
Use std::shared_ptr<const uint8_t> with a custom deleter to
represent MediaSample::data_ instead of std::vector<uint8_t>.
MediaSample::data_ can be shared by multiple MediaSamples and it is
immutable. A new data instance must be created if the clients want to
modify the underlying data. The new data instance can be transferred
to MediaSample using provided MediaSample::TransferData function.
This avoids unnecessary data copying.
Change-Id: Ib59785a9e19d0abb3283179b12eb6779ee922f79
Took the test structure for trick play and made a common test
structure that can be used for new tests. The next tests to
use this structure will be the new WebVtt pipeline.
Change-Id: I5faca306b8a2bd16b9f945d1ff5c336aed82bc35
This change create some matchers and text functions that
will be used by the upcoming webvtt work.
Change-Id: I2d7931d1bdabd761f74e8aee7524d5ecfb7cfdc4
Before the trick play handler would have one input and multiple
outputs. The normal media handler API for adding handlers had to
be ignored when setting-up the trick play handler.
This changes the trick play handler to have one input and one output.
It uses the standard AddHandler method calls. When initialized, the
trick play factor must be provided.
This included a run of 'clang-format' over all edited files.
Change-Id: I7b3cdf0a2472e2f89ab194867c4b984e26f18f24
To allow more tests to use the MediaHandlerTestBase and get access to the
helpful creation methods, this change moves the use of FakeMediaHandler
and setting up the graph to a sub class.
Change-Id: I90dd151e3c96d8fbe4bd02a1d6b11e66a279d95b
Created media handlers that make testing a handler easier by providing
methods to send messages from upstream and check messages downstream.
Change-Id: I7b3ba3a51bcf0d73bcaacd46b2dcbc16f87d8694
Moved build instructions and docker instructions to separate docs.
Removed command samples which are no longer needed as we have better
tutorials now.
Change-Id: I340c5653a6553158325970dd86d76fa11e69bcbd
In prep for changes to Trick Play, we want to make all messages
copy on write so that if the same message is sent to multiple
handlers, it is not possible for one handler to change the data
another handler is using.
Change-Id: I554166ca11c532412e4dfced5603972ca24dc2bb
Problem occurs when the media playlists are in a sub-directory under
master playlist. If base_url is not set, media segment URL should be
relative to the media playlist instead of the master playlist.
Also make sure the path separator is "/" in URL instead of "\" on
Windows.
Fixes#253
Change-Id: I8e933750276435d94dd01bfa53ee2bc050dfd193
Previously packaging completes successfully without any error
or warning.
With the fix, an error will be reported if write fails. It may
appear as "Cannot close file error" as we use threaded IO, which
could delay the error reporting until Close() call, so the user
of the File API needs to make sure Close() returns successfully.
Also fixed a deadlock in threaded_io_file if internal_file->Write
fails.
Fixes#160
Change-Id: I17f945150fb4021d2dcdbe784e557673f53ca583
Doxygen should run under src/ directory:
$ doxygen docs/Doxyfile
Outputs are created in out/doxygen.
Change-Id: I9dd1a9aa158851d37ff9a2e37f701757eb3e577c
To ensure that the style data is formatted correctly, changing the
set_style method to AppendStyle so that joining values will be
done in controlled setting.
Change-Id: Ia88e2d260b11339e6777ff018813e8aeb802a501
There are multiple functions we use to verify input from the user
and must of them would return a boolean expect for the top level
function which would return a Status. Each function would log a
message but to the user the error was not clear. This change makes
each verify function return a status so that the actual problem
will be surfaced easier.
Change-Id: I12ab43f8ca3a4d379b4309336644a70eb8cbbbe0
Implemented the other end of the webvtt timestamp parsing as it will
be needed when we write out WebVtt files. Also changed the name of
the parse function to be more inline with the writing function.
Change-Id: I1f36ddbbf80028732b4cb1b15e871dec17767f63
Inside of CreateRemuxJobs, we would take a status value from another call,
log the message, and return a boolean. The caller of CreateRemuxJobs would
then create status from the boolean.
Now we will pass the status value from the other call directly to the caller
of CreateRemuxJobs. So that failures can be better communicate in our code.
This includes a pass of clang-format over packager/packager.cc.
Change-Id: I071e7bad13f916e963641b9e53699573fe1060ed
Rewrote the RunRemuxJobs function to use waitable events so that we never
get stuck waiting for a job to finish before getting to a thread that
has already exited.
This should allow us to end execution when a job failed rather than
waiting for all jobs before the failed job to finish.
Closes#94
Change-Id: I413f62561a7a4cab83b8905e75986230b6c498bc
Created a handler called OriginHandler that can be used by the Job class
(previously called RemuxJob). Origin handlers represent the start (or
origin) of a pipeline (chain of handlers).
Change-Id: Ibd748ae0a932b6e0ebb879ea292fcb83c548214b
Support generation of fairplay key system tag: "com.apple.streamingkeydelivery" when --pssh
includes fairplay key system id:
// Unofficial fairplay system id extracted from
// https://forums.developer.apple.com/thread/6185.
const uint8_t kFairplaySystemId[] = {0x29, 0x70, 0x1F, 0xE4, 0x3C, 0xC7,
0x4A, 0x34, 0x8C, 0x5B, 0xAE, 0x90,
0xC7, 0x43, 0x9A, 0x47};
Closes#258
ImportantFileWriter::WriteFileAtomically uses mkstemp internally,
which set file permission to 0600, which is not what we want.
Update the code to not use mkstemp instead.
Also updated temporary file name logic to include process id in
the name so it can be unique across processes.
Fixes#259
Change-Id: I2d5a375925cf552bc0db5269f409d7522e63fca5
This avoids interference caused when two sockets are bound to the
same port in different multicast groups.
Fixes#241
Change-Id: Iba1b3300e3850f1f268886b7a49f5a3315d95b80
Derive file size from media ranges instead.
Also fix subsegment range reporting for WebM (internal variable,
no output affected).
Change-Id: I5f8152dff4c2cd5fbae5550992b86a669e278f7b
- The master playlist should have the display resolution instead of the
encoded resolution.
- Changed MediaPlaylist to return display resolution.
Change-Id: I162727d0bdeed0302518286e42a22b69a58fc4a3
- Use byte range format if segment_template is not set in MediaInfo.
- Change HlsNotifier::NotifyNewSegment() to take start byte offset.
- Change HlsNotifyMuxerListener to cache the segment information if
MuxerOptions::segment_template is set.
Change-Id: Ida1b8b9198acc5c43e3a5d9a641c78c81bc9a6ac
- Changed OnMediaEnd() to take (sub)segment ranges.
- Define a structure to pass the ranges.
- This requires muxers to "cache" the segment ranges but in most cases,
they already do it, e.g. 'sidx' for mp4.
Change-Id: I16b974c1f0f54ca658cc6dbe605efff84a3f52eb
In the original manifest updating process, the file was truncated first
before was written with new contents. There is a small chance that
the web server may read back empty file or incomplete file.
The new code makes the update operation atomic (by writing to a temporary
file first then replace the old file with the temporary file).
Fixes#186
Change-Id: I2fd564cb12b922b032c0e9f70d2132a5b12ff098
- Remove the TS key rotation not supported check as there is nothing
to be done for key rotation in TS for SAMPLE-AES.
- Fix IV updated problem in new segments even if crypto period does
not change.
- Avoid duplicate EXT-X-KEY tags if it does not change.
- Make EXT-X-DISCONTINUITY-SEQUENCE match with number of removed
EXT-X-DISCONTINUITY.
- Added end to end test for HLS with key rotation.
Change-Id: I73cb82e9f5575fcdf63ee643228efe78e6766302
Resolving TODO in packager/hls/base/master_playlist.cc that asked for
media playlists to be grouped when they are added.
Change-Id: I01c7687400bc74ce45f271d0242d1c3bf3610cf2
- This fixes the issue when one NAL unit has multiple
subsamples, which is allowed in spec and could happen
when the clear_bytes exceeds 2^16 bytes.
- The new algorithm updates the subsample entries in
following steps:
1) When NaluReader parses a new NAL unit, the
function splits subsamples and align them with the
NAL unit. It also finds the range of subsamples
for that NAL unit.
2) Drop or update subsamples of that NAL unit,
based on the type. Other inserted bytes are treated
as a new all-clear subsample.
3) Merge those subsamples if possible.
- This addresses #237.
Change-Id: I2cab47cd00d90e29ec52b90d4fde057c632193ac
Moved all the timestamp parsing code into its own class with its
own unit tests. This is to make testing easier.
Change-Id: I624472baba51dfa1254a9f4bae55ebb79e310855
The HLS manifests output by the packager did not have the RESOLUTION
attribute on EXT-X-STREAM-INF tags. This change adds the resolution
tag when resolution values are present.
Close#235
Change-Id: I15ca6de47cb34793cf6a940123d593261627baa4
Shared libpackager can be built by setting libpackager_type to
shared_library, e.g.
GYP_DEFINES='libpackager_type=shared_library' gclient runhooks
ninja -C out/Debug
will generate libpackager.so in out/Debug/lib directory.
Here is a few other changes to make shared_library builds and
tests pass:
- Add several test parameters to packager.h, which is needed for
testing.
- Create a protoc.gypi from build/protoc.gypi but depending on
protobuf_full_do_not_use instead of protobuf_lite, since we need
protobuf_full_do_not_use for text parsing and generation of media
info proto. Somehow shared_library build does not allow mixed use
of protobuf_full_do_not_use and protobuf_lite.
- Remove the use of LazyInstance in version/version.cc and use static
variable directly. This is because LazyInstance needs AtExitManager
which may not be easy to setup when calling GetVersion.
- Allow skipping testPackageWvmInputWithoutStrippingParameterSetNalus
with flag --shared_library, which is needed as shared_library build
does not support --strip_parameter_set_nalus flag yet.
Fixes#227
Change-Id: Iff05a50baa28134faa7218664c96114cb9e70329
Added a text sample type that can be passed between media handlers.
This will allow text samples to safely be moved between media
handlers before converting to their final format.
Bug: 36138902
Change-Id: Ic4946f774a7d37c43066b9ea46596d5c5f3c05a8
frame_duration is not available when the VideoInfo is created, but
it will become available when a media segment arrives.
Change-Id: I2c24852fca4ff77c2b86ff78dbf0963ad0a0afa6
'availability_time_offset' is not an attribute in MPD specification,
but used to calculate availability_start_time to adjust segment
availability time. A similar effect can be achieved with standard
blessed suggested_presentation_delay flag.
If suggested_presentation_delay is not set, player is expected to
choose a suitable value.
Change-Id: I1d540bd4347b4f04b6cc5ec0778e1a5392009b09
Widevine license server added support for protection_schemes recently;
If CENC request includes protection_scheme in the CENC request, server
will return with a PSSH with protection_scheme filled in.
Change-Id: I6dcac498b5e039503d6ac0f6e057737f7c53efaf
This addresses playready broken with clear lead = 0.
clear lead != 0 is still broken with playready on Edge. We think it is
likely an Edge bug, which does not support multiple sample entry boxes,
thus does not support clear lead.
b/37913785
Change-Id: I7adb77a913dccf669153b03b31b4a1e1c98d1cb0
- Implemented according to v1.0 spec @ https://www.webmproject.org/vp9/mp4/
- v0 is no longer supported
Change-Id: I189c813d788400beda797eea7da943a83dfa7d79
- This change has the muxer listener side changes.
- Also changed SetContentProtectionFields() to take vector for
default_key_id so that MuxerListener subclasses don't need to convert
it to string.
Change-Id: I8832848a0c7e34b84e2b4e6baeba364861900ff1
- Added flag --mp4_include_pssh_in_stream, default to true. If it is
set to false, the encrypted mp4 stream will not include pssh.
- Align BytesOfProtectedData to multiple of 16 bytes for cenc.
- Set TrackHeader flag to kTrackEnabled | kTrackInMovie |
kTrackInPreview
- Move mvex to after trak, required by HLS
- Add cmfc/cmfs compatible brands except for avc3/hev1, where CMAF
requires single initialization switching set which is not supported.
- Set duration to 0 in tkhd, mdhd, mvhd.
Also updated major_brand and compatible brands:
- Set major_brand to isom (iso-bmff media file format) and made dash
a compatible brand
- Replaced compatible brand iso6 with iso8 since we use sthd for text
tracks
Fixes b/36278260
Change-Id: I3cc5dd5aa1621714d517fe02fe3841d19a1a07f6
- Also fix the misleading error if an invalid input is provided
The command line app now correctly reports that the file is not
found instead of stream not available.
Change-Id: I8cfccb62d7a50de57766666f126cf061e3df5ded
This is what webm_subsample_encryption flag actually does.
- Fixes the flag not working problem introduced by previous CLs.
- Also mark flag webm_subsample_encryption as deprecated.
Closes#220
Change-Id: I03ff843786572a91c01b8bd346a3ce50b129118f
Also added a few more command line flags for end to end test:
- no-remove_temp_files_after_test: do not remove test artifacts after
test
- encryption_key, encryption_iv: allow injecting encryption key/iv from
command line
Change-Id: I62084790e10fe6a385b90cb96d9515d8436b2a49
- Add EXT-X-MAP tag for init segment.
- Do not set output field on stream descriptor if not specified on
command line. If it's set (internally) then it gets copied to
MediaInfo that gets passed to the manifest generators.
b/36279481
Change-Id: I762c55b255699ec691817dc4806b0dee2f7504b8
- The flags were overwritten in a loop and therefore only stored the
last value of the flag.
- Change the fields into arrays so that the flags are not overwritten.
Change-Id: I59762d7c84bc3d81bf3b0b5e85ffe689260d970a
This is a fix for issue #216 where the Representation::SlideWindow() logic was
still active even though a static MPD was being asked to be generated (using
the option -generate_static_mpd). This could cause static streams longer than
1800 seconds to start playing at (end-position-for-stream - 1800 seconds).
This change is in compliance with the current usage documentation for the
shaka-packager, which suggests that -time_shift_buffer_depth only is relevant
for dynamic media presentations.
Added a flag --strip_parameter_set_nalus. When enabled, parameter
set NAL units, SPS/PPS for H264 and SPS/PPS/VPS for H265, are stripped
from frames when converting NAL byte stream (AnnexB stream) to NAL
unit stream, which generates avc1/hvc1; otherwise they are not
stripped, and avc3/hev1 is generated.
Parameter set NAL units should not be stripped if they are varying
in the frames otherwise the frames may fail to be decoded.
The flag is enabled by default as we don't usually see varying
SPS/PPS/VPS and it is more space efficient with them stripped.
Set --strip_parameter_set_nalus=false to disable the flag if there
are varying SPS/PPS/VPS in the frames. This addresses #206.
Change-Id: I34bde6f33069f9f77d51a510b39f58a0f0e141aa
The issue could lead to the MPD attribute mediaPresentationDuration being
wrongly generated for WebM streams with a duration longer than INT32_MAX.
The root-cause was that StreamInfo::set_duration() accepted an int instead of a
uint64_t. This seems like a pure typo, since StreamInfo already uses a uint64_t
internally for representing duration.
This CL also removes EncryptionConfig stream data type and merges it
into StreamInfo/SegmentInfo instead.
Change-Id: Idb70ce503e61d3c951225cc78b6b15c084e16dcd
- Disable subsample encryption for VP8 in ISO-BMFF
- Apply block alignment to all subsamples for VP9 in ISO-BMFF,
instead of just superframes.
Change-Id: I8dd31cc16e87abc4d538330eaff9acb0509497df
- luma_weight_l0_flag, luma_weight_l1_flag,
chroma_weight_l0_flag, chroma_weight_l1_flag were there but not set.
- The user should use the values in pred_weight_table_l0 and
pred_weight_table_l1 instead which are set.
Change-Id: Ic9c44fb113717346938a339faf074daa32d4c2d2
Also moved MediaHandler output validation to Initialize instead.
This CL also addresses #122 with consistent chunking.
Change-Id: I60c0da6d1b33421d7828bcb827d18899e71884ce
- WebVttMediaParser uses WebVttSampleConverter to generate non
overlapping media samples.
- The media samples contains ISO BMFF boxes.
- Add kCodecWebVtt to signal that the media is webvtt and
the samples will be in ISO BMFF boxes.
Change-Id: I639902cdba7b04af75428bc20622e26b8203cfb2
Before this, HLS output did not contain language information.
Now, media playlists are properly tagged with a language in the
master playlist.
b/36134267
Change-Id: I172e946dbedd096a44cb2f917b007cc004756228
- Also sets up the packaging and verify it works.
Some of the changes are temporary to get the integration going.
Change-Id: I0cf6c379d185e157808acabb9ef58ff93d4a39ae
The spec actually requires all NAL units to be escaped, so there
is no need to escape it again.
NAL byte stream is one demarcation method; NAL unit stream is another
demarcation method. Regardless of the demarcation method used, the
requirement for NAL unit is the same. See the last paragraph of
7.4.1 NAL unit semantics on the requirement on emulation prevention bytes.
Change-Id: Icc63fcc5cf965632e331f5af5f673164c7c1663a
CENCv3 recommends only encrypting video data in slice NALs. Slice
data partition NALs should not be encrypted.
In the code, differentiate is_vcl and is_video_slice. They are the
same for H265; for H264, vcl NALs include slice data partition NALs
but video_slice NALs do not.
Change-Id: I91f4bdd76d25f0eac50e39aed350ebce3f667121
- Logic for splitting up WebVTT cues into MP4 samples.
- For any cue intervals that overlap, a new sample must be created.
Change-Id: Icf7478f42d8c5790bf6f404d512c1251dd34c405
This handler is a multi-in multi-out handler. If more than one input is
provided, there should be one and only one video stream; also, all inputs
should come from the same thread and are synchronized.
There can be multiple chunking handler running in different threads or even
different processes, we use the "consistent chunking algorithm" to make sure
the chunks in different streams are aligned without explicit communcating
with each other - which is not efficient and often difficult.
Consistent Chunking Algorithm:
1. Find the consistent chunkable boundary
Let the timestamps for video frames be (t1, t2, t3, ...). Then a
consistent chunkable boundary is simply the first chunkable boundary after
(tk / N) != (tk-1 / N), where '/' denotes integer division, and N is the
intended chunk duration.
2. Chunk only at the consistent chunkable boundary
This algorithm will make sure the chunks from different video streams are
aligned if they have aligned GoPs. However, this algorithm will only work
for video streams. To be able to chunk non video streams at similar
positions as video streams, ChunkingHandler is designed to accept one video
input and multiple non video inputs, the non video inputs are chunked when
the video input is chunked. If the inputs are synchronized - which is true
if the inputs come from the same demuxer, the video and non video chunks
are aligned.
Change-Id: Id3bad51ab14f311efdb8713b6cd36d36cf9e4639
This will move ContentProtection element from Representation
to AdaptationSet.
Shaka Player already supports AdaptationSet switching
urn:mpeg:dash:adaptation-set-switching:2016;
ExoPlayer does not support it yet. Filed
https://github.com/google/ExoPlayer/issues/2431 to track the issue.
(ExoPlayer does not like having ContentProtection in Representation
anyway)
Closes b/34691105
Change-Id: I69d0a4d0e15a912a35c8b2686620419a28e4cc99
- Deprecated command line flags --profile and --single_segment.
'segment_template' in stream descriptors implies live profile
and non-single segment.
- Added flag --generate_static_mpd_for_live_profile to generate
static mpd for live profile; if not set, dynamic mpd will be
generated.
Close#142
Change-Id: I78879297ed118f0f246c4753a16ad125bd6b5e4f
- Added two new fields in MpdOptions: dash_profile and mpd_type
- Updated MpdBuilder to support LiveProfile with static mpd
- Command line arguments will be updated in the next CL
Issue #142
Change-Id: Ibd35e5c9e1b1ef98043392e3f04a0af4700135c1
- Add MSVS related patterns to gitignore.
- Removed encryption_modes.h from media_base.gyp. It doesn't exist.
Change-Id: I44bc86ff7e79daf6f7bc61c21a558937530d8e66
Current mechanism of using AdaptationSet @group is not spec compliant.
The spec compliant mechanism is to use supplemental descriptor with
schemeIdUri: urn:mpeg:dash:adaptation-set-switching:2016 as specified
in ISO/IEC 23009-1:2014/Amd.4:2016.
Fixes#156
Change-Id: I4d97648547a23315ba9c09dcadb14e18c99a45fc
- nalu_data points to the beginning of the nalu, instead of
beginning of the slice header which excluded the nalu
header.
- nalu_size is the size of the whole nalu.
- header_bit_size calculation is corrected. The size used to exclude
emulation prevention bytes.
Change-Id: Ic00f42caeddf7b4701f3948e6ef8a3537fd7f770
- If OnEncryptionStart() is called OnMediaStart(), someone should be
calling NotifyEncrytionStart().
Fixes#169
Change-Id: I597f317d2ddeb3091b7e9d238eb9695b4cc9c69d
Three options are supported right now:
- reuse=1|0
Allow or disallow reusing UDP sockets. Default 0.
- interface=a.b.c.d
Address of the interface over which to receive UDP multicast
streams.
- timeout=microseconds
Timeout in microseconds. Default to unlimited.
A UDP url should be of the form udp://ip:port[?options], here
is an example:
udp://224.1.2.30:88?reuse=1&interface=10.11.12.13&timeout=12345
Also deprecate --udp_interface_address flag in favor udp options.
Closes#133
Change-Id: I962d35bfedc1779d67ba20ed910207c66b7c1a15
1. Allow encryption pattern of 0:0, which is treated as 1:0;
2. Remove the special handling of pattern 1:0 in pattern cryptor
which may not always be correct;
3. Allow senc/saio/saiz boxes to be absent.
Change-Id: I372e61182ec577107e39cb601c3aed80616b036d
The original code creates temporary file in the current directory,
which may not always be writable.
Change-Id: Icc278dd5db6be6206ae29defbb7423bae39cc978
- Also import the version of the protobuf bundled in packager first,
this avoids importing the installed version of the protobuf, which
may not be compatible with this script.
- Print help if no argument is supplied.
Fixes#162
Change-Id: I1b5e87ba7fb51e81b0d4dd3791aaeb37ee0fd6ce
Cues used to be generated in the end of the file; if http range
request is not supported, clients have to download the whole
file to get to the cues.
This CL updated TwoPassSingleSegmentSegmenter to write cues
after writing webm header.
This CL also updates libwebm dependency to latest.
Closes#159
Change-Id: Ic73548e1b872e6b13a37627707e7d0ff3e556877
The original code accumulates sample durations in seconds (double)
and then generates new segment / cluster by comparing the accumulated
value with configured value.
There may be loss of precision when accumulating values in double.
For example, with a GOP of 96 frames and frame rate of 24 fps; the
accumulated frame duration for 96 frames is 3.999999999999, which is
very close to 4.0 but not 4.0. Problem would arise if segment
duration is set to 4. The created segments would have a duration of
8 seconds instead of 4 seconds.
The new code accumulates sample durations in uint64_t relative to
input time scale; the configured segment duration will be converted to
timescale for comparison. This avoids loss of precision.
Change-Id: I3ae24be82a7ce45f923a6f90fea495b8b6b2e7ef
Setting this flag will tag any matching tracks with kRoleMain.
The player will then know which tracks should be default when there
is not a language match.
Closes#155
Change-Id: I827304c49e345d07b1e76f7a46421b60f228f0ba
Language tags can have subtags, such as "por-BR". While the base tag
should be 3 characters, the entire tag can be longer. Rather than
parse out the base tag and check that it is size 3 without the subtag,
we can just trust the ISO tag converter to do the right thing.
Change-Id: I2a186b5c8db0f373ee82ef68245ecc33cb068581
Some subtitle files use .webvtt instead of .vtt, probably because
ffmpeg uses -f webvtt when converting from some other format.
Change-Id: If88e8d2b90e5dea6751409568d5191e923dec4aa
- Also regenerated test content to insert an iframe every one
second: ffmpeg -i source.webm -auto-alt-ref 1 -g 30 out.webm
This makes sure the content is actually encrypted with one
second clear lead in end to end test.
Change-Id: I488143c148e2d6c45ba1586f6d0d835dc46db86e
- Add decrypt_config in media sample and stream.
- Update subsamples when converting NAL unit to bytestream.
Change-Id: I7b8975a453f81b22cf74bee3c9a58b7e458cbaae
1. Replace STLDeleter with unique_ptr in container, which is
supported in C++11
2. Replace string_as_array with &string[0] which is guaranteed
to work in C++11
Change-Id: I7f39c0e51fc8a3fcbb41313094a0ca6b33db7bf3
The current behavior of logging to a file in the same directory
as the binary will break if the binary does not have write access
to that directory.
Fixes Issue #139
Change-Id: I11adfaec676fc9bbef2a5caa3261889d1fba3b88
In H264, there may be multiple consecutive video slice NAL units
in the same frame. The original code assigns a new access unit
for every video slice NAL unit, which is incorrect.
Fixes#134.
Change-Id: I4d44271df48cb08867ddd02f7494fb3573af3356
Mimic iso-bmff behavior: either all the samples in a segment are encrypted
or all the samples are clear.
Change-Id: I03bdbbf5a4b690f4d87c4dceb0f8155e6fae941e
The right way to detect the presence of a keyframe in a BlockGroup
is to look for the absence of the ReferenceBlock element [1].
Closes b/30433206
[1] https://matroska.org/technical/specs/index.html#ReferenceBlock
Change-Id: I2b3c8704cbabcd5cbee38448e5528425865561a4
WebRTC portion (gyp build) now checked in because separate repository is no longer available due to transition to git.
Change-Id: I56436cbbc3f02a5f77112917d6229aea69f21d97
Also updated packager_test not to fill in output_format by default,
which will help catch these kinds of bugs.
Closes#130
Change-Id: I77157317de822d3f543f7e27d94bb4df7a1649d2
This flag is needed to workaround the Chromium bug
https://crbug.com/398130, which uses decoding timestamp
in buffered range.
Closes#113
Change-Id: Ib8f18be7165dd968bdc36c18ce29f694235c0c26
Packager should not care about this flag, whether it is supported
is up to the decoder.
Closes#126
Change-Id: I5e05d291eab01375f7f11c951b9dc93c1358c38a
- Add MuxerListener::OnEncryptionStart() for notifying that further
segments are encrypted.
b/29621230
Change-Id: I881b29d55baaf3d04e005a3b95d898071c3f272b
Seeing occasional build breaks when building hls_builder, e.g.
https://travis-ci.org/google/shaka-packager/jobs/143445907
Although hls_builder is dependent on widevine_pssh_data_proto
indirectly (through media_base), it does not work all the time.
Looks like the proto has to be a direct dependency.
Change-Id: Id4d9d0b9e669df0967cc767132ef29717243a722
- Try to determine container name from segment_template if output is
not specified
- segment_template descriptor is required for TS
- output descriptor should not be specified for TS
- Generate hls_playlist_name from stream number if not specified
- Update packager_test.py to include playlist validation
b/29551186
Change-Id: Ide7169f1c26c85a6e38272dda0e1af77135b4d90
- Cannot include PMT for following encrypted segments in the clear lead.
Some players consider them as conflicting codecs.
- Add EXT-X-DISCONTINUITY in front of first EXT-X-KEY to notify that the
codec is switching from e.g. "avc1" to "zavc".
b/29621230
Change-Id: I45c74813630c229d66245e992eb3a5117326bb14
Before, we converted the WebM configuration to MP4 in the video
client, however this lead to a bug when fields were missing. So
now don't convert until the cluster parser so the extra info from
the stream can be added.
This also fixes a bug where the value was not printed in the warning
logs.
b/29580732
Change-Id: If0a1a4d135f98292cdaae15a11027f42d844e85d
Since the |extra_data| field contains codec-specific configuration
data, it makes sense to call it |codec_config|.
Change-Id: If9e35165a00fe82628cf931df397a8ef06505b0d
Although superframe index is not encrypted, it should still be inserted
into subsamples list (with cipher_bytes as 0).
Change-Id: I06b38f8333a22a31a71cd67e0412218dd3aaa5c3
- Add flags and stream descriptor fields for HLS.
- Remove fields from MuxerOptions. Instead pass them directly to
HlsNotifyMuxerListener.
- Rebase segment names to the master playlist's path where it makes
sense.
Change-Id: If4f54bc56ff46dc6140859a79ed66f7b99112ed7
- Tracks other than video are protected using whole-block full-sample
encryption as specified in ISO/IEC 23001-7:2016(E) 9.7, which is
equivalent to a pattern of 1:0. This is different to the non pattern
encryption counterparts.
- Also updated the code to allow the existence of other protection
schemes in the original content, which will simply be ignored.
- The internal cryptor used by AesPatternCryptor should not use
constant iv, add a DCHECK for that.
- Optimize AesPatternCryptor handling on the special pattern 1:0.
Change-Id: Idc704e7bc6b347741336f38c6d3620fc19392960
This is part of the effort to support Opus in iso-bmff #83.
Changes in this CL:
- Add support for multiple SampleGroupDescription and SampleToGroup
boxes in TrackFragment box;
- Add support for SampleGroupDescription and SampleToGroup boxes
in SampleTable box;
- Add support for AudioRollRecoveryEntry in Sample Group and 'roll'
sample grouping type;
- Also fix a bug that default length was not set correctly in
SampleGroupDescription when constant iv is used.
Change-Id: I7e31386ea6cd17a8ee5e1dca4c1a90937d51368f
- There is a race condition during build. hls_builder should depend on
widevine_pssh_data to remove the race condition.
Change-Id: If9fa8895104a774b8ecc7545a57323ae49274936
- For encrypted segments, PMT specified by SAMPLE-AES specification is
generated.
- Remove time_scale_ field from TsWriter. It is not used.
- Move ContinuityCounter to a separate file.
- Add ProgramMapTableWriter and implementations.
- continuity_counter is incremented for PAT and PMT.
Change-Id: If0dd0990203cb6990f39a6832048bdc3ff08decd
This add several unit tests for EsParserH26x to test the various
NAL unit orders and access unit contents. Some tests are disabled
and will be fixed in another patch.
Change-Id: Id5e3291e22f1fe17ada2c03c42e2cdfe226abcb2
- BytesOfProtectedData in 'cens' SHALL also be 16-byte aligned
- Fix an incorrect box definition bug w.r.t to 'cbcs' and 'cens'
- Also add various protection scheme tests
Close#77Close#78
Change-Id: I63c8d8b01ce16ed60affa97ec95fc62bc2da06df
- The current implementation only handles VOD.
- Add #EXT-X-PLATLIST-TYPE.
- For VOD Add #EXT-X-ENDLIST at the end.
- Append comma at the end of EXTINF.
Change-Id: I16f01da66f8bbf0229395cb380fa125ffd9328a8
- Disallow packager_test.py to be executed from source directory
- Remove the building of packager target from packager_test.py. User
need to make sure packager builds successfully before running
the test.
- Remove build_type parameter which is no longer needed
Change-Id: I479d3c39b8d45730e5388d194069f1320fa5cd75
- These warning messages are not useful since the code needs to
loop through various container types to find a matching one.
Change-Id: I1d8eca563faf24e485ac71cce32e09515c174bf2
Replaces the InitializeFromH264 and InitializeFromH265 methods with
a single Initialize method that accepts a CodecType. Also moves the
CodecType enum from NaluReader to Nalu.
Change-Id: Ie33aa9c0d772a7f3c6a2e9c867e25be3edfe7828
- CBC cryptors should accept IV of size 8 bytes - it will be zero
extended to 16 bytes.
- Fixed iv() not updated problem in AesPatternCryptor.
- Replace kChainAcrossCalls with ConstantIvFlag enum flags.
Change-Id: I3fb4de0e8abbe891e6271e779373ba53f8df660d
- Sample duration wasn't scaled in TsSegmenter to calculate the current
segment's length. So segmenting length did not always respect
segment_duration.
Change-Id: I1ef4747dbd12bde7852ffcbceb971f2775aa18b3
Parsing vui_parameters in the SPS is required to be able to extract
the resolution of the video. This also adds a method to extract
the resolution to the H.265 parser.
Issue #46
Change-Id: Idf498d2afdb36a689490151a4ae6baef0b6b73f6
- Media Playlist that is non-VERSION 1 must have EXT-X-VERSION.
- Attribute for EXTINF should be CODECS instead of CODEC.
- BANDWIDTH attribute expects bitrate (not bytes per second).
- Removed unnecessary check if MediaInfo has media_info_name, in
SimpleHlsNotifier::NotifyNewStream().
Change-Id: Ia63cfa59e5e2ec24bbf1b784164e6e41176fc589
Now Finalize() will close all the files used by the segmenters. This
is important to allow WebMMuxer::FireOnMediaEndEvent to get the
correct size of the media file.
This also changes the WebM muxer to use 64-bit numbers for the init
and index ranges to correctly support >4GB files.
b/28194272
Change-Id: Ia84e4a4b0756f89644efea99a1a51968b22a1338
- Add hls_name and hls_group_id fields to MuxerOptions. This is used to
pass the NAME and GROUP-ID values for EXT-X-MEDIA tag to
MuxerListener.
- Change MuxerListener::OnEncryptionInfoReady() to take an
initialization vector.
- Change MuxerListener::OnNewSegment() to take segment name.
- Reworded and formatted MuxerListener comments to Doxygen style.
Issue #85
Change-Id: Iea06e68552a56ae180177ffd6ca315a7cf39456c
- Fixes Mac build.
- Mapping to scoped_ptr doesn't work, instead use raw pointer and
STLValueDeleter.
Change-Id: If17e43d9d4aba8c92a76d2b1ca99a64c6a2b0e84
- HlsNotifier interface, similar to MpdNotifier.
- SimpleHlsNotifier class can generate playlists.
It can also handle encrypted media with Widevine encryption.
Issue #85
Change-Id: I9305ddf42796370d3e5df360633c80997c909365
- crypt_byte_block and skip_byte_block for pattern-based encryption.
- constant_iv for constant iv (used by 'cbcs' protection scheme).
- Also renamed iv_size to per_sample_iv_size.
- Also changed "bool is_encrypted" to "uint8_t is_protected" to align
with CENC spec.
Issue #78
Change-Id: I2878a91e0ebe536a08a3e3109daf157fe4440e27
- Class to generate Master Playlist.
- Takes multiple Media Playlists to generate all the playlists.
Issue #85
Change-Id: I3e2be8763cc771d468245f64753e8a38b0746686
- Remove EncryptionMode enums
- Remove AesEncryptor::InitializeWithRandomIv, replaced with
a static function AesCryptor::GenerateRandomIv, which should
be called to generate the iv if the iv is empty.
This change is to prepare support for CBCS and CENS pattern-based
protection schemes.
Issue #77
Change-Id: Icba35089d6e451cbea7ebbf5dd5674079f206390
- FOURCC is not a mp4 only concept. It is also used elsewhere.
- RCHECK has been used outside formats/mp4.
Also updated the FOURCC enum names to match the actual fourcc code, e.g.
FOURCC_mdat for "mdat".
Change-Id: I5be02f12695882c0fb41d359305fb8ec01303a06
- Create AesCryptor class as the common base
- AesCryptor::Crypt function will serve as an Encrypt function for
encryptor and a Decrypt function for Decryptor.
Change-Id: Ie91fb14964b5091786705bf510656f40d73af160
- Simple implementation of MPEG2 TS muxer.
- This does not take MuxerListener so a manifest cannot be generated.
- Encryption is not supported.
Issue #84
Change-Id: I117b772dbbce5437398defbd564883ad758916d5
- Segmenter should not finish a segment unless the next sample is a key
frame.
- Renamed PesPacketGenerator::Finalize() to Flush().
- Use duration from the sample instead of the one copied to PesPacket.
- Remove duration field from PesPacket.
Issue #84
Change-Id: Icd90e65fd63fdeb955e7abac3473b0b54db6ac4a
- Multi-file segmenter implementation.
- The samples are passed to PesPacketGenerator to generate PesPackets.
PesPackets are passed to TsWriter to write them to file.
Issue #84
Change-Id: Ia4cae2abe2e7df46016dcdb791baaab2818aea03
The positions in the SeekHead element should be relative to the
Segment payload, not the start of the file. Also added entries for
Tracks and SegmentInfo.
Change-Id: Id692da25fffc27d78e9f1a06d061431aeb6f1e7c
This script can be used to parse and generate PSSH boxes. This
serves two purposes:
* Parse PSSH boxes into a human readable format.
* Take several arguments to generate PSSH boxes.
This is also able to parse Widevine and PlayReady PSSH data and will
print the parsed data instead of the data as binary. This can also
generate Widevine PSSH data instead of having the PSSH data passed
as an argument.
Change-Id: I245bd9abf79a259f6fda01d1fc0489b4a4a73db5
- Optimize and clean up encryption and decryption code.
- Consolidate various CBC encryption/decryption schemes into a
common class.
- Make it a constructor argument whether cipher block chain is
continuous across Encrypt/Decrypt calls.
- Also align protected region size as required in CENC spec.
Issue #77
Change-Id: I533d92ada3cd80933b532b9c3a1cca105ba66f8e
Another feature that the mac build does not support. It incorrectly
uses a copy-constructor when creating new values in a std::map. This
is incorrect and should use an in-place constructor with C++11.
Change-Id: I1e822747e97eb4ae01596d63ea05058cc383cb49
This adds a preliminary H.265 parser. The parser only support
reading the slice header. This also attaches the H.265 parser to the
VideoSliceHeaderReader so H.265 frames can have unencrypted headers.
Issue #46Closes#40
Change-Id: I8b98e1f8bb04e359d9bf896fd5b2939c326b196b
The flag --include_common_pssh will add another PSSH box in addition
to the Widevine one which will contain the key IDs of all the keys.
Closes#88
Change-Id: Ic719b19747530f0e4856cfb36471a644d572a734
- Use 'builddir_name' environment variable to create the output
path in test_env.py
- This allows running the packager test with different build
configs because it doesn't use 'out', e.g. it could use 'out_clang'
for clang builds.
Change-Id: I843343ec8e2d0a64dffbab2fdbd38aead5c97718
The new algorithm will parse NAL unit header and only starts a new
NAL unit if it is valid, otherwise it will be considered part of
the previous NAL unit.
Closes#96
Change-Id: I45f2a0f37d51841ee8345d6d0d38fcda57e0a009
KeySource is now an abstract class that is implemented in two
types WidevineKeySource and FixedKeySource. This also updates
fixed key encryption to use v1 PSSH boxes with the common system
ID.
This also updates the --pssh argument. Now it accepts full PSSH
boxes rather than data. Multiple PSSH boxes can be concatenated
together.
Issue #88
Change-Id: I4bd0290f6d8b965f7d118f075c96e0f267d7e831
- Part 1.
- Add packager command line argument "protection_scheme" to specify
protection scheme. Plumb through packager code to enable CBC
encryption/decryption.
- Add scheme type "cbc1" to sinf.
- Refactor AES encryptor and decryptor.
- Need more work in the subsample handling.
Issue #77
Change-Id: I3a9304d89adf5efbfb226b6e805a3077b6cb8c68
* EncryptionKey now contains them rather than a PSSH box.
* Outputs PSSH boxes for each entry.
* Outputs a ContentProtection element for each entry.
* Removed SystemName and UUID from KeySource.
* Removed --scheme_id_uri packager argument.
Issue #88
Change-Id: I2651784c3220fd64f5b1773fdcd70285690cf8c0
This adds a new function to key AdaptationSets based on the
characteristics that require separate sets.
With this, we are now able to package multi-codec assets at once
(AVC1/MP4 and VP9/WebM in the same manifest, but in different
AdaptationSets).
Fixes b/27534371
Change-Id: Ie82fd096d2ef99fd23e323d60fd0849b1e8d7500
When normalizing language tags to their shortest form, we need to
split off subtags and treat them as separate components that do not
get changed.
Furthermore, MP4 can only store language tags without subtags. So
only store the main language in MP4 output.
Fixes b/27533973
Change-Id: I96049e274aae9841e321c53039ef6464a2e61574
Add another overload to fetch by key ID directly, used by WebM.
Changed the one using PSSH data in favor of the entire PSSH box.
This also moves the Widevine proto to media base. Now, the
Widevine key source handles creating the Widevine specific PSSH
data.
Change-Id: I6f4633facad39207809ffbad970635d1f9d70983
This moves pssh parsing to its own class in media base. This type
can be used to parse and generate pssh boxes. This new type also
supports both v0 an v1 boxes.
This also modifies the pssh box definition to only have the |raw_box|
and moves all pssh parsing and handling into mp4 media parser.
A follow-up change will replace the remaining usages of 'raw' pssh
box data with the new type.
Issue #88
Change-Id: Ic2436ecb5df8b3558b81e600dc095b0efd122ab1
This moves common parsing code out of the H.264 parser. This is in
preparation for the H.265 parser to use.
Change-Id: Id05b55f6adc2ea31e97c8a9f22023e851808a8a7
Now the AVCDecoderConfiguration and HEVCDecoderConfiguration classes
will read the Nalu from the blocks, meaning they will store a
vector of Nalu objects for the data.
Also added H.265 to the Nalu class and added argument to NaluReader
to read H.264 vs H.265 Nalus.
Change-Id: I8d8194a90fd72a30af66e9776e01379f8428542c
This is needed to test encryption in end to end test since the video
is segmented in gop boundary by default. Due to the existence of clear
lead, the first segment is always left in clear.
Change-Id: Ie8d37a1740f9061515298cb9fb7b58983592cc5b
This member refers to the size of the payload (the data after the
header). Since the pointer is called |data|, calling this size
|data_size| is confusing.
Change-Id: I9174cf6c5c402beddaebbce7c16514470ba0c3da
Now the Nalu data pointer points to the start of the NALU header
rather than pointing to the start code. Added a new method on
NaluReader to check whether the stream starts with a start code.
Change-Id: Ifaecbe0c911aa5cffdf0a966028e6cada8621cc3
For non-video slices, the data is not encrypted. This also skips the
frame headers for H.264; support for H.265 will be added later.
Issue #40
Change-Id: Id0cb0fb9ddb6adedf63ef4aef6b3a26260a21654
Removed the code for NALU splitting in the H.264 parser and moved
it to its own class. Also added support for length-prefixed
NALU splitting for use with the encrypting fragmenter.
Change-Id: I1e91266681f1b117fb2382cf80590651efc06619
Parse channel layout value from dec3 box. Pass it through audio stream
info. MPD builder forms audio channel configuration to signal ec-3
codec.
Specs: 1) ETSI TS 102 366 V1.3.1 Digital Audio Compression (AC-3,
Enhanced AC-3) Standard E.1.3.1.8.
2) DASH-IF Interoperability Points v3.0 9.2.1.2.
Issue #64
Change-Id: Ia2c22dd3d82e757ba5a88ba1de35c5d593f5005e
Before this change, the media file will fail to be parsed if it contains
non-wvtt text streams. This change allows demuxing of video and audio
streams in these media files even though the text stream is ignored.
Fixes#74.
Change-Id: I7ef108a51125c4965fe48c39efe9638f3a0e4759
This also changes the way frames are written in the WebM muxer. Now,
frames are stored and written on the next call to AddSample. So each
call to AddSample will write the previous frame. This is needed to
determine whether the given frame is the last one in the cluster.
Closes#70
Change-Id: Ic69ebad3c4729cdaa2017c9c7f497048501ac907
Create a new MacOSX configuration for curl:
- Add a curl_config.h created on MacOSX
- Add a find_curl_ca_bundle.sh that searches default folder from Homebrew and MacPorts for a ca bundle.
To find the files either install OpenSSL from Homebrew or MacPorts, or Curl from MacPorts.
When generating ninja files we end up with two libwebm targets.
packager/media/formats/webm/webm.gyp:webm -> becomes target libwebm
packager/third_party/libwebm/libwebm.gyp:libwebm -> becomes target libwebm
Solve this conflict by renaming libwebm.gyp:libwebm to mkvmuxer.
There is a target name conflict between packager/base/base.gyp:run_all_unittests and
media_test.gyp:run_all_unittests. Change to media_test.gyp:run_tests_with_atexit_manager to
resolve the conflict.
On Linux the generated ninja files use path/libbase.a as target name.
In the Mac ninja file the target is just libbase.a.
As we have two libbase.a (base/base.gyp and media/base/media_base.gyp) the linking stage fails to include both libs.
Solve by renaming media_base.gyp's base target to media_base
For WebM contents, it is possible that we may receive some NewSampleEvent
before receiving ParserInitEvent. This is because init event is fired
after analyzing a video block which could come after an audio block.
Also modified Flush() function to return a bool to indicate whether the
the flush is successful and whether the samples are handled correctly.
And added macro to ensure Flush() and Parse() results are handled.
Fixes#71
Change-Id: I2294d6f529f54e4578344916559bb1bc116c745a
Now outputs cenc:default_KID attributes on all ContentProtection
elements for non-MP4 streams. For MP4 streams, it will still output
the default ContentProtection element.
Closes#69
Change-Id: I38b24297aa3c2ccbcbde38b44279b56c37a388f7
- Also refactor decryptor management code out of mp4_media_parser.cc
- Move decryptor managment logic to DecryptorSource class to make it
available for webm as well
- Remove data_offset member from DecryptConfig which is not useful
- Add widevine_pssh_data.proto file
Closes#72
Change-Id: I1d32baf4013ebd3382b5372c7433fae5033a260e
- dts was not set earlier, although for WebM, we could assume that
dts is the same as pts.
- Calculate block duration with the difference with the next block
if duration is not encoded, even if track default_duration is set.
- Use track default_duration as a duration estimate for the last
block.
- This also removes opus duration computation from encoded data.
Issues #67, #68
Change-Id: Icaa2769dcb2a89269ae014f44ad6a9262770aed2
- Box definitions for box type DEC3.
- Parser/muxer changes to support Enhanced-AC3 audio codecs.
- MPD signaling will come in Part 3.
Issue #64
Change-Id: Ifcd5efa1f61b470ec225127925631e4329853259
This does not support key-rotation and will give an error. This is
because WebM does not have a way to indicate a change in key ID using
media segments.
b/22463551
Change-Id: I9b3dac818dc370302a5afc0d25d8a060b64d30cd
- From DASH spec 2014, MPD@publishTime and Period@id are required for
'dynamic' MPD.
- Period@id is always set to "0".
- MPD@publishTime is set to the time when the MPD is output.
Issue #55
Change-Id: I5e50aa11067176e69a2343fbc82ca42129703e1b
Added packager tests for WebM audio. Also now only outputs the
language if it's not empty. Fixed the packager tests for that.
b/26540606
Change-Id: Ica804bd710055bcaa2113f948d723dedd78ef909
- @width, @height, @frameRate for Representation element will not be
set if the attributes are set at AdaptationSet level.
Issue #55
Change-Id: Ib4e669142874f9e8f0ca773df9f87a3fef01b729
- Box definitions for box type DAC3.
- Parser/muxer changes to support AC3 audio codecs.
- EC3 audio sample entry will come in Part 2.
- MPD signaling will come in Part 3.
Issue #64
Change-Id: I790b46ae8179b933bb8f7da9cdd38591fe8da43d
ThreadedIoFile spawns a new thread for every new file. Thread
information is stored for tracking purpose by base::tracked_objects.
The tracking object remains even if the thread itself is destroyed.
This results in memory usage increased by a couple of bytes for every
new segment created in live mode (new segments spawns new threads).
Use WorkerPool instead to avoid spawning new threads.
Fixes Issue #61.
Change-Id: Id93283903c3ba8ebf172a0d58e19b082a72c6cf0
For mp4 outputs, a metadata box with version information is added to
moov box.
For mpd outputs, a text comment with version information is added in
the beginning of mpd file.
Issue #60
Change-Id: I783ba370781c0a8f77c910ff1172bad2e7edff75
- Another minor change in segmenter_test_base: scoped_refptr can
be passed by value, remove ".Pass".
Change-Id: I31f0cf74c5ad0aba8890d3b1845155d63d7326a0
* Changed Segmenters to accept StreamInfo rather than MediaStream
to help in testing.
* Changed MemoryFile behavior to mirror local file:
* Read non-existent file is an error.
* Write deletes any existing file.
* Fixed a bug in SingleSegmentSegmenter.
Change-Id: I339e35597ca4661b7a26c6fdbbfa2f9f511c7da0
Javascript/JSON does not support int64_t or unsigned numbers. asset_id,
if greater than 0x80000000 will be converted to negative number. Use
double to represent asset_id instead as 32-bit integer can be lossless
represented using double.
Bug: 26309000
Change-Id: I3e800c396a1231375776295154fb0d96156e980b
- Replaced urn:mpeg:DASH:schema:MPD:2011 with
urn:mpeg:dash:schema:mpd:2011. The former is from the old spec.
- Replaced the MPD schema with the latest one.
- Fixed subSegmentAlignment typo with subsegmentAlignment.
Issue #55
Change-Id: Icffa9872ffc462627b81900a48f59e7b76687c11
This can be used for unit tests to reduce the need for temporary
files. A MemoryFile acts the same as any other File object, but
stores the contents in memory. The memory pointer can be accessed
directly. A file with the 'memory://' prefix will be a memory file.
Change-Id: I10ae3ed5391c8fd838e6dfb33106dec959bdc224
- Parse vp9 bitstream to get vpx codec configuration
- Add subsample encryption for vp9
- Also fixed a bug in VP9 parser if segmentation update_map is enabled
Change-Id: I69dc97088aa38c94c6d37fdbcf3d9cfc942a3df6
Profiling can be enabled by setting profiling=1 in gyp, e.g.
GYP_DEFINES="profiling=1" gclient runhooks
To turn on heap profiling, use the HEAPPROFILE environment variable
to specify a filename for the heap profile dump, e.g.
HEAPPROFILE=/tmp/heapprofile out/Release/packager ...
To turn on cpu profiling, use the CPUPROFILE environment variable
to specify a filename for the cpu profile dump, e.g.
CPUPROFILE=/tmp/cpuprofile out/Release/packager ...
Note that profiling may not work for debug builds, so use release
build if possible.
See docs/linux_profiling.md for details.
This change will help identify and resolve problem behind Issue #61.
Change-Id: I6f85a04ed82dd0cb3588e6b38e8ceb68dac6c436
- Add Box::WriteHeader and Box::HeaderSize methods. The concrete
box implementations should not need care about whether it is a
Box or FullBox normally.
- Rename Box::ReadWrite to Box::ReadWriteInternal to make it clear
that it is an internal methods. Make it private so it cannot be
incorrectly called in box implementions.
- Make an internal implementation of ComputeSizeInternal to compute
box size. |atom_size| is updated in ComputeSize, which wraps
ComputeSizeInternal.
Change-Id: I3fbcf8c527581b676d9d13f1ac1dd798da7c4d5f
Added two stream descriptors that are both used to tell the output
format. If it is not given, the packager will use the output file
extension to determine the output format.
Change-Id: Ib8d5dcf52956c2d451e77ea6a90d9502d4a77064
* Added WebM muxer that supports both single and multi-segment modes.
* WebM muxer supports MPD generation.
* Enabled WebM muxer and demuxer in packager.
b/22463220
b/25089245
b/25089714
Change-Id: I9f6e8fc51e08fbb1d685229c5cb54ea60f7bed8f
- Also set --clear_lead to 1 if encryption is enabled to make sure some
of the segments are encrypted.
- Also fixed glint errors.
Change-Id: Ie92ad3ea1a3fd95d059e424a114b863ac393e6bf
- Check whether if it can be parsed by libxml2 and check if the root
element is a 'tt' element.
- Added unit tests for determining TTML and WebVTT.
Change-Id: I4fea1881f0ab70970700601d8d7cea32cc04752c
- Since most use cases use notifier implementations that already uses
locks, removing locks from MpdBuilder.
- Added locks to MpdNotifier::Flush() implementations. Multiple threads
were able to write to the same file.
Issue #45, #49
Change-Id: I6e39824485672f40e6c947da97f1743fac174167
- Added ignores.txt file. When doing a tsan build,
tsan_blacklist gyp flag should point to the file.
Change-Id: Ia69f89f1906f0b386e081a7df5a7c867e411f8fd
Also update the code to generate CompressorName in VideoSampleEntry
for both AVC1 and various VP codecs.
Change-Id: I2355f8008a72806e852aa1ec6c80e9141b97d963
- Added factory methods for creating Representation and AdaptationSet in
tests.
- Since most tests only needed access to the constructors for
Representation and AdaptationSet, instead of using
FRIEND_TEST_ALL_PREFIXES() for each test case, added factory methods
on the test class.
- Check the 'frameRate' attribute in
CommonMpbBuilderTest.SetSampleDuration.
Change-Id: I57bd3a90b397f90c3a3a91de03eb96000f58ef58
Also includes its dependencies, like media/base/text_track_config.*
and media/filters/webvtt_util.h
Change-Id: I5b26245daf004da19b912b7c5b2c21ce4ba85688
Uses EXPECT_NEAR for comparison with an acceptable error bound. Also
increases error bound to 50ms.
Change-Id: I8fc61b3b915a50f8fe33b9ea72e123aefee25a80
- Neither live with or without key rotation did not output the right
result.
- No key rotation: did not produce any ContentProtection element.
- With key rotation: did not have DRM specific ContentProtection.
- To get key rotation working with shaka player, <cenc:pssh> elements
are removed.
Change-Id: I1f34d303ae1f3ea81820e3368ab66b8420498372
- Added --iv to inject iv for fixed key encryption.
- Added support for key rotation in fixed key encryption.
- Added --use_fake_clock_for_muxer to set timestamp to 0 in the output.
Also updates stream info dump message:
- Modified pixel_width: %d\n pixel_height: %d to pixel aspect ratio: %d:%d
- Removed language from video stream info.
These flags should only be used for testing.
Change-Id: Iedf8d6d6492226219f49fe44d932645f557010e4
- ContentProtection elements were missing when using MpdWriter,
therefore mpd_generator also did not generate ContentProtection.
- MpdWriter uses MpdNotifier now, instead of directly using MpdBuiler.
- Add mock_mpd_notifier.{h,cc} to be shared by several tests.
Change-Id: Id0f6213aa0c80e50294cd4193962054b0b5c2492
Cropping is necessary if width/height are not exactly disivible by 16 due
to 16x16 macroblock size.
The width and height fields in VisualSampleEntry must correctly document
the cropped frame dimentions (visual presentation size) of the AVC stream
that is described by that entry.
Also fixes width/height in input mp4 and wvm streams if the values in
VisualSampleEntry / metadata do not match with the values in
AVCDecoderConfigurationRecord (SPS).
Issue: 39
Change-Id: Id55c5acf245bee3f4e66302b2042eb7d9c488c19
- Because MpdBuilder no longer sets ContentProtection, live was broken.
- For key-rotation, <cenc:pssh> element is updated.
Change-Id: I45bb80a12faff39ccf5deb82439c0732ed9cea59
- Without the Flush() method the implementation had to do writes in
almost all methods. Usually the user of the notifier knows when to
write out the MPD.
Change-Id: Ic165d4594f01357a8ac7e8501eefa0f85c08d32f
- SimpleMpdNotifier didn't register the container ID to its map for VOD.
- AddContentProtection() was not implemented.
Change-Id: I5f1412c785ee5ad3d00910755ff283c404dd895c
Quotient reduction to minimal form does not work well in practice due to
rounding of the inputs.
Bug: 23432995
Change-Id: If333ab48accb7f6a3fac6ba4eb48ca7b9cee9826
This fixes a circular dependency of media_base depends on filter and
filter depends on media_base.
Also added code to extract sar in WvmMediaParser if it is not yet
extracted when setting extra_data.
Change-Id: I1b5aef2444e4134c9b6663911ac5e5c4c220404a
- subSegmentAlignment logic assumed that all Representations were
added before calling Representations::AddNewSegment() on any
Representation in an AdaptationSet. It is not a valid assumption
for VOD and is removed.
Change-Id: Ifb7e34ee103ee30027e45a804427baf281f3137c
- Width/Height in tkhd box should be display resolution instead of
coded resolution.
- pasp was not generated earlier.
- Assuming SAR as 1:1 if it failed to be extracted successfully.
- Also update test files for mp4_media_parser_unittest.
This fixes Issue #35.
Change-Id: Iedbe6a44465aac6723b6ea2de30aed180821eccd
- Call NotifyNewSegment() in MpdNotifyMuxerListener::OnMediaEnd() to
notify subsegments.
- This is necessary for MpdBuilder to calculate subsegmentAlignment.
Change-Id: I9359c408da4f616e543a6d9e568b010a90c90185
The segment duration was not reset after segment was finalized. It
was reset after adding one sample. This new sample was added to the
new fragment but was not recorded in the new segment, which resulted
in off-by-one error in the segment generation.
Change-Id: I1f9e00591be6b897773bf6caf94409b4bcd2277f
- Saves sample_duration in the media_info protobuf.
- When MpdNotifyMuxerListener::OnMediaEnd() is called, media_info passed
to MpdNotifier::NotifyNewContainer() should contain the sample
duration.
- Add tests for MpdNotifyMuxerListener.
- Moved common code from vod_media_info_dump_muxer_listener_unittest.cc
and mpd_notify_muxer_listener_unittest.cc to
muxer_listener_test_helper.cc.
Change-Id: Ibf2340076d45828d26f1af54a1ed8e95e5884c23
- AdaptationSet::OnNewSegmentForRepresentation() calculates whether the
Representations are aligned by looking at the start time and duration.
(Except that duration is not used in this implementation).
- Add RepresentationStateChangeListener to callback
AdaptationSet::OnNewSegmentForRepresentation() from Representation.
Change-Id: I3c30bd6652880dabb9d5c619d8a733ffc789eec9
- The tests couldn't run in parallel using gtest-parallel
(https://github.com/google/gtest-parallel) because all the tests
used the same file.
Change-Id: I9a3d2c20c290fbf4bee2a1d79c4ce9bf802aefcf
- ContentProtection elements should be added explicitly by the
AddContentProtection() methods.
- This is because some MPDs should have ContentProtection at
AdaptationSet level instead of Representation.
- Change SimpleMpdNotifier, which uses MpdBuilder, to add
ContentProtectionElements. The logic is moved from MuxerListener.
- Add Element class for specifying subelements for
ContentProtectionElement.
Change-Id: I9bedfb3e5a5ac0b3d5c702f1e6e4a8608c978d1d
- The UUID and DRM name can be fetched from KeySource.
- Add ProtectedContent message to MediaInfo. The message contains basic
information for the protected content, such as the default key ID for
the content.
- The message is required to separate Representations with different
content protection information into different AdaptationSets.
Change-Id: Ib9dc834ae0abf93b7ca0acdf52a865b1394a4816
- Pass pixel_height and pixel_width from VideoStreamInfo to MediaInfo.
- par can only be set if all the Representations have the same
picture aspect ratio. That is the ratio of sar_x*width:sar_y*height
should all be the same.
- If there is a Representation that does not have the sar attribute set,
par is also not set.
Change-Id: Id34c95e4a99da9ce028a9f35737cfe5bca9e5697
- Adding YCM configuration file for edash packager.
- For now the developer has to do a Debug build before YCM starts
working.
- Also PATH must be set to where 'clang' is so that the system headers
are included (e.g. for classes in std namespace).
Change-Id: I934f69fa6d70b930bde3fd684fa5d6a8204c5631
- Added logic to container parsers to extract pixel width and height.
- If the container doesn't contain the pixel width and height of the
stream and the stream is H264, this parses the SPS to get the
sar_width and sar_height.
- If extracted sar_width and sar_height are 0s then they imply 1s.
- Add hb2_v_frag.mp4 for test media. This does not have a
'pasp' box and has sar_width = 8 sar_height = 9.
Change-Id: I4a06ce95582547bec19adb7905e7612c5a1f359e
- The fields used to be repeated but there is no use case at the moment.
- DASH IOP explicitly disallows segments with multiple tracks.
Change-Id: Ia0af2048210c546dfaa178735605a81052ea1123
- DASH IOP requires AdaptationSets with video Representations to have:
@maxWidth (or @width if all the Representations have the same width).
@maxHeight (or @height if all the Representations have the same width).
@maxFrameRate (or @frameRate if all the Representations have the same
frame rate).
Change-Id: I1247b7461237255aeb70b7fb40f78d4439f9c529
- Get the first sample duration in Segmenter.
- Pass the value all the way to Representation thru MuxerListener.
Change-Id: I76fd970f8140d359863363dc347958f680cca5ae
BoringSSL removes the variants that had been used, in which the hash
is given a default value. This change makes the code more compatible
with BoringSSL.
Change-Id: I665084c2e71593db1afd6baf182224fe0d23c944
mp4 code was changed earlier which uses openssl in some way but the
dependency was not updated.
Issue: 15
Change-Id: Icababe00753f739832d61bc6792d95321cc59cf5
The input source may produce inputs with very small sizes. Demuxer
needs to accumulate enough bytes before trying to detect the container
type.
Issue: https://github.com/google/edash-packager/issues/11
Change-Id: Ie25339832a826e78f39b3b25abb98c1ad89e3021
Due to various libraries' inconsistencies with using EoF, this change
will instead continue to read a file until the number of bytes read is
0. It considers bytes_read == 0 to mean end of file.
Change-Id: I90a592d6ae2b9879fe21bd0c4f9e1c9cbc680afe
Find curl_ca_bundle script is used by linux only. Move it out of
gyp and make sure it runs only in linux.
Also make zlib deps generic for all systems, not for linux only.
This is the first step of getting edash_packager builds on Windows
and Mac.
Change-Id: I82d3e5872fd0dd6f57127890bc062490332a1b69
Increases the read/write buffers for packager input and output files to
2MB from 256KB, which helps throughput when using network file systems.
Change-Id: Iff5d1b796ba4648f1436e0559c0ca2d353b6c4b5
CreateTemporaryFileInDir uses mkstemp which works only on local files.
This change enables the use of other types of files, like network file
as temporary directory.
Bug: 19246995
Change-Id: Ic752725e56b65eec2cd10a4e5d760c6f76ba4f18
If a stream has no language metadata, or has the wrong metadata, this
field allows the user to override this on the command-line.
This also maps all specified languages on the command-line to ISO-639-2
tags as required by the MP4 muxer, so that ISO-639-1 tags specified on
the command-line do not cause a DCHECK to fail.
b/18613148
Change-Id: I473baeecbb3d388db5e06d080179ec6a332b4794
Additionally, for XML schema correctness, convert ISO-639-2 language
tags (three-letter codes) to ISO-639-1 tags (two-letter codes) when
possible. This follows BCP-47, which says to always use the shortest
language tag when there are multiple possibilities.
b/18613148
Change-Id: I120fb7b42ac4da5feb4ca046bba93d9e58acd7a9
MSE requires 'base-data-offset-present' not to be set, but FFmpeg may
generate fmp4 files with this flag though it actually uses moof as base.
Log a warning instead of failing directly in this case.
FFmpeg may also generates fmp4 files without tfdt box
(TrackFragmentDecodeTime) box, which is again non-MSE compliant. Handle
this scenario properly.
Bug: 18613712
Also fixes a test failure in SegmentTemplateTest introduced in an earlier
change.
Bug: 19235748
Change-Id: I4bcbb675b22a832a88cd33ee64c3e99a1c6e3a63
Fix "Problem with the SSL CA cert" issue @
https://github.com/google/edash-packager/issues/6. On Ubuntu, CURL_CA_BUNDLE
is located at "/etc/ssl/certs/ca-certificates.crt" while on CentOs, it is
located at "/etc/ssl/certs/ca-bundle.crt".
With this change, GYP scans the common appearing locations for SSL CA cert.
Change-Id: I838bae3954ec51f1430c900d1e82596d45a4db84
KongQun Yang
Aaron Vaage
Michael Stillwell
Haoming Chen
Kamesh Devarakonda
l-law
Joey Parrish
Widevine Buildbot
Ivan L. Isaev
David Cavar
Yohann Connell
Rintaro Kuroiwa
Haoming Chen
sylt
Zajcev Evgeny
sylt
Chun-da Chen
Kyle Alexander
sammirata
Thomas Inskip
Gabe Kopley
Jacob Trimble
Bei Li
Anders Hasselqvist
Matt Brannock
Ramji Chandramouli