Commit Graph

792 Commits

Author SHA1 Message Date
KongQun Yang feb7951f33 Add docker files for testing on various linux distros
Change-Id: Ic05e2dc326a2b70c43132118d78f6d496d8b7e18
2017-12-12 22:51:05 +00:00
Kamesh Devarakonda 4e3a24fb41 Determine chunking point
Change-Id: Ic5690f803c1a2c22b80785aefbc1f1cef2f11c38
2017-12-12 14:24:28 -05:00
Kamesh Devarakonda 7e46ad1021 Capture SCTE35 events
Change-Id: Id06239ed4820104fecbca08a4103a47ecbed5eda
2017-12-12 01:20:52 +00:00
KongQun Yang b3194866a6 [HLS] Add CHANNELS attribute to #EXT-X-MEDIA
As specified in HLS spec (https://tools.ietf.org/html/rfc8216):

All audio EXT-X-MEDIA tags SHOULD have a CHANNELS attribute.  If a
Master Playlist contains two Renditions encoded with the same
codec but a different number of channels, then the CHANNELS
attribute is REQUIRED; otherwise, it is OPTIONAL.

Fixes #299

Change-Id: Ic2308c39b170178b11cb0d94c3a8083c8b5f3353
2017-12-12 01:04:21 +00:00
KongQun Yang 8e96dd3b37 Address gcc build failures in some platforms
Seeing some failures on some platforms when compiled with clang
disabled:
  GYP_DEFINES="clang=0" gclient runhooks

Several changes to make it work:
1. Mark packager code with packager_code=1 in GYP definitions.
2. Disable a few checks in non-packager code, which we do not have
   direct control: dangling-else, deprecated-declarations,
                   unused-function
3. Fix the relevant errors in packager code.
4. Revert HAVE_STROPTS_H in curl config which is not available in
   all linux distributions.

Fixes #286
Fixes #293

Change-Id: I729b41f99403c5ad9487c6cc4a7dc06f6323cef8
2017-12-07 11:45:26 -08:00
KongQun Yang e97c89391a Address flakiness in ProducerConsumerQueueTest
1. Expect the time elapsed to be greater than kTimeout instead of
   approximately equal to kTimeout if there is a "wait";
2. Expect the time elapsed to be smaller than kTimeout instead of
   approximately equal to 0 if there is no "wait".

Also rename a variable in packager.cc to better reflect what it is.

Change-Id: I67975a6263b8dbc1124b78feae0f8e0d112bda50
2017-12-05 22:56:22 +00:00
KongQun Yang 9631ec71de Roll curl to 7.56.1
Fixes #285

Change-Id: I3cda80974e8cc66f11763c41c36d832565077374
2017-12-04 19:28:27 +00:00
KongQun Yang 7097270f13 Roll boringssl to fc9c67599d9bdeb2e0467085133b81a8e28f77a4
Change-Id: I8ff75ede6c5b6a51719163d208c96d6ec0be61bc
2017-12-04 19:27:34 +00:00
KongQun Yang 82e0fd2095 Default multicast interface address to "0.0.0.0" if absent
It aligns with what a lot of other apps are doing, e.g. tsplay.

Also added notes in docker to adjust network setting if wants to use
multicast in docker container.

Change-Id: I666a8979cc041c904f7ff8e3e022f800c0830d5d
2017-12-04 19:21:52 +00:00
KongQun Yang 0fdb0d02aa Log an error when seeing unsupported stream type in MPEG2TS
The error is logged only once to avoid log spamming.

Added definitions for the unsupported stream types for easy look up.

Change-Id: I097e2f05759bc84ef03f264cfabd2fb20da7c711
2017-12-01 14:56:22 -08:00
Kamesh Devarakonda e7540bf303 Dispatch Scte35Events.
This patch also fixes a bug which was introduced in
https://widevine-internal-review.googlesource.com/c/packager/+/38040
where it was inconsistently assumed that AdCueGenerator would be MIMO
while connecting with other handlers and at the same time it was assumed
SISO in the AdCueGenerator class. Now we assume SISO everywhere.

Change-Id: Icd8c40b5ccfe0d98f47f09a60ea1635f74dceef9
2017-12-01 02:23:36 +00:00
KongQun Yang 068e220ac7 Handle additional unused 'mdat' properly
The spec allows having more than one 'mdat' boxes even if it is not
used.

This is happening on some mp4 files in the wild:
http://www.sample-videos.com/.

Fixes #298.

Change-Id: I729cc94bee095560b5c4b3ee8511323f25f7ad5a
2017-12-01 00:28:30 +00:00
l-law 22c758ed06 Fix order of VPS, SPS, PPS in hvcC box of HEVC/MP4 (#300)
Fix order of VPS, SPS, PPS in hvcC box of HEVC/MP4

Fixes #297
2017-11-30 14:34:42 -08:00
Kamesh Devarakonda cc778f60ab Defined CueEvent struct.
Change-Id: If4ba55a7ff403248de9d0411c2ba8553b88c491b
2017-11-28 14:26:32 -05:00
Kamesh Devarakonda c4e7d02b80 Connect AdCueGenerator in the pipeline.
Change-Id: Iaae73661bc57ddecd2cc103dbd34d0c3102ffdaa
2017-11-27 21:09:26 +00:00
Kamesh Devarakonda 154e309f3f Remove unused argument.
Change-Id: I1ee3ca6527ec4ecce7dd57e18eee70a8c6a11eb0
2017-11-21 15:04:36 -05:00
Kamesh Devarakonda dac0328b47 Parse ad_cues flag.
Change-Id: I4f5897dcde20e73a01468b80a04d3f3142bc5e42
2017-11-20 16:39:16 -05:00
Kamesh Devarakonda 622ff64595 Add skeleton AdCueGenerator handler
Change-Id: I53ba982eb76c7f61122e226d295e6989fe425ae5
2017-11-17 00:06:27 +00:00
Kamesh Devarakonda dcf2d2fc0b Created a build target for files in media/public/
This new build target is explicitly listed in the packager/packager.gyp
file.

Change-Id: I49e68ea64ae9a9a30204a269a1af17744f9d25d7
2017-11-16 00:24:17 +00:00
Kamesh Devarakonda 8c877379b6 Defined Scte35Event struct and out of band cue marker flags
Change-Id: I5a4db1a691544cbedf2ccfbaf6f71169a8eed6e7
2017-11-16 00:13:29 +00:00
KongQun Yang 5cf2b17ade Add --hls_key_uri to allow setting fairplay/identity key uri
Change-Id: I52e0f56cd10390faf515170f407ea488a6b3c9fc
2017-11-14 15:49:27 +00:00
KongQun Yang 61e36d7d21 Support encrypted E-AC3 in TS
Implemented E-AC3 MPEG-2 Stream Encryption Format for HTTP Live
Streaming specified in https://goo.gl/1sgcwY

Issue #279

Change-Id: I36c1a05e3d0529ff810eaf52bdca45414baa93eb
2017-11-13 22:54:07 +00:00
Kamesh Devarakonda 09cdb2bbec Removed unused structs PeriodInfo and MediaEvent.
Change-Id: I0504853ec555fd161df21e5680d26aab50648052
2017-11-09 17:28:31 +00:00
KongQun Yang 0a69779f7c Fix a parsing problem with AC3 in TS
Audio samples per frame for AC3 was not specified correctly earlier.

Also the number of channels is not correct if LFE channel is on.

Issue #165

Change-Id: Ibf20aa4c7aec43c07ec7cd394d631c537cb387dd
2017-11-07 18:47:09 -08:00
KongQun Yang 72df5af150 Support encrypted AC3 in TS
Implemented MPEG-2 Stream Encryption Format for HTTP Live Streaming
specified in https://goo.gl/N7Tvqi.

This change also moved ProgramMapTableWriter creation from TsWriter
to TsSegment.

Issue #165

Change-Id: Ia89dd16a5e6405706dd3d85ec6b6de580f7b13a7
2017-10-30 15:22:22 -07:00
KongQun Yang d7f531fe10 Add AC3 support in TS
EC3 and encrypted AC3 in TS are not handled yet.

Partially address #165

Change-Id: If4839ee7801eac902e64e9c677dd37709ec8e88b
2017-10-30 15:11:43 -07:00
KongQun Yang ad836a5cf1 Rename EsParserAdts to EsParserAudio
Change-Id: I8d9fc265a4cf740970256e1a46b172349f1bf794
2017-10-27 23:58:50 +00:00
KongQun Yang 79a54d7748 Cleanup ProgramMapTableWriter
Change-Id: I08093a0fc35dc15a20f6e72166ed46d3ab94b334
2017-10-27 23:58:32 +00:00
KongQun Yang 2a2493ec2e Refactor EsParserAdts to handle other audio
Rename will happen in next CL

Change-Id: I28df29bb264eea7a6225b0a4a2eb38de7968f9d6
2017-10-25 23:21:35 +00:00
KongQun Yang 867244645a Implement BitWriter
Change-Id: I5b7dfc2bc07227c2fe05f6de8857042bcab944c4
2017-10-25 16:19:12 -07:00
KongQun Yang b5a8185543 Refactor ProgramMapTableWriter
To make it easier to support more codecs.

Change-Id: Ifcde0f3d7d6f74015d723a0b74401a86c8e65a72
2017-10-25 20:25:56 +00:00
KongQun Yang d66d307fc2 Rename fixed_key to raw_key
Change-Id: I1ee9398387b54aa685a5bef5b8704849e62ebae3
2017-10-24 21:52:01 +00:00
KongQun Yang 7c38963c29 [doc] Update raw key tutorial
Change-Id: Ife841975c036cf8042c9775583c409e0daa089c7
2017-10-24 21:51:39 +00:00
KongQun Yang 1bf1ec2445 Add multi-key support in fixed/raw keys
The keys can be specified in --keys option, with the form of
  label=<label>:key_id=<32-digit hex string>:key=<32-digit hex string>,label=...

There can be multiple "label=..." block. The DRM label can be one
of "AUDIO, SD, HD, UHD1, UHD2" or a custom label.

Mark --key and --key_id as deprecated. We can achieve the same result
with --keys=label=:key_id=<key_id>:key=<key>.

Also add a new optional field in stream descriptor: drm_label, which
is used to overwrite the internally generated DRM label for the stream.
If not provided, the DRM label is generated automatically based on
audio/video information, e.g. resolution.

Code changes:
- Merged RawKeyEncryptionParams and RawKeyDecryptionParams into
  RawKeyParams.
- Make FixedKeySource accepts RawKeyParams as input.

Change-Id: Ic8c2f071cc71188e13f14bc6396fc2b3ffa5cac6
2017-10-24 21:04:49 +00:00
KongQun Yang 27c1900f12 [pssh-box] Support generating Widevine PSSH without content-id
Widevine PSSH with key-id and without content-id is now be supported.

Change-Id: I356590b79f79ff4c0310b22e5dc19b6d24ec4cee
2017-10-18 21:38:23 +00:00
KongQun Yang c9cc68ba82 [Playready] Allow unencrypted client cert private key
Change-Id: I3244b428f8e3e32787520d9dd0f015e6d9301fe0
2017-10-18 21:37:36 +00:00
Joey Parrish e2bb22c633 Update docs for HLS descriptor fields
These fields are no longer required, and have sensible defaults.

Change-Id: I35bd489c8e6058f3272210cb6f3b0f504819e7c0
2017-10-17 11:37:44 -07:00
KongQun Yang e889578a2d Creates directories in the output path if not exist
Also added end to end tests for single segment HLS and multi-segment
HLS.

Fixes #276

Change-Id: I27e5d57cdc3ab79a8e5ae0c3a8da46fcf2c467b6
2017-10-11 20:19:51 +00:00
KongQun Yang c87c4c2327 [HLS] Fix single segment ULR isn't respecting folder location
Fixes #253

Change-Id: I61de1b28807ff8be9a881edd3adaa36612713dd8
2017-10-11 00:14:09 +00:00
KongQun Yang 0861452932 [Test] Rotate key_ids in PSSH as well for key rotation
Note that fixed key / raw key rotation should not be used in production.

Change-Id: I87fca41ebd00f3cadbf157ff129b8e6a9ca04e7c
2017-10-09 16:44:14 -07:00
KongQun Yang ae17159b73 [HLS] Fix init segment URL isn't respecting folder locations
Problem occurs when the media playlists are in a sub-directory under
master playlist. If base_url is not set, segment URL should be
relative to the media playlist.

This only affects fMP4 as TS does not have init segment.

Fixes #253

Change-Id: Icddd9ed500d0a705e8b3260bfd4e916ecbba3f28
2017-10-06 17:19:13 -07:00
KongQun Yang 94a64bef03 Fix WebM parsing crash when the source content contains color elements
Also updates outdated comment in packager.h related to multi-manifests.

Fixes #275

Change-Id: Ia2eb235607860f92a3c19bd791d711209da85c08
2017-10-06 22:48:04 +00:00
Aaron Vaage 65d5ecc3cf Separated Text Processing From Audio/Video
To help isolate the differences between audio/video and text in
preparation for adding the better text support, this change moves
all audio/video job creation and text job creation into their
own functions.

Change-Id: Ia71cf82ac147672b70c69123eeabdfc6b41c4a3d
2017-10-02 11:31:20 -07:00
KongQun Yang 937ecfbdcd Move language tag convertion to packager.cc
This fixes shared_library not able to compile problem.

Also removed StreamDescriptorList and replaced with expicit
comparison to make the code clearer.

Also removed an out-dated DCHECK that mpd_notifier and hls_notifier
does not co-exist.

Change-Id: I3d04b8880741fecf0931a764cc12e50aa5e392b9
2017-09-29 18:05:03 -07:00
KongQun Yang 9a60760815 Change MediaSample::CopyFrom to MediaSample::Clone
Change-Id: Ibc8e6ba149496bb62179230e95aa6803a015b12b
2017-09-29 09:58:39 -07:00
KongQun Yang 92e1e39868 Update MediaSample to avoid redundent copying
Use std::shared_ptr<const uint8_t> with a custom deleter to
represent MediaSample::data_ instead of std::vector<uint8_t>.

MediaSample::data_ can be shared by multiple MediaSamples and it is
immutable. A new data instance must be created if the clients want to
modify the underlying data. The new data instance can be transferred
to MediaSample using provided MediaSample::TransferData function.
This avoids unnecessary data copying.

Change-Id: Ib59785a9e19d0abb3283179b12eb6779ee922f79
2017-09-28 14:01:30 -07:00
Aaron Vaage 6c0f2bebef Use TrickPlay Test Structure As Common Test Structure
Took the test structure for trick play and made a common test
structure that can be used for new tests. The next tests to
use this structure will be the new WebVtt pipeline.

Change-Id: I5faca306b8a2bd16b9f945d1ff5c336aed82bc35
2017-09-27 22:10:08 +00:00
KongQun Yang 9cb4018235 Add check formatting script
The script can be set as pre-commit hook to run clang-format
check automatically on every commit.

Change-Id: I05c8e5266ebc773c7949c64d7a885809070a58ec
2017-09-27 21:52:34 +00:00
Aaron Vaage 23a2893a35 Define Text Sample Matchers and Test Functions
This change create some matchers and text functions that
will be used by the upcoming webvtt work.

Change-Id: I2d7931d1bdabd761f74e8aee7524d5ecfb7cfdc4
2017-09-27 09:44:56 -07:00
Aaron Vaage 8067fd4818 Made Trick Play A 1:1 Handler
Before the trick play handler would have one input and multiple
outputs. The normal media handler API for adding handlers had to
be ignored when setting-up the trick play handler.

This changes the trick play handler to have one input and one output.
It uses the standard AddHandler method calls. When initialized, the
trick play factor must be provided.

This included a run of 'clang-format' over all edited files.

Change-Id: I7b3cdf0a2472e2f89ab194867c4b984e26f18f24
2017-09-26 08:25:04 -07:00
Aaron Vaage 31bf6dab59 Separate MediaHandlerTestBase From FakeMediaHandler
To allow more tests to use the MediaHandlerTestBase and get access to the
helpful creation methods, this change moves the use of FakeMediaHandler
and setting up the graph to a sub class.

Change-Id: I90dd151e3c96d8fbe4bd02a1d6b11e66a279d95b
2017-09-25 16:41:06 -07:00
Aaron Vaage b5800fad77 Created Mock Input and Output Media Handlers
Created media handlers that make testing a handler easier by providing
methods to send messages from upstream and check messages downstream.

Change-Id: I7b3ba3a51bcf0d73bcaacd46b2dcbc16f87d8694
2017-09-20 15:22:12 -07:00
Aaron Vaage 304f7d2a88 Rename "crypto_handler" to "encryption_handler"
Change-Id: I44307fad6f4f0c4c2f8b61c3c6b07867859c9867
2017-09-20 19:13:01 +00:00
Aaron Vaage 3495025051 Removed unused Muxer Options
Change-Id: Ibdca7ac59cbb9728cc3308b33f9d3d4b96a7d9dc
2017-09-20 10:52:38 -07:00
KongQun Yang 21b0ca6ea5 Cleanup README
Moved build instructions and docker instructions to separate docs.

Removed command samples which are no longer needed as we have better
tutorials now.

Change-Id: I340c5653a6553158325970dd86d76fa11e69bcbd
2017-09-19 18:44:39 -07:00
KongQun Yang 327cb8df88 Update WebM error message when segment is too big
To make it clearer.

#271

Change-Id: I6abfe94b417be293320516cfa654d909412e36c0
2017-09-19 22:30:39 +00:00
Aaron Vaage 16eff80497 Made StreamData Have Const Pointers
In prep for changes to Trick Play, we want to make all messages
copy on write so that if the same message is sent to multiple
handlers, it is not possible for one handler to change the data
another handler is using.

Change-Id: I554166ca11c532412e4dfced5603972ca24dc2bb
2017-09-19 21:06:29 +00:00
KongQun Yang bc903d2d83 [HLS] Fix segment URL isn't respecting folder locations
Problem occurs when the media playlists are in a sub-directory under
master playlist. If base_url is not set, media segment URL should be
relative to the media playlist instead of the master playlist.

Also make sure the path separator is "/" in URL instead of "\" on
Windows.

Fixes #253

Change-Id: I8e933750276435d94dd01bfa53ee2bc050dfd193
2017-09-18 19:11:42 +00:00
KongQun Yang 9c861d03f7 Report an error when disk is full
Previously packaging completes successfully without any error
or warning.

With the fix, an error will be reported if write fails. It may
appear as "Cannot close file error" as we use threaded IO, which
could delay the error reporting until Close() call, so the user
of the File API needs to make sure Close() returns successfully.

Also fixed a deadlock in threaded_io_file if internal_file->Write
fails.

Fixes #160

Change-Id: I17f945150fb4021d2dcdbe784e557673f53ca583
2017-09-15 22:22:12 +00:00
KongQun Yang 7627871428 Update Doxygen config file
Doxygen should run under src/ directory:

$ doxygen docs/Doxyfile

Outputs are created in out/doxygen.

Change-Id: I9dd1a9aa158851d37ff9a2e37f701757eb3e577c
2017-09-15 22:02:32 +00:00
Aaron Vaage 28385c0fc0 Change Text Sample set_style to AppendStyle
To ensure that the style data is formatted correctly, changing the
set_style method to AppendStyle so that joining values will be
done in controlled setting.

Change-Id: Ia88e2d260b11339e6777ff018813e8aeb802a501
2017-09-14 18:37:42 +00:00
Aaron Vaage a9d957e7f0 Move Muxer Creation To Own Method
Moved all the code to do with creating a muxer to its own function to
help isolate it.

Change-Id: Ib259da7aaa41d325632d5f9bb3d54ae9df8d0e31
2017-09-11 12:23:07 -07:00
KongQun Yang ea45ce3158 Add support for callback file
Change-Id: Ieb116bf3f645a35601f1182ed139c59ddaab8ad8
2017-09-07 19:26:26 +00:00
Aaron Vaage 7c5508555c Made Verify Functions Return Status
There are multiple functions we use to verify input from the user
and must of them would return a boolean expect for the top level
function which would return a Status. Each function would log a
message but to the user the error was not clear. This change makes
each verify function return a status so that the actual problem
will be surfaced easier.

Change-Id: I12ab43f8ca3a4d379b4309336644a70eb8cbbbe0
2017-09-06 22:22:55 +00:00
Aaron Vaage f2cbe8f9c1 Moved Encryption Handler Create to Own Method
Moved all the code used to create the encryption handler for a stream
to its own function.

Change-Id: I8801a194b470d899aa9fb26b4664e36e61d5d164
2017-08-30 16:26:41 -07:00
Aaron Vaage ec83d8a73a Implement MsToWebVTTTimestamp
Implemented the other end of the webvtt timestamp parsing as it will
be needed when we write out WebVtt files. Also changed the name of
the parse function to be more inline with the writing function.

Change-Id: I1f36ddbbf80028732b4cb1b15e871dec17767f63
2017-08-30 18:30:43 +00:00
Widevine Buildbot f20ad03caf Merge remote-tracking branch 'github/master' into HEAD 2017-08-29 20:46:52 +00:00
KongQun Yang b5e6cf36ab Remove the all zero KID in manifest for key rotation
Closes b/64035941

Change-Id: I382b6478889dd73bc422decf0cbf4d32cd9bb68d
2017-08-29 20:44:32 +00:00
Ivan L. Isaev 1706b0bc3a multi-manifest (e.g. DASH + HLS) output support
Fixes #262
2017-08-29 13:42:33 -07:00
Aaron Vaage 2539920a5d Return Status from CreateRemuxJobs
Inside of CreateRemuxJobs, we would take a status value from another call,
log the message, and return a boolean. The caller of CreateRemuxJobs would
then create status from the boolean.

Now we will pass the status value from the other call directly to the caller
of CreateRemuxJobs. So that failures can be better communicate in our code.

This includes a pass of clang-format over packager/packager.cc.

Change-Id: I071e7bad13f916e963641b9e53699573fe1060ed
2017-08-28 12:57:23 -07:00
Aaron Vaage 591fd5456e Cleaned Up RunJobs
Rewrote the RunRemuxJobs function to use waitable events so that we never
get stuck waiting for a job to finish before getting to a thread that
has already exited.

This should allow us to end execution when a job failed rather than
waiting for all jobs before the failed job to finish.

Closes #94

Change-Id: I413f62561a7a4cab83b8905e75986230b6c498bc
2017-08-22 23:03:44 +00:00
KongQun Yang 675f2631b8 Move HlsParams to hls_params.h
Change-Id: I25189623512834da8bd998c388c4cb0edebc6f47
2017-08-18 11:57:41 -07:00
KongQun Yang 556ff56559 Consolidate MpdParams into MpdOptions
Change-Id: I7f37277ef87597548c2fbf219963cd8725ae4c05
2017-08-18 11:57:41 -07:00
KongQun Yang 4e82ab13bd Consolidate Mp4OutputParams into MuxerOptions
Change-Id: I533e40ef9eb84453c15cbdb7ef42fc85bef5125d
2017-08-18 11:57:34 -07:00
KongQun Yang 9eaf1dcae0 Consolidate ChunkingParams and ChunkingOptions
Change-Id: Ibbb85981d2b424432b61ca693e28334ef00d475e
2017-08-18 11:55:45 -07:00
KongQun Yang c7c201b00c Consolidate EncryptionParam and EncryptionOptions
Change-Id: I8a5b5c329d4731341a3184d50bed061aa00cdeda
2017-08-18 17:37:45 +00:00
Aaron Vaage e605203fa7 Added Origin Handler
Created a handler called OriginHandler that can be used by the Job class
(previously called RemuxJob). Origin handlers represent the start (or
origin) of a pipeline (chain of handlers).

Change-Id: Ibd748ae0a932b6e0ebb879ea292fcb83c548214b
2017-08-18 09:20:10 -07:00
David Cavar 34c5e011a5 Fairplay key system support
Support generation of fairplay key system tag: "com.apple.streamingkeydelivery" when --pssh 
includes fairplay key system id:

// Unofficial fairplay system id extracted from
// https://forums.developer.apple.com/thread/6185.
const uint8_t kFairplaySystemId[] = {0x29, 0x70, 0x1F, 0xE4, 0x3C, 0xC7,
                                     0x4A, 0x34, 0x8C, 0x5B, 0xAE, 0x90,
                                     0xC7, 0x43, 0x9A, 0x47};

Closes #258
2017-08-08 10:31:40 -07:00
KongQun Yang 085e038b89 Include /etc/ssl/ca-bundle.pem in curl ca bundle
This fixes cannot find curl ca problem on OpenSUSE.

Change-Id: I6c42abb48700d4c5c857cdeb647aa421681e6365
2017-08-04 12:28:02 -07:00
KongQun Yang 9e3dc06d20 Fix file permission issue with manifests
ImportantFileWriter::WriteFileAtomically uses mkstemp internally,
which set file permission to 0600, which is not what we want.
Update the code to not use mkstemp instead.

Also updated temporary file name logic to include process id in
the name so it can be unique across processes.

Fixes #259

Change-Id: I2d5a375925cf552bc0db5269f409d7522e63fca5
2017-08-01 15:42:55 -07:00
KongQun Yang 723079310f Disable IP_MULTICAST_ALL when setting up UDP multicast
This avoids interference caused when two sockets are bound to the
same port in different multicast groups.

Fixes #241

Change-Id: Iba1b3300e3850f1f268886b7a49f5a3315d95b80
2017-07-27 18:02:51 -07:00
KongQun Yang b451d3a7ca Consolidate various playlist_type types
Change-Id: If6fafc0867e7e1f3cd2ed612617e91c7cb144f20
2017-07-21 20:43:14 +00:00
KongQun Yang 61c58724ec Remove size parameter in OnMediaEnd
Derive file size from media ranges instead.

Also fix subsegment range reporting for WebM (internal variable,
no output affected).

Change-Id: I5f8152dff4c2cd5fbae5550992b86a669e278f7b
2017-07-21 20:42:44 +00:00
Yohann Connell 8d2b87c773 Adding the ability to provide the group_id in Widevine Cenc requests
Change-Id: I4903cc9422433f20b76d084578d8fc06d7ed45a8
2017-07-20 09:46:25 -07:00
KongQun Yang cc92cca6b9 Enable C++11 in gtest
Change-Id: I2860a34eee12bc2bdb584cc89c871da0ffb127ff
2017-07-17 20:14:23 +00:00
KongQun Yang eb98a87f7d Move packager/media/file to packager/file
Change-Id: Idc0d75a7ceab7c8ce460ef896fdefdec088124bb
2017-07-17 12:08:28 -07:00
Rintaro Kuroiwa a4c227f74a Add quotes around BYTERANGE attribute
- EXT-X-MAP BYTERANGE attribute requires quotations around it.

Change-Id: Iee073b4c40bbf41b4dc319e8a447905c004cea3b
2017-07-13 17:15:09 -07:00
Rintaro Kuroiwa 013a83c9d4 Change MediaPlaylist to return display resolution
- The master playlist should have the display resolution instead of the
  encoded resolution.
- Changed MediaPlaylist to return display resolution.

Change-Id: I162727d0bdeed0302518286e42a22b69a58fc4a3
2017-07-13 13:25:36 -07:00
Rintaro Kuroiwa d096f4b485 HLS byte range
- Use byte range format if segment_template is not set in MediaInfo.
- Change HlsNotifier::NotifyNewSegment() to take start byte offset.
- Change HlsNotifyMuxerListener to cache the segment information if
  MuxerOptions::segment_template is set.

Change-Id: Ida1b8b9198acc5c43e3a5d9a641c78c81bc9a6ac
2017-07-12 17:19:56 -07:00
Rintaro Kuroiwa 0dfdace280 Change MuxerListener::OnMediaEnd to take ranges
- Changed OnMediaEnd() to take (sub)segment ranges.
- Define a structure to pass the ranges.
- This requires muxers to "cache" the segment ranges but in most cases,
  they already do it, e.g. 'sidx' for mp4.

Change-Id: I16b974c1f0f54ca658cc6dbe605efff84a3f52eb
2017-07-11 15:59:16 -07:00
KongQun Yang 2db1867951 Write manifests atomically
In the original manifest updating process, the file was truncated first
before was written with new contents. There is a small chance that
the web server may read back empty file or incomplete file.

The new code makes the update operation atomic (by writing to a temporary
file first then replace the old file with the temporary file).

Fixes #186

Change-Id: I2fd564cb12b922b032c0e9f70d2132a5b12ff098
2017-07-07 11:05:19 -07:00
KongQun Yang 11210b400a Workaround Edge problem with non-zero clear lead
Somehow Edge requires sinf box to be put after avcC box:
https://developer.microsoft.com/en-us/microsoft-edge/platform/issues/12658991/

b/37913785

Change-Id: I201c96768328b3910001dba49a9ddda7f06f3c20
2017-07-06 22:48:09 +00:00
KongQun Yang 627af1558e Use VLOG(1) instead for unsupported stream type
Change-Id: I8ec3027edefa07f32bba3d3b39d0804d706d3798
2017-07-05 23:09:58 +00:00
KongQun Yang 3bad4ca700 Move status.h under packager/
Change-Id: I4a7f34c575c1c8b9a9a275f3889cf8641cfdd471
2017-06-30 11:35:18 -07:00
KongQun Yang e56d1faaf0 Make HLS with key rotation work
- Remove the TS key rotation not supported check as there is nothing
  to be done for key rotation in TS for SAMPLE-AES.
- Fix IV updated problem in new segments even if crypto period does
  not change.
- Avoid duplicate EXT-X-KEY tags if it does not change.
- Make EXT-X-DISCONTINUITY-SEQUENCE match with number of removed
  EXT-X-DISCONTINUITY.
- Added end to end test for HLS with key rotation.

Change-Id: I73cb82e9f5575fcdf63ee643228efe78e6766302
2017-06-29 21:03:36 +00:00
KongQun Yang 5bad2fbd5c Update the description for @bandwidth to make it clearer
Change-Id: Ib5a084c0ffea9efe0b4173248daddc53498c4498
2017-06-24 01:09:55 +00:00
KongQun Yang b4aa34b803 Support live and event HLS playlists
Closes #114

Change-Id: I6440383c3dc022fba5a9926839f79b9ed8a51f55
2017-06-24 01:09:32 +00:00
Rintaro Kuroiwa e2401f02ec Make NaluReader skip encrypted portion
Change-Id: Ibb47a1e62cd8ac3057c8f1512a88280991e48b62
2017-06-22 22:09:59 +00:00
KongQun Yang a678948db4 Update gyp to generate standalone static libpackager
Change-Id: Ic62b2e4e8a9529da936aa5c64e93faad7187a79a
2017-06-22 13:38:27 -07:00
KongQun Yang 96ced40ec2 Move command parsing tests into separate class
Change-Id: I6aeba632d69b1867c7d7745f0c8c9fd8f8e275d9
2017-06-20 22:33:55 +00:00
KongQun Yang 03889e6465 Move hex string out of packager.h
Also added validation failure tests in packager_test.py.

Change-Id: I6697a3138d57889110250404205536dd5cd53cd0
2017-06-20 22:33:47 +00:00
KongQun Yang abbd495ad4 KEYFORMATVERSION should be 1 for HLS Widevine key system
Change-Id: I59c7852727b803bcdb1e49d33e36fca7f51e507c
2017-06-20 22:33:03 +00:00
Aaron Vaage 7c48cc4800 Group MediaPlaylists in AddMediaPlaylist
Resolving TODO in packager/hls/base/master_playlist.cc that asked for
media playlists to be grouped when they are added.

Change-Id: I01c7687400bc74ce45f271d0242d1c3bf3610cf2
2017-06-20 14:35:52 +00:00
Haoming Chen 78f4fd9d3f Fix an issue in NalUnitToByteStreamConverter.
- This fixes the issue when one NAL unit has multiple
  subsamples, which is allowed in spec and could happen
  when the clear_bytes exceeds 2^16 bytes.
- The new algorithm updates the subsample entries in
  following steps:
  1) When NaluReader parses a new NAL unit, the
  function splits subsamples and align them with the
  NAL unit. It also finds the range of subsamples
  for that NAL unit.
  2) Drop or update subsamples of that NAL unit,
  based on the type. Other inserted bytes are treated
  as a new all-clear subsample.
  3) Merge those subsamples if possible.
- This addresses #237.

Change-Id: I2cab47cd00d90e29ec52b90d4fde057c632193ac
2017-06-19 20:19:11 +00:00
KongQun Yang 096c22ee63 Rename shaka::ShakaPackager to shaka::Packager
Change-Id: I493d3a6cf4c27615d1515de7c43dc3c5b8e3224e
2017-06-16 23:08:06 +00:00
Aaron Vaage 26d39db0e7 Moved Timestamp Parsing For Easier Testing
Moved all the timestamp parsing code into its own class with its
own unit tests. This is to make testing easier.

Change-Id: I624472baba51dfa1254a9f4bae55ebb79e310855
2017-06-16 18:26:16 +00:00
Aaron Vaage 0caab0a4b4 Add resolution to HLS manifests
The HLS manifests output by the packager did not have the RESOLUTION
attribute on EXT-X-STREAM-INF tags. This change adds the resolution
tag when resolution values are present.

Close #235

Change-Id: I15ca6de47cb34793cf6a940123d593261627baa4
2017-06-16 01:28:55 +00:00
KongQun Yang 1aeedd102e Use free-form strings for stream labels (track types)
Change-Id: I38489acbdfaf4bb491635fdc7f6b0cab77a53574
2017-06-13 15:04:00 -07:00
KongQun Yang 05a5a41969 Support generation of libpackager.so (shared_library)
Shared libpackager can be built by setting libpackager_type to
shared_library, e.g.

GYP_DEFINES='libpackager_type=shared_library' gclient runhooks
ninja -C out/Debug

will generate libpackager.so in out/Debug/lib directory.

Here is a few other changes to make shared_library builds and
tests pass:

- Add several test parameters to packager.h, which is needed for
  testing.
- Create a protoc.gypi from build/protoc.gypi but depending on
  protobuf_full_do_not_use instead of protobuf_lite, since we need
  protobuf_full_do_not_use for text parsing and generation of media
  info proto. Somehow shared_library build does not allow mixed use
  of protobuf_full_do_not_use and protobuf_lite.
- Remove the use of LazyInstance in version/version.cc and use static
  variable directly. This is because LazyInstance needs AtExitManager
  which may not be easy to setup when calling GetVersion.
- Allow skipping testPackageWvmInputWithoutStrippingParameterSetNalus
  with flag --shared_library, which is needed as shared_library build
  does not support --strip_parameter_set_nalus flag yet.

Fixes #227

Change-Id: Iff05a50baa28134faa7218664c96114cb9e70329
2017-06-13 20:42:32 +00:00
KongQun Yang 2816a7262e Integrate packager top level library
Change-Id: Ic2e2ef79d2f5db5aab6565835aaaf6cb1c7e44bb
2017-06-13 13:40:26 -07:00
KongQun Yang db45711750 Implement top level API: packager.cc/h
Change-Id: I2b4fbc0b9f6df5617e9f59173e144c7be1b01d2b
2017-06-13 19:23:34 +00:00
Aaron Vaage 2d2825290c Added TextSample
Added a text sample type that can be passed between media handlers.
This will allow text samples to safely be moved between media
handlers before converting to their final format.

Bug: 36138902
Change-Id: Ic4946f774a7d37c43066b9ea46596d5c5f3c05a8
2017-06-09 16:03:34 -07:00
KongQun Yang d28b19788d Remove misleading frame duration message in live mode
frame_duration is not available when the VideoInfo is created, but
it will become available when a media segment arrives.

Change-Id: I2c24852fca4ff77c2b86ff78dbf0963ad0a0afa6
2017-06-08 22:32:06 +00:00
KongQun Yang a251208a75 Report an error on encrypted stream without decryption
Change-Id: I784af57ca79aeb8a12245bd130f413ea6e3d04f1
2017-06-07 21:49:46 +00:00
KongQun Yang a0bd3dd565 The retired availability_time_offset flag should have value 0
Change-Id: I6a34c362c0c5252853f0883b54c19c434bd1d8b8
2017-06-01 14:55:16 -07:00
KongQun Yang 5ad40df810 Fix misleading error messages in WebM output
Issue #239

Change-Id: I4e6e6aafbeb0e9613769e05bd81353409576876d
2017-05-30 16:29:47 -07:00
KongQun Yang 19be04ecce Fix git not found on some Windows setups
Change-Id: I0403212f383e60ef75b426170783933f4d73b211
2017-05-26 13:52:42 -07:00
Haoming Chen 32b088c4bc Add sample usage for trick play in README.md.
Change-Id: I1c1f4af37b3573ffdbcc4368826417f1d07da437
2017-05-23 18:33:33 +00:00
KongQun Yang 18cdbd4121 Deprecate availability_time_offset MPD flag
'availability_time_offset' is not an attribute in MPD specification,
but used to calculate availability_start_time to adjust segment
availability time. A similar effect can be achieved with standard
blessed suggested_presentation_delay flag.

If suggested_presentation_delay is not set, player is expected to
choose a suitable value.

Change-Id: I1d540bd4347b4f04b6cc5ec0778e1a5392009b09
2017-05-22 23:14:16 +00:00
sylt bfe302bd5c Add support for skip_encryption stream descriptor (#219)
This option makes it possible to control encryption for specific streams,
e.g. for audio and video separately.
2017-05-22 10:17:58 -07:00
KongQun Yang 3443048e80 Fix WebM timestamp overflow if longer than 2.5 hours
Fixes #233

Change-Id: I3431f8e68bfb1de222b8ab967c9cac6a821b4f53
2017-05-17 17:00:44 -07:00
Haoming Chen 27abb208aa Remove the trick_play_factor in media_info.proto.
- Use the playback_rate instead.

Change-Id: I341c39de1c28aacebab5b31aa013bcbbcc2fa9d0
2017-05-17 22:10:49 +00:00
Haoming Chen da8877d8a2 Rename trick play rate to trick play factor.
- Add a "tpf" abbreviation in stream descriptor for
  trick_play_factor.

Change-Id: Ia09313bb9778456b53bdb0a8cc46d2ba8caa2ac2
2017-05-17 21:51:34 +00:00
Kongqun Yang 5fc6e540f3 Include protection_scheme in Widevine CENC request
Widevine license server added support for protection_schemes recently;
If CENC request includes protection_scheme in the CENC request, server
will return with a PSSH with protection_scheme filled in.

Change-Id: I6dcac498b5e039503d6ac0f6e057737f7c53efaf
2017-05-16 01:18:39 +00:00
KongQun Yang d9e7e2f1d0 Generate two sample entries only if there is clear lead
This addresses playready broken with clear lead = 0.

clear lead != 0 is still broken with playready on Edge. We think it is
likely an Edge bug, which does not support multiple sample entry boxes,
thus does not support clear lead.

b/37913785

Change-Id: I7adb77a913dccf669153b03b31b4a1e1c98d1cb0
2017-05-15 14:13:52 -07:00
Haoming Chen 4ba5bec660 Integrate trick play.
- Add trick play adaptation in mpd. This addresses #178.

Change-Id: Id6cc08c788eda074c261d0274768051837b84541
2017-05-09 13:33:11 -07:00
KongQun Yang 1e2da22c88 Add support for WebM colour element
Change-Id: I359cde97171118d3b928a9dd6650d11fade8f4a9
2017-05-02 20:22:17 +00:00
KongQun Yang 84ff940e22 Add init_segment name in --help
Closes #228

Change-Id: I596ae7519632f269e8d3e9cfcf0e96564b248d01
2017-05-02 10:25:31 -07:00
Kongqun Yang 4a47268f3d Fix avc3 codec string problem
- Added wvm decryption and demuxing test
- Added avc3 test (not strip parameter set nal units)

Change-Id: I28ff4d7f9ef539eafe45cfadfc0912cd11c72b05
2017-04-28 19:24:11 +00:00
KongQun Yang 59f393779c Support v1 vp9 in iso-bmff
- Implemented according to v1.0 spec @ https://www.webmproject.org/vp9/mp4/
- v0 is no longer supported

Change-Id: I189c813d788400beda797eea7da943a83dfa7d79
2017-04-27 19:37:14 +00:00
KongQun Yang 4d81979b16 Add -v and --vmodule flags in test script
The flags will be propagated to packager binary if set.

Change-Id: I38aa9f4fb8e23f7a802d4114b5507035ef0cc583
2017-04-26 10:25:28 -07:00
Kongqun Yang fb0790e1d5 Refactor KeySource::FetchKeys to use EME init data types
Change-Id: I517ea992a8868d7382988a8837f4f05df9f974d7
2017-04-20 20:21:29 +00:00
Rintaro Kuroiwa 9d101f85e3 Print the failing command if subprocess.call() fails
Change-Id: Ib640b590e259a1a9bd44030ae13611cd359721e0
2017-04-18 16:33:35 -07:00
Kongqun Yang 75a4b55b34 Fix AAC-HE not correctly signaled in codec string
- Also fixed a bug with dts audio in esds.

Closes #225

Change-Id: I50ca731ac81bedfec37b0e2d577c51d643b7a839
2017-04-13 23:40:23 +00:00
Zajcev Evgeny a24444cc1b [fix] kAdtsNumChannelsTable: Standard specifies 3 channels (#226)
CHANNEL_LAYOUT_SURROUND = Front L + Front R + Front C
2017-04-11 10:23:32 -07:00
Rintaro Kuroiwa a3b5e7d01a Widevine HLS new key format manifest side
- This changes the manifest side for the new key format.

Change-Id: I329f3c84605e5d2180e219643cb17282152f687b
2017-04-10 19:05:23 +00:00
Rintaro Kuroiwa c2e019ed31 Output command line when packager tests fail
- Changed python packaging test to output the command line it ran for
  tests that failed.

Change-Id: I52c06bcec725c9dce7f845ab138cfab29b582201
2017-04-05 20:05:01 +00:00
Rintaro Kuroiwa f3e91301c2 Widevine HLS new key format media event side
- This change has the muxer listener side changes.
- Also changed SetContentProtectionFields() to take vector for
  default_key_id so that MuxerListener subclasses don't need to convert
  it to string.

Change-Id: I8832848a0c7e34b84e2b4e6baeba364861900ff1
2017-04-05 11:12:57 -07:00
Kongqun Yang 437d48f041 Fix Windows 64 break (#222)
Change-Id: I88bfdee9020a6abb96d6d38397263c08773501df
2017-03-31 14:37:02 -07:00
Kongqun Yang d007ddf20b [MP4] Make outputs CMAF compatible
- Added flag --mp4_include_pssh_in_stream, default to true. If it is
  set to false, the encrypted mp4 stream will not include pssh.
- Align BytesOfProtectedData to multiple of 16 bytes for cenc.
- Set TrackHeader flag to kTrackEnabled | kTrackInMovie |
                          kTrackInPreview
- Move mvex to after trak, required by HLS
- Add cmfc/cmfs compatible brands except for avc3/hev1, where CMAF
  requires single initialization switching set which is not supported.
- Set duration to 0 in tkhd, mdhd, mvhd.

Also updated major_brand and compatible brands:
- Set major_brand to isom (iso-bmff media file format) and made dash
  a compatible brand
- Replaced compatible brand iso6 with iso8 since we use sthd for text
  tracks

Fixes b/36278260

Change-Id: I3cc5dd5aa1621714d517fe02fe3841d19a1a07f6
2017-03-30 15:33:39 -07:00
Kongqun Yang 5d71d8cc9f Move demuxer.cc/h to packager/media/demuxer
- Also fix the misleading error if an invalid input is provided
  The command line app now correctly reports that the file is not
  found instead of stream not available.

Change-Id: I8cfccb62d7a50de57766666f126cf061e3df5ded
2017-03-30 14:35:34 -07:00
Kongqun Yang 1af8f04e40 Rename webm_subsample_encryption flag to vp9_subsample_encryption
This is what webm_subsample_encryption flag actually does.

- Fixes the flag not working problem introduced by previous CLs.
- Also mark flag webm_subsample_encryption as deprecated.

Closes #220

Change-Id: I03ff843786572a91c01b8bd346a3ce50b129118f
2017-03-30 20:36:43 +00:00
Rintaro Kuroiwa 92280bf214 Full vtt in mp4 path
Change-Id: I7f49dbfed188ff7a451b0b4fbd6947590e8935d1
2017-03-29 18:52:25 +00:00
Kongqun Yang ec904f3f79 Add a new test to exercise emulation prevention after encryption
Also added a few more command line flags for end to end test:
- no-remove_temp_files_after_test: do not remove test artifacts after
  test
- encryption_key, encryption_iv: allow injecting encryption key/iv from
  command line

Change-Id: I62084790e10fe6a385b90cb96d9515d8436b2a49
2017-03-28 13:18:33 -07:00
Haoming Chen 6347b29ad7 Fix Windows build for trick play handler. (#221) 2017-03-27 23:04:29 -07:00
Rintaro Kuroiwa 4891d9a6bf HLS with fragmented MP4
- Add EXT-X-MAP tag for init segment.
- Do not set output field on stream descriptor if not specified on
  command line. If it's set (internally) then it gets copied to
  MediaInfo that gets passed to the manifest generators.

b/36279481

Change-Id: I762c55b255699ec691817dc4806b0dee2f7504b8
2017-03-27 13:20:03 -07:00
Haoming Chen cc04698460 Implement trick play handler.
Change-Id: I3d288dc7c4b77fd3aca1bad2ce96ad733384bcc8
2017-03-24 18:34:42 -07:00
Rintaro Kuroiwa 0a3ef49c11 Change {luma,chrome}_weight_flag to arrays
- The flags were overwritten in a loop and therefore only stored the
  last value of the flag.
- Change the fields into arrays so that the flags are not overwritten.

Change-Id: I59762d7c84bc3d81bf3b0b5e85ffe689260d970a
2017-03-24 21:14:18 +00:00
sylt 81beb8aeb2 Don't do timeshift SlideWindow() logic for static MPD (#218)
This is a fix for issue #216 where the Representation::SlideWindow() logic was
still active even though a static MPD was being asked to be generated (using
the option -generate_static_mpd). This could cause static streams longer than
1800 seconds to start playing at (end-position-for-stream - 1800 seconds).

This change is in compliance with the current usage documentation for the
shaka-packager, which suggests that -time_shift_buffer_depth only is relevant
for dynamic media presentations.
2017-03-24 10:37:13 -07:00
Kongqun Yang bc4c511ad4 Fix Windows build break (#217)
Change-Id: Id0a00c01077adb97f294c68357b629441321e6c7
2017-03-24 10:21:05 -07:00
Kongqun Yang 277c5b23c0 Allow generating avc3 / hev1 from TS and Wvm files
Added a flag --strip_parameter_set_nalus. When enabled, parameter
set NAL units, SPS/PPS for H264 and SPS/PPS/VPS for H265, are stripped
from frames when converting NAL byte stream (AnnexB stream) to NAL
unit stream, which generates avc1/hvc1; otherwise they are not
stripped, and avc3/hev1 is generated.

Parameter set NAL units should not be stripped if they are varying
in the frames otherwise the frames may fail to be decoded.

The flag is enabled by default as we don't usually see varying
SPS/PPS/VPS and it is more space efficient with them stripped.

Set --strip_parameter_set_nalus=false to disable the flag if there
are varying SPS/PPS/VPS in the frames. This addresses #206.

Change-Id: I34bde6f33069f9f77d51a510b39f58a0f0e141aa
2017-03-23 13:53:14 -07:00
KongQun Yang 3d7635d5de Integrate EncryptionHandler
Change-Id: I639e5707994a6601ee02078cd6f77cb38b481585
2017-03-23 18:25:59 +00:00
KongQun Yang 16615095f0 Update NalUnitToByteStreamConvert to support escaping encrypted NAL only
- Also fixed a bug if nalu_length_size != 4.

Change-Id: Ia8c67c4b1ff08a6b64fa1e309b5ecd8f5c8ea421
2017-03-23 18:21:24 +00:00
sylt d2c9a88ec5 Fix possible integer overflow in WebM duration
The issue could lead to the MPD attribute mediaPresentationDuration being
wrongly generated for WebM streams with a duration longer than INT32_MAX.

The root-cause was that StreamInfo::set_duration() accepted an int instead of a
uint64_t. This seems like a pure typo, since StreamInfo already uses a uint64_t
internally for representing duration.
2017-03-23 09:38:51 -07:00
Kongqun Yang d323d4f091 Fix break in Windows (#213) 2017-03-22 09:57:18 -07:00
KongQun Yang b09c8f6521 Update EncryptionHandler to support key rotation and SampleAes
This CL also removes EncryptionConfig stream data type and merges it
into StreamInfo/SegmentInfo instead.

Change-Id: Idb70ce503e61d3c951225cc78b6b15c084e16dcd
2017-03-21 23:50:53 +00:00
Kongqun Yang b891e0271e Make CENC behavior of VP8/9 in ISO-BMFF the same as in WebM
- Disable subsample encryption for VP8 in ISO-BMFF
- Apply block alignment to all subsamples for VP9 in ISO-BMFF,
  instead of just superframes.

Change-Id: I8dd31cc16e87abc4d538330eaff9acb0509497df
2017-03-21 23:12:57 +00:00
Kongqun Yang 0cb0f79c33 Fix build break in Windows (#211)
Change-Id: I0ed7a712fd02ea4dddd9c20541c2df653dc456cb
2017-03-17 12:45:28 -07:00
Kongqun Yang 0c3fb49eeb [WebM] Fix output truncated if using the same file name for I/O
Open output file in DoFinalize() to ensure that the input file has been
closed already.

Fixes #210

Change-Id: I935941b31c667e49be030c8da9f953d8387c7a9d
2017-03-17 01:07:49 +00:00
Rintaro Kuroiwa 15fd745fa7 Remove redundant flags in H264SliceHeader
- luma_weight_l0_flag, luma_weight_l1_flag,
  chroma_weight_l0_flag, chroma_weight_l1_flag were there but not set.
- The user should use the values in pred_weight_table_l0 and
  pred_weight_table_l1 instead which are set.

Change-Id: Ic9c44fb113717346938a339faf074daa32d4c2d2
2017-03-17 00:05:36 +00:00
Kongqun Yang 8806f78655 Use size_t as the type of stream_index
Change-Id: Ica8ffeccc45c1adf5def6fe9b2edde5708d6b432
2017-03-16 20:54:48 +00:00
Kongqun Yang 160700b452 Integrate ChunkingHandler
Also moved MediaHandler output validation to Initialize instead.

This CL also addresses #122 with consistent chunking.

Change-Id: I60c0da6d1b33421d7828bcb827d18899e71884ce
2017-03-16 20:54:36 +00:00
Rintaro Kuroiwa a3ce51785a Make WebVttMediaParser use WebVttSampleConverter
- WebVttMediaParser uses WebVttSampleConverter to generate non
  overlapping media samples.
- The media samples contains ISO BMFF boxes.
- Add kCodecWebVtt to signal that the media is webvtt and
  the samples will be in ISO BMFF boxes.

Change-Id: I639902cdba7b04af75428bc20622e26b8203cfb2
2017-03-15 20:44:36 +00:00
Joey Parrish 924d6d4693 Add HLS audio language support
Before this, HLS output did not contain language information.
Now, media playlists are properly tagged with a language in the
master playlist.

b/36134267

Change-Id: I172e946dbedd096a44cb2f917b007cc004756228
2017-03-10 16:14:14 -08:00
Kongqun Yang c6cbd73465 Rename FlushStream to OnFlushRequest; add FlushOutput
Change-Id: I2b37030b0c35aa34e9c72e43825f157924034227
2017-03-08 11:21:19 -08:00
KongQun Yang 8f2cd6da91 Implements Demuxer and Muxer media handlers
- Also sets up the packaging and verify it works.

Some of the changes are temporary to get the integration going.

Change-Id: I0cf6c379d185e157808acabb9ef58ff93d4a39ae
2017-03-08 19:11:52 +00:00
Kongqun Yang 12c61d6792 Fix pssh README to avoid confusion
Change-Id: I1ac1c4fbb75c2db6d1d960b1681afea0bac61bb8
2017-03-03 16:13:25 -08:00
Kongqun Yang 24ba12c18e Remove escape parameter in NalUnitToByteStreamConverter
The spec actually requires all NAL units to be escaped, so there
is no need to escape it again.

NAL byte stream is one demarcation method; NAL unit stream is another
demarcation method. Regardless of the demarcation method used, the
requirement for NAL unit is the same. See the last paragraph of
7.4.1 NAL unit semantics on the requirement on emulation prevention bytes.

Change-Id: Icc63fcc5cf965632e331f5af5f673164c7c1663a
2017-03-02 23:01:53 +00:00
Kongqun Yang c223bc9144 Keep slice data partition NALs in clear
CENCv3 recommends only encrypting video data in slice NALs. Slice
data partition NALs should not be encrypted.

In the code, differentiate is_vcl and is_video_slice. They are the
same for H265; for H264, vcl NALs include slice data partition NALs
but video_slice NALs do not.

Change-Id: I91f4bdd76d25f0eac50e39aed350ebce3f667121
2017-02-28 18:10:30 -08:00
Rintaro Kuroiwa 5c07ff6b1c Move and rename webvtt fragmenter to webvtt sample converter
- The class should be used to generate samples from the demuxer.

Change-Id: If7f245e5809de8773c0b1cf08673ae62dfeffe09
2017-02-14 16:33:18 -08:00
Rintaro Kuroiwa 593f513c83 Fix windows build
Change-Id: Ie23ebd9f18bf2151e52550bff62e2b043ac65ac2
2017-02-14 11:21:52 -08:00
Rintaro Kuroiwa 1879bca4bb Fragmenter for WebVTT
- Logic for splitting up WebVTT cues into MP4 samples.
- For any cue intervals that overlap, a new sample must be created.

Change-Id: Icf7478f42d8c5790bf6f404d512c1251dd34c405
2017-02-13 13:33:36 -08:00
Kongqun Yang 50787c605d Fix build break in win64
Also updated appveyor to build x64 first.

Change-Id: I1970e39bad559b3105ea31fb82375406d95f4a8b
2017-02-10 22:57:35 -08:00
Kongqun Yang 5f6e4a1dc1 Using std::shared_ptr instead of std::unique_ptr in stream data entry
Change-Id: Ib8c6f97c2329ba5b9431c301d85c1d61f89fc3d8
2017-02-11 02:41:49 +00:00
Kongqun Yang 9990524f98 Implement ChunkingHandler
This handler is a multi-in multi-out handler. If more than one input is
provided, there should be one and only one video stream; also, all inputs
should come from the same thread and are synchronized.
There can be multiple chunking handler running in different threads or even
different processes, we use the "consistent chunking algorithm" to make sure
the chunks in different streams are aligned without explicit communcating
with each other - which is not efficient and often difficult.

Consistent Chunking Algorithm:
 1. Find the consistent chunkable boundary
 Let the timestamps for video frames be (t1, t2, t3, ...). Then a
 consistent chunkable boundary is simply the first chunkable boundary after
 (tk / N) != (tk-1 / N), where '/' denotes integer division, and N is the
 intended chunk duration.
 2. Chunk only at the consistent chunkable boundary

This algorithm will make sure the chunks from different video streams are
aligned if they have aligned GoPs. However, this algorithm will only work
for video streams. To be able to chunk non video streams at similar
positions as video streams, ChunkingHandler is designed to accept one video
input and multiple non video inputs, the non video inputs are chunked when
the video input is chunked. If the inputs are synchronized - which is true
if the inputs come from the same demuxer, the video and non video chunks
are aligned.

Change-Id: Id3bad51ab14f311efdb8713b6cd36d36cf9e4639
2017-02-11 02:41:34 +00:00
Kongqun Yang 7a90ee70ab Implement EncryptionHandler
Change-Id: Iabedf3b02057d6124d5393ae9618490e5595ad89
2017-02-11 01:21:28 +00:00
Kongqun Yang 6235352bf4 Fix mpd duration not set in static live manifest
Closes #201

Change-Id: Ie9ab58ec58e1b135931a6576fb46145454b0f049
2017-02-11 00:52:32 +00:00
Kongqun Yang d592217761 Add a warning message when seeing unsupported stream type
Issue #202

Change-Id: If16fe7ccec5dfd869070dfbbb711240a89b2295b
2017-02-09 12:38:49 -08:00
Kongqun Yang bee59bc2fc Enable generate_dash_if_iop_compliant_mpd by default
This will move ContentProtection element from Representation
to AdaptationSet.

Shaka Player already supports AdaptationSet switching
urn:mpeg:dash:adaptation-set-switching:2016;

ExoPlayer does not support it yet. Filed
https://github.com/google/ExoPlayer/issues/2431 to track the issue.

(ExoPlayer does not like having ContentProtection in Representation
anyway)

Closes b/34691105

Change-Id: I69d0a4d0e15a912a35c8b2686620419a28e4cc99
2017-02-09 11:12:21 -08:00
Kongqun Yang 64c6821dc8 MediaHandler implementation
MediaHandler is the basic processing unit in shaka-packager 2.0.

Change-Id: I5f8d744e39adfff5cfebe0c50a29ead4d717cca4
2017-01-26 03:34:18 +00:00
Kongqun Yang 97fbaaa567 Replace scoped_refptr with std::shared_ptr
Change-Id: Ie8ea66e78c42c7387a97bb1abc331c636be11aa4
2017-01-26 02:20:47 +00:00
Yohann Connell 24b0e3031a Adding PlayReady support to Shaka packager
PlayReady Header Object Spec (PSSH Data VOD)
http://download.microsoft.com/download/2/3/8/238F67D9-1B8B-48D3-AB83-9C00112268B2/PlayReady%20Header%20Object%202015-08-13-FINAL-CL.PDF

PlayReady Live Streaming Spec (Consult Appendix for package acquisition information)
http://download.microsoft.com/download/2/D/D/2DD6B4E8-CABF-4DE9-8F61-895BE8F1ED33/ProtectingLiveTVServicesWithPlayReady_March2015.pdf

Issue #128
Change-Id: Ic5c0d6c642ceba3bdf9e8e413934f0fbcba242e6
2017-01-24 13:36:47 -08:00
Kongqun Yang 80c54a533a Update cmd flags to support static mpd with live profile
- Deprecated command line flags --profile and --single_segment.
  'segment_template' in stream descriptors implies live profile
  and non-single segment.
- Added flag --generate_static_mpd_for_live_profile to generate
  static mpd for live profile; if not set, dynamic mpd will be
  generated.

Close #142

Change-Id: I78879297ed118f0f246c4753a16ad125bd6b5e4f
2017-01-13 20:00:10 +00:00
Kongqun Yang 5aaae303e8 Part one of supporting live profile with static mpd
- Added two new fields in MpdOptions: dash_profile and mpd_type
- Updated MpdBuilder to support LiveProfile with static mpd
- Command line arguments will be updated in the next CL

Issue #142

Change-Id: Ibd35e5c9e1b1ef98043392e3f04a0af4700135c1
2017-01-13 19:59:49 +00:00
Chun-da Chen ce23fbacc4 Add support for udp_file in Windows. (#185)
Add support for udp_file in Windows
2017-01-10 10:05:28 -08:00
Haoming Chen 15dacf6288 Fix signed/unsigned mismatch on Windows.
Issue #168
2017-01-06 17:29:53 -08:00
Haoming Chen 93fe5457bf Fix x64 build on Windows.
- Fix integer conversion warnings.
- Fix #168.

Change-Id: Ie9aadea86a293b49e0854d6549800083143c79df
2017-01-05 14:19:35 -08:00
Rintaro Kuroiwa 655fe7b699 Add gitignores for msvs files
- Add MSVS related patterns to gitignore.
- Removed encryption_modes.h from media_base.gyp. It doesn't exist.

Change-Id: I44bc86ff7e79daf6f7bc61c21a558937530d8e66
2016-12-19 21:26:26 +00:00
Kongqun Yang af0904e428 Fix AdaptationSet switching signaling
Current mechanism of using AdaptationSet @group is not spec compliant.

The spec compliant mechanism is to use supplemental descriptor with
schemeIdUri: urn:mpeg:dash:adaptation-set-switching:2016 as specified
in ISO/IEC 23009-1:2014/Amd.4:2016.

Fixes #156

Change-Id: I4d97648547a23315ba9c09dcadb14e18c99a45fc
2016-12-19 20:12:27 +00:00
Rintaro Kuroiwa 7a4a40acb8 Change values for slice header attributes
- nalu_data points to the beginning of the nalu, instead of
  beginning of the slice header which excluded the nalu
  header.
- nalu_size is the size of the whole nalu.
- header_bit_size calculation is corrected. The size used to exclude
  emulation prevention bytes.

Change-Id: Ic00f42caeddf7b4701f3948e6ef8a3537fd7f770
2016-12-15 17:07:17 -08:00
Kyle Alexander 0c2ee4c844 Add support for 4K and 8K content.
Adds UHD1 and UHD2 track types to support 4K and 8K content.

Bug: Closes #163
Change-Id: I8fd893725cae88e9944244a48607cbaab591b401
2016-12-13 19:24:22 +00:00
Kongqun Yang d8d94da250 Fix packager_test break in Windows
Change-Id: Id43e4bd0e2067fd9ad74f8c5c60df7ddba75e840
2016-12-09 19:11:04 -08:00
Kongqun Yang c8ffa54d35 Fix --version in drive programs
Fixes #176

Change-Id: Ib092f81e1af49dfee43f7b01c0008a1b40c9098b
2016-12-08 14:45:05 -08:00
Kongqun Yang eab7910029 Fix crash when seeing unsupported es descriptor data
Addresses #175

Change-Id: Ia9c2f0c7a8c17a6614db0fe478892bd3c5efd5b0
2016-11-29 23:27:22 +00:00
Haoming Chen 9ed8303a06 Use Windows native ssl instead of openssl.
Fixes #147

Change-Id: I41193adbab5cd979ca958131cd4f148dde5e1f2e
2016-11-09 14:08:52 -08:00
Rintaro Kuroiwa e7c0dbf09d Ensure some method calls NotifyEncryptionStart()
- If OnEncryptionStart() is called OnMediaStart(), someone should be
  calling NotifyEncrytionStart().

Fixes #169

Change-Id: I597f317d2ddeb3091b7e9d238eb9695b4cc9c69d
2016-11-08 15:56:28 -08:00
Kongqun Yang 458fadc6a8 Add udp options support
Three options are supported right now:
- reuse=1|0
  Allow or disallow reusing UDP sockets. Default 0.
- interface=a.b.c.d
  Address of the interface over which to receive UDP multicast
  streams.
- timeout=microseconds
  Timeout in microseconds. Default to unlimited.

A UDP url should be of the form udp://ip:port[?options], here
is an example:
  udp://224.1.2.30:88?reuse=1&interface=10.11.12.13&timeout=12345

Also deprecate --udp_interface_address flag in favor udp options.

Closes #133

Change-Id: I962d35bfedc1779d67ba20ed910207c66b7c1a15
2016-10-31 18:40:40 +00:00
Kongqun Yang 4c6e5f4fa6 Improve support for decryption of full sample encrypted cbcs
1. Allow encryption pattern of 0:0, which is treated as 1:0;
2. Remove the special handling of pattern 1:0 in pattern cryptor
   which may not always be correct;
3. Allow senc/saio/saiz boxes to be absent.

Change-Id: I372e61182ec577107e39cb601c3aed80616b036d
2016-10-14 00:07:46 +00:00
Kongqun Yang e7393fcc22 Fix build break in Windows
Change-Id: I49815abdaa360e11e62a3193bc58d4d7d77fc8a8
2016-10-13 15:54:32 -07:00
Kongqun Yang 6dbcf27f82 [WebM] Use system temp directory if temp_dir is not specified
The original code creates temporary file in the current directory,
which may not always be writable.

Change-Id: Icc278dd5db6be6206ae29defbb7423bae39cc978
2016-10-13 21:48:00 +00:00
Kongqun Yang 472fae24f2 [pssh] Support widevine pssh generation with protection_scheme
- Also import the version of the protobuf bundled in packager first,
  this avoids importing the installed version of the protobuf, which
  may not be compatible with this script.
- Print help if no argument is supplied.

Fixes #162

Change-Id: I1b5e87ba7fb51e81b0d4dd3791aaeb37ee0fd6ce
2016-10-10 10:39:41 -07:00
sammirata 4f9bbe3769 Do not require exclusive bind of udp sockets (#154) 2016-10-04 15:51:45 -07:00
Kongqun Yang 97fc9828f0 [WebM] Move index segment after init segment
Cues used to be generated in the end of the file; if http range
request is not supported, clients have to download the whole
file to get to the cues.

This CL updated TwoPassSingleSegmentSegmenter to write cues
after writing webm header.

This CL also updates libwebm dependency to latest.

Closes #159

Change-Id: Ic73548e1b872e6b13a37627707e7d0ff3e556877
2016-10-01 00:34:56 +00:00
Kongqun Yang ac1d2692cf [WebM] Fix corner case segment generation problem
The original code accumulates sample durations in seconds (double)
and then generates new segment / cluster by comparing the accumulated
value with configured value.

There may be loss of precision when accumulating values in double.
For example, with a GOP of 96 frames and frame rate of 24 fps; the
accumulated frame duration for 96 frames is 3.999999999999, which is
very close to 4.0 but not 4.0. Problem would arise if segment
duration is set to 4. The created segments would have a duration of
8 seconds instead of 4 seconds.

The new code accumulates sample durations in uint64_t relative to
input time scale; the configured segment duration will be converted to
timescale for comparison. This avoids loss of precision.

Change-Id: I3ae24be82a7ce45f923a6f90fea495b8b6b2e7ef
2016-09-30 21:45:27 +00:00
Kongqun Yang f5dc908a0d Implement utility function DoubleToString
Implement our own DoubleToString as base::DoubleToString uses
third_party library dmg_fp.

Change-Id: I0d3d706cfd3de4f833b201110954419e585a21dd
2016-09-30 21:36:11 +00:00
Joey Parrish cda495fb57 Add default_language flag
Setting this flag will tag any matching tracks with kRoleMain.
The player will then know which tracks should be default when there
is not a language match.

Closes #155

Change-Id: I827304c49e345d07b1e76f7a46421b60f228f0ba
2016-09-29 22:59:34 +00:00
Joey Parrish 07c4b4f3bd Drop the DCHECK for language size 3
Language tags can have subtags, such as "por-BR".  While the base tag
should be 3 characters, the entire tag can be longer.  Rather than
parse out the base tag and check that it is size 3 without the subtag,
we can just trust the ISO tag converter to do the right thing.

Change-Id: I2a186b5c8db0f373ee82ef68245ecc33cb068581
2016-09-29 16:37:16 +00:00
Joey Parrish 275e60e5a9 Detect .webvtt as WebVTT files
Some subtitle files use .webvtt instead of .vtt, probably because
ffmpeg uses -f webvtt when converting from some other format.

Change-Id: If88e8d2b90e5dea6751409568d5191e923dec4aa
2016-09-28 15:02:27 -07:00
Kongqun Yang 5edd290694 Fix WebM not encrypted when clear lead is set to 0
- Also regenerated test content to insert an iframe every one
  second: ffmpeg -i source.webm -auto-alt-ref 1 -g 30 out.webm
  This makes sure the content is actually encrypted with one
  second clear lead in end to end test.

Change-Id: I488143c148e2d6c45ba1586f6d0d835dc46db86e
2016-09-28 18:17:09 +00:00
Kongqun Yang 47363dd06a Remove unused vorbis test files
Change-Id: Ifede6faac7e17dcd60d36c1e9e7c9a025f95fbdf
2016-09-13 22:06:38 +00:00
Haoming Chen 03d7a81c83 Fix a precision conversion problem.
- This causes a compilation error on Windows.

Change-Id: I53c03841565be00ebaa3acbfe621bb145f8f1ca2
2016-09-12 08:46:16 -07:00
Haoming Chen 28828b8a15 Add encryption information in demuxer.
- Add decrypt_config in media sample and stream.
  - Update subsamples when converting NAL unit to bytestream.

Change-Id: I7b8975a453f81b22cf74bee3c9a58b7e458cbaae
2016-09-08 17:13:36 +00:00
Kongqun Yang 90e3ec3f9a Some misc cleanup
- Replace DCHECK_EQ with DCHECK for compatiblity
- Add ASSERT_FILE_CONTAINS
- Remove unnecessary release on unique_ptrs

Change-Id: I2ec22c52e47cb451479ae9a59818b9df20f62e63
2016-09-07 10:15:50 -07:00
Kongqun Yang c3d1dc1733 Deprecate STLDeleter and string_as_array
1. Replace STLDeleter with unique_ptr in container, which is
   supported in C++11
2. Replace string_as_array with &string[0] which is guaranteed
   to work in C++11

Change-Id: I7f39c0e51fc8a3fcbb41313094a0ca6b33db7bf3
2016-09-01 17:39:11 +00:00
Kongqun Yang ba2a649c00 LOG_TO_SYSTEM_DEBUG_LOG instead of a file
The current behavior of logging to a file in the same directory
as the binary will break if the binary does not have write access
to that directory.

Fixes Issue #139

Change-Id: I11adfaec676fc9bbef2a5caa3261889d1fba3b88
2016-08-31 14:50:23 -07:00
Kongqun Yang 09891dcd9e Fix access unit detection problem for H264
In H264, there may be multiple consecutive video slice NAL units
in the same frame. The original code assigns a new access unit
for every video slice NAL unit, which is incorrect.

Fixes #134.

Change-Id: I4d44271df48cb08867ddd02f7494fb3573af3356
2016-08-29 11:19:52 -07:00
Kongqun Yang 52cbcb321d Add appveyor config for Windows CI
Also fixes additional Windows build failures.

Closes #22

Change-Id: I8bc416f1a44810c438f095ec137d0d102f4e08a5
2016-08-23 12:24:34 -07:00
Kongqun Yang de9667080f Do not partially encrypt a segment
Mimic iso-bmff behavior: either all the samples in a segment are encrypted
or all the samples are clear.

Change-Id: I03bdbbf5a4b690f4d87c4dceb0f8155e6fae941e
2016-08-19 14:22:40 -07:00
Kongqun Yang 23f2913248 Update DEPS to fix mac build failure
Change-Id: I1cf95d5da8e3b950300cd61cd9286e957ef653ce
2016-08-19 13:49:41 -07:00
Kongqun Yang 644629c616 Replace scoped_ptr with std::unique_ptr
Change-Id: I1ed18bf876b6154d08a1f7913aa7e76c525451f9
2016-08-18 17:12:36 -07:00
Kongqun Yang d501b457d0 Fix WebM keyframe detection in BlockGroup
The right way to detect the presence of a keyframe in a BlockGroup
is to look for the absence of the ReferenceBlock element [1].

Closes b/30433206

[1] https://matroska.org/technical/specs/index.html#ReferenceBlock

Change-Id: I2b3c8704cbabcd5cbee38448e5528425865561a4
2016-08-18 23:20:40 +00:00
Thomas Inskip db92c9a22b Implemented WebM VP9 subsample encryption and decryption.
Change-Id: I98f02c05ec8bcc976e01ff41478e92b8809c0076
2016-08-18 13:55:36 -07:00
Thomas Inskip 336ea5cb34 Windows build working
Change-Id: I6d9cfa0a310c4c6125c839e4d6a085903e981c9c
2016-08-18 08:30:26 -07:00
Thomas Inskip 224747db0b libcurl build for Windows
Change-Id: Ie14613d36509b3f45220092c5c196b14fd4b0a10
2016-08-03 16:05:40 -07:00
Thomas Inskip d059b27a69 Updated third_party/gflags.
WebRTC portion (gyp build) now checked in because separate repository is no longer available due to transition to git.

Change-Id: I56436cbbc3f02a5f77112917d6229aea69f21d97
2016-08-03 07:52:16 -07:00
Thomas Inskip f169bccdc9 Updated libxml.
Change-Id: I4f2315f10227e6b8c2545b81ea1adcf388c37cee
2016-08-02 19:11:17 -07:00
Thomas Inskip 1fbc410489 Updated protobuf library.
Change-Id: I09705df523ba4d0954925845a1619b25d70fd90f
2016-08-02 19:11:17 -07:00
Kongqun Yang 16d8583735 Update boringssl and curl(v7.50.0)
Change-Id: I89b40cf03d1aab9a13b0df449e540ab73d03451e
2016-08-02 12:00:19 -07:00
Gabe Kopley 081b6994d6 Fix broken link to "cenc" Init Data Format spec 2016-07-26 14:52:08 -07:00
Kongqun Yang 01e53ee946 [pssh script] Fix invalid hex_16_bytes error on lower case input
Change-Id: I274634f9076500156527c04bc9cbb9f86da3c6bd
2016-07-25 17:58:03 -07:00
Kongqun Yang ad95b9d8de Fix text formats not recognized problem
Also updated packager_test not to fill in output_format by default,
which will help catch these kinds of bugs.

Closes #130

Change-Id: I77157317de822d3f543f7e27d94bb4df7a1649d2
2016-07-22 20:51:16 +00:00
Kongqun Yang 2cdd54b6f7 Remove restriction on sample duration to be 0 for mp4
0 sample duration is seen occasionally in the contents in the wild.

Closes #127

Change-Id: I48b9fac4add79f39a83fc571502bedb9f0be71df
2016-07-19 18:56:45 +00:00
Kongqun Yang fae7874c4c Add a runtime flag to use dts in timeline for mp4
This flag is needed to workaround the Chromium bug
https://crbug.com/398130, which uses decoding timestamp
in buffered range.

Closes #113

Change-Id: Ib8f18be7165dd968bdc36c18ce29f694235c0c26
2016-07-19 18:56:08 +00:00
Kongqun Yang a80995538d Remove restriction on sps:gaps_in_frame_num_value_allowed_flag
Packager should not care about this flag, whether it is supported
is up to the decoder.

Closes #126

Change-Id: I5e05d291eab01375f7f11c951b9dc93c1358c38a
2016-07-18 20:25:40 +00:00
Kongqun Yang cf778f7149 Update CHANGELOG for v1.5.0
- Also updated a few wording in README and in the code.

Change-Id: I9adf297c00aee92e61e87effa3bdcfaf402638e5
2016-07-12 12:25:48 -07:00
Kongqun Yang f3d35b730c Add version information to generated HLS manifests
- Also refactor the existing version code.

Change-Id: Ib409630c4f87965a37b1d6a1bcec68178a704ce6
2016-07-11 23:26:36 +00:00
Rintaro Kuroiwa db70721e35 Fix HLS packaging failure with clear lead = 0
- Add MuxerListener::OnEncryptionStart() for notifying that further
  segments are encrypted.

b/29621230

Change-Id: I881b29d55baaf3d04e005a3b95d898071c3f272b
2016-07-11 20:43:01 +00:00
Kongqun Yang e0c5874d31 Update encrypted testing content to use common system id
- Also fix a bug in fixed key rotation with empty pssh_data.

Change-Id: I1110b839fc821c5c4778854c0b29032d3ee2d384
2016-07-11 20:10:31 +00:00
Kongqun Yang ad3c1768bb Fix dependency in hls.gyp
Seeing occasional build breaks when building hls_builder, e.g.
https://travis-ci.org/google/shaka-packager/jobs/143445907
Although hls_builder is dependent on widevine_pssh_data_proto
indirectly (through media_base), it does not work all the time.
Looks like the proto has to be a direct dependency.

Change-Id: Id4d9d0b9e669df0967cc767132ef29717243a722
2016-07-08 14:48:47 -07:00
Kongqun Yang 5f0d138ec9 Adjust hls descriptor validation
- Try to determine container name from segment_template if output is
  not specified
- segment_template descriptor is required for TS
- output descriptor should not be specified for TS
- Generate hls_playlist_name from stream number if not specified
- Update packager_test.py to include playlist validation

b/29551186

Change-Id: Ide7169f1c26c85a6e38272dda0e1af77135b4d90
2016-07-01 14:14:24 -07:00
Rintaro Kuroiwa 1095b73a44 Fix TS and Media Playlist with clear lead
- Cannot include PMT for following encrypted segments in the clear lead.
  Some players consider them as conflicting codecs.
- Add EXT-X-DISCONTINUITY in front of first EXT-X-KEY to notify that the
  codec is switching from e.g. "avc1" to "zavc".

b/29621230

Change-Id: I45c74813630c229d66245e992eb3a5117326bb14
2016-07-01 10:43:11 -07:00
Rintaro Kuroiwa 50fe17fa9f Use max bitrate for MediaPlaylist bandwidth calculation
- HLS expects max for VOD.

Change-Id: I91e9e07a27abe7167efeefc99aaada2acbed9314
2016-06-30 21:58:33 +00:00
Jacob Trimble f9bf197f2b Fix bug in VPx codec configuration.
Before, we converted the WebM configuration to MP4 in the video
client, however this lead to a bug when fields were missing.  So
now don't convert until the cluster parser so the extra info from
the stream can be added.

This also fixes a bug where the value was not printed in the warning
logs.

b/29580732

Change-Id: If0a1a4d135f98292cdaae15a11027f42d844e85d
2016-06-30 19:51:09 +00:00
Kongqun Yang f27960e00d Add fixed key support for HLS sample-aes encryption
Change-Id: I5a18e014f697561e0741b043e91f70fe2f11d2ce
2016-06-30 19:47:00 +00:00
Rintaro Kuroiwa dad07c95d7 Set MediaPlaylist version to 5
- Required for KEYFORMAT and KEYFORAMTVERSIONS attributes.

Change-Id: I834cde6e475e4f1192db0c93c4b88e2ab838532e
2016-06-29 10:15:46 -07:00
Jacob Trimble abb1abf5c5 Rename StreamInfo::extra_data to codec_config.
Since the |extra_data| field contains codec-specific configuration
data, it makes sense to call it |codec_config|.

Change-Id: If9e35165a00fe82628cf931df397a8ef06505b0d
2016-06-27 13:16:47 -07:00
Rintaro Kuroiwa b07adaeae7 Warn user when single segment and segment template are used
Change-Id: I4b85f50aa0890fb54a642c38520225df9bdbbbef
2016-06-22 15:59:00 -07:00
Jacob Trimble 6550868574 Add codec private data to VP9 in WebM.
b/29009350

Change-Id: Iaafc87340043eff77c3ef7e1c1135d8c4c4287ae
2016-06-22 16:43:57 +00:00
Kongqun Yang 47a3fb977d Fix 'senc' bug when encrypting mp4:vp9 with superframe
Although superframe index is not encrypted, it should still be inserted
into subsamples list (with cipher_bytes as 0).

Change-Id: I06b38f8333a22a31a71cd67e0412218dd3aaa5c3
2016-06-21 17:17:49 +00:00
Kongqun Yang 0a9de96eb2 Update mp4 container check function to be more robust
Consider the file in iso-bmff format if seeing two known
mp4 boxes.

Closes Issue #110

Change-Id: Ifdc76b6c8b7e26a0988ab4f8d4968e1320cb180c
2016-06-21 17:17:03 +00:00
Rintaro Kuroiwa 60419f26d0 Add driver program flags for HLS
- Add flags and stream descriptor fields for HLS.
- Remove fields from MuxerOptions. Instead pass them directly to
  HlsNotifyMuxerListener.
- Rebase segment names to the master playlist's path where it makes
  sense.

Change-Id: If4f54bc56ff46dc6140859a79ed66f7b99112ed7
2016-06-13 18:30:24 -07:00
Rintaro Kuroiwa 565affe7fb Pass hls_name and hls_group to HlsNotifyMuxerListener constructor
- Also don't Flush the notifier OnMediaEnd().

Change-Id: I2ffbbfacda74bb88678ac4e32b1f28c3e64eb85d
2016-06-13 14:36:39 -07:00
Rintaro Kuroiwa d06a9cd17c Allow encrypted TS without clear lead
Change-Id: Icc4268af94c11b004f020cfb4dd9f5f16a9169af
2016-06-13 20:49:23 +00:00
Jacob Trimble cb3b277575 Optimize the MPEG2-TS demuxer.
Change-Id: I5836fc8503fe22a3d73241dda00c36d0ffb26e65
2016-06-08 21:43:15 +00:00
Kongqun Yang d08f6ae0cc Fix cbcs/cens behavior when applied to non video tracks
- Tracks other than video are protected using whole-block full-sample
  encryption as specified in ISO/IEC 23001-7:2016(E) 9.7, which is
  equivalent to a pattern of 1:0. This is different to the non pattern
  encryption counterparts.
- Also updated the code to allow the existence of other protection
  schemes in the original content, which will simply be ignored.
- The internal cryptor used by AesPatternCryptor should not use
  constant iv, add a DCHECK for that.
- Optimize AesPatternCryptor handling on the special pattern 1:0.

Change-Id: Idc704e7bc6b347741336f38c6d3620fc19392960
2016-06-08 17:02:26 +00:00
Kongqun Yang 84f3911985 Remove top level box check in BoxReader
Change-Id: I3d0e5865851030264d7174031a98031b413e90ce
2016-06-08 00:54:11 +00:00
Kongqun Yang bd1328500c Allow uuid to be a valid top level box
Change-Id: Iec241781321a30fa8eece6a7fc6674d78a02c90f
2016-06-08 00:53:35 +00:00
Kongqun Yang 579678d6a6 Fixed a Opus in Mp4 parsing problem
- Also added end to end test in packager_test.py

Closes Issue #83

Change-Id: I7a8e695a36e654507b75d558710db1ce8867a916
2016-06-01 23:52:33 +00:00
Kongqun Yang 32482c0bce Rename filters directory to codecs
Also move codecs related classes in mp4 to codecs directory.

Change-Id: I83db2bcf9e66b405fcceed78918b762b26cd7fce
2016-05-27 16:13:19 -07:00
Kongqun Yang 6d281c6676 Rename XxxConfiguration to XxxConfigurationRecord
These are really configuration records.

Change-Id: I0d081c2e14f19bcdf8eb52dc094a1e7c096b5f98
2016-05-27 16:11:11 -07:00
Kongqun Yang 68aa1397b0 VPCodecConfiguration box should inherit from FullBox instead of Box
Change-Id: If53c6cab009bdeadc37a1bac34272056a2b7d1dc
2016-05-25 23:59:16 +00:00
Kongqun Yang 7e533ec5ca More cleanup in proto usage
Change-Id: If35f95dbff47bc9e8c1c42cb56d75719367abfc0
2016-05-25 16:57:39 +00:00
Kongqun Yang 4f9c8ea781 Rename top level namespace to shaka
Issue #103

Change-Id: I67945c8ec6d2c178494d3b5400e5ec0170e9d115
2016-05-24 21:55:55 +00:00
Kongqun Yang 72c3a76699 Update repo url in version string
Issue #103

Change-Id: Iec7db3d155c81cf989f1078ed32692f8e037aa65
2016-05-24 21:55:45 +00:00
Kongqun Yang bbcca29013 Move all code inside top level namespace
Change-Id: I83bc081f523dbc5071796a79c2d89b7367ed8d27
2016-05-24 21:55:21 +00:00
Kongqun Yang a217cdce29 Misc cleanup in MediaInfo proto usage
- Replaces SerializeToString with SerializeAsString.
- Construct StringPiece instead of doing implicit conversion when
  doing Base64Encode.

Change-Id: I9a9d4a138b0d949447396387adc599c7cde0b9f4
2016-05-17 13:28:22 -07:00
Kongqun Yang c923cb283a Fix unittest failures in previous cl
Change-Id: Ib44a8d0a2907083281a5afdd5b5277a0a0fc35c7
2016-05-16 17:09:06 -07:00
Kongqun Yang 940cc43de2 Add support for AudioRollRecoveryEntry
This is part of the effort to support Opus in iso-bmff #83.

Changes in this CL:
- Add support for multiple SampleGroupDescription and SampleToGroup
  boxes in TrackFragment box;
- Add support for SampleGroupDescription and SampleToGroup boxes
  in SampleTable box;
- Add support for AudioRollRecoveryEntry in Sample Group and 'roll'
  sample grouping type;
- Also fix a bug that default length was not set correctly in
  SampleGroupDescription when constant iv is used.

Change-Id: I7e31386ea6cd17a8ee5e1dca4c1a90937d51368f
2016-05-16 22:25:38 +00:00
Kongqun Yang cf4a2447c1 Add support for Opus specific box in iso-bmff
This is part of the effort to support Opus in iso-bmff #83.

Change-Id: Ib3678b9cb74eac76372ed83ad48ce1f203ba0c35
2016-05-16 22:25:32 +00:00
Kongqun Yang 26cb91e29b Handle preroll and codec_delay when tranmuxing opus
Closes Issue #102

Change-Id: I26aa56a63c17c85298311cc17963dd26f26e501e
2016-05-16 14:51:51 -07:00
Rintaro Kuroiwa d4e2f3c098 hls_builder should depend on widevine_pssh_data
- There is a race condition during build. hls_builder should depend on
  widevine_pssh_data to remove the race condition.

Change-Id: If9fa8895104a774b8ecc7545a57323ae49274936
2016-05-13 11:41:07 -07:00
Rintaro Kuroiwa e422b4eb0e Handle encryption in TsSegmenter
- Move GetTrackTypeForEncryption() to muxer_util.h.

Change-Id: I315957cb8983f4e7c4acff6907dfdd6ad6907c82
2016-05-11 16:21:05 -07:00
Rintaro Kuroiwa 0ba35147c8 SAMPLE-AES encryption in PesPacketGenerator
- Allow setting a key on PesPacketGenerator.
- Add SAMPLE AES mode to AesPatternCryptor.

Change-Id: Iace2381b4cd3bbce63cd5bdb4cdf3a7cea47537a
2016-05-11 09:56:36 -07:00
Rintaro Kuroiwa 7d8322377e PMT writers for H264 and AAC
- For encrypted segments, PMT specified by SAMPLE-AES specification is
  generated.
- Remove time_scale_ field from TsWriter. It is not used.
- Move ContinuityCounter to a separate file.
- Add ProgramMapTableWriter and implementations.
- continuity_counter is incremented for PAT and PMT.

Change-Id: If0dd0990203cb6990f39a6832048bdc3ff08decd
2016-05-05 15:56:54 -07:00
Jacob Trimble 9bb6c5d8d2 Add more unit tests for EsParserH26x.
This add several unit tests for EsParserH26x to test the various
NAL unit orders and access unit contents.  Some tests are disabled
and will be fixed in another patch.

Change-Id: Id5e3291e22f1fe17ada2c03c42e2cdfe226abcb2
2016-05-05 09:41:31 -07:00
Jacob Trimble 86369efc30 Add support for H.265 in MPEG2-TS demuxer.
This also changes the H.264 parser to correctly determine access
units.

Closes #46
Issue #47

Change-Id: I69a8c47ebf4fe35cef0592997460158b3131084e
2016-05-03 15:08:04 -07:00
Kongqun Yang 2abc7c60b7 Add end to end test for transport stream
- Also replaces resolution to 640x360, to reduce file size.

Change-Id: I030ceaa0c67c2b8f060d1864e2c3150f5d28b612
2016-04-29 17:21:08 -07:00
Kongqun Yang 3ff74398df Fix two bugs in 'cbcs' and 'cens' handling and add tests
- BytesOfProtectedData in 'cens' SHALL also be 16-byte aligned
- Fix an incorrect box definition bug w.r.t to 'cbcs' and 'cens'
- Also add various protection scheme tests

Close #77
Close #78

Change-Id: I63c8d8b01ce16ed60affa97ec95fc62bc2da06df
2016-04-26 17:27:26 +00:00
Kongqun Yang 976bf69294 Add decryption verification tests
Change-Id: I735dc14c618537bc4a69fcfbe2bc634bfe2a2166
2016-04-26 17:27:13 +00:00
Kongqun Yang 855d1b028f Correctly generate ContentProtection value for non-cenc protection schemes
Change-Id: Ie5469a7f192cc6167f029befd9cca7f9ab0d1f27
2016-04-26 17:27:04 +00:00
Rintaro Kuroiwa 67e0f9f31b Pass playlist type to MediaPlaylist
- The current implementation only handles VOD.
- Add #EXT-X-PLATLIST-TYPE.
- For VOD Add #EXT-X-ENDLIST at the end.
- Append comma at the end of EXTINF.

Change-Id: I16f01da66f8bbf0229395cb380fa125ffd9328a8
2016-04-25 12:36:11 -07:00
Kongqun Yang 25305b6aa3 Clarify that WVM format always uses all zero IV
Close b/22894397

Change-Id: I9f2295130d32ea5135ef0143ec7758df1114b18d
2016-04-22 13:42:37 -07:00
Kongqun Yang 797ae3e639 Copy packager_test.py to out directory on build
- Disallow packager_test.py to be executed from source directory
- Remove the building of packager target from packager_test.py. User
  need to make sure packager builds successfully before running
  the test.
- Remove build_type parameter which is no longer needed

Change-Id: I479d3c39b8d45730e5388d194069f1320fa5cd75
2016-04-22 18:16:33 +00:00
Kongqun Yang ba42fed9e5 Suppress warning messages when determining containers
- These warning messages are not useful since the code needs to
  loop through various container types to find a matching one.

Change-Id: I1d8eca563faf24e485ac71cce32e09515c174bf2
2016-04-22 09:46:21 -07:00
Jacob Trimble a789dc6d3c Replace Nalu::InitializeFrom* with Initialize.
Replaces the InitializeFromH264 and InitializeFromH265 methods with
a single Initialize method that accepts a CodecType.  Also moves the
CodecType enum from NaluReader to Nalu.

Change-Id: Ie33aa9c0d772a7f3c6a2e9c867e25be3edfe7828
2016-04-21 13:35:49 -07:00
Kongqun Yang 77fc3f6988 Fix an unintended warning message
The warning message was included accidentally for audio encryption
during rebase.

Change-Id: I105f2f01cb9016fc6c57494708ba6d9cbb2ff53c
2016-04-21 19:11:01 +00:00
Rintaro Kuroiwa f3ed07a64e Use MuxerListener in TsMuxer
- MuxerListener is used in TsMuxer so that a listener can be used to
  e.g. generate manifests.

Change-Id: I11c745e1c2b71d5ec901387fe42713d4ad69dc03
2016-04-21 10:45:43 -07:00
KongQun Yang 13202f91b6 Fix a few problems in AesCryptor classes
- CBC cryptors should accept IV of size 8 bytes - it will be zero
  extended to 16 bytes.
- Fixed iv() not updated problem in AesPatternCryptor.
- Replace kChainAcrossCalls with ConstantIvFlag enum flags.

Change-Id: I3fb4de0e8abbe891e6271e779373ba53f8df660d
2016-04-20 14:14:04 -07:00
Rintaro Kuroiwa 66b82f87dd Add HlsNotifyMuxerListener
- A MuxerListener implementation that uses HlsNotifier.

Issue #85

Change-Id: I4c94c025be112d2bd02a78123da3f275e09c3a06
2016-04-19 18:19:11 -07:00
Rintaro Kuroiwa 102f3f7062 Scale sample duration in TsSegmenter
- Sample duration wasn't scaled in TsSegmenter to calculate the current
  segment's length. So segmenting length did not always respect
  segment_duration.

Change-Id: I1ef4747dbd12bde7852ffcbceb971f2775aa18b3
2016-04-19 23:21:16 +00:00
Jacob Trimble bbf9c6849b Parse vui_parameters in H.265.
Parsing vui_parameters in the SPS is required to be able to extract
the resolution of the video.  This also adds a method to extract
the resolution to the H.265 parser.

Issue #46

Change-Id: Idf498d2afdb36a689490151a4ae6baef0b6b73f6
2016-04-19 23:15:34 +00:00
Rintaro Kuroiwa 2756902934 Fix small bugs in HLS playlist generation
- Media Playlist that is non-VERSION 1 must have EXT-X-VERSION.
- Attribute for EXTINF should be CODECS instead of CODEC.
- BANDWIDTH attribute expects bitrate (not bytes per second).
- Removed unnecessary check if MediaInfo has media_info_name, in
  SimpleHlsNotifier::NotifyNewStream().

Change-Id: Ia63cfa59e5e2ec24bbf1b784164e6e41176fc589
2016-04-18 17:14:26 -07:00
Jacob Trimble a80e16bab0 Close files in Finalize for WebM segmenters.
Now Finalize() will close all the files used by the segmenters.  This
is important to allow WebMMuxer::FireOnMediaEndEvent to get the
correct size of the media file.

This also changes the WebM muxer to use 64-bit numbers for the init
and index ranges to correctly support >4GB files.

b/28194272

Change-Id: Ia84e4a4b0756f89644efea99a1a51968b22a1338
2016-04-18 21:28:09 +00:00
Rintaro Kuroiwa a50b5787dc Use TsMuxer in driver program
- The driver program can output ts files.

Change-Id: I190ba17ded91a0be48cc0a1a8b15e44d20800eec
2016-04-18 19:13:50 +00:00
Rintaro Kuroiwa 300c23104e MuxerOptions and MuxerListener change for HLS
- Add hls_name and hls_group_id fields to MuxerOptions. This is used to
  pass the NAME and GROUP-ID values for EXT-X-MEDIA tag to
  MuxerListener.
- Change MuxerListener::OnEncryptionInfoReady() to take an
  initialization vector.
- Change MuxerListener::OnNewSegment() to take segment name.
- Reworded and formatted MuxerListener comments to Doxygen style.

Issue #85

Change-Id: Iea06e68552a56ae180177ffd6ca315a7cf39456c
2016-04-18 11:40:10 -07:00
Rintaro Kuroiwa 16ba8da295 Map to raw pointer instead of scoped_ptr
- Fixes Mac build.
- Mapping to scoped_ptr doesn't work, instead use raw pointer and
  STLValueDeleter.

Change-Id: If17e43d9d4aba8c92a76d2b1ca99a64c6a2b0e84
2016-04-17 00:12:10 -07:00
Rintaro Kuroiwa a6646841b7 HlsNotifier and implementation
- HlsNotifier interface, similar to MpdNotifier.
- SimpleHlsNotifier class can generate playlists.
  It can also handle encrypted media with Widevine encryption.

Issue #85

Change-Id: I9305ddf42796370d3e5df360633c80997c909365
2016-04-16 16:22:35 -07:00
KongQun Yang c6445b749c Change AesCbcEncryptor/AesCbcDecryptor kNoPadding behavior
Leave unaligned residual block in clear in stead of rejecting it.

Change-Id: Id5452f32f6147e26ac68d9cddb948e8b4f77d107
2016-04-15 13:31:09 -07:00
KongQun Yang e253747453 Add support for 'cbcs' and 'cens' protection schemes
Issue #78

Change-Id: I9f71b9a92067e2f6b388092494a7d6a84986cdc0
2016-04-14 15:31:11 -07:00
KongQun Yang c8819cb257 Add mp4 box fields for pattern encryption
- crypt_byte_block and skip_byte_block for pattern-based encryption.
- constant_iv for constant iv (used by 'cbcs' protection scheme).
- Also renamed iv_size to per_sample_iv_size.
- Also changed "bool is_encrypted" to "uint8_t is_protected" to align
  with CENC spec.

Issue #78

Change-Id: I2878a91e0ebe536a08a3e3109daf157fe4440e27
2016-04-14 20:06:19 +00:00