Commit Graph

647 Commits

Author SHA1 Message Date
Rintaro Kuroiwa 4891d9a6bf HLS with fragmented MP4
- Add EXT-X-MAP tag for init segment.
- Do not set output field on stream descriptor if not specified on
  command line. If it's set (internally) then it gets copied to
  MediaInfo that gets passed to the manifest generators.

b/36279481

Change-Id: I762c55b255699ec691817dc4806b0dee2f7504b8
2017-03-27 13:20:03 -07:00
Haoming Chen cc04698460 Implement trick play handler.
Change-Id: I3d288dc7c4b77fd3aca1bad2ce96ad733384bcc8
2017-03-24 18:34:42 -07:00
Rintaro Kuroiwa 0a3ef49c11 Change {luma,chrome}_weight_flag to arrays
- The flags were overwritten in a loop and therefore only stored the
  last value of the flag.
- Change the fields into arrays so that the flags are not overwritten.

Change-Id: I59762d7c84bc3d81bf3b0b5e85ffe689260d970a
2017-03-24 21:14:18 +00:00
sylt 81beb8aeb2 Don't do timeshift SlideWindow() logic for static MPD (#218)
This is a fix for issue #216 where the Representation::SlideWindow() logic was
still active even though a static MPD was being asked to be generated (using
the option -generate_static_mpd). This could cause static streams longer than
1800 seconds to start playing at (end-position-for-stream - 1800 seconds).

This change is in compliance with the current usage documentation for the
shaka-packager, which suggests that -time_shift_buffer_depth only is relevant
for dynamic media presentations.
2017-03-24 10:37:13 -07:00
Kongqun Yang bc4c511ad4 Fix Windows build break (#217)
Change-Id: Id0a00c01077adb97f294c68357b629441321e6c7
2017-03-24 10:21:05 -07:00
Kongqun Yang 277c5b23c0 Allow generating avc3 / hev1 from TS and Wvm files
Added a flag --strip_parameter_set_nalus. When enabled, parameter
set NAL units, SPS/PPS for H264 and SPS/PPS/VPS for H265, are stripped
from frames when converting NAL byte stream (AnnexB stream) to NAL
unit stream, which generates avc1/hvc1; otherwise they are not
stripped, and avc3/hev1 is generated.

Parameter set NAL units should not be stripped if they are varying
in the frames otherwise the frames may fail to be decoded.

The flag is enabled by default as we don't usually see varying
SPS/PPS/VPS and it is more space efficient with them stripped.

Set --strip_parameter_set_nalus=false to disable the flag if there
are varying SPS/PPS/VPS in the frames. This addresses #206.

Change-Id: I34bde6f33069f9f77d51a510b39f58a0f0e141aa
2017-03-23 13:53:14 -07:00
KongQun Yang 3d7635d5de Integrate EncryptionHandler
Change-Id: I639e5707994a6601ee02078cd6f77cb38b481585
2017-03-23 18:25:59 +00:00
KongQun Yang 16615095f0 Update NalUnitToByteStreamConvert to support escaping encrypted NAL only
- Also fixed a bug if nalu_length_size != 4.

Change-Id: Ia8c67c4b1ff08a6b64fa1e309b5ecd8f5c8ea421
2017-03-23 18:21:24 +00:00
sylt d2c9a88ec5 Fix possible integer overflow in WebM duration
The issue could lead to the MPD attribute mediaPresentationDuration being
wrongly generated for WebM streams with a duration longer than INT32_MAX.

The root-cause was that StreamInfo::set_duration() accepted an int instead of a
uint64_t. This seems like a pure typo, since StreamInfo already uses a uint64_t
internally for representing duration.
2017-03-23 09:38:51 -07:00
Kongqun Yang d323d4f091 Fix break in Windows (#213) 2017-03-22 09:57:18 -07:00
KongQun Yang b09c8f6521 Update EncryptionHandler to support key rotation and SampleAes
This CL also removes EncryptionConfig stream data type and merges it
into StreamInfo/SegmentInfo instead.

Change-Id: Idb70ce503e61d3c951225cc78b6b15c084e16dcd
2017-03-21 23:50:53 +00:00
Kongqun Yang b891e0271e Make CENC behavior of VP8/9 in ISO-BMFF the same as in WebM
- Disable subsample encryption for VP8 in ISO-BMFF
- Apply block alignment to all subsamples for VP9 in ISO-BMFF,
  instead of just superframes.

Change-Id: I8dd31cc16e87abc4d538330eaff9acb0509497df
2017-03-21 23:12:57 +00:00
Kongqun Yang 0cb0f79c33 Fix build break in Windows (#211)
Change-Id: I0ed7a712fd02ea4dddd9c20541c2df653dc456cb
2017-03-17 12:45:28 -07:00
Kongqun Yang 0c3fb49eeb [WebM] Fix output truncated if using the same file name for I/O
Open output file in DoFinalize() to ensure that the input file has been
closed already.

Fixes #210

Change-Id: I935941b31c667e49be030c8da9f953d8387c7a9d
2017-03-17 01:07:49 +00:00
Rintaro Kuroiwa 15fd745fa7 Remove redundant flags in H264SliceHeader
- luma_weight_l0_flag, luma_weight_l1_flag,
  chroma_weight_l0_flag, chroma_weight_l1_flag were there but not set.
- The user should use the values in pred_weight_table_l0 and
  pred_weight_table_l1 instead which are set.

Change-Id: Ic9c44fb113717346938a339faf074daa32d4c2d2
2017-03-17 00:05:36 +00:00
Kongqun Yang 8806f78655 Use size_t as the type of stream_index
Change-Id: Ica8ffeccc45c1adf5def6fe9b2edde5708d6b432
2017-03-16 20:54:48 +00:00
Kongqun Yang 160700b452 Integrate ChunkingHandler
Also moved MediaHandler output validation to Initialize instead.

This CL also addresses #122 with consistent chunking.

Change-Id: I60c0da6d1b33421d7828bcb827d18899e71884ce
2017-03-16 20:54:36 +00:00
Rintaro Kuroiwa a3ce51785a Make WebVttMediaParser use WebVttSampleConverter
- WebVttMediaParser uses WebVttSampleConverter to generate non
  overlapping media samples.
- The media samples contains ISO BMFF boxes.
- Add kCodecWebVtt to signal that the media is webvtt and
  the samples will be in ISO BMFF boxes.

Change-Id: I639902cdba7b04af75428bc20622e26b8203cfb2
2017-03-15 20:44:36 +00:00
Joey Parrish 924d6d4693 Add HLS audio language support
Before this, HLS output did not contain language information.
Now, media playlists are properly tagged with a language in the
master playlist.

b/36134267

Change-Id: I172e946dbedd096a44cb2f917b007cc004756228
2017-03-10 16:14:14 -08:00
Kongqun Yang c6cbd73465 Rename FlushStream to OnFlushRequest; add FlushOutput
Change-Id: I2b37030b0c35aa34e9c72e43825f157924034227
2017-03-08 11:21:19 -08:00
KongQun Yang 8f2cd6da91 Implements Demuxer and Muxer media handlers
- Also sets up the packaging and verify it works.

Some of the changes are temporary to get the integration going.

Change-Id: I0cf6c379d185e157808acabb9ef58ff93d4a39ae
2017-03-08 19:11:52 +00:00
Kongqun Yang 12c61d6792 Fix pssh README to avoid confusion
Change-Id: I1ac1c4fbb75c2db6d1d960b1681afea0bac61bb8
2017-03-03 16:13:25 -08:00
Kongqun Yang 24ba12c18e Remove escape parameter in NalUnitToByteStreamConverter
The spec actually requires all NAL units to be escaped, so there
is no need to escape it again.

NAL byte stream is one demarcation method; NAL unit stream is another
demarcation method. Regardless of the demarcation method used, the
requirement for NAL unit is the same. See the last paragraph of
7.4.1 NAL unit semantics on the requirement on emulation prevention bytes.

Change-Id: Icc63fcc5cf965632e331f5af5f673164c7c1663a
2017-03-02 23:01:53 +00:00
Kongqun Yang c223bc9144 Keep slice data partition NALs in clear
CENCv3 recommends only encrypting video data in slice NALs. Slice
data partition NALs should not be encrypted.

In the code, differentiate is_vcl and is_video_slice. They are the
same for H265; for H264, vcl NALs include slice data partition NALs
but video_slice NALs do not.

Change-Id: I91f4bdd76d25f0eac50e39aed350ebce3f667121
2017-02-28 18:10:30 -08:00
Rintaro Kuroiwa 5c07ff6b1c Move and rename webvtt fragmenter to webvtt sample converter
- The class should be used to generate samples from the demuxer.

Change-Id: If7f245e5809de8773c0b1cf08673ae62dfeffe09
2017-02-14 16:33:18 -08:00
Rintaro Kuroiwa 593f513c83 Fix windows build
Change-Id: Ie23ebd9f18bf2151e52550bff62e2b043ac65ac2
2017-02-14 11:21:52 -08:00
Rintaro Kuroiwa 1879bca4bb Fragmenter for WebVTT
- Logic for splitting up WebVTT cues into MP4 samples.
- For any cue intervals that overlap, a new sample must be created.

Change-Id: Icf7478f42d8c5790bf6f404d512c1251dd34c405
2017-02-13 13:33:36 -08:00
Kongqun Yang 50787c605d Fix build break in win64
Also updated appveyor to build x64 first.

Change-Id: I1970e39bad559b3105ea31fb82375406d95f4a8b
2017-02-10 22:57:35 -08:00
Kongqun Yang 5f6e4a1dc1 Using std::shared_ptr instead of std::unique_ptr in stream data entry
Change-Id: Ib8c6f97c2329ba5b9431c301d85c1d61f89fc3d8
2017-02-11 02:41:49 +00:00
Kongqun Yang 9990524f98 Implement ChunkingHandler
This handler is a multi-in multi-out handler. If more than one input is
provided, there should be one and only one video stream; also, all inputs
should come from the same thread and are synchronized.
There can be multiple chunking handler running in different threads or even
different processes, we use the "consistent chunking algorithm" to make sure
the chunks in different streams are aligned without explicit communcating
with each other - which is not efficient and often difficult.

Consistent Chunking Algorithm:
 1. Find the consistent chunkable boundary
 Let the timestamps for video frames be (t1, t2, t3, ...). Then a
 consistent chunkable boundary is simply the first chunkable boundary after
 (tk / N) != (tk-1 / N), where '/' denotes integer division, and N is the
 intended chunk duration.
 2. Chunk only at the consistent chunkable boundary

This algorithm will make sure the chunks from different video streams are
aligned if they have aligned GoPs. However, this algorithm will only work
for video streams. To be able to chunk non video streams at similar
positions as video streams, ChunkingHandler is designed to accept one video
input and multiple non video inputs, the non video inputs are chunked when
the video input is chunked. If the inputs are synchronized - which is true
if the inputs come from the same demuxer, the video and non video chunks
are aligned.

Change-Id: Id3bad51ab14f311efdb8713b6cd36d36cf9e4639
2017-02-11 02:41:34 +00:00
Kongqun Yang 7a90ee70ab Implement EncryptionHandler
Change-Id: Iabedf3b02057d6124d5393ae9618490e5595ad89
2017-02-11 01:21:28 +00:00
Kongqun Yang 6235352bf4 Fix mpd duration not set in static live manifest
Closes #201

Change-Id: Ie9ab58ec58e1b135931a6576fb46145454b0f049
2017-02-11 00:52:32 +00:00
Kongqun Yang d592217761 Add a warning message when seeing unsupported stream type
Issue #202

Change-Id: If16fe7ccec5dfd869070dfbbb711240a89b2295b
2017-02-09 12:38:49 -08:00
Kongqun Yang bee59bc2fc Enable generate_dash_if_iop_compliant_mpd by default
This will move ContentProtection element from Representation
to AdaptationSet.

Shaka Player already supports AdaptationSet switching
urn:mpeg:dash:adaptation-set-switching:2016;

ExoPlayer does not support it yet. Filed
https://github.com/google/ExoPlayer/issues/2431 to track the issue.

(ExoPlayer does not like having ContentProtection in Representation
anyway)

Closes b/34691105

Change-Id: I69d0a4d0e15a912a35c8b2686620419a28e4cc99
2017-02-09 11:12:21 -08:00
Kongqun Yang 64c6821dc8 MediaHandler implementation
MediaHandler is the basic processing unit in shaka-packager 2.0.

Change-Id: I5f8d744e39adfff5cfebe0c50a29ead4d717cca4
2017-01-26 03:34:18 +00:00
Kongqun Yang 97fbaaa567 Replace scoped_refptr with std::shared_ptr
Change-Id: Ie8ea66e78c42c7387a97bb1abc331c636be11aa4
2017-01-26 02:20:47 +00:00
Yohann Connell 24b0e3031a Adding PlayReady support to Shaka packager
PlayReady Header Object Spec (PSSH Data VOD)
http://download.microsoft.com/download/2/3/8/238F67D9-1B8B-48D3-AB83-9C00112268B2/PlayReady%20Header%20Object%202015-08-13-FINAL-CL.PDF

PlayReady Live Streaming Spec (Consult Appendix for package acquisition information)
http://download.microsoft.com/download/2/D/D/2DD6B4E8-CABF-4DE9-8F61-895BE8F1ED33/ProtectingLiveTVServicesWithPlayReady_March2015.pdf

Issue #128
Change-Id: Ic5c0d6c642ceba3bdf9e8e413934f0fbcba242e6
2017-01-24 13:36:47 -08:00
Kongqun Yang 80c54a533a Update cmd flags to support static mpd with live profile
- Deprecated command line flags --profile and --single_segment.
  'segment_template' in stream descriptors implies live profile
  and non-single segment.
- Added flag --generate_static_mpd_for_live_profile to generate
  static mpd for live profile; if not set, dynamic mpd will be
  generated.

Close #142

Change-Id: I78879297ed118f0f246c4753a16ad125bd6b5e4f
2017-01-13 20:00:10 +00:00
Kongqun Yang 5aaae303e8 Part one of supporting live profile with static mpd
- Added two new fields in MpdOptions: dash_profile and mpd_type
- Updated MpdBuilder to support LiveProfile with static mpd
- Command line arguments will be updated in the next CL

Issue #142

Change-Id: Ibd35e5c9e1b1ef98043392e3f04a0af4700135c1
2017-01-13 19:59:49 +00:00
Chun-da Chen ce23fbacc4 Add support for udp_file in Windows. (#185)
Add support for udp_file in Windows
2017-01-10 10:05:28 -08:00
Haoming Chen 15dacf6288 Fix signed/unsigned mismatch on Windows.
Issue #168
2017-01-06 17:29:53 -08:00
Haoming Chen 93fe5457bf Fix x64 build on Windows.
- Fix integer conversion warnings.
- Fix #168.

Change-Id: Ie9aadea86a293b49e0854d6549800083143c79df
2017-01-05 14:19:35 -08:00
Rintaro Kuroiwa 655fe7b699 Add gitignores for msvs files
- Add MSVS related patterns to gitignore.
- Removed encryption_modes.h from media_base.gyp. It doesn't exist.

Change-Id: I44bc86ff7e79daf6f7bc61c21a558937530d8e66
2016-12-19 21:26:26 +00:00
Kongqun Yang af0904e428 Fix AdaptationSet switching signaling
Current mechanism of using AdaptationSet @group is not spec compliant.

The spec compliant mechanism is to use supplemental descriptor with
schemeIdUri: urn:mpeg:dash:adaptation-set-switching:2016 as specified
in ISO/IEC 23009-1:2014/Amd.4:2016.

Fixes #156

Change-Id: I4d97648547a23315ba9c09dcadb14e18c99a45fc
2016-12-19 20:12:27 +00:00
Rintaro Kuroiwa 7a4a40acb8 Change values for slice header attributes
- nalu_data points to the beginning of the nalu, instead of
  beginning of the slice header which excluded the nalu
  header.
- nalu_size is the size of the whole nalu.
- header_bit_size calculation is corrected. The size used to exclude
  emulation prevention bytes.

Change-Id: Ic00f42caeddf7b4701f3948e6ef8a3537fd7f770
2016-12-15 17:07:17 -08:00
Kyle Alexander 0c2ee4c844 Add support for 4K and 8K content.
Adds UHD1 and UHD2 track types to support 4K and 8K content.

Bug: Closes #163
Change-Id: I8fd893725cae88e9944244a48607cbaab591b401
2016-12-13 19:24:22 +00:00
Kongqun Yang d8d94da250 Fix packager_test break in Windows
Change-Id: Id43e4bd0e2067fd9ad74f8c5c60df7ddba75e840
2016-12-09 19:11:04 -08:00
Kongqun Yang c8ffa54d35 Fix --version in drive programs
Fixes #176

Change-Id: Ib092f81e1af49dfee43f7b01c0008a1b40c9098b
2016-12-08 14:45:05 -08:00
Kongqun Yang eab7910029 Fix crash when seeing unsupported es descriptor data
Addresses #175

Change-Id: Ia9c2f0c7a8c17a6614db0fe478892bd3c5efd5b0
2016-11-29 23:27:22 +00:00
Haoming Chen 9ed8303a06 Use Windows native ssl instead of openssl.
Fixes #147

Change-Id: I41193adbab5cd979ca958131cd4f148dde5e1f2e
2016-11-09 14:08:52 -08:00
Rintaro Kuroiwa e7c0dbf09d Ensure some method calls NotifyEncryptionStart()
- If OnEncryptionStart() is called OnMediaStart(), someone should be
  calling NotifyEncrytionStart().

Fixes #169

Change-Id: I597f317d2ddeb3091b7e9d238eb9695b4cc9c69d
2016-11-08 15:56:28 -08:00
Kongqun Yang 458fadc6a8 Add udp options support
Three options are supported right now:
- reuse=1|0
  Allow or disallow reusing UDP sockets. Default 0.
- interface=a.b.c.d
  Address of the interface over which to receive UDP multicast
  streams.
- timeout=microseconds
  Timeout in microseconds. Default to unlimited.

A UDP url should be of the form udp://ip:port[?options], here
is an example:
  udp://224.1.2.30:88?reuse=1&interface=10.11.12.13&timeout=12345

Also deprecate --udp_interface_address flag in favor udp options.

Closes #133

Change-Id: I962d35bfedc1779d67ba20ed910207c66b7c1a15
2016-10-31 18:40:40 +00:00
Kongqun Yang 4c6e5f4fa6 Improve support for decryption of full sample encrypted cbcs
1. Allow encryption pattern of 0:0, which is treated as 1:0;
2. Remove the special handling of pattern 1:0 in pattern cryptor
   which may not always be correct;
3. Allow senc/saio/saiz boxes to be absent.

Change-Id: I372e61182ec577107e39cb601c3aed80616b036d
2016-10-14 00:07:46 +00:00
Kongqun Yang e7393fcc22 Fix build break in Windows
Change-Id: I49815abdaa360e11e62a3193bc58d4d7d77fc8a8
2016-10-13 15:54:32 -07:00
Kongqun Yang 6dbcf27f82 [WebM] Use system temp directory if temp_dir is not specified
The original code creates temporary file in the current directory,
which may not always be writable.

Change-Id: Icc278dd5db6be6206ae29defbb7423bae39cc978
2016-10-13 21:48:00 +00:00
Kongqun Yang 472fae24f2 [pssh] Support widevine pssh generation with protection_scheme
- Also import the version of the protobuf bundled in packager first,
  this avoids importing the installed version of the protobuf, which
  may not be compatible with this script.
- Print help if no argument is supplied.

Fixes #162

Change-Id: I1b5e87ba7fb51e81b0d4dd3791aaeb37ee0fd6ce
2016-10-10 10:39:41 -07:00
sammirata 4f9bbe3769 Do not require exclusive bind of udp sockets (#154) 2016-10-04 15:51:45 -07:00
Kongqun Yang 97fc9828f0 [WebM] Move index segment after init segment
Cues used to be generated in the end of the file; if http range
request is not supported, clients have to download the whole
file to get to the cues.

This CL updated TwoPassSingleSegmentSegmenter to write cues
after writing webm header.

This CL also updates libwebm dependency to latest.

Closes #159

Change-Id: Ic73548e1b872e6b13a37627707e7d0ff3e556877
2016-10-01 00:34:56 +00:00
Kongqun Yang ac1d2692cf [WebM] Fix corner case segment generation problem
The original code accumulates sample durations in seconds (double)
and then generates new segment / cluster by comparing the accumulated
value with configured value.

There may be loss of precision when accumulating values in double.
For example, with a GOP of 96 frames and frame rate of 24 fps; the
accumulated frame duration for 96 frames is 3.999999999999, which is
very close to 4.0 but not 4.0. Problem would arise if segment
duration is set to 4. The created segments would have a duration of
8 seconds instead of 4 seconds.

The new code accumulates sample durations in uint64_t relative to
input time scale; the configured segment duration will be converted to
timescale for comparison. This avoids loss of precision.

Change-Id: I3ae24be82a7ce45f923a6f90fea495b8b6b2e7ef
2016-09-30 21:45:27 +00:00
Kongqun Yang f5dc908a0d Implement utility function DoubleToString
Implement our own DoubleToString as base::DoubleToString uses
third_party library dmg_fp.

Change-Id: I0d3d706cfd3de4f833b201110954419e585a21dd
2016-09-30 21:36:11 +00:00
Joey Parrish cda495fb57 Add default_language flag
Setting this flag will tag any matching tracks with kRoleMain.
The player will then know which tracks should be default when there
is not a language match.

Closes #155

Change-Id: I827304c49e345d07b1e76f7a46421b60f228f0ba
2016-09-29 22:59:34 +00:00
Joey Parrish 07c4b4f3bd Drop the DCHECK for language size 3
Language tags can have subtags, such as "por-BR".  While the base tag
should be 3 characters, the entire tag can be longer.  Rather than
parse out the base tag and check that it is size 3 without the subtag,
we can just trust the ISO tag converter to do the right thing.

Change-Id: I2a186b5c8db0f373ee82ef68245ecc33cb068581
2016-09-29 16:37:16 +00:00
Joey Parrish 275e60e5a9 Detect .webvtt as WebVTT files
Some subtitle files use .webvtt instead of .vtt, probably because
ffmpeg uses -f webvtt when converting from some other format.

Change-Id: If88e8d2b90e5dea6751409568d5191e923dec4aa
2016-09-28 15:02:27 -07:00
Kongqun Yang 5edd290694 Fix WebM not encrypted when clear lead is set to 0
- Also regenerated test content to insert an iframe every one
  second: ffmpeg -i source.webm -auto-alt-ref 1 -g 30 out.webm
  This makes sure the content is actually encrypted with one
  second clear lead in end to end test.

Change-Id: I488143c148e2d6c45ba1586f6d0d835dc46db86e
2016-09-28 18:17:09 +00:00
Kongqun Yang 47363dd06a Remove unused vorbis test files
Change-Id: Ifede6faac7e17dcd60d36c1e9e7c9a025f95fbdf
2016-09-13 22:06:38 +00:00
Haoming Chen 03d7a81c83 Fix a precision conversion problem.
- This causes a compilation error on Windows.

Change-Id: I53c03841565be00ebaa3acbfe621bb145f8f1ca2
2016-09-12 08:46:16 -07:00
Haoming Chen 28828b8a15 Add encryption information in demuxer.
- Add decrypt_config in media sample and stream.
  - Update subsamples when converting NAL unit to bytestream.

Change-Id: I7b8975a453f81b22cf74bee3c9a58b7e458cbaae
2016-09-08 17:13:36 +00:00
Kongqun Yang 90e3ec3f9a Some misc cleanup
- Replace DCHECK_EQ with DCHECK for compatiblity
- Add ASSERT_FILE_CONTAINS
- Remove unnecessary release on unique_ptrs

Change-Id: I2ec22c52e47cb451479ae9a59818b9df20f62e63
2016-09-07 10:15:50 -07:00
Kongqun Yang c3d1dc1733 Deprecate STLDeleter and string_as_array
1. Replace STLDeleter with unique_ptr in container, which is
   supported in C++11
2. Replace string_as_array with &string[0] which is guaranteed
   to work in C++11

Change-Id: I7f39c0e51fc8a3fcbb41313094a0ca6b33db7bf3
2016-09-01 17:39:11 +00:00
Kongqun Yang ba2a649c00 LOG_TO_SYSTEM_DEBUG_LOG instead of a file
The current behavior of logging to a file in the same directory
as the binary will break if the binary does not have write access
to that directory.

Fixes Issue #139

Change-Id: I11adfaec676fc9bbef2a5caa3261889d1fba3b88
2016-08-31 14:50:23 -07:00
Kongqun Yang 09891dcd9e Fix access unit detection problem for H264
In H264, there may be multiple consecutive video slice NAL units
in the same frame. The original code assigns a new access unit
for every video slice NAL unit, which is incorrect.

Fixes #134.

Change-Id: I4d44271df48cb08867ddd02f7494fb3573af3356
2016-08-29 11:19:52 -07:00
Kongqun Yang 52cbcb321d Add appveyor config for Windows CI
Also fixes additional Windows build failures.

Closes #22

Change-Id: I8bc416f1a44810c438f095ec137d0d102f4e08a5
2016-08-23 12:24:34 -07:00
Kongqun Yang de9667080f Do not partially encrypt a segment
Mimic iso-bmff behavior: either all the samples in a segment are encrypted
or all the samples are clear.

Change-Id: I03bdbbf5a4b690f4d87c4dceb0f8155e6fae941e
2016-08-19 14:22:40 -07:00
Kongqun Yang 23f2913248 Update DEPS to fix mac build failure
Change-Id: I1cf95d5da8e3b950300cd61cd9286e957ef653ce
2016-08-19 13:49:41 -07:00
Kongqun Yang 644629c616 Replace scoped_ptr with std::unique_ptr
Change-Id: I1ed18bf876b6154d08a1f7913aa7e76c525451f9
2016-08-18 17:12:36 -07:00
Kongqun Yang d501b457d0 Fix WebM keyframe detection in BlockGroup
The right way to detect the presence of a keyframe in a BlockGroup
is to look for the absence of the ReferenceBlock element [1].

Closes b/30433206

[1] https://matroska.org/technical/specs/index.html#ReferenceBlock

Change-Id: I2b3c8704cbabcd5cbee38448e5528425865561a4
2016-08-18 23:20:40 +00:00
Thomas Inskip db92c9a22b Implemented WebM VP9 subsample encryption and decryption.
Change-Id: I98f02c05ec8bcc976e01ff41478e92b8809c0076
2016-08-18 13:55:36 -07:00
Thomas Inskip 336ea5cb34 Windows build working
Change-Id: I6d9cfa0a310c4c6125c839e4d6a085903e981c9c
2016-08-18 08:30:26 -07:00
Thomas Inskip 224747db0b libcurl build for Windows
Change-Id: Ie14613d36509b3f45220092c5c196b14fd4b0a10
2016-08-03 16:05:40 -07:00
Thomas Inskip d059b27a69 Updated third_party/gflags.
WebRTC portion (gyp build) now checked in because separate repository is no longer available due to transition to git.

Change-Id: I56436cbbc3f02a5f77112917d6229aea69f21d97
2016-08-03 07:52:16 -07:00
Thomas Inskip f169bccdc9 Updated libxml.
Change-Id: I4f2315f10227e6b8c2545b81ea1adcf388c37cee
2016-08-02 19:11:17 -07:00
Thomas Inskip 1fbc410489 Updated protobuf library.
Change-Id: I09705df523ba4d0954925845a1619b25d70fd90f
2016-08-02 19:11:17 -07:00
Kongqun Yang 16d8583735 Update boringssl and curl(v7.50.0)
Change-Id: I89b40cf03d1aab9a13b0df449e540ab73d03451e
2016-08-02 12:00:19 -07:00
Gabe Kopley 081b6994d6 Fix broken link to "cenc" Init Data Format spec 2016-07-26 14:52:08 -07:00
Kongqun Yang 01e53ee946 [pssh script] Fix invalid hex_16_bytes error on lower case input
Change-Id: I274634f9076500156527c04bc9cbb9f86da3c6bd
2016-07-25 17:58:03 -07:00
Kongqun Yang ad95b9d8de Fix text formats not recognized problem
Also updated packager_test not to fill in output_format by default,
which will help catch these kinds of bugs.

Closes #130

Change-Id: I77157317de822d3f543f7e27d94bb4df7a1649d2
2016-07-22 20:51:16 +00:00
Kongqun Yang 2cdd54b6f7 Remove restriction on sample duration to be 0 for mp4
0 sample duration is seen occasionally in the contents in the wild.

Closes #127

Change-Id: I48b9fac4add79f39a83fc571502bedb9f0be71df
2016-07-19 18:56:45 +00:00
Kongqun Yang fae7874c4c Add a runtime flag to use dts in timeline for mp4
This flag is needed to workaround the Chromium bug
https://crbug.com/398130, which uses decoding timestamp
in buffered range.

Closes #113

Change-Id: Ib8f18be7165dd968bdc36c18ce29f694235c0c26
2016-07-19 18:56:08 +00:00
Kongqun Yang a80995538d Remove restriction on sps:gaps_in_frame_num_value_allowed_flag
Packager should not care about this flag, whether it is supported
is up to the decoder.

Closes #126

Change-Id: I5e05d291eab01375f7f11c951b9dc93c1358c38a
2016-07-18 20:25:40 +00:00
Kongqun Yang cf778f7149 Update CHANGELOG for v1.5.0
- Also updated a few wording in README and in the code.

Change-Id: I9adf297c00aee92e61e87effa3bdcfaf402638e5
2016-07-12 12:25:48 -07:00
Kongqun Yang f3d35b730c Add version information to generated HLS manifests
- Also refactor the existing version code.

Change-Id: Ib409630c4f87965a37b1d6a1bcec68178a704ce6
2016-07-11 23:26:36 +00:00
Rintaro Kuroiwa db70721e35 Fix HLS packaging failure with clear lead = 0
- Add MuxerListener::OnEncryptionStart() for notifying that further
  segments are encrypted.

b/29621230

Change-Id: I881b29d55baaf3d04e005a3b95d898071c3f272b
2016-07-11 20:43:01 +00:00
Kongqun Yang e0c5874d31 Update encrypted testing content to use common system id
- Also fix a bug in fixed key rotation with empty pssh_data.

Change-Id: I1110b839fc821c5c4778854c0b29032d3ee2d384
2016-07-11 20:10:31 +00:00
Kongqun Yang ad3c1768bb Fix dependency in hls.gyp
Seeing occasional build breaks when building hls_builder, e.g.
https://travis-ci.org/google/shaka-packager/jobs/143445907
Although hls_builder is dependent on widevine_pssh_data_proto
indirectly (through media_base), it does not work all the time.
Looks like the proto has to be a direct dependency.

Change-Id: Id4d9d0b9e669df0967cc767132ef29717243a722
2016-07-08 14:48:47 -07:00
Kongqun Yang 5f0d138ec9 Adjust hls descriptor validation
- Try to determine container name from segment_template if output is
  not specified
- segment_template descriptor is required for TS
- output descriptor should not be specified for TS
- Generate hls_playlist_name from stream number if not specified
- Update packager_test.py to include playlist validation

b/29551186

Change-Id: Ide7169f1c26c85a6e38272dda0e1af77135b4d90
2016-07-01 14:14:24 -07:00
Rintaro Kuroiwa 1095b73a44 Fix TS and Media Playlist with clear lead
- Cannot include PMT for following encrypted segments in the clear lead.
  Some players consider them as conflicting codecs.
- Add EXT-X-DISCONTINUITY in front of first EXT-X-KEY to notify that the
  codec is switching from e.g. "avc1" to "zavc".

b/29621230

Change-Id: I45c74813630c229d66245e992eb3a5117326bb14
2016-07-01 10:43:11 -07:00
Rintaro Kuroiwa 50fe17fa9f Use max bitrate for MediaPlaylist bandwidth calculation
- HLS expects max for VOD.

Change-Id: I91e9e07a27abe7167efeefc99aaada2acbed9314
2016-06-30 21:58:33 +00:00
Jacob Trimble f9bf197f2b Fix bug in VPx codec configuration.
Before, we converted the WebM configuration to MP4 in the video
client, however this lead to a bug when fields were missing.  So
now don't convert until the cluster parser so the extra info from
the stream can be added.

This also fixes a bug where the value was not printed in the warning
logs.

b/29580732

Change-Id: If0a1a4d135f98292cdaae15a11027f42d844e85d
2016-06-30 19:51:09 +00:00
Kongqun Yang f27960e00d Add fixed key support for HLS sample-aes encryption
Change-Id: I5a18e014f697561e0741b043e91f70fe2f11d2ce
2016-06-30 19:47:00 +00:00
Rintaro Kuroiwa dad07c95d7 Set MediaPlaylist version to 5
- Required for KEYFORMAT and KEYFORAMTVERSIONS attributes.

Change-Id: I834cde6e475e4f1192db0c93c4b88e2ab838532e
2016-06-29 10:15:46 -07:00
Jacob Trimble abb1abf5c5 Rename StreamInfo::extra_data to codec_config.
Since the |extra_data| field contains codec-specific configuration
data, it makes sense to call it |codec_config|.

Change-Id: If9e35165a00fe82628cf931df397a8ef06505b0d
2016-06-27 13:16:47 -07:00
Rintaro Kuroiwa b07adaeae7 Warn user when single segment and segment template are used
Change-Id: I4b85f50aa0890fb54a642c38520225df9bdbbbef
2016-06-22 15:59:00 -07:00
Jacob Trimble 6550868574 Add codec private data to VP9 in WebM.
b/29009350

Change-Id: Iaafc87340043eff77c3ef7e1c1135d8c4c4287ae
2016-06-22 16:43:57 +00:00
Kongqun Yang 47a3fb977d Fix 'senc' bug when encrypting mp4:vp9 with superframe
Although superframe index is not encrypted, it should still be inserted
into subsamples list (with cipher_bytes as 0).

Change-Id: I06b38f8333a22a31a71cd67e0412218dd3aaa5c3
2016-06-21 17:17:49 +00:00
Kongqun Yang 0a9de96eb2 Update mp4 container check function to be more robust
Consider the file in iso-bmff format if seeing two known
mp4 boxes.

Closes Issue #110

Change-Id: Ifdc76b6c8b7e26a0988ab4f8d4968e1320cb180c
2016-06-21 17:17:03 +00:00
Rintaro Kuroiwa 60419f26d0 Add driver program flags for HLS
- Add flags and stream descriptor fields for HLS.
- Remove fields from MuxerOptions. Instead pass them directly to
  HlsNotifyMuxerListener.
- Rebase segment names to the master playlist's path where it makes
  sense.

Change-Id: If4f54bc56ff46dc6140859a79ed66f7b99112ed7
2016-06-13 18:30:24 -07:00
Rintaro Kuroiwa 565affe7fb Pass hls_name and hls_group to HlsNotifyMuxerListener constructor
- Also don't Flush the notifier OnMediaEnd().

Change-Id: I2ffbbfacda74bb88678ac4e32b1f28c3e64eb85d
2016-06-13 14:36:39 -07:00
Rintaro Kuroiwa d06a9cd17c Allow encrypted TS without clear lead
Change-Id: Icc4268af94c11b004f020cfb4dd9f5f16a9169af
2016-06-13 20:49:23 +00:00
Jacob Trimble cb3b277575 Optimize the MPEG2-TS demuxer.
Change-Id: I5836fc8503fe22a3d73241dda00c36d0ffb26e65
2016-06-08 21:43:15 +00:00
Kongqun Yang d08f6ae0cc Fix cbcs/cens behavior when applied to non video tracks
- Tracks other than video are protected using whole-block full-sample
  encryption as specified in ISO/IEC 23001-7:2016(E) 9.7, which is
  equivalent to a pattern of 1:0. This is different to the non pattern
  encryption counterparts.
- Also updated the code to allow the existence of other protection
  schemes in the original content, which will simply be ignored.
- The internal cryptor used by AesPatternCryptor should not use
  constant iv, add a DCHECK for that.
- Optimize AesPatternCryptor handling on the special pattern 1:0.

Change-Id: Idc704e7bc6b347741336f38c6d3620fc19392960
2016-06-08 17:02:26 +00:00
Kongqun Yang 84f3911985 Remove top level box check in BoxReader
Change-Id: I3d0e5865851030264d7174031a98031b413e90ce
2016-06-08 00:54:11 +00:00
Kongqun Yang bd1328500c Allow uuid to be a valid top level box
Change-Id: Iec241781321a30fa8eece6a7fc6674d78a02c90f
2016-06-08 00:53:35 +00:00
Kongqun Yang 579678d6a6 Fixed a Opus in Mp4 parsing problem
- Also added end to end test in packager_test.py

Closes Issue #83

Change-Id: I7a8e695a36e654507b75d558710db1ce8867a916
2016-06-01 23:52:33 +00:00
Kongqun Yang 32482c0bce Rename filters directory to codecs
Also move codecs related classes in mp4 to codecs directory.

Change-Id: I83db2bcf9e66b405fcceed78918b762b26cd7fce
2016-05-27 16:13:19 -07:00
Kongqun Yang 6d281c6676 Rename XxxConfiguration to XxxConfigurationRecord
These are really configuration records.

Change-Id: I0d081c2e14f19bcdf8eb52dc094a1e7c096b5f98
2016-05-27 16:11:11 -07:00
Kongqun Yang 68aa1397b0 VPCodecConfiguration box should inherit from FullBox instead of Box
Change-Id: If53c6cab009bdeadc37a1bac34272056a2b7d1dc
2016-05-25 23:59:16 +00:00
Kongqun Yang 7e533ec5ca More cleanup in proto usage
Change-Id: If35f95dbff47bc9e8c1c42cb56d75719367abfc0
2016-05-25 16:57:39 +00:00
Kongqun Yang 4f9c8ea781 Rename top level namespace to shaka
Issue #103

Change-Id: I67945c8ec6d2c178494d3b5400e5ec0170e9d115
2016-05-24 21:55:55 +00:00
Kongqun Yang 72c3a76699 Update repo url in version string
Issue #103

Change-Id: Iec7db3d155c81cf989f1078ed32692f8e037aa65
2016-05-24 21:55:45 +00:00
Kongqun Yang bbcca29013 Move all code inside top level namespace
Change-Id: I83bc081f523dbc5071796a79c2d89b7367ed8d27
2016-05-24 21:55:21 +00:00
Kongqun Yang a217cdce29 Misc cleanup in MediaInfo proto usage
- Replaces SerializeToString with SerializeAsString.
- Construct StringPiece instead of doing implicit conversion when
  doing Base64Encode.

Change-Id: I9a9d4a138b0d949447396387adc599c7cde0b9f4
2016-05-17 13:28:22 -07:00
Kongqun Yang c923cb283a Fix unittest failures in previous cl
Change-Id: Ib44a8d0a2907083281a5afdd5b5277a0a0fc35c7
2016-05-16 17:09:06 -07:00
Kongqun Yang 940cc43de2 Add support for AudioRollRecoveryEntry
This is part of the effort to support Opus in iso-bmff #83.

Changes in this CL:
- Add support for multiple SampleGroupDescription and SampleToGroup
  boxes in TrackFragment box;
- Add support for SampleGroupDescription and SampleToGroup boxes
  in SampleTable box;
- Add support for AudioRollRecoveryEntry in Sample Group and 'roll'
  sample grouping type;
- Also fix a bug that default length was not set correctly in
  SampleGroupDescription when constant iv is used.

Change-Id: I7e31386ea6cd17a8ee5e1dca4c1a90937d51368f
2016-05-16 22:25:38 +00:00
Kongqun Yang cf4a2447c1 Add support for Opus specific box in iso-bmff
This is part of the effort to support Opus in iso-bmff #83.

Change-Id: Ib3678b9cb74eac76372ed83ad48ce1f203ba0c35
2016-05-16 22:25:32 +00:00
Kongqun Yang 26cb91e29b Handle preroll and codec_delay when tranmuxing opus
Closes Issue #102

Change-Id: I26aa56a63c17c85298311cc17963dd26f26e501e
2016-05-16 14:51:51 -07:00
Rintaro Kuroiwa d4e2f3c098 hls_builder should depend on widevine_pssh_data
- There is a race condition during build. hls_builder should depend on
  widevine_pssh_data to remove the race condition.

Change-Id: If9fa8895104a774b8ecc7545a57323ae49274936
2016-05-13 11:41:07 -07:00
Rintaro Kuroiwa e422b4eb0e Handle encryption in TsSegmenter
- Move GetTrackTypeForEncryption() to muxer_util.h.

Change-Id: I315957cb8983f4e7c4acff6907dfdd6ad6907c82
2016-05-11 16:21:05 -07:00
Rintaro Kuroiwa 0ba35147c8 SAMPLE-AES encryption in PesPacketGenerator
- Allow setting a key on PesPacketGenerator.
- Add SAMPLE AES mode to AesPatternCryptor.

Change-Id: Iace2381b4cd3bbce63cd5bdb4cdf3a7cea47537a
2016-05-11 09:56:36 -07:00
Rintaro Kuroiwa 7d8322377e PMT writers for H264 and AAC
- For encrypted segments, PMT specified by SAMPLE-AES specification is
  generated.
- Remove time_scale_ field from TsWriter. It is not used.
- Move ContinuityCounter to a separate file.
- Add ProgramMapTableWriter and implementations.
- continuity_counter is incremented for PAT and PMT.

Change-Id: If0dd0990203cb6990f39a6832048bdc3ff08decd
2016-05-05 15:56:54 -07:00
Jacob Trimble 9bb6c5d8d2 Add more unit tests for EsParserH26x.
This add several unit tests for EsParserH26x to test the various
NAL unit orders and access unit contents.  Some tests are disabled
and will be fixed in another patch.

Change-Id: Id5e3291e22f1fe17ada2c03c42e2cdfe226abcb2
2016-05-05 09:41:31 -07:00
Jacob Trimble 86369efc30 Add support for H.265 in MPEG2-TS demuxer.
This also changes the H.264 parser to correctly determine access
units.

Closes #46
Issue #47

Change-Id: I69a8c47ebf4fe35cef0592997460158b3131084e
2016-05-03 15:08:04 -07:00
Kongqun Yang 2abc7c60b7 Add end to end test for transport stream
- Also replaces resolution to 640x360, to reduce file size.

Change-Id: I030ceaa0c67c2b8f060d1864e2c3150f5d28b612
2016-04-29 17:21:08 -07:00
Kongqun Yang 3ff74398df Fix two bugs in 'cbcs' and 'cens' handling and add tests
- BytesOfProtectedData in 'cens' SHALL also be 16-byte aligned
- Fix an incorrect box definition bug w.r.t to 'cbcs' and 'cens'
- Also add various protection scheme tests

Close #77
Close #78

Change-Id: I63c8d8b01ce16ed60affa97ec95fc62bc2da06df
2016-04-26 17:27:26 +00:00
Kongqun Yang 976bf69294 Add decryption verification tests
Change-Id: I735dc14c618537bc4a69fcfbe2bc634bfe2a2166
2016-04-26 17:27:13 +00:00
Kongqun Yang 855d1b028f Correctly generate ContentProtection value for non-cenc protection schemes
Change-Id: Ie5469a7f192cc6167f029befd9cca7f9ab0d1f27
2016-04-26 17:27:04 +00:00
Rintaro Kuroiwa 67e0f9f31b Pass playlist type to MediaPlaylist
- The current implementation only handles VOD.
- Add #EXT-X-PLATLIST-TYPE.
- For VOD Add #EXT-X-ENDLIST at the end.
- Append comma at the end of EXTINF.

Change-Id: I16f01da66f8bbf0229395cb380fa125ffd9328a8
2016-04-25 12:36:11 -07:00
Kongqun Yang 25305b6aa3 Clarify that WVM format always uses all zero IV
Close b/22894397

Change-Id: I9f2295130d32ea5135ef0143ec7758df1114b18d
2016-04-22 13:42:37 -07:00
Kongqun Yang 797ae3e639 Copy packager_test.py to out directory on build
- Disallow packager_test.py to be executed from source directory
- Remove the building of packager target from packager_test.py. User
  need to make sure packager builds successfully before running
  the test.
- Remove build_type parameter which is no longer needed

Change-Id: I479d3c39b8d45730e5388d194069f1320fa5cd75
2016-04-22 18:16:33 +00:00
Kongqun Yang ba42fed9e5 Suppress warning messages when determining containers
- These warning messages are not useful since the code needs to
  loop through various container types to find a matching one.

Change-Id: I1d8eca563faf24e485ac71cce32e09515c174bf2
2016-04-22 09:46:21 -07:00
Jacob Trimble a789dc6d3c Replace Nalu::InitializeFrom* with Initialize.
Replaces the InitializeFromH264 and InitializeFromH265 methods with
a single Initialize method that accepts a CodecType.  Also moves the
CodecType enum from NaluReader to Nalu.

Change-Id: Ie33aa9c0d772a7f3c6a2e9c867e25be3edfe7828
2016-04-21 13:35:49 -07:00
Kongqun Yang 77fc3f6988 Fix an unintended warning message
The warning message was included accidentally for audio encryption
during rebase.

Change-Id: I105f2f01cb9016fc6c57494708ba6d9cbb2ff53c
2016-04-21 19:11:01 +00:00
Rintaro Kuroiwa f3ed07a64e Use MuxerListener in TsMuxer
- MuxerListener is used in TsMuxer so that a listener can be used to
  e.g. generate manifests.

Change-Id: I11c745e1c2b71d5ec901387fe42713d4ad69dc03
2016-04-21 10:45:43 -07:00
KongQun Yang 13202f91b6 Fix a few problems in AesCryptor classes
- CBC cryptors should accept IV of size 8 bytes - it will be zero
  extended to 16 bytes.
- Fixed iv() not updated problem in AesPatternCryptor.
- Replace kChainAcrossCalls with ConstantIvFlag enum flags.

Change-Id: I3fb4de0e8abbe891e6271e779373ba53f8df660d
2016-04-20 14:14:04 -07:00
Rintaro Kuroiwa 66b82f87dd Add HlsNotifyMuxerListener
- A MuxerListener implementation that uses HlsNotifier.

Issue #85

Change-Id: I4c94c025be112d2bd02a78123da3f275e09c3a06
2016-04-19 18:19:11 -07:00
Rintaro Kuroiwa 102f3f7062 Scale sample duration in TsSegmenter
- Sample duration wasn't scaled in TsSegmenter to calculate the current
  segment's length. So segmenting length did not always respect
  segment_duration.

Change-Id: I1ef4747dbd12bde7852ffcbceb971f2775aa18b3
2016-04-19 23:21:16 +00:00
Jacob Trimble bbf9c6849b Parse vui_parameters in H.265.
Parsing vui_parameters in the SPS is required to be able to extract
the resolution of the video.  This also adds a method to extract
the resolution to the H.265 parser.

Issue #46

Change-Id: Idf498d2afdb36a689490151a4ae6baef0b6b73f6
2016-04-19 23:15:34 +00:00
Rintaro Kuroiwa 2756902934 Fix small bugs in HLS playlist generation
- Media Playlist that is non-VERSION 1 must have EXT-X-VERSION.
- Attribute for EXTINF should be CODECS instead of CODEC.
- BANDWIDTH attribute expects bitrate (not bytes per second).
- Removed unnecessary check if MediaInfo has media_info_name, in
  SimpleHlsNotifier::NotifyNewStream().

Change-Id: Ia63cfa59e5e2ec24bbf1b784164e6e41176fc589
2016-04-18 17:14:26 -07:00
Jacob Trimble a80e16bab0 Close files in Finalize for WebM segmenters.
Now Finalize() will close all the files used by the segmenters.  This
is important to allow WebMMuxer::FireOnMediaEndEvent to get the
correct size of the media file.

This also changes the WebM muxer to use 64-bit numbers for the init
and index ranges to correctly support >4GB files.

b/28194272

Change-Id: Ia84e4a4b0756f89644efea99a1a51968b22a1338
2016-04-18 21:28:09 +00:00
Rintaro Kuroiwa a50b5787dc Use TsMuxer in driver program
- The driver program can output ts files.

Change-Id: I190ba17ded91a0be48cc0a1a8b15e44d20800eec
2016-04-18 19:13:50 +00:00
Rintaro Kuroiwa 300c23104e MuxerOptions and MuxerListener change for HLS
- Add hls_name and hls_group_id fields to MuxerOptions. This is used to
  pass the NAME and GROUP-ID values for EXT-X-MEDIA tag to
  MuxerListener.
- Change MuxerListener::OnEncryptionInfoReady() to take an
  initialization vector.
- Change MuxerListener::OnNewSegment() to take segment name.
- Reworded and formatted MuxerListener comments to Doxygen style.

Issue #85

Change-Id: Iea06e68552a56ae180177ffd6ca315a7cf39456c
2016-04-18 11:40:10 -07:00
Rintaro Kuroiwa 16ba8da295 Map to raw pointer instead of scoped_ptr
- Fixes Mac build.
- Mapping to scoped_ptr doesn't work, instead use raw pointer and
  STLValueDeleter.

Change-Id: If17e43d9d4aba8c92a76d2b1ca99a64c6a2b0e84
2016-04-17 00:12:10 -07:00
Rintaro Kuroiwa a6646841b7 HlsNotifier and implementation
- HlsNotifier interface, similar to MpdNotifier.
- SimpleHlsNotifier class can generate playlists.
  It can also handle encrypted media with Widevine encryption.

Issue #85

Change-Id: I9305ddf42796370d3e5df360633c80997c909365
2016-04-16 16:22:35 -07:00
KongQun Yang c6445b749c Change AesCbcEncryptor/AesCbcDecryptor kNoPadding behavior
Leave unaligned residual block in clear in stead of rejecting it.

Change-Id: Id5452f32f6147e26ac68d9cddb948e8b4f77d107
2016-04-15 13:31:09 -07:00
KongQun Yang e253747453 Add support for 'cbcs' and 'cens' protection schemes
Issue #78

Change-Id: I9f71b9a92067e2f6b388092494a7d6a84986cdc0
2016-04-14 15:31:11 -07:00
KongQun Yang c8819cb257 Add mp4 box fields for pattern encryption
- crypt_byte_block and skip_byte_block for pattern-based encryption.
- constant_iv for constant iv (used by 'cbcs' protection scheme).
- Also renamed iv_size to per_sample_iv_size.
- Also changed "bool is_encrypted" to "uint8_t is_protected" to align
  with CENC spec.

Issue #78

Change-Id: I2878a91e0ebe536a08a3e3109daf157fe4440e27
2016-04-14 20:06:19 +00:00
Rintaro Kuroiwa 4391f247ac Fix CRC for PMT and PAT
- CRC for PMT and PAT should be CRC-32/MPEG2.

Change-Id: I9fafdbd6df46fc960c4f0e205f70be701bf7629f
2016-04-14 18:25:18 +00:00
Rintaro Kuroiwa 49e87a34a3 Expose memory file system prefix
- Memory file system prefix should be exposed for testing.

Change-Id: Id284d77084b98c8175cdd485480f2fc49241e137
2016-04-13 23:17:42 -07:00
Jacob Trimble 404660cbdb Add H.265 byte to unit stream converter.
Also renames NaluReader::NaluType to CodecType.

Issue #46

Change-Id: I37d21fcb1109659f14baf43308a94c5e2d9ac2d1
2016-04-13 10:21:52 -07:00
Rintaro Kuroiwa f1e4f74a14 HLS MasterPlaylist class
- Class to generate Master Playlist.
- Takes multiple Media Playlists to generate all the playlists.

Issue #85

Change-Id: I3e2be8763cc771d468245f64753e8a38b0746686
2016-04-12 23:55:21 +00:00
KongQun Yang 2ac57bf9b9 Use FOURCC as protection scheme parameter
- Remove EncryptionMode enums
- Remove AesEncryptor::InitializeWithRandomIv, replaced with
  a static function AesCryptor::GenerateRandomIv, which should
  be called to generate the iv if the iv is empty.

This change is to prepare support for CBCS and CENS pattern-based
protection schemes.

Issue #77

Change-Id: Icba35089d6e451cbea7ebbf5dd5674079f206390
2016-04-12 16:06:49 -07:00
KongQun Yang 0218d9c690 Move fourccs.h and rcheck.h to media/base
- FOURCC is not a mp4 only concept. It is also used elsewhere.
- RCHECK has been used outside formats/mp4.

Also updated the FOURCC enum names to match the actual fourcc code, e.g.
FOURCC_mdat for "mdat".

Change-Id: I5be02f12695882c0fb41d359305fb8ec01303a06
2016-04-12 21:28:37 +00:00
Rintaro Kuroiwa ce55fbcae2 Fix build
- Use list of raw pointers instead of scoped_ptrs.
- Use base::is_same instead of std::is_same.

Change-Id: I0eea539584acf1f34da2b1ba069ba94e53bfa785
2016-04-12 11:04:46 -07:00
KongQun Yang 1d74988159 Implement AesPatternCryptor for pattern encryption/decryption
Issue #78

Change-Id: If0fadf6f83ef67dd39af29080bab6ed71fb35290
2016-04-12 17:09:25 +00:00
Rintaro Kuroiwa 2adaf1712d HLS MediaPlaylist class
- A class to generate Media Playlists.

Issue #85

Change-Id: I689e97c767049bc21de279c743cbabf4ca4711be
2016-04-11 20:53:47 -07:00
KongQun Yang 5fc09763ce More cleanups in AesEncryptor and AesDecryptor
- Create AesCryptor class as the common base
- AesCryptor::Crypt function will serve as an Encrypt function for
  encryptor and a Decrypt function for Decryptor.

Change-Id: Ie91fb14964b5091786705bf510656f40d73af160
2016-04-11 23:30:32 +00:00
Rintaro Kuroiwa e92556658e Add TsMuxer class
- Simple implementation of MPEG2 TS muxer.
- This does not take MuxerListener so a manifest cannot be generated.
- Encryption is not supported.

Issue #84

Change-Id: I117b772dbbce5437398defbd564883ad758916d5
2016-04-11 00:41:24 -07:00
Rintaro Kuroiwa 49d1563965 Only segment before key frames
- Segmenter should not finish a segment unless the next sample is a key
  frame.
- Renamed PesPacketGenerator::Finalize() to Flush().
- Use duration from the sample instead of the one copied to PesPacket.
- Remove duration field from PesPacket.

Issue #84

Change-Id: Icd90e65fd63fdeb955e7abac3473b0b54db6ac4a
2016-04-11 00:37:10 -07:00
Rintaro Kuroiwa ccc2dc46d8 Add TsSegmenter class
- Multi-file segmenter implementation.
- The samples are passed to PesPacketGenerator to generate PesPackets.
  PesPackets are passed to TsWriter to write them to file.

Issue #84

Change-Id: Ia4cae2abe2e7df46016dcdb791baaab2818aea03
2016-04-11 00:15:45 -07:00
Rintaro Kuroiwa 25a3fec213 Add TsWriter class
- Creates PAT and PMT from StreamInfo.
- Takes PES packets and write them to file.

Issue #84

Change-Id: Id0a95e66f126cc2b0fbb6b2169a391d64e49b06a
2016-04-11 07:05:24 +00:00
KongQun Yang b7f457e368 Update CHANGELOG for v1.4.0
Change-Id: Ib4fe35fc47ce34aaae9176d8379bd01704c6bd99
2016-04-08 19:12:03 +00:00
Rintaro Kuroiwa 8ea5df820e Use list of raw pointers rather than scoped_ptr
- Containers with scoped_ptrs break mac builds.

Change-Id: Id4b9aa735511bf27c83aa8465f64a627fb7fa1d2
2016-04-07 12:28:08 -07:00
Rintaro Kuroiwa cfd782ff40 Add PES packet related classes
- Define PesPacket class.
- PesPacketGenerator creates PesPackets from samples.

Change-Id: Icfd3656b498e0075f83ff3c789f95658f98c6144
2016-04-06 18:25:17 -07:00
Jacob Trimble 40e1cc87d1 Fix bug in WebM SeekHead.
The positions in the SeekHead element should be relative to the
Segment payload, not the start of the file.  Also added entries for
Tracks and SegmentInfo.

Change-Id: Id692da25fffc27d78e9f1a06d061431aeb6f1e7c
2016-04-04 15:50:55 -07:00
Jacob Trimble fe6775a509 Fixed off-by-one bug in WebM muxer
There was a bug in the single-segment WebM muxer where the index
range was incorrect.

Closes #99

Change-Id: I7cde98f9b5e88615fcf0ab406b22f65f0b14e15f
2016-04-01 16:26:41 -07:00
Jacob Trimble c9645bc21b Add a utility script used to generate/parse PSSH boxes.
This script can be used to parse and generate PSSH boxes.  This
serves two purposes:

* Parse PSSH boxes into a human readable format.
* Take several arguments to generate PSSH boxes.

This is also able to parse Widevine and PlayReady PSSH data and will
print the parsed data instead of the data as binary.  This can also
generate Widevine PSSH data instead of having the PSSH data passed
as an argument.

Change-Id: I245bd9abf79a259f6fda01d1fc0489b4a4a73db5
2016-04-01 16:19:45 +00:00
KongQun Yang a9e5a2ff4f Refactor and optimize cbc encryption/decryption
- Optimize and clean up encryption and decryption code.
- Consolidate various CBC encryption/decryption schemes into a
  common class.
- Make it a constructor argument whether cipher block chain is
  continuous across Encrypt/Decrypt calls.
- Also align protected region size as required in CENC spec.

Issue #77

Change-Id: I533d92ada3cd80933b532b9c3a1cca105ba66f8e
2016-03-30 17:22:13 -07:00
KongQun Yang a2438554f6 Fix gpylint errors on packager_test.py
Change-Id: I28074c4e9270279ebb83f64b2f9f615b4f705079
2016-03-29 17:37:58 -07:00
Jacob Trimble e7926e7ccb Fixed mac build for H.265 parser.
Another feature that the mac build does not support.  It incorrectly
uses a copy-constructor when creating new values in a std::map.  This
is incorrect and should use an in-place constructor with C++11.

Change-Id: I1e822747e97eb4ae01596d63ea05058cc383cb49
2016-03-29 15:52:03 -07:00
Jacob Trimble cdcfc4c44b Added H.265 parser.
This adds a preliminary H.265 parser.  The parser only support
reading the slice header.  This also attaches the H.265 parser to the
VideoSliceHeaderReader so H.265 frames can have unencrypted headers.

Issue #46
Closes #40

Change-Id: I8b98e1f8bb04e359d9bf896fd5b2939c326b196b
2016-03-29 20:21:46 +00:00
Jacob Trimble 6f8cbf90b9 Add packager flag to include common PSSH with Widevine.
The flag --include_common_pssh will add another PSSH box in addition
to the Widevine one which will contain the key IDs of all the keys.

Closes #88

Change-Id: Ic719b19747530f0e4856cfb36471a644d572a734
2016-03-28 22:27:51 +00:00
Rintaro Kuroiwa 94b4c52bf5 packager test should respect builddir_name
- Use 'builddir_name' environment variable to create the output
  path in test_env.py
- This allows running the packager test with different build
  configs because it doesn't use 'out', e.g. it could use 'out_clang'
  for clang builds.

Change-Id: I843343ec8e2d0a64dffbab2fdbd38aead5c97718
2016-03-26 16:42:00 -07:00
KongQun Yang 0c46943177 Improve handling of unescaped NAL units in byte stream
The new algorithm will parse NAL unit header and only starts a new
NAL unit if it is valid, otherwise it will be considered part of
the previous NAL unit.

Closes #96

Change-Id: I45f2a0f37d51841ee8345d6d0d38fcda57e0a009
2016-03-24 13:15:17 -07:00
Jacob Trimble d5cdd00ba1 Move fixed key encryption code to new FixedKeySource.
KeySource is now an abstract class that is implemented in two
types WidevineKeySource and FixedKeySource.  This also updates
fixed key encryption to use v1 PSSH boxes with the common system
ID.

This also updates the --pssh argument.  Now it accepts full PSSH
boxes rather than data.  Multiple PSSH boxes can be concatenated
together.

Issue #88

Change-Id: I4bd0290f6d8b965f7d118f075c96e0f267d7e831
2016-03-21 10:30:47 -07:00
Bei Li e39c3572af Add support for "AES 128-bit Cipher Block Chaining (CBC-128) Encryption"
- Part 1.

- Add packager command line argument "protection_scheme" to specify
  protection scheme. Plumb through packager code to enable CBC
  encryption/decryption.
- Add scheme type "cbc1" to sinf.
- Refactor AES encryptor and decryptor.
- Need more work in the subsample handling.

Issue #77

Change-Id: I3a9304d89adf5efbfb226b6e805a3077b6cb8c68
2016-03-17 16:51:04 -07:00
KongQun Yang ef81be5f7b Fix a subsample incorrect size bug in mp4 encryption
Clear size was not correctly calculated for non-video slice nal units.

Change-Id: I8371973a4ae346cbe4c1891ab274fb7a86983db8
2016-03-14 17:42:46 -07:00
Jacob Trimble 1863f5a569 Renamed SPS/PPS to Sps/Pps.
H264SPS -> H264Sps
H264PPS -> H264Pps
ParseSPS -> ParseSps
ParsePPS -> ParsePps
GetSPS -> GetSps
GetPPS -> GetPps

Change-Id: Ib658c05d78baabc698af4e52d8b4c77dbfbd6870
2016-03-14 12:44:05 -07:00
Jacob Trimble 43dc6248a3 Renamed H264BitReader to H26xBitReader.
Change-Id: I960a636541a5cadcce5f41e1a75c3ddfbbfaa803
2016-03-14 12:02:21 -07:00
Jacob Trimble 144cdc5e59 Change to use ProtectionSystemSpecificInfo.
* EncryptionKey now contains them rather than a PSSH box.
  * Outputs PSSH boxes for each entry.
  * Outputs a ContentProtection element for each entry.
* Removed SystemName and UUID from KeySource.
* Removed --scheme_id_uri packager argument.

Issue #88

Change-Id: I2651784c3220fd64f5b1773fdcd70285690cf8c0
2016-03-11 17:56:25 +00:00
Joey Parrish f3e19fc002 Split AdaptationSets by content type, container, codec, and language
This adds a new function to key AdaptationSets based on the
characteristics that require separate sets.

With this, we are now able to package multi-codec assets at once
(AVC1/MP4 and VP9/WebM in the same manifest, but in different
AdaptationSets).

Fixes b/27534371

Change-Id: Ie82fd096d2ef99fd23e323d60fd0849b1e8d7500
2016-03-10 15:53:17 -08:00
Joey Parrish b6718a37c5 Accept language subtags
When normalizing language tags to their shortest form, we need to
split off subtags and treat them as separate components that do not
get changed.

Furthermore, MP4 can only store language tags without subtags.  So
only store the main language in MP4 output.

Fixes b/27533973

Change-Id: I96049e274aae9841e321c53039ef6464a2e61574
2016-03-09 23:23:12 +00:00
Jacob Trimble 591650e61d Removed the use of emplace_back.
This does not currently work on our mac builds, so this replaces it
with push_back.

Change-Id: Iecd292969daac059bdd23d1ad937ff4366a7943f
2016-03-09 10:51:57 -08:00
Jacob Trimble c4246d04fd Change FetchKey usages.
Add another overload to fetch by key ID directly, used by WebM.
Changed the one using PSSH data in favor of the entire PSSH box.

This also moves the Widevine proto to media base.  Now, the
Widevine key source handles creating the Widevine specific PSSH
data.

Change-Id: I6f4633facad39207809ffbad970635d1f9d70983
2016-03-08 21:45:05 +00:00
Rintaro Kuroiwa 90731d79df Unit stream to byte stream converter
Change-Id: Iafe1989019e306281f17d80faa348262043cd3b8
2016-03-07 19:10:42 -08:00
Jacob Trimble 9d67f545ed Add support for v1 PSSH boxes.
This moves pssh parsing to its own class in media base.  This type
can be used to parse and generate pssh boxes.  This new type also
supports both v0 an v1 boxes.

This also modifies the pssh box definition to only have the |raw_box|
and moves all pssh parsing and handling into mp4 media parser.

A follow-up change will replace the remaining usages of 'raw' pssh
box data with the new type.

Issue #88

Change-Id: Ic2436ecb5df8b3558b81e600dc095b0efd122ab1
2016-03-07 14:45:01 -08:00
Jacob Trimble 0ca1160474 Moved common code out of H264Parser.
This moves common parsing code out of the H.264 parser.  This is in
preparation for the H.265 parser to use.

Change-Id: Id05b55f6adc2ea31e97c8a9f22023e851808a8a7
2016-03-04 13:25:54 -08:00
Jacob Trimble 0803e31836 Read SPS/PPS Nalus from decoder configurations.
Now the AVCDecoderConfiguration and HEVCDecoderConfiguration classes
will read the Nalu from the blocks, meaning they will store a
vector of Nalu objects for the data.

Also added H.265 to the Nalu class and added argument to NaluReader
to read H.264 vs H.265 Nalus.

Change-Id: I8d8194a90fd72a30af66e9776e01379f8428542c
2016-03-04 18:41:50 +00:00
KongQun Yang 91b43773fd Update hevc test file to have more than one key frames
This is needed to test encryption in end to end test since the video
is segmented in gop boundary by default. Due to the existence of clear
lead, the first segment is always left in clear.

Change-Id: Ie8d37a1740f9061515298cb9fb7b58983592cc5b
2016-03-03 17:34:24 -08:00
KongQun Yang cef1729652 Replace vector_as_array(...) with vector::data()
vector::data() introduced in C++11 does exactly what vector_as_array does.

Change-Id: Iab436b6f1ce7db7678bd240256d9ae80208be07b
2016-03-02 10:22:15 -08:00
Jacob Trimble 53e6d2d149 Renamed Nalu |data_size| to |payload_size|.
This member refers to the size of the payload (the data after the
header).  Since the pointer is called |data|, calling this size
|data_size| is confusing.

Change-Id: I9174cf6c5c402beddaebbce7c16514470ba0c3da
2016-02-26 16:11:30 -08:00
Jacob Trimble e0e0925b31 Add error messages for when a WebM Cluster is too large.
Issue #90

Change-Id: I7fe8393df931a12e999724e3604648b97de24da7
2016-02-26 19:21:52 +00:00
KongQun Yang a01b07fbfd Log an error when PushSample fails
Change-Id: I7911881a42af0edac10b36ad82158e103418714a
2016-02-24 12:44:10 -08:00
Jacob Trimble 0d3951ff74 Moved Nalu data pointer position.
Now the Nalu data pointer points to the start of the NALU header
rather than pointing to the start code.  Added a new method on
NaluReader to check whether the stream starts with a start code.

Change-Id: Ifaecbe0c911aa5cffdf0a966028e6cada8621cc3
2016-02-23 10:48:14 -08:00
Jacob Trimble 96abd90ca2 Only encrypt video slice NAL units.
For non-video slices, the data is not encrypted.  This also skips the
frame headers for H.264; support for H.265 will be added later.

Issue #40

Change-Id: Id0cb0fb9ddb6adedf63ef4aef6b3a26260a21654
2016-02-22 21:52:05 +00:00
KongQun Yang 0b0a8a721a Fix unittest break in Mac due to C++11 incompatibility
Fixes Issue #80

Change-Id: I17bb0706cf99b77fa879f86805017462e5027537
2016-02-17 16:41:35 -08:00
Jacob Trimble 890c601dce Split NALU enumeration into its own class.
Removed the code for NALU splitting in the H.264 parser and moved
it to its own class.  Also added support for length-prefixed
NALU splitting for use with the encrypting fragmenter.

Change-Id: I1e91266681f1b117fb2382cf80590651efc06619
2016-02-17 00:59:38 +00:00
KongQun Yang 6f3e5c77b7 Fix generate_version_string errors with old versions of git
Change-Id: I870bc65849906a52a94f8e562e780c8bf7924118
2016-02-11 18:01:00 -08:00
Bei Li 9ddf9276ce MPD signaling for Dolby Enhanced AC3 audio.
Parse channel layout value from dec3 box. Pass it through audio stream
info. MPD builder forms audio channel configuration to signal ec-3
codec.
Specs: 1) ETSI TS 102 366 V1.3.1 Digital Audio Compression (AC-3,
Enhanced AC-3) Standard E.1.3.1.8.
2) DASH-IF Interoperability Points v3.0 9.2.1.2.

Issue #64

Change-Id: Ia2c22dd3d82e757ba5a88ba1de35c5d593f5005e
2016-02-08 16:22:01 -08:00
KongQun Yang 38c30bd7a1 Allow other text sample entries than wvtt
Before this change, the media file will fail to be parsed if it contains
non-wvtt text streams. This change allows demuxing of video and audio
streams in these media files even though the text stream is ignored.

Fixes #74.

Change-Id: I7ef108a51125c4965fe48c39efe9638f3a0e4759
2016-02-04 11:36:44 -08:00
Jacob Trimble bb3918e62b Sets the duration of the last frame in WebM Cluster.
This also changes the way frames are written in the WebM muxer.  Now,
frames are stored and written on the next call to AddSample.  So each
call to AddSample will write the previous frame.  This is needed to
determine whether the given frame is the last one in the cluster.

Closes #70

Change-Id: Ic69ebad3c4729cdaa2017c9c7f497048501ac907
2016-01-27 20:08:08 +00:00
Anders Hasselqvist d19e8a01f1 MacOSX: Import apple_apsl
Import from 67e43c31a2/apple_apsl/
2016-01-22 18:50:05 -08:00
Anders Hasselqvist 8ac8190d07 MacOSX: Configure curl for MacOSX
Create a new MacOSX configuration for curl:
- Add a curl_config.h created on MacOSX
- Add a find_curl_ca_bundle.sh that searches default folder from Homebrew and MacPorts for a ca bundle.
  To find the files either install OpenSSL from Homebrew or MacPorts, or Curl from MacPorts.
2016-01-22 18:50:05 -08:00
Anders Hasselqvist cb8de4c2f4 MacOSX: Solve conflicting libwebm targets
When generating ninja files we end up with two libwebm targets.
packager/media/formats/webm/webm.gyp:webm -> becomes target libwebm
packager/third_party/libwebm/libwebm.gyp:libwebm -> becomes target libwebm

Solve this conflict by renaming libwebm.gyp:libwebm to mkvmuxer.
2016-01-22 18:50:05 -08:00
Anders Hasselqvist 4195c5e505 MacOSX: Fix run_all_unittests target name conflict
There is a target name conflict between packager/base/base.gyp:run_all_unittests and
media_test.gyp:run_all_unittests. Change to media_test.gyp:run_tests_with_atexit_manager to
resolve the conflict.
2016-01-22 18:50:05 -08:00
Anders Hasselqvist db7bebe3fa MacOSX: Solve libbase.a conflict link error
On Linux the generated ninja files use path/libbase.a as target name.
In the Mac ninja file the target is just libbase.a.
As we have two libbase.a (base/base.gyp and media/base/media_base.gyp) the linking stage fails to include both libs.

Solve by renaming media_base.gyp's base target to media_base
2016-01-22 18:50:05 -08:00
KongQun Yang 2c3aed4842 Handle possible NewSampleEvent before ParserInitEvent
For WebM contents, it is possible that we may receive some NewSampleEvent
before receiving ParserInitEvent. This is because init event is fired
after analyzing a video block which could come after an audio block.

Also modified Flush() function to return a bool to indicate whether the
the flush is successful and whether the samples are handled correctly.

And added macro to ensure Flush() and Parse() results are handled.

Fixes #71

Change-Id: I2294d6f529f54e4578344916559bb1bc116c745a
2016-01-22 17:50:54 -08:00
Jacob Trimble cd74066bb4 Outputs default_KID for non-MP4 ContentProtection elements.
Now outputs cenc:default_KID attributes on all ContentProtection
elements for non-MP4 streams.  For MP4 streams, it will still output
the default ContentProtection element.

Closes #69

Change-Id: I38b24297aa3c2ccbcbde38b44279b56c37a388f7
2016-01-22 21:47:42 +00:00
KongQun Yang 22498e125a Add WebM decryption support
- Also refactor decryptor management code out of mp4_media_parser.cc
- Move decryptor managment logic to DecryptorSource class to make it
  available for webm as well
- Remove data_offset member from DecryptConfig which is not useful
- Add widevine_pssh_data.proto file

Closes #72

Change-Id: I1d32baf4013ebd3382b5372c7433fae5033a260e
2016-01-21 16:59:58 -08:00
KongQun Yang 940c3571aa WebMParser: set duration and dts correctly
- dts was not set earlier, although for WebM, we could assume that
  dts is the same as pts.
- Calculate block duration with the difference with the next block
  if duration is not encoded, even if track default_duration is set.
- Use track default_duration as a duration estimate for the last
  block.
- This also removes opus duration computation from encoded data.

Issues #67, #68

Change-Id: Icaa2769dcb2a89269ae014f44ad6a9262770aed2
2016-01-20 11:33:55 -08:00
Bei Li 8563db4cff Support Dolby audio Enhanced AC3 in ISO BMFF (Part 2)
- Box definitions for box type DEC3.
- Parser/muxer changes to support Enhanced-AC3 audio codecs.
- MPD signaling will come in Part 3.

Issue #64

Change-Id: Ifcd5efa1f61b470ec225127925631e4329853259
2016-01-15 14:44:06 -08:00
Jacob Trimble d1d75f477c Add encryption support to WebM.
This does not support key-rotation and will give an error.  This is
because WebM does not have a way to indicate a change in key ID using
media segments.

b/22463551

Change-Id: I9b3dac818dc370302a5afc0d25d8a060b64d30cd
2016-01-15 11:51:41 -08:00
Rintaro Kuroiwa 5ad076d92c Fix schematron validation problem for live
- From DASH spec 2014, MPD@publishTime and Period@id are required for
  'dynamic' MPD.
- Period@id is always set to "0".
- MPD@publishTime is set to the time when the MPD is output.

Issue #55

Change-Id: I5e50aa11067176e69a2343fbc82ca42129703e1b
2016-01-14 16:53:25 -08:00
Jacob Trimble 61a8d4f09e Add codec private data to WebM muxer.
Added packager tests for WebM audio.  Also now only outputs the
language if it's not empty.  Fixed the packager tests for that.

b/26540606

Change-Id: Ica804bd710055bcaa2113f948d723dedd78ef909
2016-01-14 12:36:17 -08:00
KongQun Yang 27a6e31595 Fix test break in previous cl
- Golden files were not updated correctly.

Change-Id: I4f2bd92261b1e5ee789f5125db05d83bc9335a59
2016-01-13 14:37:44 -08:00
KongQun Yang 4b111a030e Update end to end test with vpx packaging
Change-Id: I70886693b5ad7d4c52170e57901c2437eba69214
2016-01-13 19:34:50 +00:00
KongQun Yang b4e9b5ac2b Set codec string for webm contents correctly
The new codec strings may not be accepted by old versions of browsers.

Change-Id: Ic1bdde80f204f56674fca1f959a3897146953bcf
2016-01-13 19:29:04 +00:00
Rintaro Kuroiwa 2909a53568 Remove redundant attributes from Representation
- @width, @height, @frameRate for Representation element will not be
  set if the attributes are set at AdaptationSet level.

Issue #55

Change-Id: Ib4e669142874f9e8f0ca773df9f87a3fef01b729
2016-01-08 16:42:10 -08:00
KongQun Yang a040fb711c Handle side data properly in webm
Change-Id: Ic8c81154ea66374df62a4a46d66c45b3035a7829
2016-01-09 00:32:08 +00:00
Bei Li 58b95fd3d5 Support Dolby audio AC3 in ISO BMFF (Part 1)
- Box definitions for box type DAC3.
- Parser/muxer changes to support AC3 audio codecs.
- EC3 audio sample entry will come in Part 2.
- MPD signaling will come in Part 3.

Issue #64

Change-Id: I790b46ae8179b933bb8f7da9cdd38591fe8da43d
2016-01-08 15:56:48 -08:00
KongQun Yang b17d240060 Fix a memory leak due to thread object tracking
ThreadedIoFile spawns a new thread for every new file. Thread
information is stored for tracking purpose by base::tracked_objects.
The tracking object remains even if the thread itself is destroyed.
This results in memory usage increased by a couple of bytes for every
new segment created in live mode (new segments spawns new threads).

Use WorkerPool instead to avoid spawning new threads.

Fixes Issue #61.

Change-Id: Id93283903c3ba8ebf172a0d58e19b082a72c6cf0
2016-01-08 23:41:41 +00:00
Jacob Trimble db9460ca4e Added packager version to WebM muxer.
Also fixed WebM unit tests to match.

Change-Id: Ia0825b13f074667bc2fb13c60f3a1a9c4e4bf67a
2016-01-08 11:20:31 -08:00
KongQun Yang e0040a4910 Add version information in generated outputs
For mp4 outputs, a metadata box with version information is added to
moov box.

For mpd outputs, a text comment with version information is added in
the beginning of mpd file.

Issue #60

Change-Id: I783ba370781c0a8f77c910ff1172bad2e7edff75
2016-01-08 19:08:56 +00:00
KongQun Yang 43d5a9ef6e Add new option build_type to packager_test.py
Change-Id: Ifb1109b48e2d93e81561c3e71c5512052851bb56
2016-01-07 20:36:19 +00:00
Rintaro Kuroiwa 62a70e71ee Add MPD schema validation for subsegmentAlignment
- No tests checked the MPD output with AdaptationSet@subsegmentAlignment
  set.

Change-Id: Id45d2bf6865c4550b57c6be1d423b527d3f45cfb
2016-01-07 20:28:20 +00:00
KongQun Yang 458d376343 Block align frames in superframe for vp9 cenc
Change-Id: I065151c05d55c993ffbd0c30f2aa521e8476833c
2016-01-07 20:08:03 +00:00
KongQun Yang 7c61aa15ed Remove webm_tools which is not needed
- Another minor change in segmenter_test_base: scoped_refptr can
  be passed by value, remove ".Pass".

Change-Id: I31f0cf74c5ad0aba8890d3b1845155d63d7326a0
2016-01-07 10:33:10 -08:00
Jacob Trimble 7b52f0a3ed Added unit tests for WebM Segmenters.
* Changed Segmenters to accept StreamInfo rather than MediaStream
  to help in testing.
* Changed MemoryFile behavior to mirror local file:
  * Read non-existent file is an error.
  * Write deletes any existing file.
* Fixed a bug in SingleSegmentSegmenter.

Change-Id: I339e35597ca4661b7a26c6fdbbfa2f9f511c7da0
2016-01-07 17:07:02 +00:00
KongQun Yang b3e85ff810 Enable '-Wimplicit-fallthrough' and fix issues
Change-Id: I9e1e09f3924dcd8f59af2fbc952456b81e2d7c4e
2016-01-06 17:12:49 -08:00
Bei Li bb073cef51 Add support for DTS in ESDS. Needed to handle DTS audio generated by FFMPEG.
Issue #52

Change-Id: Ibd97054bce0b7ccb344f4bb34370399b8e051f30
2016-01-06 22:30:52 +00:00
KongQun Yang a4659c40dd Fix AssetId overflow for classic WVM decryption
Javascript/JSON does not support int64_t or unsigned numbers. asset_id,
if greater than 0x80000000 will be converted to negative number. Use
double to represent asset_id instead as 32-bit integer can be lossless
represented using double.

Bug: 26309000

Change-Id: I3e800c396a1231375776295154fb0d96156e980b
2016-01-06 20:18:42 +00:00
Rintaro Kuroiwa a9b039c3f9 Fix MPD validation problems
- Replaced urn:mpeg:DASH:schema:MPD:2011 with
  urn:mpeg:dash:schema:mpd:2011. The former is from the old spec.
- Replaced the MPD schema with the latest one.
- Fixed subSegmentAlignment typo with subsegmentAlignment.

Issue #55

Change-Id: Icffa9872ffc462627b81900a48f59e7b76687c11
2016-01-05 13:50:32 -08:00
KongQun Yang 098e087459 Implement SampleEncryption box parsing and generation
Issue #41

Change-Id: Ic57436b6e6c8b58fc264037fd81c98627c525932
2016-01-05 01:56:06 +00:00
Jacob Trimble c0df6b3239 Cleaned up Status class.
Change-Id: I1ff470410b38d32ad828618f756487585660a272
2016-01-05 01:02:34 +00:00
Jacob Trimble 8a0458c17b Added MemoryFile, a File type stored in memory.
This can be used for unit tests to reduce the need for temporary
files.  A MemoryFile acts the same as any other File object, but
stores the contents in memory.  The memory pointer can be accessed
directly.  A file with the 'memory://' prefix will be a memory file.

Change-Id: I10ae3ed5391c8fd838e6dfb33106dec959bdc224
2016-01-04 22:50:07 +00:00
KongQun Yang 0c870b7c02 Define SampleEncryption 'senc' box
Issue #41

Change-Id: Ib28c079209f3a58ecd8d7b86d720a1c8c774b669
2015-12-31 01:26:18 +00:00
KongQun Yang 814d2414e3 Implement vp8 bit stream parser.
Change-Id: I8b7997ceb8e7484399442f7c1072f4d5aa33ff4e
2015-12-30 21:40:10 +00:00
KongQun Yang 94401d750a VP9 codec string from bitstream and subsample encryption support
- Parse vp9 bitstream to get vpx codec configuration
- Add subsample encryption for vp9
- Also fixed a bug in VP9 parser if segmentation update_map is enabled

Change-Id: I69dc97088aa38c94c6d37fdbcf3d9cfc942a3df6
2015-12-30 21:39:34 +00:00
KongQun Yang 9c95309c12 Pull profiling dependency from Chromium
Profiling can be enabled by setting profiling=1 in gyp, e.g.
  GYP_DEFINES="profiling=1" gclient runhooks

To turn on heap profiling, use the HEAPPROFILE environment variable
to specify a filename for the heap profile dump, e.g.
  HEAPPROFILE=/tmp/heapprofile out/Release/packager ...

To turn on cpu profiling, use the CPUPROFILE environment variable
to specify a filename for the cpu profile dump, e.g.
  CPUPROFILE=/tmp/cpuprofile out/Release/packager ...

Note that profiling may not work for debug builds, so use release
build if possible.

See docs/linux_profiling.md for details.

This change will help identify and resolve problem behind Issue #61.

Change-Id: I6f85a04ed82dd0cb3588e6b38e8ceb68dac6c436
2015-12-29 17:14:02 -08:00
Rintaro Kuroiwa 09d1c2ce9f Box definitions for WebVTT
Change-Id: I94d38bef918eed18df7cd312126e2d589aadf9c7
2015-12-22 14:05:50 -08:00
KongQun Yang c8e3345959 Some code clean ups in box definitions
- Add Box::WriteHeader and Box::HeaderSize methods. The concrete
  box implementations should not need care about whether it is a
  Box or FullBox normally.
- Rename Box::ReadWrite to Box::ReadWriteInternal to make it clear
  that it is an internal methods. Make it private so it cannot be
  incorrectly called in box implementions.
- Make an internal implementation of ComputeSizeInternal to compute
  box size. |atom_size| is updated in ComputeSize, which wraps
  ComputeSizeInternal.

Change-Id: I3fbcf8c527581b676d9d13f1ac1dd798da7c4d5f
2015-12-21 18:40:05 +00:00
Jacob Trimble 05f5682728 Added stream descriptor for output format.
Added two stream descriptors that are both used to tell the output
format.  If it is not given, the packager will use the output file
extension to determine the output format.

Change-Id: Ib8d5dcf52956c2d451e77ea6a90d9502d4a77064
2015-12-16 15:10:35 -08:00
KongQun Yang af7d6a7921 Estimate duration of last sample in cluster from next cluster
Change-Id: I7dbc4045d366bbfb0c12f9652ffe97b8fcf447cf
2015-12-15 22:34:01 +00:00
Jacob Trimble 07378e806c Added WebM muxer.
* Added WebM muxer that supports both single and multi-segment modes.
* WebM muxer supports MPD generation.
* Enabled WebM muxer and demuxer in packager.

b/22463220
b/25089245
b/25089714

Change-Id: I9f6e8fc51e08fbb1d685229c5cb54ea60f7bed8f
2015-12-15 20:34:44 +00:00
Rintaro Kuroiwa 1cdce29343 WebVttMediaParser and TextStreamInfo
- Add WebVttMediaParser which parses and creates MediaSamples from text
  WebVtt inputt

Change-Id: Ia7bb7474df7f15e454e887b8c291fdfdc3195e46
2015-12-14 13:58:21 -08:00
KongQun Yang e1bf322b83 s/::STLElementDeleter/STLElementDeleter/ to improve portability
Change-Id: I5234b415a6ae88ba26da571f56cc8b16748203ae
2015-12-11 15:14:26 -08:00
Bei Li aa805b9632 Support DTS audio in MP4 - Part 2
Issue #52

Change-Id: Ib71077c5063aced23f9d060b35facbf8e6eb0a3a
2015-12-10 20:49:13 +00:00
KongQun Yang e7a4dde677 Replace "pixel aspect ratio" with "pixel_aspect_ratio" in stream info msg
Change-Id: Ia70b084df9b1b4bdf94a308bcc746f3ad2b5ad8e
2015-12-09 23:55:01 +00:00
Bei Li 2806dffb47 Support DTS audio in MP4 - Part 1
Issue #52

Change-Id: I5061e2a674d5c1b039a49729e53d4d15a4120a5c
2015-12-08 23:06:52 +00:00
Thomas Inskip 329afd0ade Added Seek & Tell functionality to ThreadedIoFile.
Change-Id: I3c714014e961d08110ce194ad865322e0b487fcb
2015-12-08 22:55:08 +00:00
Bei Li c647c25d5f Revert "Support DTS audio in MP4 - Part 1"
This reverts commit 9b0d4abece.

Change-Id: I3897fffde2ab162638876df1eafa7a4058573e49
2015-12-07 19:35:11 +00:00
Bei Li 9b0d4abece Support DTS audio in MP4 - Part 1
Change-Id: I4464f869552f5cd0cb77d1b3e97c8efc4b00c552
2015-12-04 16:03:16 -08:00
KongQun Yang 6ca2ea9a24 Update end to end packager_test with multiple inputs
- Also set --clear_lead to 1 if encryption is enabled to make sure some
  of the segments are encrypted.
- Also fixed glint errors.

Change-Id: Ie92ad3ea1a3fd95d059e424a114b863ac393e6bf
2015-11-20 11:31:25 -08:00
Rintaro Kuroiwa ef7b136116 Better TTML input check
- Check whether if it can be parsed by libxml2 and check if the root
  element is a 'tt' element.
- Added unit tests for determining TTML and WebVTT.

Change-Id: I4fea1881f0ab70970700601d8d7cea32cc04752c
2015-11-18 19:02:04 +00:00
KongQun Yang c393b71b6f Implement vp9 bitstream parser
Bug: 25586821

Change-Id: I42d77a85c7214262842f49fec83689b6b58cad3d
2015-11-18 00:21:48 +00:00
Rintaro Kuroiwa d3c52cffd9 Use alias template for ScopedXmlPtr
- Also renamed to scoped_xml_ptr to match other smart pointers.

Change-Id: Idb998aa3252d4f3a50068a09e26a05f124e94a2f
2015-11-17 22:07:10 +00:00
KongQun Yang 8072fd1eaf Disable HAVE_STROPTS_H which is not available in CentOS
Change-Id: I98c2a4f2ecbbb39dff7a76522243e4f227911c91
2015-11-17 19:55:32 +00:00
Thomas Inskip 7b74ca12de Fixed a miniscule odds race condition found by tsan (Thread sanitizer)
in ThreadedIofile.
Added loop to handle partial writes in ThreadedIoFile.

Change-Id: Ib62855ab849ffbfd00afc5b226dd81d4cd38ff51
2015-11-17 18:42:10 +00:00
Rintaro Kuroiwa 981041a3a8 Remove locks from MpdBuilder and lock at MpdNotifier level
- Since most use cases use notifier implementations that already uses
  locks, removing locks from MpdBuilder.
- Added locks to MpdNotifier::Flush() implementations. Multiple threads
  were able to write to the same file.

Issue #45, #49

Change-Id: I6e39824485672f40e6c947da97f1743fac174167
2015-11-16 11:29:06 -08:00
Rintaro Kuroiwa e251863130 Add ignore file for TSan build
- Added ignores.txt file. When doing a tsan build,
  tsan_blacklist gyp flag should point to the file.

Change-Id: Ia69f89f1906f0b386e081a7df5a7c867e411f8fd
2015-11-13 18:57:54 +00:00
KongQun Yang d60cc9416f Fix a possible crash if a file fails to be closed
Change-Id: I6bc806a68b81ea5bde09bada1175f257c296afcd
2015-11-13 10:07:13 -08:00
Rintaro Kuroiwa 29e14a3d6b VOD text support
Change-Id: Ic69169c31d0a54de4895d49ff8b4a758039733a7
2015-11-11 12:26:25 -08:00
KongQun Yang 4c10755d40 Support HEVC in mp4 parser and muxer
Issue #46

Change-Id: I36bf8418a335181ad71509e6e0cccdce210467f0
2015-11-04 17:18:41 -08:00
Rintaro Kuroiwa 8c53995335 Text support for MpdBuilder
Change-Id: I75e1da391356e9edfcb520941029341941c31332
2015-11-03 17:55:50 -08:00
KongQun Yang c577e6132f Implement initial support for vpx in iso-bmff
Also update the code to generate CompressorName in VideoSampleEntry
for both AVC1 and various VP codecs.

Change-Id: I2355f8008a72806e852aa1ec6c80e9141b97d963
2015-11-04 00:34:00 +00:00
Rintaro Kuroiwa 749571fc07 Do MpdBuilder test TODOs
- Added factory methods for creating Representation and AdaptationSet in
  tests.
- Since most tests only needed access to the constructors for
  Representation and AdaptationSet, instead of using
  FRIEND_TEST_ALL_PREFIXES() for each test case, added factory methods
  on the test class.
- Check the 'frameRate' attribute in
  CommonMpbBuilderTest.SetSampleDuration.

Change-Id: I57bd3a90b397f90c3a3a91de03eb96000f58ef58
2015-10-30 14:48:30 -07:00
KongQun Yang cf0db383a3 Enable verbose logging through --v or --vmodule
Change-Id: I36d8d3dd11abe10ce598ef92f38a2a33a15d8f8e
2015-10-28 22:15:50 +00:00
KongQun Yang 8fc7f51d81 Move AVCDecoderConfiguration parsing to media/filters
Change-Id: I7daa616e53bdef2206ce145f907b8f55cde87740
2015-10-28 00:31:43 +00:00
KongQun Yang 3842a5b43c Use StrEq for string comparison in webm_parser_unittest
Change-Id: Iaa02b7e8f318508b4de38bd258453095686c431d
2015-10-22 17:40:11 -07:00
KongQun Yang 95d2dbf68d Relax requirement on reserved bits when parsing AVCC
Issue #44

Change-Id: I84235a4d94225e2bf1777443611ad19429b958d4
2015-10-22 19:09:40 +00:00
Jacob Trimble 074bce8425 Added libwebm and webm-tools to imports
b/25089850

Change-Id: If97fc4dec70c3c94054b0f7ec5f243d9d5bd676e
2015-10-22 11:16:29 -07:00
Thomas Inskip 05f80f961c Fix for race condition when flushing ThreadedIoFile.
Change-Id: I46f26fa9fddf53ca5231c31d6442053ab5202ade
2015-10-20 16:44:15 +00:00
KongQun Yang afbab86399 Additional clean ups on comments and TODOs
Also rename WebMStreamParser to WebMMediaParser.

Change-Id: I0defca481f9d7774dd84810a5abd52902d5f47b6
2015-10-19 17:23:46 +00:00
KongQun Yang 845766f69a Make webm parser compile and unittest pass
Change-Id: Ib93fbb3eba6f0638f3aa57a9a47e73e8738792d1
2015-10-16 15:51:56 -07:00
KongQun Yang 5a4234f4da Update webm_cluster_parser to emit samples
Change-Id: I02436cfcb53b96210d6f683227cdabb994f4c01f
2015-10-16 15:46:36 -07:00
KongQun Yang 732e06fde0 Adjust webm code for packager
- Adding namespace edash_packager
- Fix includes
- (u)int(8|16|32|64) to (u)int(8|16|32|64)_t
- Remove MEDIA_EXPORT
- Remove MediaLog

Change-Id: Ifa116d966ab70ae1b3c299ae1f6f7044d7ae0cc6
2015-10-16 15:46:36 -07:00
KongQun Yang 87993c5dc7 Add WebM parser code from Chromium
Also includes its dependencies, like media/base/text_track_config.*
and media/filters/webvtt_util.h

Change-Id: I5b26245daf004da19b912b7c5b2c21ce4ba85688
2015-10-16 15:46:36 -07:00
KongQun Yang b6db8b9867 Get rid of svn dependencies
- Generate local copies of libevent, libxml and protobuf.

Change-Id: Ibb29f604191519267f0e6a3ee810cad0c1f80b27
2015-10-16 20:11:20 +00:00
Kongqun Yang 571d713fec Boringssl gyp and patches from Chromium
Bug: 22463853

Change-Id: Ib108016160d2c6ecdd56e369b585ab3c3ae82601
2015-10-16 20:11:04 +00:00
Kongqun Yang 9c0ae378bc Switch to boringssl. Also update curl accordingly.
Boringssl and yasm gyp and patches will follow in a separate cl.

Bug: 22463853

Change-Id: I87b91ac089866db58ac433def006fe9ac4744bd6
2015-10-16 20:10:38 +00:00
Kongqun Yang 47c79d08ce Update code to resolve problems due to base updates.
Notably deprecate of OVERRIDE, to be replaced with C++11 override.

Bug: 22463130
Bug: 22664127

Change-Id: I12c82e092e9e8eb0870da4363307c3563a3531b7
2015-10-16 20:09:56 +00:00
Thomas Inskip a9b6fcedd7 Added fallback to computed AVCDecoderConfigurationRecord if the one
stored in the WVM metadata is invalid.

Change-Id: I9e3c7ba4677f1ac7c19ec4d51f600d28bb427acb
2015-10-14 09:46:14 -07:00
Thomas Inskip cb7173d280 Added support for multiple audio & video configurations in WVM parser.
BUG=24614172

Change-Id: I197a0ebdfa6df90abc27358762cae041e4f8fd45
2015-10-13 10:42:49 -07:00
KongQun Yang 93e6c4fc92 Reduce test file sizes and remove unused test files
Change-Id: I7e3b24e92b2a16de857e0424c85232cc31444182
2015-10-12 20:16:03 +00:00
Thomas Inskip d0fb8ced4b Use default audio and video PES stream IDs (0xc0 and 0xe0 respectively)
if not specified in the WVM metadata.

BUG=24753064

Change-Id: I4c4cd0b30f0e153d46639382682da13b127dd305
2015-10-08 13:16:00 -07:00
KongQun Yang 3677042a7c Cleanup producer_consumer_queue_unittest
Uses EXPECT_NEAR for comparison with an acceptable error bound. Also
increases error bound to 50ms.

Change-Id: I8fc61b3b915a50f8fe33b9ea72e123aefee25a80
2015-10-01 10:31:27 -07:00
KongQun Yang 2080973f17 Fix an xml include in xml_compare.cc
Change-Id: I06867c8405b0b7ff55f17622e409a42d3eceacc0
2015-09-30 17:45:46 -07:00
Rintaro Kuroiwa 76e5be3756 More fix for live
- Neither live with or without key rotation did not output the right
  result.
- No key rotation: did not produce any ContentProtection element.
- With key rotation: did not have DRM specific ContentProtection.
- To get key rotation working with shaka player, <cenc:pssh> elements
  are removed.

Change-Id: I1f34d303ae1f3ea81820e3368ab66b8420498372
2015-09-29 15:12:46 -07:00
KongQun Yang 8a850af9cd Add end to end tests for edash-packager
Test files can be updated with --test_update_golden_files if needed.

Change-Id: I5aa904634450f8963f55c46ab5fe0b63f67fef46
2015-09-29 20:38:22 +00:00
KongQun Yang 8c202047fb Make edash-packager outputs predictable for testing
- Added --iv to inject iv for fixed key encryption.
- Added support for key rotation in fixed key encryption.
- Added --use_fake_clock_for_muxer to set timestamp to 0 in the output.

Also updates stream info dump message:
- Modified pixel_width: %d\n pixel_height: %d to pixel aspect ratio: %d:%d
- Removed language from video stream info.

These flags should only be used for testing.

Change-Id: Iedf8d6d6492226219f49fe44d932645f557010e4
2015-09-29 17:19:59 +00:00
KongQun Yang 5c788c2dd5 Use StrEq for string comparison in mpd notifier unittests
Change-Id: I9960908764aea29adbe8feae2a71359f9ff4ecee
2015-09-22 16:17:28 -07:00
Rintaro Kuroiwa 6a7b32641a Get MpdWriter working again
- ContentProtection elements were missing when using MpdWriter,
  therefore mpd_generator also did not generate ContentProtection.
- MpdWriter uses MpdNotifier now, instead of directly using MpdBuiler.
- Add mock_mpd_notifier.{h,cc} to be shared by several tests.

Change-Id: Id0f6213aa0c80e50294cd4193962054b0b5c2492
2015-09-22 15:00:03 -07:00