Commit Graph

93 Commits

Author SHA1 Message Date
KongQun Yang 7f7843d241 Clean up output format determination
Requiring output format determined from 'output' to be consistent with
output format determined from 'segment_template'.

Change-Id: I32cbd63fcd6e2a4272dd0db531c1d5b385315445
2018-05-17 17:33:47 +00:00
Aaron Vaage 8f3a45c497 Respect Output Format In Packager Test
Make sure to use the output format when given when setting up our
tests. Not doing so results in text always being set to "vtt" when
the output format is "mp4".

Change-Id: I11c5f861091598a67fc76dc19b1b16a9a773a2e0
2018-05-15 23:35:54 +00:00
Aaron Vaage d3fd4e9c05 Create Test For Ad Cues
Create an end-to-end test for ad cues. This test's final result is not
correct but illustrates the problem we have in the cue insertion and will
be fixed by a later CL.

Change-Id: Ia8b43a53848941be52cf9ade018668e6477e8df2
2018-05-15 21:55:40 +00:00
KongQun Yang bbd5e074ee Add a test outputing DASH and HLS at the same time
Change-Id: I38c0ca9ed206536a4fd84e8c9c074609250013cc
2018-05-14 23:55:30 +00:00
KongQun Yang 2d919b5a31 Fix potential partial DASH segments during live packaging
Packager uses ThreadedIO to write media segments and manifest /
playlists. There was a possibility that media segments write being
delayed and scheduled after updating manifest / playlists.

This CL fixes the race condition.

Also added a note on how segments can be synced to cloud storage to
avoid the race condition during file sync.

Also added a live WebM test.

Fixes #386.

Change-Id: Icf9c38cdec715fa3dc2836eab1511131e129fe41
2018-05-03 09:26:23 -07:00
KongQun Yang a7463f60c1 Enable segment auto removal for live packaging
The number of preserved segments outside live window can be
configured using flag --preserved_segments_outside_live_window,
which is default to 50, i.e. 5 minutes for 6s segment.

Note that the segment removal will be disabled if it is set to 0.

Only HLS live playlist and DASH dynamic MPD are affected by this flag.

- Also add end to end tests.

Fixes #223.

Change-Id: I8a566efebe2f1552c7d9509ab017bade5a4a1c98
2018-05-02 00:18:42 +00:00
KongQun Yang b6f0da246f Add support for FLAC codec
Implemented according to spec:
  https://github.com/xiph/flac/blob/master/doc/isoflac.txt

Closes #345.

Change-Id: If2e277d8eac4baff5965faa0e13c44c334f6184d
2018-05-02 00:16:51 +00:00
Aaron Vaage 902dfb905a Create Tests For Bandwidth
Created HLS and DASH tests that include overriding the
computed bandwidth value.

Issue #389

Change-Id: I6063fe780f6a68e2e7a7ad4440c9d76bacbbffe3
2018-04-30 15:44:55 -07:00
KongQun Yang 0ef078a23b Fix default_language not effective with 2-char code
Two-character ISO-639 code in --default_language was ignored due to
a bug in language code matching as the language code in stream is
always converted to 3-character code.

Fixes #371.

Change-Id: I8618938af583a417446636ff9efe1c72ce822c33
2018-04-13 20:08:12 +00:00
KongQun Yang 222737d5b5 Support UTCTiming
UTCTiming schemeIdUri and value pairs can be provided to packager using
--utc_timings flag. It should be comma separated list of
schemeIdUri=value pairs.

Note that urn:mpeg:dash:utc:direct:2014 scheme is not supported as it
requires the MPD to be dynamically generated on the fly when MPD is
served to client.

Fixes #311.

Change-Id: Ibc07af8a6d8b2b6261ba3ecd2c02f23809f96614
2018-04-04 01:37:01 +00:00
Aaron Vaage 631bbdc883 Update Encrypted Tests to Use DiffDir
This changes almost all encrypted tests to use DiffDir. The remaining
un-updated tests are tests that won't match every time.

Change-Id: I858bbaa0beea34887cc3100eb784aa2aecd5a0a5
2018-03-13 16:37:10 +00:00
Aaron Vaage fb2dfe129c Update CheckTestResults to sanitize media info
Change MediaInfo media paths in test output to only use the filename so
that they can be compared between test runs.

Change-Id: I26bb3f5ce79a83400b8a162c61cf5b54e41f9c70
2018-03-13 16:37:10 +00:00
Aaron Vaage 5fee0391fb Change all HLS tests to generate iframe playlists
iframe playlists will be given a name based on the output name of
the main stream. This is done to ensure unique filenames between
different video output streams.

Change-Id: Id0de9c5834599e9d896243e30569c55f34e7cd68
2018-03-07 17:29:47 -08:00
Aaron Vaage b2ce6322b6 Remove Test File Index in packager_test.py
Instead of using the test file's index in a list to create the output
name, use the original filename and the descriptor.

This caused some problems with file name collisions when some tests
were using the same name. That was fixed by changing the names. This
will go away once they are transitioned to use DiffDir like the other
tests.

Change-Id: I0a4c480406705ca63fcea61c86c67d4a5f739295
2018-03-06 21:38:04 +00:00
Aaron Vaage 7dd88ca5b2 Create CheckLiveTestResults
Created a method to compare test output for live tests. This wraps
CheckTestResults but first replaces the times that would diff between
manifests.

Change-Id: Iafb51a28a9bcb9f32b210c6d76bf23e2b9a3e0d6
2018-03-02 21:00:03 +00:00
Aaron Vaage 74c53239dd Updated Nested Directories Test
This require a change to the update gold method as it would not
create nested directories. Instead it deletes the old directory
and copies the whole new directory over. This made the code simplier
and far-less error prone.

Change-Id: I1da5d9bda171b0f106d1425b204139dfbcb39b42
2018-03-02 12:21:39 -08:00
Aaron Vaage 4ace2754fd Avoid Full Path until Needed in packager_test.py
To make working with file names easier in packager_test.py _GetStream
the full path is not created until needed so that multiple paths can
use the same root filename.

Change-Id: I23a9231433b22ab5cef9b88bd6359f351da56eda
2018-03-02 10:27:35 -08:00
Aaron Vaage cd16e95d79 Make All Output Filenames Similar
Changed all output file names (not paths) to follow the pattern that
any qualifier (e.g. trick play) will be join the current name using '-'
but will use '_' within itself (e.g. trick_play_1).

Change-Id: Ib0247bf1ca6d94815fedaaf73d3a400d31c20c40
2018-03-02 10:27:34 -08:00
Aaron Vaage c7929b1505 Remove "with" from test names
Remove the leading "with" from test directories.

Bug: 73830478
Change-Id: I45a0914b5a0e56ec9d3340ec81eb5048c1d653a3
2018-02-27 17:43:16 -08:00
Aaron Vaage bbdcf5ee91 Fix Wrong Remove Function in Packager Test
Change-Id: I58350c2041abfd3b9c533003d408d447b9213123
2018-02-27 22:30:30 +00:00
Aaron Vaage a097aa4d3b Update Many Tests to Use CheckTestResults
Updated as many tests as we easily could. The tests that were not updated
all use live manifests or encryption that all require some "help", either
by dealing with times differing or with verifying decryption.

Bug: 73830478
Change-Id: I6803c2d960b71b459eb57b7a5e562164bb713e2a
2018-02-27 13:32:32 -08:00
Aaron Vaage 60160fe602 Change Old Test Names to Current Standard
Change-Id: Ib74b1726f6515ec562fdb82a2af390e43c36b94f
2018-02-27 13:32:32 -08:00
Aaron Vaage 6e0088c4c0 Change "AudioVideo" test to use DiffDir
Changed the end-to-end test "AudioVideo" to use diff dir
instead of manually checking each file.

Change-Id: Ic1ce1e3664cde7b2b2db262a9fde0cbe2f5b400a
2018-02-27 17:12:50 +00:00
KongQun Yang b81b832522 Create GetStream
Created GetStream, a function used to get the stream descriptor for a
single stream. This allows for finer control over the stream compared
to the batch GetStreams.

Added "trick_play_factor" to GetStream so that the trick play factor
did not have to be added the stream descriptor value and "injected"
into the stream.

Change-Id: Ifbde64b961a673aafa5f6d21f33a68d02fb46610
2018-02-27 17:12:36 +00:00
Aaron Vaage 9547000eb6 Fix Update Golden With DiffDir
Change-Id: I30c79a2acaae2d3066982d84776aaa922e183515
2018-02-27 17:12:28 +00:00
Aaron Vaage a82ac20dbb Updated "testFirstStream" to use DiffDir
Change-Id: I8c279ba0c208f1e56d53460ae134b3ed8c5f552c
2018-02-23 10:10:34 -08:00
Aaron Vaage 7096ca8998 HLS WebVtt End-to-End Tests
Created end-to-end tests for the segmented webvtt text in HLS. The
text stream is longer than both audio and video, so there are more
text segments than video/audio.

Change-Id: Ib860ccd3f2ac0048983fdeebe3847a0b9dcbc160
2018-02-23 16:55:50 +00:00
Aaron Vaage 393f989839 Create DiffDir in packager_test.py
Created a method to diff the output dir against the golden copy
of the output dir. The goal with this is to simplify our end-to-end
test structure by removing the need to track the output files within
the test runner.

With all the golden output in one directory, this makes it easier to
setup end-to-end tests and even allows the golden files to be playable.

This should not affect the repo size too much as duplicate files will
only be stored in the git object store once.

Tests will migrate to this in separate CLs.

Change-Id: Id1be9f8f60c9e362b9b615445dd8ca7da996af4c
2018-02-20 11:09:06 -08:00
KongQun Yang 74f6cefb82 Change "live" setting to be "segmented"
Changed the "live" setting in _GetStreams to be "segmented" so that it
make sense in a more general sense.

Change-Id: I2ceaad67653e1e2237533b68fccbc9fe3fabfc9b
2018-02-17 00:24:24 +00:00
Aaron Vaage a82edb5228 Make StreamDescriptor Object
Create a StreamDescriptor class that can be used to build the stream
descriptor command line argument.

Change-Id: Ifcba4f37d69a1a36c4add51a2a10c3f0c8b2ccd7
2018-02-07 19:22:05 +00:00
KongQun Yang 2d5e2b0903 Fix DCHECK of SimpleThread on invalid command line options
- Construct SimpleThreads only after all workers have been successfully
  initialized

b/72999680

Change-Id: I4016321a4bdd27b1de746c50c5f1d7a60d7ae9f1
2018-02-06 22:41:38 +00:00
KongQun Yang 9ecee46658 Fix SAMPLE-AES EC3 encryption
The IV was incorrectly updated across samples.

Created a new cryptor SampleAesEc3Cryptor specially for SAMPLE-AES
EC3 encryption / decryption. The new cryptor uses constant-iv, and
makes sure the IV is reset to the initial value at the beginning
of each audio frame and chained across syncframes within the the
audio frame.

Also added E-AC3 end to end test.

Fixes #279

Change-Id: I0aa60c17836daeef5ba433a05e5ff0906191d9ac
2018-01-24 03:02:49 +00:00
KongQun Yang 1455f43c02 Share chunkers from the same input except for WVM
Updated test files and added end to end tests for AdCues.

Change-Id: I28f57c2f0d15f65fa04f599fae446a5c1373bf47
2018-01-22 17:43:25 -08:00
KongQun Yang 53892c7467 Add end to end test for ac3
Also updated bear-640x360.ts test files.

Change-Id: Iff53800b7b4706fb2f6d94682b9446406a94cd6f
2018-01-18 17:23:20 -08:00
KongQun Yang 2d025dae2d Include shared_library and mpd_generator in ci and releases
Change-Id: Ie9449cbca711324831cb15e83d0efaf7a43cf9ef
2017-12-12 23:11:01 +00:00
KongQun Yang 5cf2b17ade Add --hls_key_uri to allow setting fairplay/identity key uri
Change-Id: I52e0f56cd10390faf515170f407ea488a6b3c9fc
2017-11-14 15:49:27 +00:00
KongQun Yang d66d307fc2 Rename fixed_key to raw_key
Change-Id: I1ee9398387b54aa685a5bef5b8704849e62ebae3
2017-10-24 21:52:01 +00:00
KongQun Yang 1bf1ec2445 Add multi-key support in fixed/raw keys
The keys can be specified in --keys option, with the form of
  label=<label>:key_id=<32-digit hex string>:key=<32-digit hex string>,label=...

There can be multiple "label=..." block. The DRM label can be one
of "AUDIO, SD, HD, UHD1, UHD2" or a custom label.

Mark --key and --key_id as deprecated. We can achieve the same result
with --keys=label=:key_id=<key_id>:key=<key>.

Also add a new optional field in stream descriptor: drm_label, which
is used to overwrite the internally generated DRM label for the stream.
If not provided, the DRM label is generated automatically based on
audio/video information, e.g. resolution.

Code changes:
- Merged RawKeyEncryptionParams and RawKeyDecryptionParams into
  RawKeyParams.
- Make FixedKeySource accepts RawKeyParams as input.

Change-Id: Ic8c2f071cc71188e13f14bc6396fc2b3ffa5cac6
2017-10-24 21:04:49 +00:00
KongQun Yang e889578a2d Creates directories in the output path if not exist
Also added end to end tests for single segment HLS and multi-segment
HLS.

Fixes #276

Change-Id: I27e5d57cdc3ab79a8e5ae0c3a8da46fcf2c467b6
2017-10-11 20:19:51 +00:00
KongQun Yang e56d1faaf0 Make HLS with key rotation work
- Remove the TS key rotation not supported check as there is nothing
  to be done for key rotation in TS for SAMPLE-AES.
- Fix IV updated problem in new segments even if crypto period does
  not change.
- Avoid duplicate EXT-X-KEY tags if it does not change.
- Make EXT-X-DISCONTINUITY-SEQUENCE match with number of removed
  EXT-X-DISCONTINUITY.
- Added end to end test for HLS with key rotation.

Change-Id: I73cb82e9f5575fcdf63ee643228efe78e6766302
2017-06-29 21:03:36 +00:00
KongQun Yang b4aa34b803 Support live and event HLS playlists
Closes #114

Change-Id: I6440383c3dc022fba5a9926839f79b9ed8a51f55
2017-06-24 01:09:32 +00:00
KongQun Yang 96ced40ec2 Move command parsing tests into separate class
Change-Id: I6aeba632d69b1867c7d7745f0c8c9fd8f8e275d9
2017-06-20 22:33:55 +00:00
KongQun Yang 03889e6465 Move hex string out of packager.h
Also added validation failure tests in packager_test.py.

Change-Id: I6697a3138d57889110250404205536dd5cd53cd0
2017-06-20 22:33:47 +00:00
KongQun Yang 05a5a41969 Support generation of libpackager.so (shared_library)
Shared libpackager can be built by setting libpackager_type to
shared_library, e.g.

GYP_DEFINES='libpackager_type=shared_library' gclient runhooks
ninja -C out/Debug

will generate libpackager.so in out/Debug/lib directory.

Here is a few other changes to make shared_library builds and
tests pass:

- Add several test parameters to packager.h, which is needed for
  testing.
- Create a protoc.gypi from build/protoc.gypi but depending on
  protobuf_full_do_not_use instead of protobuf_lite, since we need
  protobuf_full_do_not_use for text parsing and generation of media
  info proto. Somehow shared_library build does not allow mixed use
  of protobuf_full_do_not_use and protobuf_lite.
- Remove the use of LazyInstance in version/version.cc and use static
  variable directly. This is because LazyInstance needs AtExitManager
  which may not be easy to setup when calling GetVersion.
- Allow skipping testPackageWvmInputWithoutStrippingParameterSetNalus
  with flag --shared_library, which is needed as shared_library build
  does not support --strip_parameter_set_nalus flag yet.

Fixes #227

Change-Id: Iff05a50baa28134faa7218664c96114cb9e70329
2017-06-13 20:42:32 +00:00
KongQun Yang 19be04ecce Fix git not found on some Windows setups
Change-Id: I0403212f383e60ef75b426170783933f4d73b211
2017-05-26 13:52:42 -07:00
sylt bfe302bd5c Add support for skip_encryption stream descriptor (#219)
This option makes it possible to control encryption for specific streams,
e.g. for audio and video separately.
2017-05-22 10:17:58 -07:00
Haoming Chen da8877d8a2 Rename trick play rate to trick play factor.
- Add a "tpf" abbreviation in stream descriptor for
  trick_play_factor.

Change-Id: Ia09313bb9778456b53bdb0a8cc46d2ba8caa2ac2
2017-05-17 21:51:34 +00:00
KongQun Yang d9e7e2f1d0 Generate two sample entries only if there is clear lead
This addresses playready broken with clear lead = 0.

clear lead != 0 is still broken with playready on Edge. We think it is
likely an Edge bug, which does not support multiple sample entry boxes,
thus does not support clear lead.

b/37913785

Change-Id: I7adb77a913dccf669153b03b31b4a1e1c98d1cb0
2017-05-15 14:13:52 -07:00
Haoming Chen 4ba5bec660 Integrate trick play.
- Add trick play adaptation in mpd. This addresses #178.

Change-Id: Id6cc08c788eda074c261d0274768051837b84541
2017-05-09 13:33:11 -07:00
KongQun Yang 1e2da22c88 Add support for WebM colour element
Change-Id: I359cde97171118d3b928a9dd6650d11fade8f4a9
2017-05-02 20:22:17 +00:00