Commit Graph

97 Commits

Author SHA1 Message Date
Caitlin O'Callaghan cd018a71c3
Low latency DASH support (#979)
# LL-DASH Support
These changes add support for LL-DASH streaming. 

**NOTE:** LL-HLS support is still in progress, but it's coming. :) 

## Testing
`./chunking_unittest --gtest_filter="ChunkingHandlerTest.LowLatencyDash"`

`./media_event_unittest --gtest_filter="MpdNotifyMuxerListenerTest.LowLatencyDash"`

`./mpd_unittest --gtest_filter="PeriodTest.LowLatencyDashMpdGetXml"`
`./mpd_unittest --gtest_filter="SimpleMpdNotifierTest.NotifyAvailabilityTimeOffset"`
`./mpd_unittest --gtest_filter="SimpleMpdNotifierTest.NotifySegmentDuration"`
`./mpd_unittest --gtest_filter="LowLatencySegmentTest.LowLatencySegmentTemplate"`

Note, packager_test must be run from the main project directory
`./out/Release/packager_test --gtest_filter="PackagerTest.LowLatencyDashEnabledAndUtcTimingNotSet"`
`./out/Release/packager_test --gtest_filter="PackagerTest.LowLatencyDashEnabledAndUtcTimingNotSet"`
2021-08-25 08:38:05 -07:00
Joey Parrish ac125564b9 build: Fix pylint 2.10 issues, use python3 where possible
The newest pylint release complained about several issues that the
older release did not.  This resolves those issues:

 - removes unneeded "u" prefix from strings
 - adds "encoding" parameter for all open() calls
 - because "encoding" is a python3-only parameter, use python3 in all
   the scripts that we control

Unfortunately, python2 is required for any scripts that import modules
from the ancient Chromium build system we're using (referenced by
DEPS), as well as kokoro scripts.

Change-Id: I2e9f97af508efe58b5a71de21740e59b1528affd
2021-08-24 12:27:03 -07:00
Joey Parrish 68b50f656d build: Stop using hermetic clang, libc++, etc
This brings our default build config more in line with what is
necessary for some platforms anyway: using the system-installed
toolchain and sysroot to build everything.

We will no longer fetch source or binaries for any specific build
tools, such as libc++, clang, gold, binutils, or valgrind.

The main part of this change is the changing of default gyp settings
in gyp_packager.py.  For this, a bug in gyp_packager.py had to be
fixed, in which similar GYP_DEFINE key names (such as clang and
host_clang) would conflict, causing some defaults not to be installed
properly.

In order to enable clang=0 by default, some changes had to be made in
common.gypi:
  - compiler macros added to fix a compatibility issue between
    Chromium's base/mac/ folder and the actual OSX SDK
	- replaced clang_warning_flags variables with standard cflags
	  settings, plus xcode_settings for OSX
  - turned off warnings-as-errors for non-shaka code, rather than
		allow-listing specific warning types, since we can't actually fix
    those warnings on any platform
  - disabled two specific warnings in shaka code, both of which are
    caused by headers from our non-shaka dependencies

Also, one warning (missing "override" keyword) has been fixed in
vod_media_info_dump_muxer_listener.h.

Although these changes were done to make building simpler on a wider
array of platforms (arm64, for example), it seems to make the build a
bit faster, too.  For me, at least, on my main Linux workstation:
  - "gclient sync" now runs 20-30% faster
  - "ninja -C out/Release" now runs 5-13% faster

The following environment variables are no longer required:
  - DEPOT_TOOLS_WIN_TOOLCHAIN
  - MACOSX_DEPLOYMENT_TARGET
Documentation, Dockerfiles, and GitHub Actions workflows have been
updated to reflect this.

The following GYP_DEFINES are no longer required for anyone:
  - clang=0
  - host_clang=0
  - clang_xcode=1
  - use_allocator=none
  - use_experimental_allocator_shim=0
Documentation, Dockerfiles, and GitHub Actions workflows have been
updated to reflect this.

The following repos are no longer dependencies in gclient:
  - binutils
  - clang
  - gold
  - libc++
  - libc++abi
  - valgrind

The following gclient hooks have been removed:
  - clang
  - mac_toolchain
  - sysroot

Change-Id: Ie94ccbeec722ab73c291cb7df897d20761a09a70
2021-07-29 13:54:44 -07:00
Joey Parrish 0f8749a211 CI overhaul based on GitHub Actions
This replaces Travis (for Linux & Mac) and Appveyor (for Windows) with
GitHub Actions.  In addition to using GitHub Actions to test PRs, this
also expands the automation of releases so that the only manual steps
are:

 1. Create a new CHANGELOG.md entry
 2. Create a release tag

Workflows have been create for building and testing PRs and releases,
for publishing releases to GitHub, NPM, and Docker Hub, and for
updating documentation on GitHub Pages.

When a new PR is created, GitHub Actions will:
 - Build and test on all combinations of OS, release type, and library
   type

Appveyor's workflow took ~2 hours, whereas the new GitHub Actions
workflow takes ~30 minutes.

When a new release tag is created, GitHub Actions will:
 - Create a draft release on GitHub
 - Extract release notes from CHANGELOG.md & attach them to the
   draft release
 - Build and test on all combinations of OS, release type, and library
   type, aborting if any build or test fails
 - Attach release artifacts to the draft release, aborting if any
   one artifact can't be prepared
 - Fully publish the draft release on GitHub
 - Publish the same release to NPM (triggered by GitHub release)
 - Publish the same release to Docker Hub (triggered by GitHub release)
 - Update the docs on GitHub pages

Closes #336 (GitHub Actions workflow to replace Travis and Appveyor)

b/190743862 (internal; tracking replacement of Travis)

Change-Id: Ic53eef60a8587c5d1487769a0cefaa16eb9b46e7
2021-06-16 11:52:02 -07:00
Joey Parrish 2526c61e9e
Merge pull request #955 from joeyparrish/master
At this point static_library builds are working in MSVS 2019. shared_library builds are still not working.

Closes #867 (MSVS 2019)
Issue #318 (progress toward shared_library support on Windows)
Issue #336 (progress toward replacing Travis & Appveyor with GitHub Actions, which uses MSVS 2019)
b/190743862 (internal; tracking replacement of Travis)
2021-06-14 15:25:39 -07:00
Joey Parrish 60464334c1 Update build instructions for Windows
The build instructions have been updated with necessary environment
variables for newer MSVS versions.

Issue #867 (MSVS 2019)
Issue #336 (progress toward replacing Travis & Appveyor with GitHub
  Actions, which uses MSVS 2019)
b/190743862 (internal; tracking replacement of Travis)

Change-Id: Ic80ba22a750946a508803c52af6b7869964d595b
2021-06-14 12:17:31 -07:00
sr90 d84260ca78
Properly format the documentation for dash_only and hls_only 2021-06-10 14:21:23 -07:00
KongQun Yang 6c8ad30217 Update Sphinx config for markdown handling
The config format has changed in v3.0 in the handling of markdown.

See https://www.sphinx-doc.org/en/master/usage/markdown.html.

Change-Id: I3f2dad962824e1d5f410f8420795c68715b03635
2021-04-09 21:18:06 -07:00
Suraj kumar (sk) aa046a8d9e
Fix the build instructions on Windows
For windows running `GYP_DEFINES='target_arch=X64'` shows an error GYP_DEFINES is
not recognized as an internal or external command.

The actual command is to set the value of GYP_DEFINES. so it should be 
`SET GYP_DEFINES='target_arch=X64'`.
2021-03-22 11:32:26 -07:00
Jacob Trimble 00af192626 Cleanup HttpFile and related PR.
This implements many of the comments made on the PR and cleans up those
files.

Closes #149

Change-Id: Ice73fe3c04a6f595da6986a4c070e50cb20f9435
2021-03-02 17:43:47 +00:00
Jacob Trimble c4ea028d11 Update documentation for text support.
Change-Id: I43acad9ae62f89b06491a3b56fb846da1edd4002
2021-02-17 18:39:07 +00:00
Jacob Trimble a0f3f2cd3a Add cc_index to stream descriptor.
This also allows setting the language of different text streams from
the same input.  Multiple streams can use the same input stream
using different cc_index values and can each use a different language.

This also will try to pull the language from the input if not
specified.

Change-Id: I7078710b509b7d77dad8cb4299a82f954af7e9e7
2021-02-17 18:33:53 +00:00
Alen Vrečko 449945dfba
Added IDE support in build instructions
Fixes #772.
2021-02-03 11:02:13 -08:00
Ole Andre Birkedal aa17521268
HTTP PUT output support (#737)
Issue #149

Co-authored-by: Andreas Motl <andreas.motl@elmyra.de>
Co-authored-by: Rintaro Kuroiwa <rkuroiwa@google.com>
Co-authored-by: Ole Andre Birkedal <o.birkedal@sportradar.com>
2021-02-02 10:51:50 -08:00
KongQun Yang 3b04881a04 Update docker file for Arch Linux and OpenSUSE
Change-Id: Ia573a6ff84ff9a9789061813ef47137b086bd597
2020-12-14 15:15:05 -08:00
KongQun Yang 7f9fa8de9a Update build instruction notes for MAC 10.15 or later
Change-Id: I29ae342b3d82d13f92c3c2695e605eb2e11d9d09
2020-12-11 19:08:56 +00:00
KongQun Yang 10daa39901 [MP4] Allow not to generate 'sidx' box for single-segment too
I.e. the flag --generate_sidx_in_media_segments,
--nogenerate_sidx_in_media_segments work for both single-segment
and multi-segment mode with this change.

Related to #862.

Change-Id: Icd27fd00e8e036ba0c4709b48650372429cc0351
2020-12-11 19:08:37 +00:00
Jacob Trimble 97cad858bf Add docs for --playready_extra_header_data.
Issue #756

Change-Id: Iedcf801c4c3e1e05e63857f3fe546ed2080c7005
2020-09-01 17:55:45 +00:00
Joe Foraci 6b036b9bb1
Support different IVs for each track (#793)
This is supported by allowing IV to be specified in the "key_info_string".

Fixes #543.
2020-06-25 20:37:50 -07:00
sr90 7aefbeb27f
Adding check for creating captions with dash_only and hls_only. (#761)
Fix the problems in the previous CL for #651.
2020-04-29 17:39:12 -07:00
sr90 55349aa4c8
Adding --allow_codec_switching (#726)
To allow adaptive switching between different codecs.

Closes #542.
2020-03-17 19:33:44 -07:00
sr90 a1dd82d478
Support dash_only and hls_only parameters (#721)
This allows conditional stream descriptors which apply to DASH or HLS only.

Closes #651.
2020-03-06 10:19:47 -08:00
KongQun Yang 4028bf727b Allow specifying protection pattern for pattern encryption
Added --crypt_byte_block, --skip_byte_block to allow users to specify
protection pattern other than 1:9 for pattern based encryption scheme,
e.g. cbcs and cens.

Closes #710.
b/147307451

Change-Id: I9f64a7639170c737f138572689b28d17286325c7
2020-02-28 00:50:58 +00:00
Widevine Buildbot 888be2d1ee Merge remote-tracking branch 'github/master' into HEAD 2020-02-10 07:11:40 +00:00
Daniel Cantarín 7aab7a8b50
Add --hls_media_sequence_number to support custom value in HLS
HLS uses the EXT-X-MEDIA-SEQUENCE tag at the start of a live playlist in
order to specify the first segment sequence number. This is because any
live playlist have a limited number of segments, and they also keep
updating with new segments while removing old ones. When a player refreshes
the playlist, this information is important for keeping track of segments
positions.

When the packager starts, it naturally starts this count from zero. However,
there are many situations where the packager may be restarted, without this
meaning starting this value from zero (but continuing a previous sequence).
The most common situations are problems in the encoder feeding the packager.

With those cases in mind, this parameter allows to set the initial
EXT-X-MEDIA-SEQUENCE value. This way, it's possible to continue the sequence
number from previous packager run.

Closes #691.
2020-01-31 10:25:19 -08:00
KongQun Yang 4d01283456 Clean up some documentations
Change-Id: Id643740950aab703db9a1b42757a44c75fdfde88
2020-01-23 12:18:44 -08:00
KongQun Yang 9fa56b675d Switch to python3 for sphinx document generator
Python2.7 has just been officially deprecated on Jan 1 2020.

Change-Id: I5b7b5d3955f72c9281b2f602047e07c9d2b9c2a5
2020-01-22 01:27:25 +00:00
Tomohiro IKEDA 0b53b40428 Change --generate_static_mpd to --generate_static_live_mpd (#672) 2019-11-15 11:53:47 -08:00
KongQun Yang b70da0ed93 Add --quiet to suppress LOG(INFO) outputs
Closes #661.

Change-Id: Ic75479fba4f8dc26146b42b85ab95af6151302b6
2019-10-08 10:20:51 -07:00
Herlon Aguiar 443519aeb3 Update general_encryption_options with a better clear_lead description (#638) 2019-09-06 13:24:24 -07:00
KongQun Yang 796974d2a1 [DASH] Support custom Accessibility and Role elements
Add dash_accessibilities stream descriptor, which is a semi-colon
separated list of accessibility_scheme_id_uri=value. It is optional.

Add dash_roles stream descriptor, which is a semi-colon separated
list of strings. It is optional.

Closes #565.

Change-Id: Idb1c20bb410fdd016db07e11fe507c102a3dd8ea
2019-06-13 15:43:32 -07:00
KongQun Yang 3f7ecd4e29 Workaround extra AUD in the access unit
VLC seems to generate access units with extra AUDs. In #526, the below
sequence is seen:

  AUD | SPS | PPS | SPS | PPS | AUD | SEI | SEI | SEI | IDR_SLICE

Previously we exit early when seeing AUD, which results in delayed
processing of the access unit.

The behavior is changed to continue processing the following NAL units
to workaround the content issue.

Closes #526.

Change-Id: I80f571c0711c6db1337eb393fce36fae5432b6c5
2018-12-15 00:32:06 +00:00
KongQun Yang 4b97a6d8a2 Add support for --default_text_language
It allows users to override the default language for text tracks.

If not specified, --default_language applies to both audio and text
tracks.

Issue #430.

Change-Id: I86a9baba2072be27b6661fa7b65a8bc8b6adb3cc
2018-11-26 13:42:58 -08:00
KongQun Yang 273ab09f05 Support HLS characteristics
Add hls_characteristics stream descriptor, which is a colon or semi-colon
separated list of strings. It is optional.

Fixes #430.

Change-Id: Ifcf79316e68768ff065891933de565cd0ff32ec4
2018-11-26 20:02:43 +00:00
KongQun Yang 1aeabc10f1 Use -x264-params instead of -x264opts in the docs
-x264opts is deprecated: https://trac.ffmpeg.org/wiki/Encode/H.264.

Fixes #507.

Change-Id: Idb6cda03fbdfb88804e2297df4b9986ea87e4e64
2018-11-19 21:11:47 +00:00
KongQun Yang 70dfced819 Fix incorrect spelling of utc_timings in the doc
Should be utc_timings but was spelled as utc_timing.

Fixes #496.

Change-Id: I77aeb3caf54493ff8c1367f6f078de192860c39a
2018-11-08 14:08:12 -08:00
KongQun Yang 8d11e5ea64 Support Marlin DRM signaling in DASH
Only Marlin Adaptive Streaming Specification – Simple Profile is
supported.

Two additional updates:
- Remove FairPlay ContentProtection element from DASH mpd as FairPlay
  does not define a signaling in DASH.
- Updated end to end test to include all DRMs we support.

Closes #381.

Change-Id: Id12269b471ea34983b782cbd92f687332292ef59
2018-10-01 20:22:58 +00:00
KongQun Yang 9b655189cf Disable bundled_binutils and bundled_gold on Linux
Keep bundled binutil scripts but not downloading actual binaries by default.

Automatic downloading of binutils has been causing problems for some users:
see #164, #412, #440.

Using bundled binutils helps reduce linking time, but packager codebase is
relatively small, so the gain is not significant.

Local testing shows that the full build time increases from 1m42s to 1m50s, i.e.
an increase of about 8s, or ~8%.

User can still enable the usage of bundled binutils by running
'python src/packager/third_party/binutils/download.py' and set
'linux_use_bundled_binutils' and 'linux_use_bundled_gold' to 1 in GYP_DEFINES.

Change-Id: I7ecae2333861f41054f957d0a6081dae948167bd
2018-08-24 21:14:19 +00:00
KongQun Yang f49b89280c Support STYLE and REGION in WebVTT
Note that STYLE and REGION are not supported in mp4 container due to
spec limitation as 14496-30:2014 does not specify a way to signal
styles/regions inside mp4.

Closes #344.

Change-Id: I05c14df916f7b2c7ca4364ee9407e0eda4dc7a3f
2018-08-20 14:12:07 -07:00
KongQun Yang 715ed939f1 Add instructions to build on Alpine Linux
- Also fixed compilations in Alpine Linux and other flavors of Linux.
- Added container versions in docker files to always use a verified
  version.

Closes #164.

Change-Id: I949a8709e4d70c49129c9c2e8608dd78193d964c
2018-08-17 20:46:11 +00:00
KongQun Yang 59cc6ad511 Clean up in documentation
Bug: 112707139

Change-Id: Ifae98ad3dc94562918c8fdd254c5bbeb7a444586
2018-08-16 20:08:03 +00:00
KongQun Yang 451406ad3f Update DRM documentations
Fixes: #245.
Fixes: #306.

Change-Id: Ib847f5cfbf01b98f7a7e088dfcbc4d25cbbc15f2
2018-08-15 17:18:57 -07:00
KongQun Yang 9c08808c7f Add documentation for Ad Insertion
Change-Id: Ia001a0a624c49854a31fd26008b2c7b3e81012f7
2018-08-16 00:14:07 +00:00
KongQun Yang 8612798de1 Change --additional_protection_systems to --protection_systems
Issue: #245.

Change-Id: Ib7da3b874207623e3ee0b1a942ecb2509e750669
2018-08-14 22:12:55 +00:00
KongQun Yang c503db8e4d Add instructions for missing curl CA bundle on mac
Change-Id: I39f36cb08cd816a0373713b21ed1f8bad0d86124
2018-08-14 21:30:45 +00:00
KongQun Yang db076d6892 Support Fairplay in --additional_protection_systems
Issue: #245.

Change-Id: I15187c1d3463534bf5e11ff97032311bb1d0c3bf
2018-08-09 23:59:40 +00:00
KongQun Yang 0fe31423cb Multi-DRM flag refactoring
- Added flag --additional_protection_systems to specify multi-DRM
  options.
- Deprecated old multi-DRM flags: --generate_widevine_pssh
                                  --generate_playready_pssh
                                  --generate_common_pssh

Issue: #245.

Change-Id: If926bc9d481e941aa2269b6672bf2ba5411c3a50
2018-08-09 22:01:46 +00:00
KongQun Yang a47f79a014 Update pssh sample to remove provider
provider has been deprecated from Widevine PSSH.

Bug: 110944320

Change-Id: Ib8223a63a6379e46d9cdc33eff901b92ee54acca
2018-08-02 14:25:36 -07:00
KongQun Yang 51d39d96a1 Add offset to tranport streams (MPEG2-TS, HLS Packed Audio)
Configurable with --transport_stream_offset_ms.

This is needed to compensate for possible negative timestamps in
inputs, which could happen on ISO-BMFF with EditLists.

Issue #112.

Change-Id: I0fce8766c9df2911b9bb859c1e54052a8ed2abfb
2018-07-26 23:20:21 +00:00
Haoming Chen d15ee877cb Add an example command for using trick play.
Change-Id: Ibf1f455355f72f703e7732833a7a11d99874ce31
2018-07-25 23:58:07 -07:00