Commit Graph

434 Commits

Author SHA1 Message Date
Cosmin Stejerean 6b68c138cc
feat: port demuxer and app to cmake (#1260)
Related to issue #1047

After this it should be possible to build a working `packager` application from the `cmake` branch. Some further logging improvements may be needed to get full parity with the `main` branch in terms of ability to do verbose debug logging, but other than that everything is expected to work.
2023-09-08 17:44:17 -07:00
Cosmin Stejerean 274c885253 Merge branch 'main' into cmake 2023-09-03 10:01:08 -07:00
Cosmin Stejerean 3afe761637
feat: port media/formats/* to cmake (#1258)
Issue #1047
2023-08-31 16:59:46 -07:00
Caitlin O'Callaghan f264befe86
feat: Write colr atom to muxed mp4 (#1261)
This PR is an extension of the full AV1 codec string feature: [PR
1205](https://github.com/shaka-project/shaka-packager/pull/1205) and
relates to [Issue
1007](https://github.com/shaka-project/shaka-packager/issues/1007) and
[Issue
1202](https://github.com/shaka-project/shaka-packager/issues/1202).

As per the AV1 spec, the codec string may contain optional color values.
These color values are critical for detecting HDR video streams - see
[Issue
1007](https://github.com/shaka-project/shaka-packager/issues/1007).
Color information is extracted from the input mp4's `colr` atom and used
to generate the full AV1 codec string. This PR preserves the color
information by writing the `colr` atom to the muxed mp4.

**References**:
- [AV1 Codec ISO Media File Format
Binding](https://aomediacodec.github.io/av1-isobmff/#codecsparam)
- [AV1 Bitstream & Decoding Process
Specification - Section 6.4.2 Color config semantics (page
117)](https://aomediacodec.github.io/av1-spec/av1-spec.pdf)
- [QuickTime File Format
Specification](https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/QTFFChap3/qtff3.html#//apple_ref/doc/uid/TP40000939-CH205-125526)
2023-08-29 18:46:19 -07:00
Cosmin Stejerean 682ac3c990 Merge branch 'main' into cmake 2023-08-21 17:22:35 -07:00
Prakash Duggaraju dcf32258ff
fix: Fix handling of non-interleaved multi track FMP4 files (#1214)
Do not assume that each fragment contains all tracks. 
Use track id instead of index to pick the correct timestamp.

Fixes #1213
2023-08-21 16:34:32 -07:00
Cosmin Stejerean 793518c73d Merge branch 'main' into cmake 2023-08-04 22:23:26 -07:00
Caitlin O'Callaghan cc9a691aef
feat: Generate the entire AV1 codec string when the colr atom is present (#1205)
As per the AV1 spec, the codec string may contain optional color values.

This extracts the missing color information from the mp4 `colr` atom, if
present, and generates the full AV1 codec string. 

Closes #1007
2023-08-04 09:00:59 -07:00
Cosmin Stejerean 97cf7c0a5b Merge branch 'main' into cmake 2023-08-02 12:38:01 -07:00
Joey Parrish ba51270b0e
feat: Port app/job_manager to cmake and absl (#1249)
Issue #1047 (cmake)
Issue #346 (absl)
2023-07-21 12:41:49 +02:00
sr90 520926c27a
fix(MP4): Add compatible brand dby1 for Dolby content. (#1211)
This PR adds dby1 compatible brand to dolby content as per
https://professional.dolby.com/siteassets/content-creation/dolby-vision-for-content-creators/dolby_vision_bitstreams_within_the_iso_base_media_file_format_dec2017.pdf
2023-07-18 19:50:33 -07:00
Joey Parrish c29c03c6e3
test: Fix bad assertions in formats/mp4 and formats/mp2t tests (#1247)
PR #1242 introduced some bad assertions that went unnoticed because we
are not yet building those modules in the cmake branch. This fixes those
so that ASSERT_ macros are not used in non-void functions, and so that
assumptions about a successful parse are always after an ASSERT_ that
the data was loaded properly.

All ASSERT_ macros from #1242 were audited, and the others were all
fine.
2023-07-18 11:30:05 -07:00
Joey Parrish 868a8c5d8e
ci: Collect and report test results (#1244)
All gtest-based tests should now use add_gtest. Results from these tests
will be surfaced more prominently in GitHub Actions. Before this, it was
hard to find test failures in the log.
2023-07-17 16:11:41 -07:00
Joey Parrish 60fb057f1a
test: Avoid crashing a test executable (#1242)
LOG(FATAL) aborts the executable, which means gtest can't write any
reports it might be configured to write when tests are complete. This
will interfere with reporting.

This converts the only use of LOG(FATAL), which was in
ReadTestDataFile(), to LOG(ERROR). This also updates test cases to avoid
crashing when ReadTestDataFile() returns an empty buffer.
2023-07-17 13:16:22 -07:00
Joey Parrish 98490edc5f Merge remote-tracking branch 'upstream/main' into cmake 2023-07-05 19:51:52 -07:00
Marcus Spangenberg 494769ca86
fix: TTML generator timestamp millisecond formatting (#1179)
Fix bug where milliseconds were formatted with two digits instead of
three, resulting in incorrect timestamps in TTML cues.

Fixes #1180
2023-07-05 14:28:57 -07:00
Carlos Bentzen 56d3304045
feat: port media/formats/webm to CMake (#1147)
Issue #1047 (CMake port)

Co-authored-by: Joey Parrish <joeyparrish@users.noreply.github.com>
2022-12-16 21:40:00 -08:00
Carlos Bentzen e9bf0c6de4
feat: port media/codecs to CMake (#1143)
Issue #1047 (CMake port)
Issue #346 (absl port)
2022-12-15 16:53:36 -08:00
Joey Parrish b1095f6bc3
feat: port media/base (#1127)
This removes all chromium dependencies from media/base/ and completes
the build system in CMake.

The ClosureThread class and its classes were removed, as they were
specific to chromium base. ClosureThread has been replaced by
std::thread.

The byte-swapping utilities in network_util.cc have been removed and
replaced with absl.

generate_unique_temp_path() was split out of file_unittest.cc into
file_test_util.cc, where other test suites could make use of it.

WARN_UNUSED_RESULT was replaced with the C++ standard attribute
[[nodiscard]].

The base::Clock interface was replaced with a typedef for a function
pointer that returns the current time.

This re-enables the tests in http_key_fetcher_unittest.cc by using
httpbin.org.

Issue #1047 (CMake porting)
Issue #346 (absl porting)
2022-11-04 15:46:41 -07:00
Joey Parrish 0ee4af438d Merge branch 'main' into cmake 2022-11-03 07:58:47 -07:00
Bartek Zdanowski b221aa9caf
fix: Parse one frame mpeg-ts video (#1015)
Closes #1013

Co-authored-by: Joey Parrish <joeyparrish@users.noreply.github.com>
2022-10-27 20:22:17 -07:00
Bartek Zdanowski ab8ab12d09
fix: PTS diverge DTS when DTS close to 2pow33 and PTS more than 0 (#1050)
Fixes #1049

Co-authored-by: Joey Parrish <joeyparrish@users.noreply.github.com>
2022-10-27 14:21:03 -07:00
Joey Parrish 1131bf7eea
chore: Mass-update copyright headers (#1086)
Closes #1076
2022-08-26 08:44:59 -07:00
Vishal Shah b9d477b969
fix: webvtt single cue do not fail on EOS (#1061)
While Parsing cue body check for the block size. 
If it's the last block do not error if it doesn't have a newline.

Fixes #1018
2022-06-02 09:27:47 -07:00
Joey Parrish f577e2a0cf
chore: Update URLs after moving projects (#1042)
Since a project URL is encoded into outputs, this means also updating
the golden output files.

Closes #1043
2022-03-07 11:56:34 -08:00
Vishal Shah e1b0c7c454
Fix WEBVTT Region parse 100 precent (#1006) 2021-11-15 21:28:15 -08:00
Joey Parrish efbca399c0 fix: Add missing limits header
In many places, we used std::numeric_limits without including the
proper header.  This would build on some Linux distributions, but not
others.

This adds the missing includes, fixing the build on Fedora, among
other distros.

Change-Id: I63e9e37e5973fe23bbdf9868552db51062b1dae4
2021-10-13 12:25:34 -07:00
Caitlin O'Callaghan c87c5bcdef
Fix for gap size warning in Low Latency mode (#985)
## The issue
- With LL-DASH mode enabled, the gap size warning was hit and printed to the console every time a new segment was registered to the manifest.
- This occurred because the first chunk's size and duration were being stored for each segment, rather than the full segment size and duration. Note, only the first chunk's metrics are known at first because in low latency mode, the segment is registered to the manifest before it is finished being processed and written.
- Because of this, the gap size check was comparing the end time of the first chunk in the previous segment to the beginning time of the current segment, causing the check to fail every time.

## The Fix
- Update a low latency segment's duration and size once the segment file has been fully written.
- The full segment size and duration will be used to update the bandwidth estimator and the segment info list. 
- Updating the segment info list to hold the full duration is necessary for satisfying [the gap size check found in Represenation.cc](https://github.com/google/shaka-packager/blob/master/packager/mpd/base/representation.cc#L391).
- NOTE: bandwidth estimation is currently only used in HLS
2021-09-03 09:57:43 -07:00
Caitlin O'Callaghan cd018a71c3
Low latency DASH support (#979)
# LL-DASH Support
These changes add support for LL-DASH streaming. 

**NOTE:** LL-HLS support is still in progress, but it's coming. :) 

## Testing
`./chunking_unittest --gtest_filter="ChunkingHandlerTest.LowLatencyDash"`

`./media_event_unittest --gtest_filter="MpdNotifyMuxerListenerTest.LowLatencyDash"`

`./mpd_unittest --gtest_filter="PeriodTest.LowLatencyDashMpdGetXml"`
`./mpd_unittest --gtest_filter="SimpleMpdNotifierTest.NotifyAvailabilityTimeOffset"`
`./mpd_unittest --gtest_filter="SimpleMpdNotifierTest.NotifySegmentDuration"`
`./mpd_unittest --gtest_filter="LowLatencySegmentTest.LowLatencySegmentTemplate"`

Note, packager_test must be run from the main project directory
`./out/Release/packager_test --gtest_filter="PackagerTest.LowLatencyDashEnabledAndUtcTimingNotSet"`
`./out/Release/packager_test --gtest_filter="PackagerTest.LowLatencyDashEnabledAndUtcTimingNotSet"`
2021-08-25 08:38:05 -07:00
Joey Parrish cfbe5c08c2 cleanup: Convert all time parameters to signed
This converts all time parameters to signed, finishing a cleanup that
was started in 2018 in b4256bf0.  This changes the type of:
  - timestamps
    - PTS specifically
  - timestamp offsets
  - timescales
  - durations

This excludes:
  - MP4 box definitions
  - DTS specifically

This is meant to address signed/unsigned conversion issues on arm64
that caused some test cases to fail.

Change-Id: Ic752a20cbc6e31fea6bc0894d1771833171e7cbe
2021-08-05 18:24:15 +00:00
nvincen f018c9a9bf
Added MPEG-H support (mha1, mhm1)
Implemented according to `Audio Amendment to Guidelines for Implementation:
DASH-IF Interoperability Points, Version 4.3`
(https://dashif.org/docs/Audio%20Amendment%20to%20DASH%20IOP%204.3.pdf).

Closes #930.
2021-06-29 23:10:53 -07:00
Mattias Wadman 62f37eb3b7
Ignore matroska projection metadata
Warn instead of fail parsing.

Closes #932.
2021-05-07 10:13:02 -07:00
KongQun Yang 2e521c8413 Remove another use of regex library
It is not working correctly in gcc 4.8 or earlier, which is still
popular (bundled by default in CentOS 7).

Issue #865, #929.

Change-Id: I136446a70831bd0237cd29646dd349fe7558176b
2021-05-05 18:01:27 +00:00
Vishal Shah d9124d6aaa
[WEBVTT] Fix missing text alignment tags from output
Legacy players, e.g. older versions of ExoPlayer, do not handle default webvtt text alignment correctly. Need to specify `align:center` explicitly cues without text alignment for backwards compatibility.

Fixes #925.
2021-05-04 22:57:43 -07:00
KongQun Yang 4528bdb330 Remove the use of regex library
It is not working correctly in gcc 4.8 or earlier, which is still
popular (e.g. bundled by default in CentOS 7).

Fixes #865, #929.

Change-Id: I55a42428dbd2a12fc2c3b1e6a49fdd662a295dca
2021-05-04 02:09:08 +00:00
Daniel Cantarín f6c02e629d
Generate object type properly for MPEG-1 audio
Fix #905.
2021-04-04 22:47:31 -07:00
Jacob Trimble c1f64e5350 Fix transparency case in DVB-SUB.
This fixes some math errors in the color conversions and handles the
case of Y=0.

Fixes #903

Change-Id: I796246e4d62a3161b44916f97e9e98f9203ad338
2021-03-29 16:34:39 +00:00
Daniel Cantarín dd935f6dc3
TTML: change "imagetype" attribute to camel case
Fixes #908
2021-03-09 10:10:32 -08:00
Sergio Garcia Murillo f9908362f8
Prevent seg fault if webm fragment is not initialized or last frame is EOS
Fixes #900
2021-03-07 15:01:08 -08:00
Sergio Garcia Murillo b8ce44aba0
Prevent seg fault if mp4 fragment is not initialized
Related to #900.
2021-03-02 23:53:58 -08:00
Jacob Trimble 00af192626 Cleanup HttpFile and related PR.
This implements many of the comments made on the PR and cleans up those
files.

Closes #149

Change-Id: Ice73fe3c04a6f595da6986a4c070e50cb20f9435
2021-03-02 17:43:47 +00:00
Jacob Trimble a0f3f2cd3a Add cc_index to stream descriptor.
This also allows setting the language of different text streams from
the same input.  Multiple streams can use the same input stream
using different cc_index values and can each use a different language.

This also will try to pull the language from the input if not
specified.

Change-Id: I7078710b509b7d77dad8cb4299a82f954af7e9e7
2021-02-17 18:33:53 +00:00
Jacob Trimble 78be14c092 Add DVB-sub parser
Note that this only supports a single page within the DVB-sub stream.
Multiple pages will be merged together.  A follow-up will allow
selecting a specific page.

This only supports outputting using TTML or MP4+TTML; you cannot have
DVB-sub output nor can you output it in WebVTT.  Since DVB-sub
uses images, it is hard to impossible to do this with WebVTT.

This also only supports interlaced images, not progressive images
nor text.

Closes #832

Change-Id: Id6dbb6393c7b9a05722e61c6bd255bef5e69a7d8
2021-02-17 18:32:03 +00:00
Jacob Trimble 95089593fc Don't re-open WebVTT file to determine size.
Change-Id: Id92226adce813b7d0c4c741e47e36dbf8f208797
2021-02-08 20:31:13 +00:00
JPeMu 36ef7ec945
[MPEG-TS] Fix PCR reserved bits not being set correctly
Fixes #893.
2021-02-03 12:09:07 -08:00
Ole Andre Birkedal aa17521268
HTTP PUT output support (#737)
Issue #149

Co-authored-by: Andreas Motl <andreas.motl@elmyra.de>
Co-authored-by: Rintaro Kuroiwa <rkuroiwa@google.com>
Co-authored-by: Ole Andre Birkedal <o.birkedal@sportradar.com>
2021-02-02 10:51:50 -08:00
Jacob Trimble 5bcda6b88b Use TsStreamType for MP2T parser.
This also changes some of the logs to error so the user can see why
the parsing failed.

Change-Id: Ib8b7a5076462bccc718e17ef9e0a57d172d1f7b4
2021-02-01 20:13:13 +00:00
Jacob Trimble 2eb32ee177 Propagate Flush errors in MP2T parser.
Issue #832

Change-Id: I59f31ff491437b81ffc22ab5760ad0c059e9933e
2021-01-20 18:27:31 +00:00
Jacob Trimble 89d407f9ae Add subtitle composition to DVB-sub parser.
Issue #832

Change-Id: Iababe884619e1e48f1abe0806e8b863c95a3c1ef
2021-01-20 18:26:28 +00:00
Jacob Trimble 32c5393fba Add helpers for DVB-sub colors.
Issue #832

Change-Id: I6350306c7d9a6450d82994bbd9a9a239986bc3fa
2021-01-20 18:25:43 +00:00