Commit Graph

412 Commits

Author SHA1 Message Date
rlaphoenix 2a8307b98d Decrypt DASH downloads after merging all segments
Since DASH doesn't have the ability to change keys dynamically per-track (Representation), there's no need for the DASH downloader to decrypt segments as they are downloaded (like HLS).

This halves the amount of processes needing to be opened as well as the I/O usage. It may result in noticeably lower CPU usage. Since the IOPS is lowered, you may even see an increase in download speed if downloading to something like a meh HDD.

This also fixes decryption in some weird edge-cases where decrypting each segment individually resulted in timestamp anomalies causing shaka to fail.
2023-05-16 21:55:53 +01:00
rlaphoenix bdc1203514 Only verify download size in requests downloader if possible
Some Servers may not response with the Content-Length header, even if it's from segmented media. I.e., if it's a subtitle URL. The requests downloader required the header to be present as it downloads each URL, which is not possible.

Now it tries to get it if possible, and verifies the download size with the Content-Length value if it could be obtained.
2023-05-16 20:49:43 +01:00
rlaphoenix 2a4e9505f1 Remove unnecessary HEAD calls in requests downloader
HEAD requests were made to sum a total file size of the download operation. However, the downloader is may be used on URLs where the content is not segmented media. Therefore, the server may not support or respond with the Content-Length header which causes the requests downloader to crash before it even gets a chance to begin downloading.

Even still, this total size value isn't really necessary, and would cause possibly 100s of HEAD requests (in quick succession of each other) on segmented sources. It would also add up-front delay before it actually starts to download.
2023-05-16 20:47:26 +01:00
rlaphoenix e7dc138c0f Improve readability and documentation of DASH's to_tracks function 2023-05-15 16:19:53 +01:00
rlaphoenix e079febe79 Ensure output directory exists in requests downloader 2023-05-15 13:33:59 +01:00
rlaphoenix 95802d1e64 Fix regression with downloader mapper on aria2c and saldl
The setup I had for using asyncio.run with functools.partial didn't actually pan out. A full pass-through lambda is required.

I've also moved the mapped downloader variable to the root of the downloaders package.
2023-05-12 12:19:34 +01:00
rlaphoenix be403bbff4 Implement a Python-requests-based downloader 2023-05-12 07:02:39 +01:00
rlaphoenix cb82febb7c Add ability to choose downloader via config 2023-05-12 06:42:33 +01:00
rlaphoenix b92708ef45 Alter behaviour of --skip-dl to allow DRM licensing
Most people used --skip-dl just to license the DRM pre-v1.3.0. Which makes sense, --skip-dl is otherwise a pointless feature. I've fixed it so that --skip-dl worked like before, allowing license calls, while still supporting the new per-segment features post-v1.3.0.

Fixes #37
2023-05-11 22:17:41 +01:00
rlaphoenix 3ec317e9d6 Pass manifest to DASH downloader instead of re-requesting it
Fixes #51
2023-05-11 20:46:37 +01:00
rlaphoenix 5ca2f256d5 Fix URL used on final chance to get Track KID on DASH downloads
segments[0] is the first tuple, of two values. The URL and an optional byte range. So this accidentally passed the tuple rather than the URL within the tuple.

Fixes #54
2023-05-09 13:04:20 +01:00
rlaphoenix 1668647e4d Change ConstError for ValidationError when ignoring tenc errors
The change to pymp4 may result in it complaining when validating parameters of the tenc box data, i.e., if the version byte is not 0 or 1, e.t.c. It shouldn't do a ConstError on tenc boxes anymore as the definition of the tenc box has been much improved in pymp4 v1.4.0.
2023-05-08 17:41:55 +01:00
rlaphoenix bf82065400 Move back to the official pymp4 dependency
All the latest changes in my fork is now available on the official pymp4 repository, and a new PyPI release v1.4.0 is available with the fixes to use.
2023-05-07 18:30:42 +01:00
rlaphoenix 3ae0fb3454 Update Actions in GitHub Actions CI/CD workflows 2023-05-03 03:05:11 +01:00
rlaphoenix 1c5099440b Add FLAC to the Audio Codecs enum and relevant methods 2023-05-01 18:49:25 +01:00
rlaphoenix e3941e4640 Remove now unnecessary explicit version checks on tenc boxes
pymp4 was updated to automatically do this during parsing and building of tenc boxes. Therefore it would instead fail parsing if the version is not 0 or 1.
2023-04-23 22:39:14 +01:00
rlaphoenix 2b07399f5a Update dependency on my fork of pymp4
It may look like I'm downgrading, but I'm not.

1.5.0 was an incorrect version bump based on TrueDread's fork. v1.4.0 is the next release after, even though it's a lower version. v1.4.0 should be used instead.
2023-04-23 22:36:41 +01:00
rlaphoenix b5263491ab Update Changelog for v2.2.0 2023-04-23 18:38:57 +01:00
rlaphoenix bd40c38d23 Bump to v2.2.0 2023-04-23 18:38:47 +01:00
rlaphoenix 630832e434 Ignore failed parsing of tenc boxes
Some services accidentally (? I presume) mix up the `tenc` box's data with that of an `avc1` box or similar. This causes total failure and crashing. However, in these scenarios there's usually a 2nd box further down the stream that is not an error and will parse correctly. So just skip these errors and continue.
2023-04-23 17:38:12 +01:00
rlaphoenix 86322159b6 Fix multiplexing of downloads without a video track
E.g., --subs-only and --audio-only
2023-04-17 14:09:36 +01:00
rlaphoenix 96aa7c1e0a Fix segmented vtt merging code
This got 'broken' after moving to my fork of pymp4 because my fork has commits by TrueDread that add support for the vttc, payl, and sttg boxes, therefore they no longer contain `data` fields but rather specifically parsed fields. I also no longer need to parse the data stream of vttc boxes, as they are already parsed as `children`.
2023-04-04 20:15:18 +01:00
rlaphoenix 62965f8e21 Skip tenc boxes that are not version 0 or 1
It seems tenc boxes should only be 0 or 1, yet sometimes it can be above that. It seems some services accidentally use the `tenc` atom code over another. E.g., Netflix sometimes using `tenc` instead of `avc1`.
2023-03-28 21:27:43 +01:00
rlaphoenix eb39c8eba6 Ensure QualityList returns resolutions in descending order 2023-03-28 20:46:13 +01:00
rlaphoenix b301fb4390 Use tenc boxes within uuid boxes as another fallback for get_key_id 2023-03-28 20:25:30 +01:00
rlaphoenix 5b7fadbc55 Use -nobom with CCExtractor 2023-03-27 19:25:21 +01:00
rlaphoenix 527cd4cca1 Fix regression where only last mux would be moved to dl folder 2023-03-27 18:51:32 +01:00
rlaphoenix 8c14b73bc1 Replace abandoned pymp4 with my fork of pymp4
My fork contains various fixes by beardypig and truedread that do not have a release on PyPI yet.
2023-03-27 18:41:28 +01:00
rlaphoenix 0a128e1f70 Fix regression where no-video dls fail, improve multiplex progress 2023-03-26 23:13:11 +01:00
rlaphoenix 8f5bbeb8e3 Split the download finished log into dl/title finish logs 2023-03-26 22:42:16 +01:00
rlaphoenix 0b2e3e2255 Remove the muxed download path log
It's not very appealing, nor is it info the user realistically needs for every download. They could check `devine env info` to see where downloads go if they are unsure.
2023-03-26 22:41:23 +01:00
rlaphoenix 2a8e86b057 Delete video tracks as they are muxed
This reduces the total cumulative temp download folder size.
2023-03-26 22:40:05 +01:00
rlaphoenix 0c6d0986e4 Remove now unused and superseded with_resolution method 2023-03-26 20:21:31 +01:00
rlaphoenix 63eeeca910 Fix regression where all videos are selected if --quality isnt used 2023-03-26 20:18:15 +01:00
Hollander_1908 d894e5bbe0
Was not able to use the initialization from a DASH segment_list (#47)
* Was not able to use the initialization from a DASH segment_list

* Check if initialization in DASH has attribute range
2023-03-26 20:01:17 +01:00
rlaphoenix 33a9c307f3 Add ability to download multiple resolutions per title
Closes #26
2023-03-26 19:28:46 +01:00
rlaphoenix 71cf2b4016 Fix rare issue where DASH/HLS dl speed divides by 0 2023-03-26 14:30:12 +01:00
rlaphoenix 1c73e8d7fa Fix CCExtractor failing in edge cases by repacking first 2023-03-26 12:04:16 +01:00
rlaphoenix bf3219b4e8 Ignore empty files when decrypting with Widevine
This happens because the input file has no actual useful data, likely just a bunch of headers and nothing of use. Therefore shaka skips and returns OK, yet didnt make any file at the decrypted path.

This fixes a crash when it tries to move the non-existent decrypted file to the input file location, causing the rest of the download to halt.
2023-03-17 21:09:09 +00:00
rlaphoenix f4a9d6c0b1 Replace negative size values in TTML text with 0
Negative size values are not allowed by the spec basically anywhere in the document. Some services seem to accidentally specify a negative value which puts pycaption on a fritz.
2023-03-17 19:28:55 +00:00
rlaphoenix 41018d4574 Don't absorb error messages on Caption Syntax Errors 2023-03-17 18:56:53 +00:00
rlaphoenix 2a4dfb3e93 Improve anti-duplicate checks in Widevine tree logs
While it already has anti-duplicate checks, these checks did not take into account the `*` indicator, or `from x vault` e.t.c. This rework of the duplicate check ignores all messages.
2023-03-17 01:52:47 +00:00
rlaphoenix df4dd73271 Bump to v2.1.0 2023-03-16 20:51:45 +00:00
rlaphoenix 6e888a095e Silence SubtitleEdit when stripping SDH 2023-03-16 20:49:23 +00:00
rlaphoenix c778a890cf Fix calls for Audio & Subtitle's OnMultiplex event 2023-03-15 05:50:12 +00:00
rlaphoenix 0ac1955db6 Ignore aria2's "If errors see the log file" logs 2023-03-15 03:13:41 +00:00
rlaphoenix d3cfc722dc
Merge pull request #43 from Hollander-1908/patch-1
DASH: improved forced subtitle recognition
2023-03-14 12:39:21 +00:00
rlaphoenix cd2831fe82 Add Hollander-1908 to Contributors list 2023-03-14 12:39:05 +00:00
Hollander_1908 5eedbe1f59
DASH: improved forced subtitle recognition
Some manifests uses value `forced_subtitle` instead of the regular `forced-subtitle`.
This way both are recognized.
2023-03-14 12:56:34 +01:00
rlaphoenix 36c530ccc6 Add support for JS-style 13-char timestamps to Cacher 2023-03-13 22:48:49 +00:00