Commit Graph

288 Commits

Author SHA1 Message Date
rlaphoenix eebe76b6f6 Fix segment download merging on Linux machines
It seems on Windows the pathlib.iterdir() function is always in order. However, on Linux or at least some machines this is not the case. This change fixes the order.

If you think you were affected, check your previous downloads that used DASH or HLS segmentation and make sure they dont randomly change scenes out of order.
2023-02-24 23:10:51 +00:00
rlaphoenix c6976a7112 Move --log from the dl cmd to the root cmd 2023-02-24 19:54:05 +00:00
rlaphoenix 4f1cff681c Increase connection pool limit to accommodate Byte-Range dls 2023-02-24 19:46:26 +00:00
rlaphoenix ad82eab712 Pad/depad data when decrypting with ClearKey DRM 2023-02-23 18:19:51 +00:00
rlaphoenix 45c9ba5198 Add support for data URIs from HLS playlists in ClearKey 2023-02-23 18:19:51 +00:00
rlaphoenix 221cd1c283 Fix JOC check on HLS playlists with no audio channel info 2023-02-23 18:19:51 +00:00
rlaphoenix 1f86775ac9 Add support for segment downloads with byte-ranges
Adds support for HLS's EXT-X-BYTERANGE and DASH's SegmentBase.
2023-02-23 18:19:51 +00:00
rlaphoenix 55da41c74f Remove byte_range param on aria2c downloader
Turns out, even if you manually set the Range header AND the server has full support, it does not work. It will act like it works, but it seems internally aria2c gets confused on what bytes it requested, what it returned, and it will either just download the full file, or the range requested (but still complain, and freeze!).

Yikes.
2023-02-23 16:47:14 +00:00
rlaphoenix 725480adf0 Update Changelog for v1.3.1 2023-02-23 09:25:46 +00:00
rlaphoenix da1beae28f Bump to v1.3.1 2023-02-23 09:25:34 +00:00
rlaphoenix 739c17bf4c Initialize config when adding auth if non-existant 2023-02-23 09:09:09 +00:00
rlaphoenix 9932fa3f7a Fix mistake when logging where the cookie file was moved 2023-02-23 09:07:21 +00:00
rlaphoenix 3b5a199f31 Fix regression of missing title & track when licensing with Widevine
For some reason I had the partial removed and replaced with the direct function, meaning services using `title` and `track` during the license and service cert functions would crash.
2023-02-23 08:56:37 +00:00
rlaphoenix 5624232e5e
Merge pull request #33 from varyg1001/master
always define track.path if track.descriptor is URL
2023-02-23 08:45:06 +00:00
varyg fc19358de4 Fix regression of missing track.path on URL downloads 2023-02-23 08:44:50 +00:00
rlaphoenix d8ae543d63 Update Changelog for v1.3.0 2023-02-22 04:39:55 +00:00
rlaphoenix 2aea4549ce Bump to v1.3.0 2023-02-22 04:39:42 +00:00
rlaphoenix 2ad3f04a5e Use the session DRM as the initial DRM for HLS
The variant-playlist may have DRM information that won't be in the invariant (stream) playlist. We need to begin with this DRM data if available.
2023-02-22 04:30:24 +00:00
rlaphoenix c1f716cb6c Drop support for Python 3.8 2023-02-22 03:43:22 +00:00
rlaphoenix f21aa5aac5 Silence aria2c when downloading segmented streams
It conflicts with the TQDM progress bar, it's also not really useful information.
2023-02-22 03:06:53 +00:00
rlaphoenix 4406e3bbab Add ability to silence aria2c's output 2023-02-22 03:06:17 +00:00
rlaphoenix 0913b0dda6 Add ability to set extra arguments with aria2c 2023-02-22 03:01:54 +00:00
rlaphoenix 1443dfaecc Replace Threading Pool system with Thread Storage in Vaults
This fixes the usage of vaults across different threads. It now makes a truly unique connection for each thread. The previous code did this as well, but put back the connection from x thread, to re-use in y thread. Now it simply creates and reuses the connection on their own thread.

Once the thread is closed, the data is now also garbage collected. This now reduces the risk of filling up memory over time.
2023-02-22 01:31:03 +00:00
rlaphoenix 4e875f5ffc Multi-thread the new DASH download system, improve redundency
Just like the commit for HLS multi-threading, this mimics the -j=16 system of aria2c, but manually via a ThreadPoolExecutor. Benefits of this is we still keep support for the new system, and we now get a useful progress bar via TQDM on segmented downloads, unlike aria2c which essentially fills the terminal with jumbled download progress stubs.
2023-02-22 01:06:24 +00:00
rlaphoenix 9e6f5b25f3 Multi-thread the new HLS download system
This mimics the -j=16 system of aria2c, but manually via a ThreadPoolExecutor. Benefits of this is we still keep support for the new system, and we now get a useful progress bar via TQDM on segmented downloads, unlike aria2c which essentially fills the terminal with jumbled download progress stubs.
2023-02-21 16:16:12 +00:00
rlaphoenix 314079c75f Pass save path to DRM decrypt functions directly
This is required in segmented scenarios when multi-threaded where the same `track.path` would be get and set from possibly at the same time. It's also just better logically to do it this way.
2023-02-21 16:09:35 +00:00
rlaphoenix 8268825ba8 Use Widevine.from_init_data when downloading DASH
This is required as some DASH manifests do not explicitly list the PSSH with the Widevine ContentProtection, only listing that its a supported.
2023-02-21 08:02:32 +00:00
rlaphoenix 8c312440a3 Add func to get Widevine PSSH and KID from Init Data
This is just a more direct alternative to Widevine.from_track where you may have the init data but not the track itself.
2023-02-21 08:00:42 +00:00
rlaphoenix 50193856c2 Add missing tqdm progress bar to DASH downloads 2023-02-21 06:35:40 +00:00
rlaphoenix 61270d3af4 Fix expired check in some cases in Cacher class
It seems if a service sets the expiration time in seconds or by a timestamp, then this resolves as expired up to 14 hrs too early, or 14 hours too late, depending on what your timezone is relative to UTC+00:00.

However, I haven't fully confirmed if this is the right fix to make. As in, I'm not sure if changing datetime.utcnow() to datetime.now() should be done, or the other way around. However, I've had multiple people tell me changing it this way worked for them so I'm just going to roll with it.
2023-02-21 06:13:02 +00:00
rlaphoenix 42aaa03941 Completely rewrite downloading system
The new system now downloads and decrypts segments individually instead of downloading all segments, merging them, and then decrypting. Overall the download system now acts more like a normal player.

This fixes #23 as the new HLS download system detects changes in keys and init segments as segments are downloaded. DASH still only supports one period, and one period only, but hopefully I can change that in the future.

Downloading code is now also moved from the Track classes to the manifest classes. Download progress is now also actually helpful for segmented downloads (all HLS, and most DASH streams). It uses TQDM to show a progress bar based on how many segments it needs to download, and how fast it downloads them.

There's only one down side currently. Downloading of segmented videos no longer have the benefit of aria2c's -j parameter. Where it can download n URLs concurrently. Aria2c is still used but only -x and -s is going to make a difference.

In the future I will make HLS and DASH download in a multi-threaded way, sort of a manual version of -j.
2023-02-21 06:00:39 +00:00
rlaphoenix c925cb8af9 Remove AtomicSQL, use Connection Pools in Vaults
This allows the use of vaults in any thread, while keeping database-file level thread safety. AtomicSQL doesn't actually do anything that useful.
2023-02-21 05:38:39 +00:00
rlaphoenix 707469d252 Add ability to pass a proxy to ClearKey in HLS.get_drm 2023-02-21 01:37:39 +00:00
rlaphoenix 5197961d91 Add doc-string to HLS.get_drm 2023-02-21 01:36:01 +00:00
rlaphoenix 09989f8b94 Raise error if no supported drm was found in HLS.get_drm 2023-02-21 01:31:52 +00:00
rlaphoenix c3f2d0d9cc Support HLS's EXT-X-KEY NONE
If EXT-X-KEY of Method=NONE is encountered, it assumes no DRM should be used, even if other supported DRM may have already been or is going to be iterated.
2023-02-21 01:24:04 +00:00
rlaphoenix 7025b5cef3 Only convert AES-128 EXT-X-KEYs to ClearKey objects
There's other AES methods like AES-CBC and such that is not currently supported by the ClearKey DRM class.
2023-02-21 01:21:28 +00:00
rlaphoenix 88a5bc8ad5
Merge pull request #30 from varyg1001/master
Check if cookies exists
2023-02-14 22:38:24 +00:00
varyg f412b0e37f
Check if cookies exists 2023-02-14 21:56:55 +01:00
rlaphoenix 989c24788b Update Changelog for v1.2.0 2023-02-13 20:12:49 +00:00
rlaphoenix be87d897e3 Bump to v1.2.0 2023-02-13 20:04:47 +00:00
rlaphoenix 6d3caffa32 Backport Path.is_relative_to to Python 3.8.x
Further Fix to #28.
2023-02-13 19:07:24 +00:00
rlaphoenix d926b4fe9a Fix annotation type-hints in some files on Python 3.x
Fixes #28.
2023-02-13 19:06:25 +00:00
rlaphoenix af5e6acbb8 Add byte-range option to the aria2c downloader 2023-02-13 18:43:36 +00:00
rlaphoenix 0c02b1513a
Merge pull request #27 from varyg1001/master
Fix cookie import across drives in `auth add`
2023-02-13 18:36:07 +00:00
rlaphoenix f06ca768c3 Add varyg1001 to the Contributors list 2023-02-13 18:33:51 +00:00
varyg bb9e85d777 Use shutil.move to move data instead of Path.rename
Path.rename() cannot move data to different drives. It can only rename the path reference on the same file system. However, shutil.move() will move the data while also changing it's name.
2023-02-13 18:33:02 +00:00
rlaphoenix b25e6c5ce5
Merge pull request #25 from Arias800/patch-1
Detect redirection
2023-02-12 17:55:22 +00:00
rlaphoenix 6a3559bc0f Add Arias800 to the Contributor list in the README 2023-02-12 17:54:02 +00:00
Arias800 7169eaa885
Detect redirection
If the manifest is hidden behind a redirect, the url is not updated and the segments are created with the old url.
I found a French service where this situation occurred.

After this change, it was possible to correctly parse the mpd.
2023-02-12 11:01:16 +01:00