Commit Graph

136 Commits

Author SHA1 Message Date
rlaphoenix 8f5bbeb8e3 Split the download finished log into dl/title finish logs 2023-03-26 22:42:16 +01:00
rlaphoenix 0b2e3e2255 Remove the muxed download path log
It's not very appealing, nor is it info the user realistically needs for every download. They could check `devine env info` to see where downloads go if they are unsure.
2023-03-26 22:41:23 +01:00
rlaphoenix 2a8e86b057 Delete video tracks as they are muxed
This reduces the total cumulative temp download folder size.
2023-03-26 22:40:05 +01:00
rlaphoenix 63eeeca910 Fix regression where all videos are selected if --quality isnt used 2023-03-26 20:18:15 +01:00
rlaphoenix 33a9c307f3 Add ability to download multiple resolutions per title
Closes #26
2023-03-26 19:28:46 +01:00
rlaphoenix 2a4dfb3e93 Improve anti-duplicate checks in Widevine tree logs
While it already has anti-duplicate checks, these checks did not take into account the `*` indicator, or `from x vault` e.t.c. This rework of the duplicate check ignores all messages.
2023-03-17 01:52:47 +00:00
rlaphoenix 90818f201d Include the mkvmerge return code on error 2023-03-12 00:10:54 +00:00
rlaphoenix 7bb215d496 Prevent licensing Widevine DRM a second time
If you use --cdm-only, you will end up licensing multiple times if the PSSH has more than one Key ID. While this is checked for, with the KID check now being more lenient, it will end up continuing and licensing again.

However, even with the original check code this would have been pointless. If the first license did not return a content key for a KID, then the next license call with the exact same parameters wouldn't have either
2023-03-08 22:48:36 +00:00
rlaphoenix da7acb0417 Specify URL Track Key IDs to prepare_drm 2023-03-08 22:42:25 +00:00
rlaphoenix 923cb71f81 Only raise error if the Track's KID was not found when licensing
For ex., if a service has the same PSSH or license call for 720p and 1080p video tracks, but it doesn't return a KID for the 1080p track, then the previous code would return an error, even though it has enough content key data to continue.

With this change it now only raises the error if the track's exact KID was not licensed. This adds support to prepare_drm for specifying the track's KID.
2023-03-08 22:41:13 +00:00
rlaphoenix 8337162991 Prepare DRM on URL tracks if they already have DRM
Previously it would only prepare the DRM, if it had to find DRM from the init data. Now it prepares DRM if already pre-provided with DRM objects.
2023-03-08 21:35:14 +00:00
rlaphoenix d73256f1b3 Fix storing of DRM to be before preparation on URL tracks 2023-03-08 21:31:44 +00:00
rlaphoenix fa84ef53e7 Don't space the `*` that denotes KIDs within PSSH
This prevents it from being 1 character over the size limit and wrapping to the next line.
2023-03-08 13:43:11 +00:00
rlaphoenix 0bceb772c2 Handle exceptions in user's Service license funcs 2023-03-04 11:23:58 +00:00
rlaphoenix d9471f886f Raise exceptions in prepare_drm instead of using sys.exit(1) 2023-03-04 11:22:51 +00:00
rlaphoenix 9fff14af30 Fix regression that broke pproxy 2023-03-03 08:53:28 +00:00
rlaphoenix 714e9af99a Don't print traceback of subprocess errors on download failures
Since we now have pretty logs for them, the exception (which would be a CalledProcessError) would be generally pointless. However, the return code may be useful so that is kept.
2023-03-03 07:52:13 +00:00
rlaphoenix d75996f6e4 Add title download time elapsed to finish log 2023-03-01 16:06:55 +00:00
rlaphoenix 7ee5e71075 Move download time elapsed code to utilities 2023-03-01 16:06:20 +00:00
rlaphoenix 7b7be47f7d Clean up residual files on download stops and fails 2023-03-01 11:19:32 +00:00
rlaphoenix fbe78308eb Handle download worker exceptions outside thread loop
This is so that I can start to log information after the track listing. It's also not necessary to have the try catch within the loop, when both methods will have exited the loop.
2023-03-01 11:08:52 +00:00
rlaphoenix 3a98c93f03 Support CTRL+C on URL downloads, use FAILED/STOPPED messages 2023-03-01 11:08:52 +00:00
rlaphoenix 6a65617179 Reduce the download stop check to one if check in dl 2023-03-01 08:55:40 +00:00
rlaphoenix 840db6e689 Move segment merging from dl to DASH/HLS classes 2023-03-01 08:54:35 +00:00
rlaphoenix d07fedbbe1 Move Widevine DRM prep for URL downloads before download 2023-03-01 08:44:50 +00:00
rlaphoenix b482f86bb3 Skip post-download operations if dl stop event is set 2023-02-28 18:05:04 +00:00
rlaphoenix 383e7d9647 Add full support for CTRL+C on HLS and DASH 2023-02-28 18:05:04 +00:00
rlaphoenix ad1990cc42 Print a download cancelled message on CTRL+C 2023-02-28 18:05:04 +00:00
rlaphoenix 53c005f727 Remove unnecessary dl stop event set on CTRL+C 2023-02-28 18:05:04 +00:00
rlaphoenix 9cfda3bb9c Don't shutdown pool or the for loop will lock
Since I'm using `futures.as_completed()`, it will never ever for loop over all tracks and segments and will forever be stuck in the primary thread of the operation. I.e., main thread for the download track threads, or the track thread for the download segment threads.

I've also removed all future cancelled checks as they will never be cancelled before they get the chance to run, because no future cancel calls are made anymore.
2023-02-28 18:05:03 +00:00
rlaphoenix 51fb7920c9 Mark track as skipped if it never got a chance to start downloading 2023-02-28 16:39:33 +00:00
rlaphoenix acead803bd Remove unnecessary sleep calls at start of download threads 2023-02-28 16:38:10 +00:00
rlaphoenix ce53a1b636 Don't run aria2c under asyncio, further improve progress updates
I've removed asyncio usage as it's generally unnecessary. If you want to run aria2c under a thread, run it under a thread. In the case for devine, this would take another thread, and would be another thread layer deep. Pointless. Would affect speed.

With this change I've been able to improve the aria2c progress capture code quite a bit.
2023-02-28 08:21:55 +00:00
rlaphoenix 7560ee96c9 Replace console.log calls with padded console.prints
While console.log is currently removing the need for `Padding(..., (0, 5))` as it is overwritten to do it automatically, but in terms of purpose the `console.print` function is more logical.

I hope to find a way to automate the padding within console.print in the future, but for now this will work.
2023-02-26 23:35:40 +00:00
rlaphoenix eb3f268d64 Revert back to using logging over console.log where possible
It seems the commit I made to do this change initially seemed to help, it was actually pointless and issues I had were caused by other problems.

For consistency it seems best to stick with the logging module with the RichHandler applied. Using just console.log means being unable to control the log level and which level of logs appear.
2023-02-26 21:20:43 +00:00
Arias800 d028957e9c Implement `add` and `delete` sub-commands to `wvd`
This pull request adds the features that are detailed in this issue:
https://github.com/devine-dl/devine/issues/2

Also changes some debug logs to info logs, as the information would generally be wanted. Also changes some logging logs to console.logs.
2023-02-26 18:46:09 +00:00
rlaphoenix 6419c27e0a Remove spacer from Spinner of progress bars 2023-02-26 15:57:10 +00:00
rlaphoenix 73b68fe7fe Use OnMultiplex for auto SDH-strip subtitle 2023-02-25 22:00:44 +00:00
rlaphoenix 2635538205 Move repack to post-download, use rich status 2023-02-25 21:25:24 +00:00
rlaphoenix 7d1af8bd8c Move sub conversion to post-download, use rich status 2023-02-25 21:20:50 +00:00
rlaphoenix 8b405b6e02 Skip CC extraction if the binary isn't found 2023-02-25 21:06:11 +00:00
rlaphoenix a5c6052292 Move CC extraction to be post-download, use rich status 2023-02-25 21:04:04 +00:00
rlaphoenix b535715166 Print download thread exceptions with rich traceback 2023-02-25 17:47:59 +00:00
rlaphoenix 70106d32ce Log DRM license info under track downloads 2023-02-25 17:21:13 +00:00
rlaphoenix cd0c419142 Print available tracks in a Panel 2023-02-25 14:01:46 +00:00
rlaphoenix 4a5aebbca7 Add a total elapsed timer to the final log of dl command 2023-02-25 14:01:25 +00:00
rlaphoenix 09d6c4e1c3 Add a log when download finishes, listing file path 2023-02-25 14:00:39 +00:00
rlaphoenix a5da7c8fbd Add rich progress bar for the mux operation
Update dl.py
2023-02-25 13:58:58 +00:00
rlaphoenix 58673590df Remove unnecessary if check for --skip-dl
This would always be a true statement as this part of the code is already in an if else statement checking skip_dl.
2023-02-25 13:55:44 +00:00
rlaphoenix 92895426b3 Replace tqdm progress bars with rich progress bars 2023-02-25 13:45:17 +00:00
rlaphoenix cc69423374 Remove logs stating available/selected tracks
These are now unnecessary to distinguish as only one of the two will appear in the log depending what args are used.
2023-02-25 13:19:45 +00:00
rlaphoenix 96f408ca49 Only log available tracks if --list is used
Update dl.py
2023-02-25 13:12:46 +00:00
rlaphoenix 984582d19d Replace Tracks.print() calls with new rich tree 2023-02-25 13:10:49 +00:00
rlaphoenix 97b3dbeed2 Re-implement title listing with rich tree
Note that the full title list will no longer be printed unless --list-titles is used.
2023-02-25 13:02:00 +00:00
rlaphoenix 389fa6e979 Use console.rule between sections of dl's new logs
These are handy to separate the logs to be per-title, and per section of initialization.
2023-02-25 12:52:51 +00:00
rlaphoenix b2bbc808c4 Use console.status across dl command 2023-02-25 12:22:32 +00:00
rlaphoenix 62d91a3e77 Replace log.info calls with console.log calls
I've moved log.info calls to console.log calls to reduce conflicts of logs at the same time as console refreshes, but also to reduce unnecessary log level text being printed to the console.

We don't need to know if a log is an `info` level log, but I've kept log.error's and such as we would want to know if a log is an error log and such.
2023-02-25 12:11:17 +00:00
rlaphoenix 570e0aba00 Fix printing of download worker exceptions 2023-02-25 14:46:43 +00:00
rlaphoenix eebe76b6f6 Fix segment download merging on Linux machines
It seems on Windows the pathlib.iterdir() function is always in order. However, on Linux or at least some machines this is not the case. This change fixes the order.

If you think you were affected, check your previous downloads that used DASH or HLS segmentation and make sure they dont randomly change scenes out of order.
2023-02-24 23:10:51 +00:00
rlaphoenix c6976a7112 Move --log from the dl cmd to the root cmd 2023-02-24 19:54:05 +00:00
rlaphoenix 739c17bf4c Initialize config when adding auth if non-existant 2023-02-23 09:09:09 +00:00
rlaphoenix 9932fa3f7a Fix mistake when logging where the cookie file was moved 2023-02-23 09:07:21 +00:00
rlaphoenix 3b5a199f31 Fix regression of missing title & track when licensing with Widevine
For some reason I had the partial removed and replaced with the direct function, meaning services using `title` and `track` during the license and service cert functions would crash.
2023-02-23 08:56:37 +00:00
varyg fc19358de4 Fix regression of missing track.path on URL downloads 2023-02-23 08:44:50 +00:00
rlaphoenix c1f716cb6c Drop support for Python 3.8 2023-02-22 03:43:22 +00:00
rlaphoenix 314079c75f Pass save path to DRM decrypt functions directly
This is required in segmented scenarios when multi-threaded where the same `track.path` would be get and set from possibly at the same time. It's also just better logically to do it this way.
2023-02-21 16:09:35 +00:00
rlaphoenix 42aaa03941 Completely rewrite downloading system
The new system now downloads and decrypts segments individually instead of downloading all segments, merging them, and then decrypting. Overall the download system now acts more like a normal player.

This fixes #23 as the new HLS download system detects changes in keys and init segments as segments are downloaded. DASH still only supports one period, and one period only, but hopefully I can change that in the future.

Downloading code is now also moved from the Track classes to the manifest classes. Download progress is now also actually helpful for segmented downloads (all HLS, and most DASH streams). It uses TQDM to show a progress bar based on how many segments it needs to download, and how fast it downloads them.

There's only one down side currently. Downloading of segmented videos no longer have the benefit of aria2c's -j parameter. Where it can download n URLs concurrently. Aria2c is still used but only -x and -s is going to make a difference.

In the future I will make HLS and DASH download in a multi-threaded way, sort of a manual version of -j.
2023-02-21 06:00:39 +00:00
rlaphoenix c925cb8af9 Remove AtomicSQL, use Connection Pools in Vaults
This allows the use of vaults in any thread, while keeping database-file level thread safety. AtomicSQL doesn't actually do anything that useful.
2023-02-21 05:38:39 +00:00
varyg f412b0e37f
Check if cookies exists 2023-02-14 21:56:55 +01:00
varyg bb9e85d777 Use shutil.move to move data instead of Path.rename
Path.rename() cannot move data to different drives. It can only rename the path reference on the same file system. However, shutil.move() will move the data while also changing it's name.
2023-02-13 18:33:02 +00:00
rlaphoenix 47448aac3c Print traceback when download worker fails 2023-02-11 22:16:54 +00:00
rlaphoenix 067130990c Add missing exit when --abitrate returns no tracks 2023-02-10 21:11:28 +00:00
rlaphoenix d612599e27 Add Audio Channels Selection option to dl command CLI
Sub-channel layouts like 5.1, 7.1, will match with e.g., 6.0 and 8.0 respectively.

Closes #9.
2023-02-10 21:10:09 +00:00
rlaphoenix 84bf9fde92 Add Bitrate Selection options to dl command CLI
It defaults to highest available. Closes #8.
2023-02-10 20:19:54 +00:00
rlaphoenix 268509fb49 Remove unused f-strings with no placeholders 2023-02-10 19:26:46 +00:00
rlaphoenix d982e37ee5 Re-order all imports across project with isort 2023-02-10 19:25:25 +00:00
rlaphoenix faabfb550c Replace use of with_stem + with_suffix to with_name 2023-02-10 19:13:31 +00:00
rlaphoenix 18e2d8617e Create credentials dict prior to assignment in auth add
Fixes #17.
2023-02-07 21:32:21 +00:00
rlaphoenix c5d6ba09f2 Check if Cookie file for profile already exists during auth add
Fixes #16.
2023-02-07 21:25:26 +00:00
rlaphoenix 6619c29fb5 Create parent dirs when adding cookies with auth add
Fixes #21.
2023-02-07 21:21:15 +00:00
rlaphoenix 0446c44a42 Only iterate over cookie dir if exists in auth list
Fixes #18.
2023-02-07 21:08:45 +00:00
rlaphoenix 794de8b516 Efficiently store data in auth list, sort the data 2023-02-07 21:07:58 +00:00
rlaphoenix 1fa3ba61c8 Fix crash when using cfg on empty/new configs
Fixes #19.
2023-02-07 20:44:34 +00:00
rlaphoenix 5301ac2924 Add util to test decoding of the video/audio streams with FFmpeg 2023-02-07 00:58:53 +00:00
rlaphoenix 00f85f7206 Add util to change video range flag losslessly
This is useful for some services. Some times there's a random stream with the wrong video range.
2023-02-06 23:49:28 +00:00
rlaphoenix 7fd87b8aa2 Initial commit 2023-02-06 02:41:29 +00:00