ninjajiraiya
/
devine
forked from DRMTalks/devine
1
0
Fork 0
Code Pull Requests Activity
261 Commits 6 Branches 14 Tags 1.1 MiB
Commit Graph

232 Commits

Author SHA1 Message Date
rlaphoenix a549cc6afb Specify DASH Track Key IDs to prepare_drm 2023-03-08 22:41:58 +00:00
rlaphoenix 923cb71f81 Only raise error if the Track's KID was not found when licensing 
For ex., if a service has the same PSSH or license call for 720p and 1080p video tracks, but it doesn't return a KID for the 1080p track, then the previous code would return an error, even though it has enough content key data to continue.

With this change it now only raises the error if the track's exact KID was not licensed. This adds support to prepare_drm for specifying the track's KID.
 2023-03-08 22:41:13 +00:00
rlaphoenix 73bd17ec94 Implement new get_key_id() method to Track 2023-03-08 22:36:21 +00:00
rlaphoenix 853a021ac0 Fix regression in new get_init_segment, change fallback_size to maximum_size 
When the Content-Size is successfully determined, it still uses the (now called maximum_size) as it could otherwise be MBs or GBs worth of data.
 2023-03-08 22:27:38 +00:00
rlaphoenix 573dd8cd49 Don't immediately license DASH DRM until used 
This is unnecessary as the DASH track may get converted into an URL track, which will also prepare the DRM.
 2023-03-08 21:42:05 +00:00
rlaphoenix 8337162991 Prepare DRM on URL tracks if they already have DRM 
Previously it would only prepare the DRM, if it had to find DRM from the init data. Now it prepares DRM if already pre-provided with DRM objects.
 2023-03-08 21:35:14 +00:00
rlaphoenix d73256f1b3 Fix storing of DRM to be before preparation on URL tracks 2023-03-08 21:31:44 +00:00
rlaphoenix 32c118ab57 Rewrite Track's get_init_segment method, now more dynamic 
- DASH and HSL tracks must now explicitly provide the URL to download as the init segment. This is because the original code assumed the first segment, or first init segment was the one that the caller wants, which may not be the case (e.g., and ad is the first init segment).

It now supports explicitly providing a byte-range to download, as well as modifying the fallback content size of 20KB.

It now also checks if the server supports the HTTP Range header and uses it over the hacky request-streaming method. It also checks for the file size and uses that over the fallback size as long as it's not bigger than 100 KB.

Overall it's now more dynamic to specific use-cases, and more efficient in various ways.
 2023-03-08 21:08:50 +00:00
rlaphoenix 4f1d94dd7b Remove list unpack from Widevine's from_track for HLS tracks 
This isn't actually necessary. Likely hasn't been necessary since either v1.0.0, or v1.2.0.
 2023-03-08 20:43:25 +00:00
rlaphoenix cbd796463d Ignore "aria2 will resume download" logs 
These only happen if we intentionally cancel the process, or failed. However, this is something that is generally obvious given the args, and when cancelling a wall of these logs would appear.
 2023-03-08 13:46:55 +00:00
rlaphoenix fa84ef53e7 Don't space the `*` that denotes KIDs within PSSH 
This prevents it from being 1 character over the size limit and wrapping to the next line.
 2023-03-08 13:43:11 +00:00
rlaphoenix b3fdafcf06 Simplify Base URL joining and calculation on DASH 
This also fixes some DASH manifests where it uses multiple BaseURL definitions that must be joined together.
 2023-03-07 12:36:00 +00:00
rlaphoenix eaf7752dde Bump to v2.0.1 2023-03-07 10:28:07 +00:00
rlaphoenix d175ffaf15 Add support for byte-range on HLS init maps 2023-03-04 12:21:28 +00:00
rlaphoenix 1b1412d498 Fix byte range calculation on HLS downloads 
It was off by one. The final calculation for the right-side range needed to be converted from one-index to zero-index.
 2023-03-04 12:18:19 +00:00
rlaphoenix 318832e6b2 Store DRM in the track.drm property in HLS and DASH 2023-03-04 11:49:53 +00:00
rlaphoenix f8166f098c Apply threading lock to HLS DRM preparation 
Without this, if two threads started at the same time there was a very good chance they would run the code and license twice, which is unnecessary.
 2023-03-04 11:41:10 +00:00
rlaphoenix 0bceb772c2 Handle exceptions in user's Service license funcs 2023-03-04 11:23:58 +00:00
rlaphoenix d9471f886f Raise exceptions in prepare_drm instead of using sys.exit(1) 2023-03-04 11:22:51 +00:00
rlaphoenix 4b330c0478 Implement CEKNotFound and EmptyLicense exceptions to Widevine 2023-03-04 11:18:28 +00:00
rlaphoenix c3a22431f0 Fix possible soft-lock in HLS if Queue is left empty after error 2023-03-04 11:11:20 +00:00
rlaphoenix 7df6aa42b4 Ignore Insufficient bits warning from shaka 2023-03-04 02:31:23 +00:00
rlaphoenix 9fff14af30 Fix regression that broke pproxy 2023-03-03 08:53:28 +00:00
rlaphoenix 19ca567019 Only use captured aria2c output if available 2023-03-03 07:54:31 +00:00
rlaphoenix d964dde4d5 Don't pre-allocate file-space for segmented downloads 2023-03-03 07:52:13 +00:00
rlaphoenix a3efadf00b Fix aria2c's segmented check for DASH/HLS 2023-03-03 07:52:13 +00:00
rlaphoenix 714e9af99a Don't print traceback of subprocess errors on download failures 
Since we now have pretty logs for them, the exception (which would be a CalledProcessError) would be generally pointless. However, the return code may be useful so that is kept.
 2023-03-03 07:52:13 +00:00
rlaphoenix 9d6adec707 Fix printing of shaka-packager logs 2023-03-03 07:52:13 +00:00
rlaphoenix 9e23ee13bb Remove silent args in aria2c calls for HLS/DASH 2023-03-03 07:52:13 +00:00
rlaphoenix 432a1122c5 Fix printing of aria2c logs when capturing progress 2023-03-03 07:52:13 +00:00
rlaphoenix b2bcaf97a2 Remove double newline after ASCII banner 2023-03-02 16:24:54 +00:00
rlaphoenix cb1a7988f4 Fix centering of project url in ASCII banner 2023-03-02 16:24:01 +00:00
rlaphoenix 3456e24846 State full 'Episode' text if there's no episode name 2023-03-02 16:19:46 +00:00
rlaphoenix f8a8309628 Fix verbose episode listings if there's no episode name 2023-03-02 16:17:20 +00:00
rlaphoenix 46cb1ba0fa Bump to v2.0.0 2023-03-01 22:05:52 +00:00
rlaphoenix d75996f6e4 Add title download time elapsed to finish log 2023-03-01 16:06:55 +00:00
rlaphoenix 7ee5e71075 Move download time elapsed code to utilities 2023-03-01 16:06:20 +00:00
rlaphoenix 7b7be47f7d Clean up residual files on download stops and fails 2023-03-01 11:19:32 +00:00
rlaphoenix 9f48aab80c Shutdown HLS & DASH dl pool, pass exceptions to dl 
This results in a noticeably faster speed cancelling segmented track downloads on CTRL+C and Errors. It's also reducing code duplication as the dl code will now handle the exception and cleanup for them.

This also simplifies the STOPPING/STOPPED and FAILING/FAILED status messages by quite a bit.
 2023-03-01 11:08:52 +00:00
rlaphoenix fbe78308eb Handle download worker exceptions outside thread loop 
This is so that I can start to log information after the track listing. It's also not necessary to have the try catch within the loop, when both methods will have exited the loop.
 2023-03-01 11:08:52 +00:00
rlaphoenix 624bb6fe75 Only calculate DASH/HLS dl speed if dl sizes are available 2023-03-01 11:08:52 +00:00
rlaphoenix 3a98c93f03 Support CTRL+C on URL downloads, use FAILED/STOPPED messages 2023-03-01 11:08:52 +00:00
rlaphoenix 6a65617179 Reduce the download stop check to one if check in dl 2023-03-01 08:55:40 +00:00
rlaphoenix 840db6e689 Move segment merging from dl to DASH/HLS classes 2023-03-01 08:54:35 +00:00
rlaphoenix d07fedbbe1 Move Widevine DRM prep for URL downloads before download 2023-03-01 08:44:50 +00:00
rlaphoenix f4ad7a2e6c Mark track as stopping when skipping segments 2023-02-28 18:14:03 +00:00
rlaphoenix b482f86bb3 Skip post-download operations if dl stop event is set 2023-02-28 18:05:04 +00:00
rlaphoenix 383e7d9647 Add full support for CTRL+C on HLS and DASH 2023-02-28 18:05:04 +00:00
rlaphoenix 8365d798a4 Pass shaka-packager & aria CTRL+C to caller as KeyboardInterrupt()s 2023-02-28 18:05:04 +00:00
rlaphoenix ad1990cc42 Print a download cancelled message on CTRL+C 2023-02-28 18:05:04 +00:00
rlaphoenix 53c005f727 Remove unnecessary dl stop event set on CTRL+C 2023-02-28 18:05:04 +00:00
rlaphoenix 9cfda3bb9c Don't shutdown pool or the for loop will lock 
Since I'm using `futures.as_completed()`, it will never ever for loop over all tracks and segments and will forever be stuck in the primary thread of the operation. I.e., main thread for the download track threads, or the track thread for the download segment threads.

I've also removed all future cancelled checks as they will never be cancelled before they get the chance to run, because no future cancel calls are made anymore.
 2023-02-28 18:05:03 +00:00
rlaphoenix 51fb7920c9 Mark track as skipped if it never got a chance to start downloading 2023-02-28 16:39:33 +00:00
rlaphoenix acead803bd Remove unnecessary sleep calls at start of download threads 2023-02-28 16:38:10 +00:00
rlaphoenix b6d3c8368a Update CONFIG docs to use proxy_providers 
Also removes an old unused `proxies` config option.
 2023-02-28 16:32:13 +00:00
rlaphoenix 961747b74c Set the default aria2c --file-allocation to prealloc 
Falloc is faster, but supports less systems/environments, and usually require admin perms on Windows.
 2023-02-28 16:23:34 +00:00
rlaphoenix ce53a1b636 Don't run aria2c under asyncio, further improve progress updates 
I've removed asyncio usage as it's generally unnecessary. If you want to run aria2c under a thread, run it under a thread. In the case for devine, this would take another thread, and would be another thread layer deep. Pointless. Would affect speed.

With this change I've been able to improve the aria2c progress capture code quite a bit.
 2023-02-28 08:21:55 +00:00
rlaphoenix d427ec8472 Fix yet another startup crash when loading the config 2023-02-28 06:30:12 +00:00
rlaphoenix 3cfc679294 Center ASCII banner without using U+2800 2023-02-28 06:03:42 +00:00
rlaphoenix dc55f6ffeb Calculate DASH and HLS download speed in an alternate way 
Also fixes getting download sizes for Subtitle tracks
 2023-02-28 06:03:27 +00:00
rlaphoenix f4122f1ae6 Add mapping for Netflix profiles starting with `h264` 
e.g., the new -qc profiles.
 2023-02-28 06:02:36 +00:00
rlaphoenix e5e3f4687d Fix printing of aria2c errors if progress is used 
Also improves the general process of ingesting aria2c progress information.
 2023-02-27 16:52:00 +00:00
rlaphoenix 18449c4777 Ignore ERROR prints from aria2c progress checks 2023-02-27 00:00:55 +00:00
rlaphoenix 7560ee96c9 Replace console.log calls with padded console.prints 
While console.log is currently removing the need for `Padding(..., (0, 5))` as it is overwritten to do it automatically, but in terms of purpose the `console.print` function is more logical.

I hope to find a way to automate the padding within console.print in the future, but for now this will work.
 2023-02-26 23:35:40 +00:00
rlaphoenix eb3f268d64 Revert back to using logging over console.log where possible 
It seems the commit I made to do this change initially seemed to help, it was actually pointless and issues I had were caused by other problems.

For consistency it seems best to stick with the logging module with the RichHandler applied. Using just console.log means being unable to control the log level and which level of logs appear.
 2023-02-26 21:20:43 +00:00
rlaphoenix 401d0481df Implement verbose arg on tree method of Movies and Album 2023-02-26 19:04:54 +00:00
Arias800 d028957e9c Implement `add` and `delete` sub-commands to `wvd` 
This pull request adds the features that are detailed in this issue:
https://github.com/devine-dl/devine/issues/2

Also changes some debug logs to info logs, as the information would generally be wanted. Also changes some logging logs to console.logs.
 2023-02-26 18:46:09 +00:00
rlaphoenix 6419c27e0a Remove spacer from Spinner of progress bars 2023-02-26 15:57:10 +00:00
rlaphoenix a8c1612eb5 Remove unused TextColumn from Track progress bar 2023-02-26 15:56:04 +00:00
rlaphoenix 73b68fe7fe Use OnMultiplex for auto SDH-strip subtitle 2023-02-25 22:00:44 +00:00
rlaphoenix fe320e177d Add OnMultiplex event, for pre-multiplex 2023-02-25 21:58:32 +00:00
rlaphoenix 2635538205 Move repack to post-download, use rich status 2023-02-25 21:25:24 +00:00
rlaphoenix 7d1af8bd8c Move sub conversion to post-download, use rich status 2023-02-25 21:20:50 +00:00
rlaphoenix 8b405b6e02 Skip CC extraction if the binary isn't found 2023-02-25 21:06:11 +00:00
rlaphoenix a5c6052292 Move CC extraction to be post-download, use rich status 2023-02-25 21:04:04 +00:00
rlaphoenix b535715166 Print download thread exceptions with rich traceback 2023-02-25 17:47:59 +00:00
rlaphoenix 70106d32ce Log DRM license info under track downloads 2023-02-25 17:21:13 +00:00
rlaphoenix 178bd01069 Add a rich horizontal rule print on Service construction 2023-02-25 14:04:13 +00:00
rlaphoenix e9b3b3a588 Use rich status when checking for proxy geofence 2023-02-25 14:03:41 +00:00
rlaphoenix cd0c419142 Print available tracks in a Panel 2023-02-25 14:01:46 +00:00
rlaphoenix 4a5aebbca7 Add a total elapsed timer to the final log of dl command 2023-02-25 14:01:25 +00:00
rlaphoenix 09d6c4e1c3 Add a log when download finishes, listing file path 2023-02-25 14:00:39 +00:00
rlaphoenix a5da7c8fbd Add rich progress bar for the mux operation 
Update dl.py
 2023-02-25 13:58:58 +00:00
rlaphoenix 58673590df Remove unnecessary if check for --skip-dl 
This would always be a true statement as this part of the code is already in an if else statement checking skip_dl.
 2023-02-25 13:55:44 +00:00
rlaphoenix 92895426b3 Replace tqdm progress bars with rich progress bars 2023-02-25 13:45:17 +00:00
rlaphoenix cc69423374 Remove logs stating available/selected tracks 
These are now unnecessary to distinguish as only one of the two will appear in the log depending what args are used.
 2023-02-25 13:19:45 +00:00
rlaphoenix 96f408ca49 Only log available tracks if --list is used 
Update dl.py
 2023-02-25 13:12:46 +00:00
rlaphoenix 984582d19d Replace Tracks.print() calls with new rich tree 2023-02-25 13:10:49 +00:00
rlaphoenix 48e35fb4c4 Add rich tree generator to tracks wrapper 2023-02-25 13:05:56 +00:00
rlaphoenix 97b3dbeed2 Re-implement title listing with rich tree 
Note that the full title list will no longer be printed unless --list-titles is used.
 2023-02-25 13:02:00 +00:00
rlaphoenix f7c4d72108 Add rich tree generator to sorted title wrappers 2023-02-25 13:01:09 +00:00
rlaphoenix 92774dcfe6 Simplify str representation of Sorted title containers 2023-02-25 12:54:37 +00:00
rlaphoenix 389fa6e979 Use console.rule between sections of dl's new logs 
These are handy to separate the logs to be per-title, and per section of initialization.
 2023-02-25 12:52:51 +00:00
rlaphoenix b2bbc808c4 Use console.status across dl command 2023-02-25 12:22:32 +00:00
rlaphoenix 62d91a3e77 Replace log.info calls with console.log calls 
I've moved log.info calls to console.log calls to reduce conflicts of logs at the same time as console refreshes, but also to reduce unnecessary log level text being printed to the console.

We don't need to know if a log is an `info` level log, but I've kept log.error's and such as we would want to know if a log is an error log and such.
 2023-02-25 12:11:17 +00:00
rlaphoenix 3e1a067724 Override the default traceback with rich traceback 2023-02-25 11:53:44 +00:00
rlaphoenix b8f3118775 Replace version logs with new ASCII banner log 2023-02-25 11:51:03 +00:00
rlaphoenix f1864ad63c Replace logging handler with ComfyRichHandler 
This changes all logs via logging to be printed to the rich console instead. Specifically, using my custom ComfyRichHandler making the logs padded horizontally.

In this change the time is no longer printed. The log level is still printed but it's now the full log name in bold.
 2023-02-25 11:47:39 +00:00
rlaphoenix 8d626822cb Remove coloredlogs and now unused log constants 2023-02-25 11:46:03 +00:00
rlaphoenix 77c16f557c Replace logging FileHandler with console.save_text 2023-02-25 11:38:02 +00:00