Commit Graph

497 Commits

Author SHA1 Message Date
rlaphoenix 10285c3819 feat(dl): Add *new* --workers to set download threads/workers
The previously named --workers which is now --downloads specified how many tracks to download, not how many threads/workers are used per-download.

It defaults to nothing, which each downloader then has their own defaults. All current downloaders though currently default to `min(32, (os.cpu_count() or 1) + 4)`, which is also the default for `ThreadPoolExecutor` in general.

This also brings a side effect of changing DASH and HLS's forced max_workers of 16 to now a more appropriate default but more importantly actually configurable. You can set a default in your config under `dl.workers`.
2024-04-03 00:58:47 +01:00
rlaphoenix 0cf20f84a9 refactor(dl): Change --workers to --downloads 2024-04-02 23:34:45 +01:00
rlaphoenix fb5580882b docs(changelog): Add v3.3.0 Changes 2024-04-02 21:59:43 +01:00
rlaphoenix 6d18402807 docs(version): Bump to v3.3.0 2024-04-02 21:53:07 +01:00
rlaphoenix 1db2230892 build(poetry): Update dependencies 2024-04-02 21:52:22 +01:00
rlaphoenix c3d50cf12c ci(pre-commit): Update hooks 2024-04-02 21:51:31 +01:00
rlaphoenix 5a12cb33e2 refactor(Track): Move from OnXyz callables to Event observer
Fixes #85
2024-04-02 18:01:03 +01:00
rlaphoenix 226b609ff5 feat(Events): Add new global Event Observer API 2024-04-02 13:00:38 +01:00
rlaphoenix c194bb5b3a fix(Basic): Fix variable typo regression 2024-04-02 11:06:34 +01:00
rlaphoenix 3b3345964a fix(WVD): Add exists/empty checks to WVD folder dumps 2024-04-01 18:36:51 +01:00
rlaphoenix f99fad8e15 refactor(WVD): Seperate logs in loop for visual clarity 2024-04-01 18:35:56 +01:00
rlaphoenix f683be01d4 fix(WVD): Move log with path before Device load
This is so if the Device.load() call fails, we can know which file of the many files failed.
2024-04-01 18:35:00 +01:00
rlaphoenix 9f4c4584da fix(WVD): Move log out of loop to save performance 2024-04-01 18:34:03 +01:00
rlaphoenix 117a1188cd fix(WVD): Fix empty path to WVDs folder check
It seems a change in click made it so this is now just an empty tuple rather than a one-item tuple with Path("") resolved.
2024-04-01 18:33:02 +01:00
rlaphoenix a053423d23 refactor(WVD): Print error if path to parse doesn't exist 2024-04-01 18:29:56 +01:00
rlaphoenix 3659c81d6a fix(WVD): Ensure WVDs dir exists before moving WVD file
Fixes #83
2024-04-01 18:28:21 +01:00
rlaphoenix 491a0b3a5a feat(Basic): Allow proxy selection by index (one-indexed) 2024-04-01 17:50:08 +01:00
rlaphoenix b36befb296 feat(Basic): Allow single string URIs for countries 2024-04-01 17:47:51 +01:00
rlaphoenix 03b8945273 docs(config): Explain brief usage of proxy_provider data 2024-04-01 17:46:58 +01:00
rlaphoenix 6121cc0896 refactor(Basic): Improve proxy format checks 2024-04-01 17:38:04 +01:00
rlaphoenix bd8309e1d7 fix(Basic): Make query case-insensitive
Fixes #88
2024-04-01 17:32:52 +01:00
rlaphoenix f25d2419cf fix(curl-impersonate): Set Cert-Authority Bundle for HTTPS Proxies
For some reason curl-impersonate (curl_cffi project) does not set the certificate-authority bundle for proxies, which to be fair is for some reason seperated into two curl-options.

Doing this change as well as removing the https->http scheme enforcement on proxies, fixes HTTPS proxies on the curl-impersonate downloaders. I also simplified the seperate http and https proxy definitions to the `all` definition which was not originally supported but does seem to be supported as of v0.6.2.

I tested this on NordVPN proxies which are explicitly HTTPS-only and it does work.
2024-04-01 16:54:21 +01:00
rlaphoenix 45ccc129ce feat(dl): Try find SSAv4 fonts in System OS fonts folder
Currently only Windows is supported. Feel free to make a pull request to add Linux or mac OS support.
2024-03-27 06:01:57 +00:00
rlaphoenix eeab8a4f39 feat(dl): Automatically attach fonts used within SSAv4 subs
The fonts must be within the /devine/fonts folder. This folder location can be changed in the config. If a font is missing it will warn the user and continue.

Closes #82
2024-03-27 06:01:57 +00:00
rlaphoenix 057e4efb56 feat: Add support for MKV Attachments via Attachment class
You add these new Attachment objects to the Tracks object just like you would with Video, Audio, and Subtitle objects.
2024-03-27 06:01:56 +00:00
rlaphoenix a51e1b4f3c docs(changelog): Add v3.2.0 Changes 2024-03-25 03:54:14 +00:00
rlaphoenix 7715a3e844 docs(version): Bump to v3.2.0 2024-03-25 03:53:42 +00:00
rlaphoenix 16faa7dadf build(poetry): Update dependencies 2024-03-25 03:52:38 +00:00
rlaphoenix d9873dac25 fix(HLS): Delete video/audio segments after FFmpeg merge 2024-03-24 22:28:15 +00:00
rlaphoenix 774fec6d77 fix(HLS): Delete subtitle segments as they are merged 2024-03-24 22:27:32 +00:00
rlaphoenix e7294c95d1 fix(requests): Block until connection freed if too many connections 2024-03-13 17:15:13 +00:00
rlaphoenix 36b070f729 fix(requests): Manually compute default max_workers or pool size is None 2024-03-13 17:12:06 +00:00
rlaphoenix 458ad70fae fix(Video): Delete original file after using remove_eia_cc() 2024-03-12 11:08:15 +00:00
rlaphoenix 9fce56cc66 fix(Video): Delete original file after using change_color_range() 2024-03-12 11:07:40 +00:00
rlaphoenix 1bff87bd70 fix(requests): Set HTTP pool connections/maxsize to max workers
This allows requests to open and save/cache up to *max_workers* amount of TCP connections. In most situations it will still only save and re-use one TCP Connection since it always tries to re-use the connection if one is available.

However, in situations where downloads are from more than 10 Host/Port combinations (the default pool connections/maxsize) then this will improve download speeds.
2024-03-12 01:06:42 +00:00
rlaphoenix 5376e4c042 refactor(Service): Go back to the default pool_maxsize in Session
The pool_maxsize value here isn't actually doing much. It should have also been applied to pool_connections. What we realistically needed was just pool_block to prevent opening too much connections (causing a warning). The default pool_connections=10 and pool_maxsize=10 is fine. The downloader doesn't currently use this value.
2024-03-12 00:59:30 +00:00
rlaphoenix c77d521a42 refactor(Track): Default the track name to it's lang's script/territory
This allows you to override the whole track name instead of just prefixing before the script/territory. If you want no track name at all, you can set the track name to an empty string.

The script "Zzzz" (placeholder?) and territory "ZZ" (placeholder?) are not used. The script/territory values are only used if available and if necessary. I.e., fr-CA will use "Canada" but fr-FR will NOT use "France", it will be blank.
2024-03-10 15:19:39 +00:00
rlaphoenix f0b589c8a5 refactor(Track): Remove TERRITORY_MAP constant, trim SAR China manually
e.g., Hong Kong SAR China, Macao SAR China
2024-03-10 15:13:01 +00:00
rlaphoenix 4f79550301 fix(Track): Fix order of operation mistake in get_track_name 2024-03-09 19:56:41 +00:00
rlaphoenix 73d9bc4f94 fix(HLS): Remove save dir even if final merge wasn't needed 2024-03-09 19:44:40 +00:00
rlaphoenix 35501bdb9c fix(DASH): Fix merge regression from recent commit
An else tree was used in 4d6c72ba30 when it shouldn't have been.

Fixes #81
2024-03-09 17:52:50 +00:00
rlaphoenix 1d5d4fd347 fix(dl): Use click.command() instead of click.group() 2024-03-09 01:40:21 +00:00
rlaphoenix 4d6c72ba30 fix(DASH/HLS): Don't merge folders, skip final merge if only 1 segment 2024-03-09 01:37:55 +00:00
rlaphoenix 77e663ebee feat(search): New Search command, Service method, SearchResult Class 2024-03-08 21:32:55 +00:00
rlaphoenix 10a01b0b47 fix(Track): Compute Track ID from the `this` variable, not `self` 2024-03-08 19:22:33 +00:00
rlaphoenix 4c395edc53 fix(dl): Add single mux job if there's no video tracks
Fixes regression from v3.1.0 with --audio-only, --subs-only and --chapters-only.
2024-03-08 19:06:21 +00:00
rlaphoenix eeccdc37cf fix(MultipleChoice): Simplify super() call and value types
It was using the wrong instance, leaving the convert() method to seemingly default to str() for the returned chosen value types (or something, I don't really see why this works).
2024-03-08 17:09:20 +00:00
rlaphoenix 423ff289db feat(Track): Allow Track to choose downloader to use
The downloader property must be a Callable of the same signature as the aria2c, curl_impersonate, and requests downloader functions. You can pass it these functions by importing, or a custom function of a matching signature.

Note: It will still override the chosen downloader and use a fallback one in the case of using aria2c downloader but the download uses the HTTP Range header.

Closes #70
2024-03-08 16:48:44 +00:00
rlaphoenix ba801739fe fix(aria2c): Support aria2(c) 1.37.0 by handling upstream regression
From aria2c's changelog (2007-09-02):

```
Now *.aria2 contorol file is first saved to *.aria2__temp and if it is successful, then renamed to *.aria2.
This prevents *.aria2 file from being truncated or corrupted when file system becomes out of space.
```

It seems something went wrong in 1.37.0 resulting in these files sometimes not being renamed back to `.aria2` and then being left there for good. The fix for devine would be to simply detect `.aria2__temp` and delete them once all segments finish downloading. My only worry here is the root cause for why it has failed to rename. Did the download actually complete without error? According to aria2c's RPC, no errors occurred. There's no way to add support for Aria2(c) 1.37.0 without this sort of change as the files to seem to download correctly regardless of the file not being renamed and then deleted.

Fixes #71
2024-03-08 16:15:50 +00:00
rlaphoenix 79506dda75 chore(HLS): Remove commented-out code from get_supported_key()
This is code I forgot to remove while testing the HLS rework which released in v3.0.0.
2024-03-08 15:48:39 +00:00