DRMTalks
/
devine
mirror of https://github.com/devine-dl/devine.git
1
0
Fork 1
Code Issues Packages Projects Releases Wiki Activity

Rework Profile/Authentication System 

- Removed `devine auth` command and sub-commands due to lack of support, risk of data, and general quirks of it.
- Removed `profiles` config data, you must now specify which profile you wish to use each time with -p/--profile. If you use a specific profile a lot more than others, you should make it the default. See below.
- Added a `default` key to each service mapping in `credentials` that will be used if -p/--profile is not specified.
- Each service mapping in `credentials` is no longer forced to use profiles. You can now simply specify `Service: username:password` if you only use one credential.
- Auth-less Services now simply have to specify no credential and have no cookie file.
- There is no longer an error for not having a cookie and/or credential for the chosen profile, as a profile no longer has to be chosen.
- Cookies are now checked for in 3 different locations in the following order:
1. `/Cookies/{Service Name}.txt`
2. `/Cookies/Service Name/{profile}.txt`
3. `/Cookies/Service Name/default.txt`
This means you now have more options on organization and layout of Cookie files, similarly to the new Credentials config.
Note: `/Cookies/Service Name/.txt` also works as an alternative to `default.txt`. The benefit of this is `.txt` will always be at the top of your folder.
This commit is contained in:
rlaphoenix 2024-01-29 06:34:22 +00:00
parent 1c6e91b6f9
commit 837061cf91
5 changed files with 90 additions and 352 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

51
CONFIG.md
View File

@ -67,25 +67,30 @@ DSNP:
default: chromecdm_903_l3 default: chromecdm_903_l3
``` ```
## credentials (dict) ## credentials (dict[str, str|list|dict])
Specify login credentials to use for each Service by Profile as Key (case-sensitive). Specify login credentials to use for each Service, and optionally per-profile.
The value should be `email:password` or `username:password` (with some exceptions).
The first section does not have to be an email or username. It may also be a Phone number.
For example, For example,
```yaml ```yaml
AMZN: ALL4: jane@gmail.com:LoremIpsum100 # directly
AMZN: # or per-profile, optionally with a default
default: jane@example.tld:LoremIpsum99 # <-- used by default if -p/--profile is not used
james: james@gmail.com:TheFriend97 james: james@gmail.com:TheFriend97
jane: jane@example.tld:LoremIpsum99
john: john@example.tld:LoremIpsum98 john: john@example.tld:LoremIpsum98
NF: NF: # the `default` key is not necessary, but no credential will be used by default
john: john@gmail.com:TheGuyWhoPaysForTheNetflix69420 john: john@gmail.com:TheGuyWhoPaysForTheNetflix69420
``` ```
Credentials must be specified per-profile. You cannot specify a fallback or default credential. The value should be in string form, i.e. `john@gmail.com:password123` or `john:password123`.
Any arbitrary values can be used on the left (username/password/phone) and right (password/secret).
You can also specify these in list form, i.e., `["john@gmail.com", ":PasswordWithAColon"]`.
If you specify multiple credentials with keys like the `AMZN` and `NF` example above, then you should
use a `default` key or no credential will be loaded automatically unless you use `-p/--profile`. You
do not have to use a `default` key at all.
Please be aware that this information is sensitive and to keep it safe. Do not share your config. Please be aware that this information is sensitive and to keep it safe. Do not share your config.
## curl_impersonate (dict) ## curl_impersonate (dict)
@ -260,34 +265,6 @@ together.
- `set_title` - `set_title`
Set the container title to `Show SXXEXX Episode Name` or `Movie (Year)`. Default: `true` Set the container title to `Show SXXEXX Episode Name` or `Movie (Year)`. Default: `true`
## profiles (dict)
Pre-define Profiles to use Per-Service.
For example,
```yaml
AMZN: jane
DSNP: john
```
You can also specify a fallback value to pre-define if a match was not made.
This can be done using `default` key. This can help reduce redundancy in your specifications.
```yaml
AMZN: jane
DSNP: john
default: james
```
If a Service doesn't require a profile (as it does not require Credentials or Authorization of any kind), you can
disable the profile checks by specifying `false` as the profile for the Service.
```yaml
ALL4: false
CTV: false
```
## proxy_providers (dict) ## proxy_providers (dict)
Enable external proxy provider services. Enable external proxy provider services.

35
README.md
View File

@ -252,22 +252,33 @@ sure that the version of devine you have locally is supported by the Service cod
> automatically download. Python importing the files triggers the download to begin. However, it may cause a delay on > automatically download. Python importing the files triggers the download to begin. However, it may cause a delay on
> startup. > startup.
## Profiles (Cookies & Credentials) ## Cookies & Credentials
Just like a streaming service, devine associates both a cookie and/or credential as a Profile. You can associate up to Devine can authenticate with Services using Cookies and/or Credentials. Credentials are stored in the config, and
one cookie and one credential per-profile, depending on which (or both) are needed by the Service. This system allows Cookies are stored in the data directory which can be found by running `devine env info`.
you to configure multiple accounts per-service and choose which to use at any time.
Credentials are stored in the config, and Cookies are stored in the data directory. You can find the location of these To add a Credential to a Service, take a look at the [Credentials Config](CONFIG.md#credentials-dictstr-strlistdict)
by running `devine env info`. However, you can manage profiles with `devine auth --help`. E.g. to add a new John for information on setting up one or more credentials per-service. You can add one or more Credential per-service and
profile to Netflix with a Cookie and Credential, take a look at the following CLI call, use `-p/--profile` to choose which Credential to use.
`devine auth add John NF --cookie "C:\Users\John\Downloads\netflix.com.txt --credential "john@gmail.com:pass123"`
You can also delete a credential with `devine auth delete`. E.g., to delete the cookie for John that we just added, run To add a Cookie to a Service, use a Cookie file extension to make a `cookies.txt` file and move it into the Cookies
`devine auth delete John --cookie`. Take a look at `devine auth delete --help` for more information. directory. You must rename the `cookies.txt` file to that of the Service tag (case-sensitive), e.g., `NF.txt`. You can
also place it in a Service Cookie folder, e.g., `/Cookies/NF/default.txt` or `/Cookies/NF/.txt`.
> __Note__ Profile names are case-sensitive and unique per-service. They also have no arbitrary character or length You can add multiple Cookies to the `/Cookies/NF/` folder with their own unique name and then use `-p/--profile` to
> limit, but for convenience I don't recommend using any special characters as your terminal may get confused. choose which one to use. E.g., `/Cookies/NF/sam.txt` and then use it with `--profile sam`. If you make a Service Cookie
folder without a `.txt` or `default.txt`, but with another file, then no Cookies will be loaded unless you use
`-p/--profile` like shown. This allows you to opt in to authentication at whim.
> [!TIP]
> - If your Service does not require Authentication, then do not define any Credential or Cookie for that Service.
> - You can use both Cookies and Credentials at the same time, so long as your Service takes and uses both.
> - If you are using profiles, then make sure you use the same name on the Credential name and Cookie file name when
> using `-p/--profile`.
> [!WARNING]
> Profile names are case-sensitive and unique per-service. They have no arbitrary character or length limit, but for
> convenience sake I don't recommend using any special characters as your terminal may get confused.
### Cookie file format and Extensions ### Cookie file format and Extensions

266
devine/commands/auth.py
View File

@ -1,266 +0,0 @@
import logging
import shutil
import sys
import tkinter.filedialog
from collections import defaultdict
from pathlib import Path
from typing import Optional
import click
from ruamel.yaml import YAML
from devine.core.config import Config, config
from devine.core.constants import context_settings
from devine.core.credential import Credential
@click.group(
short_help="Manage cookies and credentials for profiles of services.",
context_settings=context_settings)
@click.pass_context
def auth(ctx: click.Context) -> None:
"""Manage cookies and credentials for profiles of services."""
ctx.obj = logging.getLogger("auth")
@auth.command(
name="list",
short_help="List profiles and their state for a service or all services.",
context_settings=context_settings)
@click.argument("service", type=str, required=False)
@click.pass_context
def list_(ctx: click.Context, service: Optional[str] = None) -> None:
"""
List profiles and their state for a service or all services.
\b
Profile and Service names are case-insensitive.
"""
log = ctx.obj
service_f = service
auth_data: dict[str, dict[str, list]] = defaultdict(lambda: defaultdict(list))
if config.directories.cookies.exists():
for cookie_dir in config.directories.cookies.iterdir():
service = cookie_dir.name
for cookie in cookie_dir.glob("*.txt"):
if cookie.stem not in auth_data[service]:
auth_data[service][cookie.stem].append("Cookie")
for service, credentials in config.credentials.items():
for profile in credentials:
auth_data[service][profile].append("Credential")
for service, profiles in dict(sorted(auth_data.items())).items(): # type:ignore
if service_f and service != service_f.upper():
continue
log.info(service)
for profile, authorizations in dict(sorted(profiles.items())).items():
log.info(f' "{profile}": {", ".join(authorizations)}')
@auth.command(
short_help="View profile cookies and credentials for a service.",
context_settings=context_settings)
@click.argument("profile", type=str)
@click.argument("service", type=str)
@click.pass_context
def view(ctx: click.Context, profile: str, service: str) -> None:
"""
View profile cookies and credentials for a service.
\b
Profile and Service names are case-sensitive.
"""
log = ctx.obj
service_f = service
profile_f = profile
found = False
for cookie_dir in config.directories.cookies.iterdir():
if cookie_dir.name == service_f:
for cookie in cookie_dir.glob("*.txt"):
if cookie.stem == profile_f:
log.info(f"Cookie: {cookie}")
log.debug(cookie.read_text(encoding="utf8").strip())
found = True
break
for service, credentials in config.credentials.items():
if service == service_f:
for profile, credential in credentials.items():
if profile == profile_f:
log.info(f"Credential: {':'.join(list(credential))}")
found = True
break
if not found:
raise click.ClickException(
f"Could not find Profile '{profile_f}' for Service '{service_f}'."
f"\nThe profile and service values are case-sensitive."
)
@auth.command(
short_help="Check what profile is used by services.",
context_settings=context_settings)
@click.argument("service", type=str, required=False)
@click.pass_context
def status(ctx: click.Context, service: Optional[str] = None) -> None:
"""
Check what profile is used by services.
\b
Service names are case-sensitive.
"""
log = ctx.obj
found_profile = False
for service_, profile in config.profiles.items():
if not service or service_.upper() == service.upper():
log.info(f"{service_}: {profile or '--'}")
found_profile = True
if not found_profile:
log.info(f"No profile has been explicitly set for {service}")
default = config.profiles.get("default", "not set")
log.info(f"The default profile is {default}")
@auth.command(
short_help="Delete a profile and all of its authorization from a service.",
context_settings=context_settings)
@click.argument("profile", type=str)
@click.argument("service", type=str)
@click.option("--cookie", is_flag=True, default=False, help="Only delete the cookie.")
@click.option("--credential", is_flag=True, default=False, help="Only delete the credential.")
@click.pass_context
def delete(ctx: click.Context, profile: str, service: str, cookie: bool, credential: bool):
"""
Delete a profile and all of its authorization from a service.
\b
By default this does remove both Cookies and Credentials.
You may remove only one of them with --cookie or --credential.
\b
Profile and Service names are case-sensitive.
Comments may be removed from config!
"""
log = ctx.obj
service_f = service
profile_f = profile
found = False
if not credential:
for cookie_dir in config.directories.cookies.iterdir():
if cookie_dir.name == service_f:
for cookie_ in cookie_dir.glob("*.txt"):
if cookie_.stem == profile_f:
cookie_.unlink()
log.info(f"Deleted Cookie: {cookie_}")
found = True
break
if not cookie:
for key, credentials in config.credentials.items():
if key == service_f:
for profile, credential_ in credentials.items():
if profile == profile_f:
config_path = Config._Directories.user_configs / Config._Filenames.root_config
yaml, data = YAML(), None
yaml.default_flow_style = False
data = yaml.load(config_path)
del data["credentials"][key][profile_f]
yaml.dump(data, config_path)
log.info(f"Deleted Credential: {credential_}")
found = True
break
if not found:
raise click.ClickException(
f"Could not find Profile '{profile_f}' for Service '{service_f}'."
f"\nThe profile and service values are case-sensitive."
)
@auth.command(
short_help="Add a Credential and/or Cookies to an existing or new profile for a service.",
context_settings=context_settings)
@click.argument("profile", type=str)
@click.argument("service", type=str)
@click.option("--cookie", type=str, default=None, help="Direct path to Cookies to add.")
@click.option("--credential", type=str, default=None, help="Direct Credential string to add.")
@click.pass_context
def add(ctx: click.Context, profile: str, service: str, cookie: Optional[str] = None, credential: Optional[str] = None):
"""
Add a Credential and/or Cookies to an existing or new profile for a service.
\b
Cancel the Open File dialogue when presented if you do not wish to provide
cookies. The Credential should be in `Username:Password` form. The username
may be an email. If you do not wish to add a Credential, just hit enter.
\b
Profile and Service names are case-sensitive!
Comments may be removed from config!
"""
log = ctx.obj
service = service.upper()
profile = profile.lower()
if cookie:
cookie = Path(cookie)
if not cookie.is_file():
log.error(f"No such file or directory: {cookie}.")
sys.exit(1)
else:
print("Opening File Dialogue, select a Cookie file to import.")
cookie = tkinter.filedialog.askopenfilename(
title="Select a Cookie file (Cancel to skip)",
filetypes=[("Cookies", "*.txt"), ("All files", "*.*")]
)
if cookie:
cookie = Path(cookie)
else:
log.info("Skipped adding a Cookie...")
if credential:
try:
credential = Credential.loads(credential)
except ValueError as e:
raise click.ClickException(str(e))
else:
credential = input("Credential: ")
if credential:
try:
credential = Credential.loads(credential)
except ValueError as e:
raise click.ClickException(str(e))
else:
log.info("Skipped adding a Credential...")
if cookie:
final_path = (config.directories.cookies / service / profile).with_suffix(".txt")
final_path.parent.mkdir(parents=True, exist_ok=True)
if final_path.exists():
log.error(f"A Cookie file for the Profile {profile} on {service} already exists.")
sys.exit(1)
shutil.move(cookie, final_path)
log.info(f"Moved Cookie file to: {final_path}")
if credential:
config_path = Config._Directories.user_configs / Config._Filenames.root_config
yaml, data = YAML(), None
yaml.default_flow_style = False
data = yaml.load(config_path)
if not data:
data = {}
if "credentials" not in data:
data["credentials"] = {}
if service not in data["credentials"]:
data["credentials"][service] = {}
data["credentials"][service][profile] = credential.dumps()
yaml.dump(data, config_path)
log.info(f"Added Credential: {credential}")

89
devine/commands/dl.py
View File

@ -28,6 +28,7 @@ from pymediainfo import MediaInfo
from pywidevine.cdm import Cdm as WidevineCdm from pywidevine.cdm import Cdm as WidevineCdm
from pywidevine.device import Device from pywidevine.device import Device
from pywidevine.remotecdm import RemoteCdm from pywidevine.remotecdm import RemoteCdm
from requests.cookies import RequestsCookieJar
from rich.console import Group from rich.console import Group
from rich.live import Live from rich.live import Live
from rich.padding import Padding from rich.padding import Padding
@ -68,7 +69,7 @@ class dl:
token_normalize_func=Services.get_tag token_normalize_func=Services.get_tag
)) ))
@click.option("-p", "--profile", type=str, default=None, @click.option("-p", "--profile", type=str, default=None,
help="Profile to use for Credentials and Cookies (if available). Overrides profile set by config.") help="Profile to use for Credentials and Cookies (if available).")
@click.option("-q", "--quality", type=QUALITY_LIST, default=[], @click.option("-q", "--quality", type=QUALITY_LIST, default=[],
help="Download Resolution(s), defaults to the best available resolution.") help="Download Resolution(s), defaults to the best available resolution.")
@click.option("-v", "--vcodec", type=click.Choice(Video.Codec, case_sensitive=False), @click.option("-v", "--vcodec", type=click.Choice(Video.Codec, case_sensitive=False),
@ -155,17 +156,14 @@ class dl:
self.log = logging.getLogger("download") self.log = logging.getLogger("download")
self.service = Services.get_tag(ctx.invoked_subcommand) self.service = Services.get_tag(ctx.invoked_subcommand)
self.profile = profile
with console.status("Preparing Service and Profile Authentication...", spinner="dots"): if self.profile:
if profile: self.log.info(f"Using profile: '{self.profile}'")
self.profile = profile
self.log.info(f"Profile: '{self.profile}' from the --profile argument")
else:
self.profile = self.get_profile(self.service)
self.log.info(f"Profile: '{self.profile}' from the config")
with console.status("Loading Service Config...", spinner="dots"):
service_config_path = Services.get_path(self.service) / config.filenames.config service_config_path = Services.get_path(self.service) / config.filenames.config
if service_config_path.is_file(): if service_config_path.exists():
self.service_config = yaml.safe_load(service_config_path.read_text(encoding="utf8")) self.service_config = yaml.safe_load(service_config_path.read_text(encoding="utf8"))
self.log.info("Service Config loaded") self.log.info("Service Config loaded")
else: else:
@ -289,14 +287,11 @@ class dl:
else: else:
vaults_only = not cdm_only vaults_only = not cdm_only
if self.profile: with console.status("Authenticating with Service...", spinner="dots"):
with console.status("Authenticating with Service...", spinner="dots"): cookies = self.get_cookie_jar(self.service, self.profile)
cookies = self.get_cookie_jar(self.service, self.profile) credential = self.get_credentials(self.service, self.profile)
credential = self.get_credentials(self.service, self.profile) service.authenticate(cookies, credential)
if not cookies and not credential: if cookies or credential:
self.log.error(f"The Profile '{self.profile}' has no Cookies or Credentials, Check for typos")
sys.exit(1)
service.authenticate(cookies, credential)
self.log.info("Authenticated with Service") self.log.info("Authenticated with Service")
with console.status("Fetching Title Metadata...", spinner="dots"): with console.status("Fetching Title Metadata...", spinner="dots"):
@ -663,13 +658,9 @@ class dl:
)) ))
# update cookies # update cookies
cookie_file = config.directories.cookies / service.__class__.__name__ / f"{self.profile}.txt" cookie_file = self.get_cookie_path(self.service, self.profile)
if cookie_file.exists(): if cookie_file.exists():
cookie_jar = MozillaCookieJar(cookie_file) self.save_cookies(cookie_file, service.session.cookies)
cookie_jar.load()
for cookie in service.session.cookies:
cookie_jar.set_cookie(cookie)
cookie_jar.save(ignore_discard=True)
dl_time = time_elapsed_since(start_time) dl_time = time_elapsed_since(start_time)
@ -954,10 +945,24 @@ class dl:
return profile return profile
@staticmethod @staticmethod
def get_cookie_jar(service: str, profile: str) -> Optional[MozillaCookieJar]: def get_cookie_path(service: str, profile: Optional[str]) -> Optional[Path]:
"""Get Profile's Cookies as Mozilla Cookie Jar if available.""" """Get Service Cookie File Path for Profile."""
cookie_file = config.directories.cookies / service / f"{profile}.txt" direct_cookie_file = config.directories.cookies / f"{service}.txt"
if cookie_file.is_file(): profile_cookie_file = config.directories.cookies / service / f"{profile}.txt"
default_cookie_file = config.directories.cookies / service / "default.txt"
if direct_cookie_file.exists():
return direct_cookie_file
elif profile_cookie_file.exists():
return profile_cookie_file
elif default_cookie_file.exists():
return default_cookie_file
@staticmethod
def get_cookie_jar(service: str, profile: Optional[str]) -> Optional[MozillaCookieJar]:
"""Get Service Cookies for Profile."""
cookie_file = dl.get_cookie_path(service, profile)
if cookie_file:
cookie_jar = MozillaCookieJar(cookie_file) cookie_jar = MozillaCookieJar(cookie_file)
cookie_data = html.unescape(cookie_file.read_text("utf8")).splitlines(keepends=False) cookie_data = html.unescape(cookie_file.read_text("utf8")).splitlines(keepends=False)
for i, line in enumerate(cookie_data): for i, line in enumerate(cookie_data):
@ -972,17 +977,29 @@ class dl:
cookie_file.write_text(cookie_data, "utf8") cookie_file.write_text(cookie_data, "utf8")
cookie_jar.load(ignore_discard=True, ignore_expires=True) cookie_jar.load(ignore_discard=True, ignore_expires=True)
return cookie_jar return cookie_jar
return None
@staticmethod @staticmethod
def get_credentials(service: str, profile: str) -> Optional[Credential]: def save_cookies(path: Path, cookies: RequestsCookieJar):
"""Get Profile's Credential if available.""" cookie_jar = MozillaCookieJar(path)
cred = config.credentials.get(service, {}).get(profile) cookie_jar.load()
if cred: for cookie in cookies:
if isinstance(cred, list): cookie_jar.set_cookie(cookie)
return Credential(*cred) cookie_jar.save(ignore_discard=True)
return Credential.loads(cred)
return None @staticmethod
def get_credentials(service: str, profile: Optional[str]) -> Optional[Credential]:
"""Get Service Credentials for Profile."""
credentials = config.credentials.get(service)
if credentials:
if isinstance(credentials, dict):
if profile:
credentials = credentials.get(profile) or credentials.get("default")
else:
credentials = credentials.get("default")
if credentials:
if isinstance(credentials, list):
return Credential(*credentials)
return Credential.loads(credentials) # type: ignore
@staticmethod @staticmethod
def get_cdm(service: str, profile: Optional[str] = None) -> WidevineCdm: def get_cdm(service: str, profile: Optional[str] = None) -> WidevineCdm:

1
devine/core/config.py
View File

@ -60,7 +60,6 @@ class Config:
self.key_vaults: list[dict[str, Any]] = kwargs.get("key_vaults", []) self.key_vaults: list[dict[str, Any]] = kwargs.get("key_vaults", [])
self.muxing: dict = kwargs.get("muxing") or {} self.muxing: dict = kwargs.get("muxing") or {}
self.nordvpn: dict = kwargs.get("nordvpn") or {} self.nordvpn: dict = kwargs.get("nordvpn") or {}
self.profiles: dict = kwargs.get("profiles") or {}
self.proxy_providers: dict = kwargs.get("proxy_providers") or {} self.proxy_providers: dict = kwargs.get("proxy_providers") or {}
self.serve: dict = kwargs.get("serve") or {} self.serve: dict = kwargs.get("serve") or {}
self.services: dict = kwargs.get("services") or {} self.services: dict = kwargs.get("services") or {}