Commit graph

650 commits

Author SHA1 Message Date
coletdjnz 641ad5d813
[youtube] Extract error messages from HTTPError response (#644)
Authored by: coletdjnz
2021-08-13 11:48:26 +05:30
Felix S 0715f7e19b
Revert erroneous use of the Content-Length header (#637)
This reverts commit 6c907eb33f

The use of the Content-Length value here is erroneous and may lead
to truncated downloads if a compression scheme is specified in the
Content-Encoding header, as the Content-Length header refers to the
size of encoded data, not of the raw bytestream. This has been noticed
in the wild with WebVTT subtitle segments.

Authored by: fstirlitz
2021-08-11 21:09:17 +05:30
pukkandan a8731fcc1d minor bugfixes
bugs due to be2fc5b212, e9f4ccd19e
2021-08-11 20:27:30 +05:30
pukkandan ade6dc5e9e
[version] update
:ci skip all
2021-08-10 20:51:47 +05:30
jhwgh1968 c196640ff1
[eroprofile] Add album downloader (#658)
Authored by: jhwgh1968
2021-08-10 19:21:12 +05:30
SsSsS 60c8fc73c6
[instagram] Fix comments extraction (#660)
Authored-by: u-spec-png <miloradkalabasdt@gmail.com>
2021-08-10 18:45:32 +05:30
Ashish bc8745480e
[BandCamp] Add BandcampMusicIE (#668)
Authored by Ashish0804
2021-08-10 18:42:11 +05:30
The Hatsune Daishi ff5e16f2f6
[mirrativ] Add extractors (#657)
Authored by: nao20010128nao
2021-08-10 08:54:58 +05:30
pukkandan be2fc5b212
[extractor] Detect sttp as subtitles in MPD
Closes #656
Solution by: fstirlitz
2021-08-10 04:46:48 +05:30
pukkandan 7be9ccff0b [utils] Fix InAdvancePagedList.__getitem__
Since it didn't have any cache, the page was re-fetched for each video.
* Also generalized the cache code
2021-08-10 04:45:25 +05:30
funniray 245d43cacf
[crunchyroll] Fix thumbnail (#650)
Authored by: funniray
2021-08-10 03:09:20 +05:30
mzbaulhaque 246fb276e0
[blackboardcollaborate] Add new extractor (#646)
Authored by: Ashish0804
2021-08-10 02:03:12 +05:30
shirt 6e6e0d95b3
[paramountplus] Separate extractor and fix some titles (#652)
Co-authored-by: shirt, pukkandan
2021-08-10 01:54:50 +05:30
Felix S 25a3f4f5d6
[webvtt] Merge daisy-chained duplicate cues (#638)
Fixes: https://github.com/yt-dlp/yt-dlp/issues/631#issuecomment-893338552

Previous deduplication algorithm only removed duplicate cues with
identical text, styles and timestamps.  This change also merges
cues that come in ‘daisy chains’, where sequences of cues with
identical text and styles appear in which the ending timestamp of
one equals the starting timestamp of the next.

This deduplication algorithm has the somewhat unfortunate side effect
that NOTE blocks between cues, if found, will be emitted in a different
order relative to their original cues.  This may be unwanted if perfect
fidelity is desired, but then so is daisy-chain deduplication itself.
NOTE blocks ought to be ignored by WebVTT players in any case.

Authored by: fstirlitz
2021-08-10 01:52:30 +05:30
pukkandan ad3dc496bb Misc fixes - See desc
* Remove unnecessary uses of _list_from_options_callback
* Fix download tests - Bug from 6e84b21559
* Rename ExecAfterDownloadPP to ExecPP and refactor its tests
* Ensure _write_ytdl_file closes file handle on error - Potential fix for #517
2021-08-10 01:22:55 +05:30
pukkandan 2831b4686c Show libraries present in verbose head 2021-08-10 01:22:55 +05:30
pukkandan 8c0ae192a4 [ffmpeg] Fix --ffmpeg-location when directory is given
Bug introduced in 89efdc15dd
Closes #654
2021-08-10 01:22:55 +05:30
pukkandan e9f4ccd19e
Add option --replace-in-metadata 2021-08-10 01:22:55 +05:30
pukkandan a38bd1defa
[viki] Print error message from API request
Closes #651
2021-08-10 01:21:22 +05:30
Ashish b6a35ad83b
[HotStar] Use API for metadata and extract subtitles (#640)
The API is not rate-limited unlike the webpage

Authored by: Ashish0804
2021-08-08 09:45:06 +05:30
SsSsS bfd56b74b9
[peertube] Fix videos without description (#639)
Authored by: u-spec-png
2021-08-08 09:26:44 +05:30
PSlava 858a65ecc1
[youtube] Improve signature function detection (#641)
Authored by: PSlava (Slava <slash@i-slash.com>)
2021-08-08 09:24:37 +05:30
Wes 3b34e38813
[aenetworks] Update _THEPLATFORM_KEY and _THEPLATFORM_SECRET (#643)
Original PR: https://github.com/ytdl-org/youtube-dl/pull/29749
Fixes: https://github.com/ytdl-org/youtube-dl/issues/29300

Authored by: wesnm
2021-08-08 09:22:31 +05:30
pukkandan 3448870205
[docs] Fix some mistakes and improve doc 2021-08-07 21:41:48 +05:30
pukkandan b868936cd6 [cleanup] Misc 2021-08-07 21:17:07 +05:30
pukkandan c681cb5d93 Allow multiple --exec and --exec-before-download 2021-08-07 21:17:07 +05:30
pukkandan 379e44ed3c [youtube] Raise appropriate error when API pages can't be downloaded 2021-08-07 21:17:06 +05:30
pukkandan 28f436bad0 [extractor] Reset non-repeating warnings per video 2021-08-07 21:17:05 +05:30
pukkandan 2b8a2973bd Allow entire infodict to be printed using %()s
Makes `--dump-json` redundant
2021-08-07 21:17:04 +05:30
pukkandan b7b04c782e Add option --no-simulate to not simulate even when --print or --list... are used
* Deprecates `--print-json`
* Some listings like `--list-extractors` are handled by `yt_dlp` and so are not affected by this. These have been documented as such

Addresses: https://github.com/ytdl-org/youtube-dl/issues/29675, https://github.com/ytdl-org/youtube-dl/issues/29580#issuecomment-882046305
2021-08-07 21:17:03 +05:30
pukkandan 6e84b21559
Fix bugs related to sanitize_info
Related: 8012d892bd (r54555230)
2021-08-07 21:16:55 +05:30
pukkandan 575e17a1b9
[utils] Fix traverse_obj depth when is_user_input 2021-08-07 20:08:22 +05:30
pukkandan 57015a4a3f
[youtube] extractor-arg to show live dash formats
If replay is enabled, these formats can be used to download the last 4 hours
2021-08-07 12:47:54 +05:30
pukkandan 9cc1a3130a
Fix resuming when using --no-part
Closes #576
2021-08-06 00:55:04 +05:30
pukkandan b51d2ae3ca
Add compat-option no-keep-subs
Closes #630
2021-08-06 00:55:04 +05:30
Jesse fee5f0c909
[adobepass] Add MSO Cablevision (#635)
Authored by: Jessecar96
2021-08-06 00:53:37 +05:30
funniray 7bb6434767
[vrv] Fix thumbnail extraction (#634)
Authored by: funniray
2021-08-05 21:49:28 +05:30
pukkandan 124bc071ee
Fix wrong extension for intermediate files
Closes #632
2021-08-05 19:51:14 +05:30
pukkandan a047eeb6d2
Add regex to --match-filter
This does not fully deprecate `--match-title`/`--reject-title`
since `--match-filter` is only checked after the extraction is complete,
while `--match-title` can often be checked from the flat playlist.

Fixes: https://github.com/ytdl-org/youtube-dl/issues/9092, https://github.com/ytdl-org/youtube-dl/issues/23035
2021-08-05 04:10:26 +05:30
Max Teegen 77b87f0519 Add all format filtering operators also to --match-filter
PR: https://github.com/ytdl-org/youtube-dl/pull/27361

Authored by: max-te
2021-08-05 03:37:20 +05:30
pukkandan 678da2f21b [twitch:clips] Extract display_id
PR: https://github.com/ytdl-org/youtube-dl/pull/29684
Fixes: https://github.com/ytdl-org/youtube-dl/issues/29666

Authored by: dirkf
2021-08-05 03:37:20 +05:30
pukkandan cc3fa8d39d Handle BrokenPipeError
PR: https://github.com/ytdl-org/youtube-dl/pull/29505
Fixes: https://github.com/ytdl-org/youtube-dl/issues/29082

Authored by: kikuyan
2021-08-05 03:37:20 +05:30
pukkandan 89efdc15dd [ffpmeg] Allow --ffmpeg-location to be a file with different name 2021-08-05 03:37:18 +05:30
pukkandan 8012d892bd
Ensure sanitization of infodict before printing to stdout
* `filter_requested_info` is renamed to a more appropriate name `sanitize_info`
2021-08-05 03:37:16 +05:30
Stavros Ntentos 9d65e7bd6d
Fix --compat-options filename (#629)
The correct default filename is `%(title)s-%(id)s.%(ext)s`

Authored by: stdedos
2021-08-04 23:31:37 +05:30
SsSsS 36576d7c4c
[Newgrounds] Improve extractor and fix playlist (#627)
Authored by: u-spec-png
2021-08-04 21:18:54 +05:30
nikhil bb36a55c41
[nbcolympics:stream] Fix extractor
PR: https://github.com/ytdl-org/youtube-dl/pull/29688
Closes: #617, https://github.com/ytdl-org/youtube-dl/issues/29665

* Livestreams are untested
* If using ffmpeg as downloader, v4.3+ is needed since `-http_seekable` option is necessary
* Instead of making a seperate key for each arg that needs to be passed to ffmpeg, I made `_ffmpeg_args`
* This deprecates `_seekable`, but the option is kept for compatibility

Authored by: nchilada, pukkandan
2021-08-04 20:41:59 +05:30
MinePlayersPE 3dbb2a9dcb
[RCTIPlus] Support events and TV (#625)
Authored by: MinePlayersPE
2021-08-04 18:42:15 +05:30
The Hatsune Daishi 9997eee4af
[openrec] Add extractors (#624)
Authored by: nao20010128nao
2021-08-04 14:44:37 +05:30
Wes 3e376d183e
[nbcolympics] Update extractor for 2020 olympics (#621)
Fixes: https://github.com/yt-dlp/yt-dlp/issues/617#issuecomment-891834323

Authored by: wesnm
2021-08-04 09:49:44 +05:30
Sam 888299e6ca
[VrtNU] Fix XSRF token (#588)
PR: https://github.com/ytdl-org/youtube-dl/pull/29614
Authored-by: pgaig
2021-08-04 00:11:26 +05:30
pukkandan e5611e8eda
[ffmpeg] Fix streaming mp4 to stdout 2021-08-03 00:05:16 +05:30
SsSsS 8e6cc12c80
[Vine] Remove invalid formats (#614)
Authored by: u-spec-png
2021-08-02 23:37:59 +05:30
pukkandan e9d9efc0f2
[version] update
:ci skip all
2021-08-02 10:41:58 +05:30
pukkandan cf87314d4e
[youtube] Extract SAPISID only once 2021-08-02 08:00:08 +05:30
pukkandan 1bd3639f69
[tenplay] Add MA15+ age limit (#606)
Authored by: pento
2021-08-02 07:52:11 +05:30
LE 68f5867cf0
[CBS] Add fallback (#579)
Related: https://github.com/ytdl-org/youtube-dl/issues/29564
Authored-by: llacb47, pukkandan
2021-08-02 07:46:12 +05:30
Ashish 605cad0be7
[Vimeo] Better extraction of original file (#599)
Authored by: Ashish0804
2021-08-02 07:23:12 +05:30
Ashish e8384376c0
[CBS] Add ParamountPlusSeriesIE (#603)
Authored by: Ashish0804
2021-08-02 02:58:47 +05:30
David e7e94f2a5c
[youtube] Add age-gate bypass for unverified accounts (#600)
Adds `_creator` variants for each client

Authored by: zerodytrash, colethedj, pukkandan
2021-08-02 02:43:46 +05:30
pukkandan a46a815b05
[cleanup] Fix linter in 96fccc101f 2021-08-01 12:52:09 +05:30
pukkandan 96fccc101f [downloader] Allow streaming unmerged formats to stdout using ffmpeg
For this to work:
1. The downloader must be ffmpeg
2. The selected formats must have the same protocol
3. The formats must be downloadable by ffmpeg to stdout

Partial solution for: https://github.com/ytdl-org/youtube-dl/issues/28146, https://github.com/ytdl-org/youtube-dl/issues/27265
2021-08-01 12:38:06 +05:30
pukkandan dbf5416a20 [cleanup] Refactor some code 2021-08-01 12:38:05 +05:30
pukkandan d74a58a186
Set home: as the default key for -P 2021-08-01 12:13:40 +05:30
pukkandan f5510afef0
[FormatSort] Fix bug for audio with unknown codec 2021-08-01 12:13:40 +05:30
pukkandan e4f0275711
Add compat-option no-clean-infojson 2021-08-01 12:13:40 +05:30
pukkandan e0f2b4b47d
[utils] Fix slicing of reversed LazyList
Closes #589
2021-08-01 12:13:40 +05:30
coletdjnz eca330cb88
[youtube] Fix default global API key
bug introduced in 000c15a4ca
2021-08-01 06:12:26 +00:00
Wes d24734daea
[adobepass] Add MSO Sling TV (#596)
Original PR: ytdl-org/youtube-dl#29686
Closes: #300, ytdl-org/youtube-dl#18132

Authored by: wesnm
2021-07-31 03:35:56 +05:30
MinePlayersPE d9e6e9481e
[RCTIPlus] Remove PhantomJS dependency (#595)
Authored by: MinePlayersPE
2021-07-31 03:22:52 +05:30
pukkandan 3619f78d2c
[youtube] Misc cleanup (#577)
Authored by: pukkandan, colethedj
2021-07-31 03:01:49 +05:30
pukkandan 65c2fde23f
[youtube] Add thirdParty to agegate clients (#577)
* This allows more videos like `tf2U5Vyj0oU` to become embeddable
    See https://github.com/yt-dlp/yt-dlp/pull/575#issuecomment-888837000
* Also added tests for all types of age-gate

Closes #581
2021-07-31 02:20:21 +05:30
pukkandan 000c15a4ca
[youtube] simplify and de-duplicate client definitions (#577) 2021-07-31 02:14:15 +05:30
colethedj 9275f62cf8
[youtube] Improve age-gate detection (#577)
Authored by: colethedj
2021-07-31 02:13:55 +05:30
coletdjnz 6552469433
[youtube] Force hl=en for comments (#594)
Closes #532
2021-07-31 01:06:00 +05:30
MinePlayersPE 11cc45718c
[vidio] Fix login error detection (#582)
Authored by: MinePlayersPE
2021-07-29 10:11:05 +05:30
Ashish fe07e2c69f
[Hotstar] Support cookies (#584)
Closes #583 
Authored by: Ashish0804
2021-07-29 10:06:38 +05:30
Ashish 89ce723edd
[Mxplayer] Add h265 formats (#572)
Authored by: Ashish0804
2021-07-29 09:57:09 +05:30
Sipherdrakon 45d1f15725
[dplay] Add ScienceChannelIE (#567)
Authored by: Sipherdrakon
2021-07-29 09:55:00 +05:30
rigstot a318f59d14
[generic] Support KVS player (#549)
* Replaces the extractor for thisvid

Fixes: https://github.com/ytdl-org/youtube-dl/issues/2077
Authored-by: rigstot
2021-07-29 09:33:01 +05:30
pukkandan 7d1eb38af1
Add format types j, l, q for outtmpl
Closes #345
2021-07-29 08:47:25 +05:30
pukkandan 901130bbcf Expand and escape environment variables correctly in outtmpl
Fixes: https://www.reddit.com/r/youtubedl/comments/otfmq3/ytdlp_same_parameters_different_results
2021-07-29 08:38:18 +05:30
MinePlayersPE c0bc527bca
[YouTube] Age-gate bypass implementation (#575)
* Calling the API with `clientScreen=EMBED` allows access to most age-gated videos - discovered by @ccdffddfddfdsfedeee (https://github.com/yt-dlp/yt-dlp/issues/574#issuecomment-887171136)
* Adds clients: (web/android/ios)_(embedded/agegate), mweb_embedded
* Renamed mobile_web to mweb

Closes #574

Authored by pukkandan, MinePlayersPE
2021-07-27 15:10:44 +05:30
pukkandan 2a9c6dcd22
[youtube] Fix format sorting when using alternate clients 2021-07-26 03:50:13 +05:30
coletdjnz 5a1fc62b41
[youtube] Add mobile_web client (#557)
Authored by: colethedj
2021-07-26 03:48:36 +05:30
pukkandan b4c055bac2
[youtube] Add player_client=all 2021-07-26 03:38:18 +05:30
pukkandan ea05b3020d
Remove asr appearing twice in -F 2021-07-26 03:38:15 +05:30
pukkandan 9536bc072d
[bilibili] Improve _VALID_URL 2021-07-26 03:38:10 +05:30
Ashish 8242bf220d
[HotStarSeriesIE] Fix regex (#569)
Authored by: Ashish0804
2021-07-25 22:43:43 +05:30
Ashish 4bfa401d40
[UtreonIE] Add extractor (#562)
Authored by: Ashish0804
2021-07-25 22:41:45 +05:30
nixxo 0222620725
[mediaset] Fix extraction (#564)
Closes #365
Authored by: nixxo
2021-07-24 20:06:55 +05:30
pukkandan 1fe3c4c27e
[version] update
:ci skip all
2021-07-24 20:02:12 +05:30
pukkandan f703a88055
Release 2021.07.24 2021-07-24 07:03:14 +05:30
pukkandan a353beba83
[youtube:tab] Extract video duration early
Based on: https://github.com/ytdl-org/youtube-dl/pull/29487 by glenn-slayden
2021-07-24 06:59:20 +05:30
pukkandan 052e135029
[youtube] Simplify _get_text early 2021-07-24 06:59:20 +05:30
xtkoba cb89cfc14b
[test] Add Python 3.10 (#480)
Authored-by: pukkandan, xtkoba
2021-07-23 20:32:48 +05:30
pukkandan 063c409dfb
[cookies] Handle errors when importing keyring
Workaround for #551
2021-07-23 19:58:27 +05:30
Matt Broadway 767b02a99b
[cookies] Handle sqlite ImportError gracefully (#554)
Closes #544
Authored by: mbway
2021-07-23 19:56:19 +05:30
pukkandan f45e6c1126
[downloader] Pass same status object to all progress_hooks 2021-07-23 09:46:55 +05:30
pukkandan 3944e7af92
[youtube] Fix subtitles only being extracted from the first client
Closes #547
2021-07-23 09:46:55 +05:30