mirrors/yt-dlp
1
0
Fork 0
mirror of https://github.com/yt-dlp/yt-dlp.git synced 2024-11-09 01:43:37 +00:00
Code Issues Packages Projects Releases Wiki Activity
21599 commits 3 branches 99 tags 102 MiB
Commit graph

368 commits

Author SHA1 Message Date
Lesmiscore 4d37720a0c
[extractor/youtube] Download post_live videos from start (#5091) 
* The fragments are generated as a `LazyList`. So only the required formats are expanded during download, but all fragment lists are printed/written in infojson.
* The m3u8 formats which cannot be downloaded from start are not extracted by default, but can be enabled with an extractor-arg. The extractor-arg `include_live_dash` is renamed to `include_incomplete_formats` to account for this new use-case.

Closes #1564
Authored by: Lesmiscore, pukkandan
 2022-10-04 08:18:31 +05:30
pukkandan a057779d5e
[cleanup] Minor fixes 
Closes #5129, Closes #4982
 2022-10-04 01:48:14 +05:30
pukkandan 7a32c70d13
[cleanup] Fix flake8 and minor refactor 
Issues from ab029d7e92, 1fb53b946c
 2022-09-27 08:32:57 +05:30
pukkandan 709ee21417
[extractor/youtube] Do not warn on duplicate chapters 
Eg: vYbaM8w8yzw
 2022-09-27 08:26:26 +05:30
pukkandan 1fb53b946c
[extractor/youtube:tab] Improve continuation items extraction 2022-09-27 04:44:54 +05:30
pukkandan 1dd18a8808
[extractor/YoutubeShortsAudioPivot] Support source URLs 
`ytshortsap:` is no longer needed
 2022-09-27 04:44:50 +05:30
pukkandan 0a5095fe8d
[extractor/youtube:tab] Support reporthistory page 
Closes #4929
 2022-09-27 04:44:50 +05:30
coletdjnz 0ca0f88121
[extractor/heise] Fix extractor (#5029) 
Fixes https://github.com/yt-dlp/yt-dlp/issues/1520
Authored by: coletdjnz
 2022-09-26 00:58:06 +00:00
coletdjnz 80eb0bd9b9
[extractor/youtube] Add support for Shorts audio pivot feed (#4932) 
This feed shows Shorts using the audio of a given video. 

ytshortsap: prefix can be used as a shortcut until YouTube
implements an official view. 

Closes #4911
Authored by: coletdjnz
 2022-09-22 05:39:02 +00:00
coletdjnz c26f9b991a
[extractor/youtube] Support changing extraction language (#4470) 
Adds `--extractor-args youtube:lang=<supported lang code>` extractor arg to prefer translated fields (e.g. title and description) of that language, if available, for all YouTube extractors. See README or error message for list of supported language codes.

Closes https://github.com/yt-dlp/yt-dlp/issues/387

Authored by: coletdjnz
 2022-09-09 05:16:46 +00:00
coletdjnz 3ffb2f5bea
[extractor/youtube] Fix video like count extraction 
Support new combined button layout
Authored by: coletdjnz
 2022-09-09 12:34:39 +12:00
pukkandan 17ffed1842
[docs] Improvements 
* Move detailed installation instructions to https://github.com/yt-dlp/yt-dlp/wiki/Installation
* Link to wiki where applicable
* Fix some mistakes. Closes #4853, Closes #4855, Closes #4852
* Improve some error messages
 2022-09-07 17:38:05 +05:30
pukkandan 7c6eb424d3
[extractor/youtube] Detect lazy-load-for-videos embeds 
Closes #4812
 2022-09-02 02:01:57 +05:30
pukkandan 05deb747bb
[jsinterp] Fix escape in regex 2022-09-01 16:46:32 +05:30
pukkandan b505e8517a
[extractor/youtube] Fallback regex for nsig code extraction 2022-09-01 16:46:32 +05:30
coletdjnz 1ff88b7aec
[extractor/youtube] Add no-youtube-prefer-utc-upload-date compat option (#4771) 
This option reverts 992f9a730b and 17322130a9 to prefer the non-UTC upload date in microformats.

Authored by: coletdjnz, pukkandan
 2022-09-01 10:02:28 +00:00
pukkandan da4db748fa
[utils] Add deprecation_warning 
See https://github.com/yt-dlp/yt-dlp/pull/2173#issuecomment-1097021515
 2022-08-30 21:03:07 +05:30
pukkandan d81ba7d491
[jsinterp, extractor/youtube] Minor fixes 2022-08-30 18:13:37 +05:30
pukkandan c4b2df872d
[jsinterp] Fix _separate 
Ref: https://github.com/yt-dlp/yt-dlp/issues/4635#issuecomment-1231126941
 2022-08-30 16:06:40 +05:30
Samantaz Fox 224b5a35f7
[extractor/youtube] Update iOS Innertube clients (#4792) 
Authored by: SamantazFox
 2022-08-29 03:36:55 +00:00
coletdjnz 50ac0e5416
[extractor/youtube] Use device-specific user agent (#4770) 
Thwart latest fingerprinting attempt (see https://github.com/iv-org/invidious/issues/3230#issuecomment-1226887639)

Authored by: coletdjnz
 2022-08-28 22:59:54 +00:00
pukkandan 5e01315aa1
[cache, extractor/youtube] Invalidate old cache 2022-08-27 07:25:14 +05:30
pukkandan 992dc6b486
[jsinterp] Implement timeout 
Workaround for #4716
 2022-08-22 06:19:06 +05:30
pukkandan b25cac650f
[extractor/youtube] Fix bug in format sorting 2022-08-21 00:56:27 +05:30
pukkandan 90a1df305b
[test] Fix test_youtube_signature 2022-08-21 00:51:03 +05:30
pukkandan a831c2ea90
[cleanup] Misc 2022-08-19 05:08:21 +05:30
pukkandan 25836db6be
[extractor/youtube] Add fallback to phantomjs 
Related #4635
 2022-08-18 21:35:18 +05:30
pukkandan 580ce00782
[youtube] Improve signature caching 
and refactor related functions
 2022-08-18 21:33:30 +05:30
pukkandan f6ca640b12
[jsinterp] Fix for youtube player 1f7d5369 
Closes #4635 again
 2022-08-18 16:38:35 +05:30
pukkandan 3ce2933693
[youtube] Fix error reporting of "Incomplete data" 
Related: #4669
 2022-08-16 22:01:48 +05:30
pukkandan 5c6d2ef9d1
[youtube] Improve format sorting for IOS formats 
When no itag/resolution is available for reference, use the closest resolution
 2022-08-15 14:04:05 +05:30
Lesmiscore 62b58c0936
[docs] Consistent use of e.g. (#4643) 
Authored by: Lesmiscore
 2022-08-14 17:34:13 +05:30
pukkandan 8f53dc44a0
[jsinterp] Handle new youtube signature functions 
Closes #4635
 2022-08-14 05:12:32 +05:30
pukkandan 7e798d725e
[extractor] Fix format sorting of channels 2022-08-11 07:23:46 +05:30
coletdjnz c7dcf0b31e
[extractor/youtube] Add androidSdkVersion parameter to Android Innertube clients 
Required to prevent YouTube returning a bad player response in some cases.

See: https://github.com/yt-dlp/yt-dlp/pull/4593, https://github.com/TeamNewPipe/NewPipe/issues/8713, https://github.com/iv-org/invidious/issues/3230, https://github.com/Tyrrrz/YoutubeExplode/issues/647

Authored by: coletdjnz
 2022-08-08 12:03:10 +12:00
pukkandan a416623436
[extractor/youtube] Extract more format info 2022-08-08 01:47:07 +05:30
coletdjnz a3e9642116
[extractor/youtube] Prevent redirect to unwanted videos (#4593) 
Example: https://www.youtube.com/watch?v=aQvGIIdgFDM

Authored by: coletdjnz
 2022-08-07 19:13:20 +05:30
coletdjnz a0c830f488
[extractor/youtube] Bump Innertube client versions 
YouTube may be requiring new versions soon. See https://github.com/iv-org/invidious/issues/3230, https://github.com/TeamNewPipe/NewPipe/issues/8713

Authored by: coletdjnz
 2022-08-02 19:02:05 +12:00
pukkandan be5c1ae862
Standardize retry mechanism (#1649) 
* [utils] Create `RetryManager`
* Migrate all retries to use the manager
* [extractor] Add wrapper methods for convenience
* Standardize console messages for retries
* Add `--retry-sleep` for extractors
 2022-08-02 01:43:18 +05:30
pukkandan bfd973ece3 [extractors] Use new framework for existing embeds (#4307) 
`Brightcove` is difficult to migrate because it's subclasses may depend
on the signature of the current functions. So it is left as-is for now

Note: Tests have not been migrated
 2022-08-02 01:08:16 +05:30
pukkandan f2e8dbcc00 [extractor, test] Basic framework for embed tests (#4307) 
and split download tests so they can be more easily run in CI

Authored by: coletdjnz
 2022-08-02 01:08:16 +05:30
lazypete365 e325a21a1f
[extractor/youtube] Add live_status=post_live (#4495) 
Related: https://github.com/yt-dlp/yt-dlp/issues/1564
Authored by: lazypete365
 2022-08-01 01:42:04 +05:30
pukkandan 31b532a1f2
[cleanup] Misc 2022-07-31 03:35:16 +05:30
pukkandan c646d76f67
[webvtt, extractor/youtube] Extract auto-subs from livestream VODs 
Closes #4130

Authored by: pukkandan, fstirlitz
 2022-07-31 02:20:11 +05:30
pukkandan 07b47084ba
[extractor/youtube] Parse translated subtitles only when requested 
Closes #4274
 2022-07-31 02:20:11 +05:30
pukkandan 693f060040
[youtube,twitch] Allow waiting for channels to become live 
Closes #2597
 2022-07-26 09:33:19 +05:30
ftk 45e8a04e48
[extractor/youtube] More metadata for storyboards (#4334) 
Authored by: ftk
 2022-07-12 20:46:45 +05:30
pukkandan a3fb1ca5ab
[extractor/youtube] Fix duration check for post-live manifestless mode 2022-07-10 09:59:32 +05:30
Andrew 12a1b2254d
[extractor/youtube, cleanup] Fix tests (#4293) 
Authored by: sheerluck
 2022-07-08 02:50:02 +05:30
pukkandan a3976e0760
Improve chapter sanitization 2022-07-07 11:28:56 +05:30
pukkandan ae61d108dd
[cleanup] Misc cleanup 2022-06-29 06:43:27 +05:30
crazymoose77756 962ffcf89c
[cleanup] Fix some typos (#4194) 
Authored by: crazymoose77756
 2022-06-26 17:50:06 -07:00
pukkandan 14f25df2b6
[compat] Remove deprecated functions from core code 2022-06-25 00:14:12 +05:30
pukkandan ac66811112
[compat] Remove more functions 
Removing any more will require changes to a large number of extractors
 2022-06-25 00:08:55 +05:30
Brett824 06cc8f103b
[extractor/youtube] Mark videos as fully watched (#4146) 
* Also fixes videos appearing as shorts in watch history

Closes #2555
Authored by: Brett824
 2022-06-23 16:30:17 -07:00
pukkandan 9809740ba5
[extractor, cleanup] Reduce direct use of _downloader 2022-06-23 09:57:26 +05:30
pukkandan 471d0367c7
[youtube:clips] Support downloading clips 
Closes #2543
 2022-06-22 02:50:55 +05:30
pukkandan a70635b8a1
[cleanup, utils] Don't use kwargs for format_field 2022-06-18 08:13:22 +05:30
pukkandan 0df111a371
[youtube] Extract comment_count from webpage 
Closes #4091
 2022-06-17 12:00:55 +05:30
coletdjnz 4ce05f5759
[extractor/youtube] Fix live chat for videos with content warning 
Fixes #4051
Authored by: coletdjnz
 2022-06-12 17:56:50 +12:00
pukkandan 56ba69e4c9
[cleanup] Misc fixes 
Closes #4027
 2022-06-11 05:00:12 +05:30
coletdev 2ae778b8fc
[extractor/youtube] Add innertube_host and innertube_key extractor args (#3916) 
Allows user to override Innertube API host or key for all requests
Authored by: coletdjnz
 2022-06-08 22:18:01 +00:00
MMM 36195c4461
[dash] Show fragment count with --live-from-start (#3493) 
Authored by: flashdagger
 2022-06-07 05:44:08 -07:00
coletdjnz 65141660ab
[extractor/youtube] Fix bug in b7c47b7438 
Closes #3997

Authored by: coletdjnz
 2022-06-07 12:26:36 +12:00
pukkandan b7c47b7438
[extractor] Add _search_json 
All fetching of JSON objects should eventually be done with this function
but only `youtube` is being refactored for now
 2022-06-06 19:46:45 +05:30
pukkandan 1890fc6389
[cleanup] Misc fixes 
Cherry-picks from: #3498, #3947
Related: #3949, https://github.com/yt-dlp/yt-dlp/issues/1839#issuecomment-1140313836
Authored by: pukkandan, flashdagger, gamer191
 2022-06-03 21:45:35 +05:30
coletdev c7a7baaa13
[extractor/youtube] Fix :ytnotifications extractor (#3775) 
Still some issues, see https://github.com/yt-dlp/yt-dlp/pull/3775

Authored by: coletdjnz
 2022-06-03 07:04:39 +00:00
pukkandan 6b0b0a289a
[extractor/youtube:tab] Detect videoRenderer in _post_thread_continuation_entries 2022-06-01 02:03:24 +05:30
coletdjnz ee27297f82
[extractor/youtube] Fix initial player response extraction 
Authored by: pukkandan, coletdjnz
 2022-05-29 19:54:22 +12:00
pukkandan 0fe51254cb
[extractor/youtube] Bring back _extract_chapters_from_description 
Closes #3886
 2022-05-29 01:00:41 +05:30
pukkandan 52023f1291
[extractor/youtube] Make signature extraction non-fatal 
and reduce verbosity of it's warning

Closes #3882
 2022-05-29 00:00:24 +05:30
pukkandan 829bbd1d05
[youtube] Add warning for PostLiveDvr 
Closes #3746, Related #1564
 2022-05-27 05:07:00 +05:30
pukkandan 2762dbb17e
[compat] Add functools.cached_property 2022-05-20 21:06:37 +05:30
pukkandan 2414649192
[cleanup] Misc cleanup 2022-05-17 00:07:43 +05:30
Jordan Weatherby d1c4f6d4da
[youtube] Added piped instance urls (#3714) 
Closes #3661
Authored by: JordanWeatherby
 2022-05-12 13:52:13 -07:00
pukkandan 82d020804d
[extractor] Use classmethod/property where possible 
and refactor lazy extractors accordingly.

This reduces the need to create extractor instances
 2022-05-13 00:23:26 +05:30
pukkandan 0f06bcd759
[cleanup] Minor fixes (See desc) 
* [youtube] Fix `--youtube-skip-dash-manifest`
* [build] Use `$()` in `Makefile`. Closes #3684
* Fix bug in 385ffb467b
* Fix bug in 43d7f5a5d0
* [cleanup] Remove unnecessary `utf-8` from `str.encode`/`bytes.decode`
* [utils] LazyList: Expose unnecessarily "protected" attributes
and other minor cleanup
 2022-05-09 17:59:26 +05:30
pukkandan 91e5e839d3
[youtube] Deprioritize format 22 
Reduces chance of encountering #3372
 2022-05-07 08:05:32 +05:30
coletdev 6e634cbe42
[youtube] Add YoutubeStoriesIE (#3362) 
Get channel stories with `ytstories:<channel UCID>`

Authored-by: coletdjnz
 2022-05-01 06:46:28 +00:00
pukkandan bfec31bec8
[youtube] De-prioritize auto-generated thumbnails 
Closes #3112
 2022-04-29 07:41:39 +05:30
pukkandan 1d485a1a79
[cleanup] Misc fixes 
Closes #3565, https://github.com/yt-dlp/yt-dlp/issues/3514#issuecomment-1105944364
 2022-04-29 07:39:33 +05:30
pukkandan 0a41f331cc
[doc] Minor improvements 
Closes #3518, Closes #3560
 2022-04-29 06:53:36 +05:30
pukkandan 19a0394044
[cleanup] Misc cleanup and refactor (#2173) 2022-04-18 02:28:28 +05:30
coletdjnz affc4fefea
[youtube] Fix episode metadata extraction 2022-04-15 16:22:03 +12:00
pukkandan f82711587c
[cleanup] Sort imports 
Using https://github.com/PyCQA/isort

    isort -m VERTICAL_HANGING_INDENT --py 36 -l 80 --rr -n --tc .
 2022-04-12 05:32:52 +05:30
pukkandan 86e5f3ed2e
[cleanup] Upgrade syntax 
Using https://github.com/asottile/pyupgrade

1. `__future__` imports and `coding: utf-8` were removed
2. Files were rewritten with `pyupgrade --py36-plus --keep-percent-format`
3. f-strings were cherry-picked from `pyupgrade --py36-plus`

Extractors are left untouched (except removing header) to avoid unnecessary merge conflicts
 2022-04-12 05:32:51 +05:30
krichbanana ca5300c7ed
[youtube] Add :ytnotifications extractor (#3347) 
Authored by: krichbanana
 2022-04-09 12:55:24 -07:00
pukkandan a25bca9f89
[youtube, cleanup] Minor refactoring 
Authored by: coletdjnz, pukkandan
 2022-04-10 01:21:34 +05:30
coletdev 61d3665d9d
[youtube] Fix uploader for collaborative playlists (#3332) 
Authored by: coletdjnz
 2022-04-07 01:11:16 -07:00
pukkandan a44ca5a470
[cleanup] Misc fixes 
Closes https://github.com/yt-dlp/yt-dlp/pull/3213, Closes https://github.com/yt-dlp/yt-dlp/pull/3117

Related: https://github.com/yt-dlp/yt-dlp/issues/3146#issuecomment-1077323114, https://github.com/yt-dlp/yt-dlp/pull/3277#discussion_r841019671, a825ffbffa (commitcomment-68538986), https://github.com/yt-dlp/yt-dlp/issues/2360, 5fa3c9a88f (r70393519), 5fa3c9a88f (r70393254)
 2022-04-05 18:12:18 +05:30
pukkandan a17526e427
[youtube:tab] Minor improvements (See desc) 
* Support shorts on channel homepage
* Extract thumbnail of OLAK playlists
 2022-04-03 19:01:03 +05:30
coletdev ad210f4fd4
[youtube:search] Support hashtag entries (#3265) 
Authored-by: coletdjnz
 2022-04-02 06:11:14 +00:00
David e7870111e8
[YouTube] Add new age-gate bypass (#3233) 
Closes #3182
Authored by: zerodytrash, pukkandan
 2022-03-29 03:05:31 -07:00
coletdev 1c1b2f96ae
[youtube:tab] Fix duration extraction for shorts (#3171) 
Related: https://github.com/TeamNewPipe/NewPipe/issues/8034
Authored-by: coletdjnz
 2022-03-28 00:49:42 +00:00
pukkandan 1235d333ab
[youtube] Fix auto-translated automatic captions 
d49669acad only covered ASR

Closes #2956
 2022-03-27 14:06:26 +05:30
pukkandan 18e4940825
[youtube] Add extractor-arg to skip auto-translated subs 2022-03-27 14:04:20 +05:30
shirt 727029c508
[youtube] Detect DRM better 
Authored by: shirt-dev
 2022-03-27 11:27:27 +05:30
coletdev fd2ad7cb24
[youtube:tab] Return shorts url if video is a short (#3168) 
Allows filtering out shorts from feeds with `--match-filter`
Closes #3165
Authored-by: coletdjnz
 2022-03-27 05:20:25 +00:00
pukkandan ae72962643
[youtube] Try embedded client variants before agegate 
agegate variants appears to be broken, but don't remove them for the time-being
 2022-03-25 05:00:41 +05:30
pukkandan 52efa4b312
[extractor] Add _perform_login function (#2943) 
* Adds new functions `_initialize_pre_login` and `_perform_login` as part of the extractor API
* Adds `ie.supports_login` to the public API
 2022-03-18 13:53:33 -07:00