Commit graph

1298 commits

Author SHA1 Message Date
coletdjnz 3999a510f7
Working websockets HTTP/S proxy 2024-04-06 15:14:59 +13:00
coletdjnz fddf9e0577
Merge remote-tracking branch 'coletdjnz/networking/add-http-proxy-tests' into networking/websockets-http-proxy 2024-04-06 12:32:59 +13:00
coletdjnz 6c3140a8c1
try this 2024-04-04 19:34:28 +13:00
coletdjnz 41add1d7af
be gone unclosed socket 2024-04-01 14:37:21 +13:00
coletdjnz bff727c043
Fix unclosed socket errors 2024-04-01 14:02:15 +13:00
coletdjnz 39a45d48f9
somewhat working implementation 2024-03-31 22:04:21 +13:00
coletdjnz a14bb53ab5
remove debug 2024-03-31 16:29:44 +13:00
coletdjnz 14505063ec
cleanup 2024-03-31 16:25:07 +13:00
coletdjnz b44e0f8b98
[test] Add http proxy tests 2024-03-31 14:50:29 +13:00
coletdjnz 52f5be1f1e
[rh:curlcffi] Add support for curl_cffi
Authored by: coletdjnz, Grub4K, pukkandan, bashonly

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
Co-authored-by: bashonly <bashonly@protonmail.com>
2024-03-16 23:15:11 -05:00
coletdjnz 0b81d4d252
Add new options --impersonate and --list-impersonate-targets
Authored by: coletdjnz, Grub4K, pukkandan, bashonly

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
Co-authored-by: bashonly <bashonly@protonmail.com>
2024-03-16 23:14:13 -05:00
coletdjnz f849d77ab5
[test] Workaround websocket server hanging (#9467)
Authored by: coletdjnz
2024-03-16 16:57:21 +13:00
pukkandan 615a84447e
[cleanup] Misc (#8968)
Authored by: pukkandan, bashonly, seproDev
2024-03-11 00:52:28 +05:30
pukkandan 47ab66db0f
[docs] Misc Cleanup (#8977)
Closes #8355, #8944

Authored by: bashonly, Grub4k, Arthurszzz, seproDev, pukkandan

Co-authored-by: sepro <4618135+seproDev@users.noreply.github.com>
Co-authored-by: bashonly <bashonly@protonmail.com>
Co-authored-by: Arthurszzz <minecraftgamerarthur@gmail.com>
Co-authored-by: Simon Sawicki <accounts@grub4k.xyz>
Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2024-03-11 00:48:47 +05:30
sepro ac340d0745
[test:websockets] Fix timeout test on Windows (#9344)
Authored by: seproDev
2024-03-04 17:47:38 +01:00
Lev 104a7b5a46
[ie] Migrate commonly plural fields to lists (#8917)
Authored by: llistochek, pukkandan
Related: #3944
2024-02-20 12:49:24 +05:30
coletdjnz 0085e2bab8
[rh] Remove additional logging handlers on close (#9032)
Fixes https://github.com/yt-dlp/yt-dlp/issues/8922

Authored by: coletdjnz
2024-02-18 11:32:34 +13:00
coletdjnz 35f4f764a7
[rh:requests] Apply remove_dot_segments to absolute redirect locations
Fixes https://github.com/yt-dlp/yt-dlp/issues/9020

Authored by: coletdjnz
2024-01-21 10:03:33 +13:00
coletdjnz 811d298b23
[networking] Remove _CompatHTTPError (#8871)
Use `yt_dlp.networking.exceptions.HTTPError`.
`_CompatHTTPError` was to help with transition to the networking framework.

Authored by: coletdjnz
2024-01-20 15:26:50 +13:00
coletdjnz 69d3191495
[test] Skip source address tests if the address cannot be bound to (#8900)
Fixes https://github.com/yt-dlp/yt-dlp/issues/8890

Authored by: coletdjnz
2024-01-20 10:39:49 +13:00
Simon Sawicki ffbd4f2a02
[utils] traverse_obj: Support xml.etree.ElementTree.Element (#8911)
Authored by: Grub4K
2024-01-05 21:26:17 +01:00
Simon Sawicki f9fb3ce86e
[cleanup] Misc (#8598)
Authored by: bashonly, pukkandan, seproDev, Grub4K

Co-authored-by: bashonly <bashonly@protonmail.com>
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
Co-authored-by: sepro <4618135+seproDev@users.noreply.github.com>
2023-12-30 22:27:36 +01:00
Simon Sawicki 00cdda4f6f
[core] Fix format selection parse error for CPython 3.12 (#8797)
Authored by: Grub4K
2023-12-24 22:09:01 +01:00
bashonly 632b8ee54e
[core] Release workflow and Updater cleanup (#8640)
- Only use trusted publishing with PyPI and remove support for PyPI tokens from release workflow
- Clean up improper actions syntax in the build workflow inputs
- Refactor Updater to allow for consistent unit testing with `UPDATE_SOURCES`

Authored by: bashonly
2023-12-21 21:06:26 +00:00
bashonly 37755a037e
[test:networking] Update tests for OpenSSL 3.2 (#8814)
Authored by: bashonly
2023-12-20 19:03:54 +00:00
coletdjnz 196eb0fe77
[networking] Strip whitespace around header values (#8802)
Fixes https://github.com/yt-dlp/yt-dlp/issues/8729
Authored by: coletdjnz
2023-12-20 19:15:38 +13:00
Simon Sawicki 0b6f829b1d
[utils] traverse_obj: Move is_user_input into output template (#8673)
Authored by: Grub4K
2023-12-06 21:46:45 +01:00
pukkandan 993edd3f6e
[outtmpl] Support multiplication
Related: #8683
2023-12-06 03:44:11 +05:30
sepro 1732eccc0a
[core] Parse release_year from release_date (#8524)
Closes #7263
Authored by: seproDev
2023-11-26 02:12:05 +00:00
coletdjnz ccfd70f4c2
[rh:websockets] Migrate websockets to networking framework (#7720)
* Adds a basic WebSocket framework
* Introduces new minimum `websockets` version of 12.0
* Deprecates `WebSocketsWrapper`

Fixes https://github.com/yt-dlp/yt-dlp/issues/8439

Authored by: coletdjnz
2023-11-20 08:04:04 +00:00
sepro 45d82be65f
[ie/nebula] Overhaul extractors (#8566)
Closes #4300, Closes #5814, Closes #7588, Closes #6334, Closes #6538
Authored by: elyse0, pukkandan, seproDev

Co-authored-by: Elyse <26639800+elyse0@users.noreply.github.com>
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
2023-11-20 01:03:33 +00:00
bashonly f4b95acafc
Remove Python 3.7 support (#8361)
Closes #7803
Authored by: bashonly
2023-11-16 18:39:00 +00:00
bashonly f04b5bedad
[ie] Do not smuggle http_headers
See: https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-3ch3-jhc6-5r8x

Authored by: coletdjnz
2023-11-14 22:04:25 +01:00
bashonly 87264d4fda
[test:update] Implement simple updater unit tests
Authored by: bashonly
2023-11-12 18:30:55 -06:00
Simon Sawicki 4e38e2ae9d
[rh:requests] Handle both bytes and int for IncompleteRead.partial (Fix 8a8b54523a) (#8348)
Authored by: bashonly, coletdjnz, Grub4K
2023-10-15 10:54:38 +02:00
coletdjnz 8a8b54523a
[rh:requests] Add handler for requests HTTP library (#3668)
Adds support for HTTPS proxies and persistent connections (keep-alive)

Closes https://github.com/yt-dlp/yt-dlp/issues/1890
Resolves https://github.com/yt-dlp/yt-dlp/issues/4070
Resolves https://github.com/ytdl-org/youtube-dl/issues/32549
Resolves https://github.com/ytdl-org/youtube-dl/issues/14523
Resolves https://github.com/ytdl-org/youtube-dl/issues/13734

Authored by: coletdjnz, Grub4K, bashonly
2023-10-13 23:33:00 +00:00
Awal Garg 9d7ded6419
[utils] js_to_json: Fix Date constructor parsing (#8295)
Authored by: awalgarg, Grub4K
2023-10-08 01:57:23 +02:00
Simon Sawicki 088add9567
[cleanup] Misc
Authored by: Grub4K
2023-09-24 02:35:23 +02:00
Simon Sawicki de015e9307
[core] Prevent RCE when using --exec with %q (CVE-2023-40581)
The shell escape function is now using `""` instead of `\"`. `utils.Popen` has been patched to properly quote commands.

Prior to this fix using `--exec` together with `%q` when on Windows could cause remote code to execute. See https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-42h4-v29r-42qg for reference.

Authored by: Grub4K
2023-09-24 02:29:01 +02:00
Simon Sawicki 61bdf15fc7
[core] Raise minimum recommended Python version to 3.8 (#8183)
Authored by: Grub4K
2023-09-24 02:24:47 +02:00
bashonly 5ca095cbcd
[cleanup] Misc (#8182)
Closes #7796, Closes #8028
Authored by: barsnick, sqrtNOT, gamer191, coletdjnz, Grub4K, bashonly
2023-09-23 20:00:31 +00:00
bashonly 661c9a1d02
[test:download] Test for expected_exception
Authored by: at-wat

Co-authored-by: Atsushi Watanabe <atsushi.w@ieee.org>
2023-09-21 17:48:57 -05:00
bashonly 52414d64ca
[utils] js_to_json: Handle Array objects
Authored by: Grub4K, std-move

Co-authored-by: std-move <26625259+std-move@users.noreply.github.com>
Co-authored-by: Simon Sawicki <accounts@grub4k.xyz>
2023-09-21 16:51:57 -05:00
coletdjnz 20fbbd9249
[networking] Fix various socks proxy bugs (#8065)
- Fixed support for IPv6 socks proxies
- Fixed support for IPv6 over socks5
- Fixed --source-address not being obeyed for socks4 and socks5
- Fixed socks4a when the destination address is an IPv4 address

Closes https://github.com/yt-dlp/yt-dlp/issues/7959
Fixes https://github.com/ytdl-org/youtube-dl/issues/15368

Authored by: coletdjnz
Co-authored-by: Simon Sawicki <accounts@grub4k.xyz>
Co-authored-by: bashonly <bashonly@bashonly.com>
2023-09-18 07:33:26 +00:00
coletdjnz fcd6a76adc
[tests] Add tests for socks proxies (#7908)
Authored by: coletdjnz
2023-08-25 07:10:44 +00:00
coletdjnz db7b054a61
[networking] Add request handler preference framework (#7603)
Preference functions that take a request and a request handler instance can be registered to prioritize different request handlers per request.

Authored by: coletdjnz
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
2023-08-04 22:17:48 +00:00
pukkandan 6148833f5c
[cleanup] Misc 2023-07-30 04:06:18 +05:30
pukkandan 3f7965105d
[utils] HTTPHeaderDict: Handle byte values 2023-07-30 03:18:10 +05:30
pukkandan de20687ee6
[test] Fix test_load_certifi
Closes #7688, #7675
2023-07-29 21:53:00 +05:30
coletdjnz 4bf912282a
[networking] Remove dot segments during URL normalization (#7662)
This implements RFC3986 5.2.4 remove_dot_segments during the URL normalization process.

Closes #3355, #6526

Authored by: coletdjnz
2023-07-28 22:40:20 +00:00