Commit graph

59 commits

Author SHA1 Message Date
Simon Sawicki edbe5b589d
Bugfixes for 4823ec9f46
Hotfix for fragmented downloads

Authored by: bashonly
2023-05-25 22:52:44 +02:00
pukkandan 4823ec9f46
Update to ytdl-commit-d1c6c5
[YouTube] [core] Improve platform debug log, based on yt-dlp
d1c6c5c4d6

Except:
    * 6ed34338285f722d0da312ce0af3a15a077a3e2a [jsinterp] Add short-cut evaluation for common expression
        * There was no performance improvement when tested with https://github.com/ytdl-org/youtube-dl/issues/30641
    * e8de54bce50f6f77a4d7e8e80675f7003d5bf630 [core] Handle `/../` sequences in HTTP URLs
        * We plan to implement this differently
2023-05-24 23:30:43 +05:30
pukkandan 46f1370e9a
[devscripts/cli_to_api] Add script 2023-05-24 23:29:30 +05:30
pukkandan 5b28cef72d
[cleanup] Misc 2023-02-28 23:51:06 +05:30
pukkandan f34804b2f9
[extractor/youtube] Fix 5038f6d713
* [fragment] Fix `request_data`
* [youtube] Don't use POST for now. It may be easier to break in future

Authored by: bashonly, coletdjnz
2023-02-28 23:34:43 +05:30
Simon Sawicki 6839ae1f6d
[utils] traverse_obj: Fix more bugs
and cleanup uses of `default=[]`

Continued from b1bde57bef
2023-02-10 19:36:55 +05:30
bashonly 7e68567e50
[downloader/hls] Allow extractors to provide AES key (#6158)
and related cleanup

Authored by: bashonly, Grub4K

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
2023-02-08 11:09:32 +05:30
pukkandan 814bba3933
[downloader/fragment] HLS download can continue without first fragment
Closes #5274
2022-10-18 19:20:51 +05:30
pukkandan da4db748fa
[utils] Add deprecation_warning
See https://github.com/yt-dlp/yt-dlp/pull/2173#issuecomment-1097021515
2022-08-30 21:03:07 +05:30
pukkandan be5c1ae862
Standardize retry mechanism (#1649)
* [utils] Create `RetryManager`
* Migrate all retries to use the manager
* [extractor] Add wrapper methods for convenience
* Standardize console messages for retries
* Add `--retry-sleep` for extractors
2022-08-02 01:43:18 +05:30
pukkandan ac66811112
[compat] Remove more functions
Removing any more will require changes to a large number of extractors
2022-06-25 00:08:55 +05:30
pukkandan 998a3cae0c
[cleanup] Misc fixes 2022-06-22 03:47:41 +05:30
MMM 36195c4461
[dash] Show fragment count with --live-from-start (#3493)
Authored by: flashdagger
2022-06-07 05:44:08 -07:00
pukkandan 1890fc6389
[cleanup] Misc fixes
Cherry-picks from: #3498, #3947
Related: #3949, https://github.com/yt-dlp/yt-dlp/issues/1839#issuecomment-1140313836
Authored by: pukkandan, flashdagger, gamer191
2022-06-03 21:45:35 +05:30
pukkandan 666c36d58d
Bugfix for 23326151c4 2022-05-20 21:03:19 +05:30
pukkandan 23326151c4
Add option --retry-sleep (#3059)
Closes #2852
2022-05-19 20:00:31 +05:30
pukkandan 3fe75fdc80
[cleanup] Misc fixes (see desc)
* Do not warn when fixup is skipped for existing file
* [fragment] Fix `--skip-unavailable-fragments` for HTTP Errors
* [utils] write_string: Fix bug in 59f943cd50
* [utils] parse_codecs: Subtitle codec is generally referred to as `scodec`. https://github.com/yt-dlp/yt-dlp/pull/2174#discussion_r790156048
* [docs] Remove note about permissions. Closes #3597
2022-05-01 04:58:38 +05:30
pukkandan 1d485a1a79
[cleanup] Misc fixes
Closes #3565, https://github.com/yt-dlp/yt-dlp/issues/3514#issuecomment-1105944364
2022-04-29 07:39:33 +05:30
pukkandan 19a0394044
[cleanup] Misc cleanup and refactor (#2173) 2022-04-18 02:28:28 +05:30
Lesmiscore (Naoya Ozaki) c854208ccf
[downloader/fragment] Make single thread download work for --live-from-start (#3446)
Authored by: Lesmiscore
2022-04-16 21:11:09 +09:00
pukkandan e06bd8800f
Fix --skip-unavailable-fragments
Bug in d71fd41249
Closes #3437
2022-04-15 23:47:37 +05:30
pukkandan 3b9d9f4374
Do not change fragment chunk-size when --test
Closes #3434
2022-04-15 17:05:56 +05:30
pukkandan f82711587c
[cleanup] Sort imports
Using https://github.com/PyCQA/isort

    isort -m VERTICAL_HANGING_INDENT --py 36 -l 80 --rr -n --tc .
2022-04-12 05:32:52 +05:30
pukkandan 86e5f3ed2e
[cleanup] Upgrade syntax
Using https://github.com/asottile/pyupgrade

1. `__future__` imports and `coding: utf-8` were removed
2. Files were rewritten with `pyupgrade --py36-plus --keep-percent-format`
3. f-strings were cherry-picked from `pyupgrade --py36-plus`

Extractors are left untouched (except removing header) to avoid unnecessary merge conflicts
2022-04-12 05:32:51 +05:30
pukkandan a44ca5a470
[cleanup] Misc fixes
Closes https://github.com/yt-dlp/yt-dlp/pull/3213, Closes https://github.com/yt-dlp/yt-dlp/pull/3117

Related: https://github.com/yt-dlp/yt-dlp/issues/3146#issuecomment-1077323114, https://github.com/yt-dlp/yt-dlp/pull/3277#discussion_r841019671, a825ffbffa (commitcomment-68538986), https://github.com/yt-dlp/yt-dlp/issues/2360, 5fa3c9a88f (r70393519), 5fa3c9a88f (r70393254)
2022-04-05 18:12:18 +05:30
Lesmiscore (Naoya Ozaki) d71fd41249
[fragment] Read downloaded fragments only when needed (#3069)
Authored by: Lesmiscore
2022-03-15 12:27:41 +09:00
Emanuel Hoogeveen 45806d44a7
[downloader] Obey --file-access-retries when deleting/renaming (#2224)
Authored by: ehoogeveen-medweb
2022-03-03 06:33:32 -08:00
Lesmiscore (Naoya Ozaki) 93c8410d33
[downloader/fragment] Fix bugs around resuming with Range (#2901)
Authored by: Lesmiscore
2022-02-28 13:10:54 +09:00
Lesmiscore 195c22840c
[downloader/fragment] Ignore FileNotFoundError when downloading livestreams
when `--live-from-start` is used for YouTube and the live ends, request for the last segment prematurely ends (or 404, 403).
this is causing lack of the file and `FileNotFoundError`
lacking segment doesn't have any data, so it's safe to ignore
2022-02-26 12:34:36 +09:00
Lesmiscore f0734e1190
[downloader/fragment] Fix problem where multiple threads can share one iterator
which causes "ValueError: generator already executing" error

Closes #2881
2022-02-25 13:22:17 +09:00
Lesmiscore (Naoya Ozaki) a539f06570
[downloader/fragment] Improve --live-from-start for YouTube livestreams (#2870) 2022-02-25 02:00:46 +09:00
pukkandan 1d3586d0d5
[aes] Add unpad_pkcs7 2022-02-01 00:29:36 +05:30
pukkandan 185bf31070
[youtube] End live-from-start properly when stream ends with 403
Closes #2089
2021-12-26 16:14:00 +05:30
Emanuel Hoogeveen 205a0654c0
Add option --file-access-retries (#2066)
Closes #517
Authored by: ehoogeveen-medweb
2021-12-23 07:59:03 +05:30
The Hatsune Daishi adbc4ec4bb
[dash,youtube] Download live from start to end (#888)
* Add option `--live-from-start` to enable downloading live videos from start
* Add key `is_from_start` in formats to identify formats (of live videos) that downloads from start
* [dash] Create protocol `http_dash_segments_generator` that allows a function to be passed instead of fragments
* [fragment] Allow multiple live dash formats to download simultaneously
* [youtube] Implement fragment re-fetching for the live dash formats
* [youtube] Re-extract dash manifest every 5 hours (manifest expires in 6hrs)
* [postprocessor/ffmpeg] Add `FFmpegFixupDuplicateMoovPP` to fixup duplicated moov atoms

Known issue: Ctrl+C doesn't work on Windows when downloading multiple formats

Closes #1521
Authored by: nao20010128nao, pukkandan
2021-12-20 11:36:46 +05:30
pukkandan 59a7a13ef9
[docs] Minor documentation improvements
Closes #1583, #1599
2021-11-10 04:19:52 +05:30
pukkandan bd93fd5d45
[fragment] Fix progress display in fragmented downloads
Closes #1517
2021-11-03 16:45:58 +05:30
pukkandan 49a57e70a9
[cleanup] misc 2021-10-23 02:09:10 +05:30
pukkandan b4b855ebc7
[fragment] Print error message when skipping fragment 2021-10-19 22:58:26 +05:30
pukkandan 2cda6b401d
Revert "[fragments] Pad fragments before decrypting (#1298)"
This reverts commit 373475f035.
2021-10-19 22:58:25 +05:30
shirt 373475f035
[fragments] Pad fragments before decrypting (#1298)
Closes #197, #1297, #1007
Authored by: shirt-dev
2021-10-18 08:14:20 +05:30
pukkandan 819e05319b
Improved progress reporting (See desc) (#1125)
* Separate `--console-title` and `--no-progress`
* Add option `--progress` to show progress-bar even in quiet mode
* Fix and refactor `minicurses`
* Use `minicurses` for all progress reporting
* Standardize use of terminal sequences and enable color support for windows 10
* Add option `--progress-template` to customize progress-bar and console-title
* Add postprocessor hooks and progress reporting

Closes: #906, #901, #1085, #1170
2021-10-09 00:41:59 +05:30
shirt 7687c8ac6e
[HLS] Fix decryption issues (#1117)
* Unpad HLS fragments with PKCS#7 according to datatracker.ietf.org/doc/html/rfc8216
* media_sequence should only be incremented in for media fragments
* The native decryption should only be used if ffmpeg is unavailable since it is significantly slower. Closes #1086

Authored by: shirt-dev, pukkandan
2021-09-29 00:23:24 +05:30
The Hatsune Daishi bd50a52b0d
Basic framework for simultaneous download of multiple formats (#1036)
Authored by: nao20010128nao
2021-09-22 19:42:04 +05:30
pukkandan 1009f67c2a
[fragment,aria2c] Generalize and refactor some code 2021-09-22 05:27:07 +05:30
pukkandan d9d8b85747
[fragment] Fix range header when using -N and media sequence (#1048)
Authored by: shirt
2021-09-22 04:19:45 +05:30
pukkandan daf7ac2b92
[fragment] Avoid repeated request for AES key 2021-09-22 01:15:16 +05:30
pukkandan edf65256aa
[hls,aes] Fallback to native implementation for AES-CBC
and detect `Cryptodome` in addition to `Crypto`

Closes #935
Related: #938
2021-09-18 00:55:58 +05:30
Felix S 25a3f4f5d6
[webvtt] Merge daisy-chained duplicate cues (#638)
Fixes: https://github.com/yt-dlp/yt-dlp/issues/631#issuecomment-893338552

Previous deduplication algorithm only removed duplicate cues with
identical text, styles and timestamps.  This change also merges
cues that come in ‘daisy chains’, where sequences of cues with
identical text and styles appear in which the ending timestamp of
one equals the starting timestamp of the next.

This deduplication algorithm has the somewhat unfortunate side effect
that NOTE blocks between cues, if found, will be emitted in a different
order relative to their original cues.  This may be unwanted if perfect
fidelity is desired, but then so is daisy-chain deduplication itself.
NOTE blocks ought to be ignored by WebVTT players in any case.

Authored by: fstirlitz
2021-08-10 01:52:30 +05:30
pukkandan ad3dc496bb Misc fixes - See desc
* Remove unnecessary uses of _list_from_options_callback
* Fix download tests - Bug from 6e84b21559
* Rename ExecAfterDownloadPP to ExecPP and refactor its tests
* Ensure _write_ytdl_file closes file handle on error - Potential fix for #517
2021-08-10 01:22:55 +05:30