Commit graph

15 commits

Author SHA1 Message Date
pukkandan 8c04f0be96 batch-file enumeration improvements (https://github.com/ytdl-org/youtube-dl/pull/26813)
Co-authored by: glenn-slayden
Modified from c9a9ccf8a3

These improvements apply to reading the list of URLs from the file supplied via the `--batch-file` (`-a`) command line option.

1. Skip blank and empty lines in the file. Currently, lines with leading whitespace are only skipped when that whitespace is followed by a comment character (`#`, `;`, or `]`). This means that empty lines and lines consisting only of whitespace are returned as (trimmed) empty strings in the list of URLs to process.

2. [bug fix] Detect and remove the Unicode BOM when the file descriptor is already decoding Unicode.

With Python 3, the `batch_fd` enumerator returns the lines of the file as Unicode. For UTF-8, this means that the raw BOM bytes from the file `\xef \xbb \xbf` show up converted into a single `\ufeff` character prefixed to the first enumerated text line.

This fix solves several buggy interactions between the presence of BOM, the skipping of comments and/or blank lines, and ensuring the list of URLs is consistently trimmed. For example, if the first line of the file is blank, the BOM is incorrectly returned as a URL standing alone. If the first line contains a URL, it will be prefixed with this unwanted single character--but note that its being there will have inhibited the proper trimming of any leading whitespace. Currently, the `UnicodeBOMIE` helper attempts to recover from some of these error cases, but this fix prevents the error from happening in the first place (at least on Python3). In any case, the `UnicodeBOMIE` approach is flawed, because it is clearly illogical for a BOM to appear in the (non-batch) URL(s) specified directly on the command line (and for that matter, on URLs *after the first line* of a batch list, also)

3. Adds proper trimming of the " #" into the read_batch_urls processing so that the URLs it enumerates are cleaned and trimmed more consistently.
2021-01-09 18:08:03 +05:30
pukkandan 00dd0cd573 Update to ytdl-2021.01.08 2021-01-08 21:59:10 +05:30
pukkandan c76eb41bb9 Preparing for release 2021-01-07 17:22:45 +05:30
pukkandan dbbbe555d7 Add duration_string to info_dict 2021-01-06 22:37:55 +05:30
pukkandan 735d865ece Disable Updates 2021-01-06 17:43:27 +05:30
pukkandan 91ebc64068 Change defaults
* Enabled --ignore by default
* Disabled --video-multistreams and --audio-multistreams by default
* Changed default format selection to 'bv*+ba/b' when --audio-multistreams is disabled
* Changed default format sort order to 'res,fps,codec,size,br,asr,proto,ext,has_audio,source,format_id'
* Changed default output template to '%(title)s [%(id)s].%(ext)s'
* Enabled `--list-formats-as-table` by default
2021-01-05 02:36:09 +05:30
pukkandan 29f7c58aaf Update to ytdl-2021.01.03 2021-01-05 00:02:27 +05:30
pukkandan 76d321f68f Option to present -F output to a more tabular form 2021-01-04 23:03:04 +05:30
pukkandan 732044afb2 Add --write-*-link by h-h-h-h
Authored-by: h-h-h-h
2020-12-13 20:05:04 +05:30
pukkandan a0566bbf5c Updated to release 2020.11.21.1 2020-11-22 03:51:26 +05:30
pukkandan 8bdd16b499 Merge 'ytdl-org/youtube-dl/master' release 2020.11.19
Old Extractors left behind:
	VLivePlaylistIE
	YoutubeSearchURLIE
	YoutubeShowIE
	YoutubeFavouritesIE

If removing old extractors, make corresponding changes in
	docs/supportedsites.md
	youtube_dlc/extractor/extractors.py

Not merged:
	.github/ISSUE_TEMPLATE/1_broken_site.md
	.github/ISSUE_TEMPLATE/2_site_support_request.md
	.github/ISSUE_TEMPLATE/3_site_feature_request.md
	.github/ISSUE_TEMPLATE/4_bug_report.md
	.github/ISSUE_TEMPLATE/5_feature_request.md
	test/test_all_urls.py
	youtube_dlc/version.py
	Changelog
2020-11-20 13:03:32 +05:30
Unknown 7fb5f2f29d [skip travis] templates and urls 2020-10-31 09:26:04 +01:00
Unknown b6e0c7d2e3 [mtv] fix mtv.com and more(?) 2020-10-09 07:06:49 +02:00
Unknown 39e7107d7f Merge branch 'ytdl-org-master' 2020-09-12 05:08:57 +02:00
Unknown cefecac12c [skip travis] renaming
to avoid using same folder when using pip install for example
2020-09-02 20:25:25 +02:00
Renamed from youtube_dl/utils.py (Browse further)