Commit Graph

10500 Commits

Author SHA1 Message Date
Sergey M․ 651ad35ce0
[tagesschau] Relax _VALID_URL 2016-05-01 06:57:19 +06:00
Sergey M․ 6a0f9a24d0
[tagesschau] Separate player extractor 2016-05-01 06:45:44 +06:00
remitamine 9cf79e8f4b [ccc] improve extraction 2016-05-01 01:45:17 +01:00
Sergey M․ 2844b09336
[tagesschau] Fix article media ids 2016-05-01 04:42:05 +06:00
Sergey M․ 1a2b377cc2
[tagesschau] Fix audio support 2016-05-01 04:38:46 +06:00
Sergey M․ 4c1b2e5c0e
[tagesschau] Add support for playlists 2016-05-01 04:18:56 +06:00
Sergey M․ 9e1b96ae40
[rtlnl] Match formats only by height 2016-05-01 03:20:36 +06:00
Sergey M․ fc35cd9e0c
[tagesschau] Relax _VALID_URL 2016-05-01 02:56:32 +06:00
Sergey M․ 339fe7228a
[tagesschau] Update _FORMATS map 2016-05-01 02:56:32 +06:00
remitamine ea7e7fecbd [discovery] remove unused imports 2016-04-30 21:55:28 +01:00
remitamine d00b93d58c [discovery] extract more info using BrightcoveNewIE 2016-04-30 21:49:32 +01:00
remitamine 93f7a31bf3 [discovery] extract subtitle 2016-04-30 20:51:32 +01:00
remitamine 33a1ec950c [discovery] extract http formats 2016-04-30 20:51:32 +01:00
Sergey M․ 4e0c0c1508
[xiami] Improve extraction (Closes #9079)
* Switch to JSON source
* Add abstract IE for playlists
* Extract more track related metadata
2016-04-30 21:50:23 +06:00
BlahGeek 89c0dc9a5f
[xiami] Add xiami extractor 2016-04-30 21:48:40 +06:00
remitamine f628d800fb [ted] add support for youtube embeds and update tests 2016-04-30 16:34:57 +01:00
remitamine 11fa3d7f99 [ted] extract all http formats 2016-04-30 15:44:30 +01:00
Sergey M․ d41ee7b774
[vlive] Pass Referer as bytestring (Closes #9352) 2016-04-30 19:22:42 +06:00
remitamine e0e9bbb0e9 [pbs] extract srt and vtt subtitles 2016-04-30 14:02:17 +01:00
remitamine 7691184a31 [pbs] remove duplicate format 2016-04-30 12:57:30 +01:00
remitamine 35cd2f4c25 [pbs] extract only the formats that we know that they will be available as http format
https://projects.pbs.org/confluence/display/coveapi/COVE+Video+Specifications
2016-04-30 11:32:13 +01:00
remitamine 350d7963db [pbs] fix the least bitrate http url construction 2016-04-30 11:12:11 +01:00
remitamine cbc032c8b7 [pbs] extract all http formats 2016-04-30 01:24:36 +01:00
remitamine 69c4cde4ba [wsj] improve extraction 2016-04-29 21:37:05 +01:00
Sergey M․ ca278a182b
[rtlnl] Replace test 2016-04-30 02:07:29 +06:00
Sergey M․ 373e1230e4
[rtlnl] Clarify tests 2016-04-30 01:50:26 +06:00
Sergey M․ cd63d091ce
[rtlnl] Fix tests 2016-04-30 01:48:14 +06:00
Sergey M․ 0571ffda7d
[rtlnl] Improve extraction (Closes #9329)
* Make hls extraction non fatal and revert ext
* Extract progressive formats' metadata from corresponding hls formats
2016-04-30 01:43:39 +06:00
Reino17 5556047465
[rtlnl] Update 720p PG_URL_TEMPLATE
- Fixed the format_id for the 720p progressive videostream and added the video's resolution.
- The adaptive videostreams have the m3u8-extension, so I removed the confusing mp4-extension in order to make a better distinction between the these and the progressive videostreams.
2016-04-30 01:43:13 +06:00
remitamine 65a3bfb379 [dfb] extract m3u8 formats 2016-04-29 19:21:17 +01:00
Yen Chi Hsuan cef3f3011f
[funimation] Detect blocking and support CloudFlare cookies 2016-04-30 00:17:09 +08:00
Yen Chi Hsuan e9c6cdf4a1
[common] Fix format_id construction for HLS 2016-04-29 22:50:16 +08:00
Sergey M․ 00a17a9e12
[crunchyroll] Sort formats 2016-04-29 19:44:10 +06:00
Sergey M․ 8312b1a3d1
[crunchyroll] Add even more relaxed fmt fallback 2016-04-29 19:43:53 +06:00
Sergey M․ 6ff4469528
[crunchyroll] Relax fmt regex 2016-04-29 19:39:27 +06:00
Yen Chi Hsuan 68835d687a
Merge branch 'Kagami-vlive-hls' 2016-04-29 19:30:51 +08:00
Yen Chi Hsuan 9d186afac8
[vlive] Coding style and PEP8 2016-04-29 19:29:50 +08:00
Yen Chi Hsuan 151d98130b Merge branch 'vlive-hls' of https://github.com/Kagami/youtube-dl into Kagami-vlive-hls 2016-04-29 19:26:39 +08:00
Kagami Hiiragi b24d6336a7 [vlive] Add support for live videos 2016-04-29 14:22:50 +03:00
remitamine 065216d94f [crunchyroll] reduce requests for formats extraction 2016-04-29 11:46:42 +01:00
remitamine 67167920db [viewlift] replace SnagFilms extractors
- add support for other sites that use the same logic
- improve format extraction and sorting
2016-04-29 11:24:10 +01:00
Yen Chi Hsuan 14638e2915
[sexykarma] Rename to WatchIndianPornIE and fix extraction 2016-04-29 18:17:08 +08:00
Yen Chi Hsuan 1910077ed7
Revert "[sexykarma] Remove the extractor"
This reverts commit 31ff3c074e.
2016-04-29 17:59:23 +08:00
Yen Chi Hsuan 5819edef03
[ooyala] Skip an invalid test
Ooyala is used by lots of extractors and its correctness can be verified
by these websites.
2016-04-29 14:27:15 +08:00
Yen Chi Hsuan f5535ed0e3
[orf] Skip the expired test 2016-04-29 14:24:07 +08:00
Yen Chi Hsuan 31ff3c074e
[sexykarma] Remove the extractor
Its domain name is on sale.

Closes #9317
2016-04-29 13:36:52 +08:00
Sergey M․ 72670c39de
[arte:+7] Fix typo in _VALID_URL 2016-04-29 04:46:23 +06:00
Sergey M․ 683d892bf9
[viewster] Remove unused import 2016-04-29 01:30:53 +06:00
Sergey M․ 497971cd4a
[yandexmusic] Clarify blockage even more 2016-04-29 01:28:07 +06:00
remitamine e757fb3d05 [crunchyroll] improve extraction
- extract more metadata(series, episode, episode_number)
- reduce duplicate requests for extracting formats
- remove duplicate formats
2016-04-28 18:42:20 +01:00
remitamine 0ba9e3ca22 [viewster] extract formats for videos with multiple audios/subtitles 2016-04-28 17:45:09 +01:00
Sergey M․ 4b53762914
[yandexmusic] Clarify blockage 2016-04-28 21:45:33 +06:00
Sergey M․ eebe6b382e
[yandexmusic] Improve error handling 2016-04-28 21:37:34 +06:00
Yen Chi Hsuan 0cbcbdd89d
[nuvid] Fix extraction
Closes #7620
2016-04-28 17:51:20 +08:00
Yen Chi Hsuan 7f776fa4b5
[yandexmusic] Skip tests as Travis CI blocked 2016-04-28 17:08:41 +08:00
Yen Chi Hsuan a5941305b6
[mwave] Coding style 2016-04-28 16:03:08 +08:00
Yen Chi Hsuan f8dddaf456 Merge branch 'mwave-meetgreet' of https://github.com/pmrowla/youtube-dl into pmrowla-mwave-meetgreet 2016-04-28 15:56:32 +08:00
Yen Chi Hsuan 618c71dc64
[cloudy] New domain name for the test_cloudy_1
I'm sure whether videoraj.ch still works or not, so keep it.
2016-04-28 15:46:00 +08:00
Sergey M․ 52af8f222b
[cwtv] Relax _VALID_URL (Closes #9327) 2016-04-28 04:01:21 +06:00
Yen Chi Hsuan 3cc8649c9d
[20min] Detect embedded YouTube videos
Fixes #9331
2016-04-28 02:58:11 +08:00
Yen Chi Hsuan dcf094d626
[theplatform] Fix for Python 3.2
test_AENetworks{,_1} fails as in Python < 3.3, binascii.a2b_* functions
accepts only bytes-like objects
2016-04-27 18:35:33 +08:00
Peter Rowlands 5b5d7cc11e [mwave] Add Mwave Meet & Greet extractor 2016-04-27 15:57:17 +09:00
Yen Chi Hsuan 2ac2cbc0a3
[malemotion] Remove the extractor
Announcement from their homepage:

```
MaleMotion is closed

After another system crash, I'm forced to close the site

This week all content will be erased

Don't forget to cancel your subscription if any !
```

Closes #9311.
2016-04-27 13:55:32 +08:00
Yen Chi Hsuan a7e03861e8
[scivee] Skip the test
Not accessible from either Travis CI or my machine.

Closes #9315
2016-04-27 13:52:04 +08:00
Sergey M․ 175c2e9ec3
[youtube:search_url] Reimplement in terms of youtube:playlistbase 2016-04-26 22:29:29 +06:00
remitamine f1f879098a [viewster] extract more metadata for http formats 2016-04-26 13:40:40 +01:00
Sergey M․ c9fd530670
[ok] Extract start time 2016-04-25 22:15:15 +06:00
Sergey M․ 749b0046a8
[ok] Allow embeds without title (Closes #9303) 2016-04-25 22:05:47 +06:00
Yen Chi Hsuan e3de3d6f2f
[normalboots] Fix extraction
Now it's using ScreenwaveMedia
2016-04-25 23:49:12 +08:00
Yen Chi Hsuan ad58942d57
[muzu] Remove extractor
MUZU is shutting down in October 2015. [1]

[1] http://www.musicbusinessworldwide.com/youtube-rival-muzu-is-heading-into-liquidation/
2016-04-25 23:35:05 +08:00
Yen Chi Hsuan 4645432d7a
[eagleplatform] Checking direct HTTP links
Sometimes they fail with 404
2016-04-25 22:48:17 +08:00
Yen Chi Hsuan 6bdc2d5358
[mitele] Comment out unstable MD5
Also Akamai f4f fragments
2016-04-25 22:27:25 +08:00
Yen Chi Hsuan 2beff95da5
[nrk] Comment out unstable MD5 checksums
Both are Akamai f4f fragments.
2016-04-25 22:26:19 +08:00
Yen Chi Hsuan abc1723edd
[unistra] Sort formats
Originally URLs are passed to set() and not sorted, so the result is not
deterministic, causing occasional FAILs on Travis CI.
2016-04-25 22:24:40 +08:00
Yen Chi Hsuan d6712378e7
Merge branch 'akamai_pv' of https://github.com/remitamine/youtube-dl into remitamine-akamai_pv 2016-04-25 21:02:02 +08:00
remitamine fb72ec58ae [extractor/common] do not process f4m manifest that contain akamai playerVerificationChallenge 2016-04-25 13:37:03 +01:00
Sergey M․ c83a352227
[openload] Make thumbnail optional 2016-04-25 00:26:06 +06:00
Sergey M․ e9063b5de9
[openload] Add test 2016-04-25 00:22:55 +06:00
Sergey M․ 594b0c4c69
[openload] Fix ext extraction 2016-04-25 00:03:29 +06:00
Sergey M․ eb9ee19422
[utils] Allow None mimetypes in mimetype2ext 2016-04-25 00:03:12 +06:00
Sergey M․ a1394b820d
[openload] Fix title extraction (Closes #9298) 2016-04-25 00:01:37 +06:00
Yen Chi Hsuan aa9dc24f5a
[douyutv] Improve extraction and update tests
The JSON API sometimes return HTML pages with errors
2016-04-24 23:52:17 +08:00
Yen Chi Hsuan 51762e1a31
[xminus] Fix extraction (closes #9228) 2016-04-24 23:21:45 +08:00
Philipp Hagemeister 8b38f2ac40 release 2016.04.24 2016-04-24 17:06:46 +02:00
Yen Chi Hsuan a82398bd72
[kwuo:song] Fix extraction and update the test 2016-04-24 22:20:45 +08:00
remitamine c14dc00df3 [viewster] improve http formats extraction 2016-04-24 14:34:28 +01:00
Yen Chi Hsuan 03dd60ca41
[kuwo:category] Fix the test
Sometimes there are 24 songs and sometimes 30 lol
2016-04-24 21:16:06 +08:00
Yen Chi Hsuan 0738187f9b
[ThePlatform] Fix tests failed since 79ba9140dc 2016-04-24 20:46:06 +08:00
Yen Chi Hsuan a956cb6306
[onionstudios] Fix description extraction
\1 does not work in []. Fixes test_Generic_75
(http://www.clickhole.com/video/dont-understand-bitcoin-man-will-mumble-explanatio-2537)
2016-04-24 20:41:17 +08:00
Yen Chi Hsuan a8062eabcd
[mwave] Skip checking unstable MD5
On my PC the checksum is 02eda6d09fb63131a17a8d44e6237463, while a
recent Travis CI build
(https://travis-ci.org/rg3/youtube-dl/jobs/125341081) shows it's
c930e27b7720aaa3c9d0018dfc8ff6cc
2016-04-24 20:05:24 +08:00
Yen Chi Hsuan 2a7dee8cc5
[yahoo] Improve error detection and update tests 2016-04-24 18:12:16 +08:00
Yen Chi Hsuan d9ed362116
[yahoo] Extract all <iframe>s
Fixes test_yahoo_6

(https://ca.finance.yahoo.com/news/hackers-sony-more-trouble-well-154609075.html)
2016-04-24 17:46:25 +08:00
Yen Chi Hsuan 4f54958097
[yahoo] Update some tests
One has new fields as ThePlatformIE changed, and others have changed
files.
2016-04-24 17:29:01 +08:00
Yen Chi Hsuan 2a7c38831c
[yahoo] Extend _VALID_URL and fix extraction
Closes #9271
2016-04-24 17:01:18 +08:00
Yen Chi Hsuan 949b6497cc
[generic] Unescape the video URL
Fixes #9279
2016-04-24 16:25:37 +08:00
remitamine fda9a1ca9e [viewster] simplify qualities_basename regex 2016-04-24 03:06:46 +01:00
remitamine 864d5e7231 [viewster] extract all http formats 2016-04-24 02:32:56 +01:00
Sergey M․ 5448b781f6
[dplay] Sign unsigned final download hls URLs 2016-04-23 17:28:45 +06:00
Sergey M․ e239413fbc
[dplay] Extract subtitles (Closes #9284) 2016-04-23 16:50:31 +06:00
Sergey M․ fd0ff8bad8
[dplay] Improve extraction and document workarounds and tests 2016-04-23 16:36:17 +06:00
Sergey M․ 397ec446f3
[dplay] Try secure api for no tld (Closes #9282) 2016-04-23 15:59:30 +06:00
remitamine 29a7e8f6f8 [nhl] Add new extractor(closes #8419)(closes #8798) 2016-04-22 20:18:27 +01:00
Yen Chi Hsuan eb01e97e10
[youku] Skip streams with channel_type=tail
Fixes #9275

These video segments look like ads and they don't appear in the web
player.
2016-04-23 02:54:09 +08:00
remitamine cb7d4d0efd [nbc] add support for today.com(closes #2909) 2016-04-22 18:08:20 +01:00
Yen Chi Hsuan c80037918b
[iqiyi] Improve error detection (#9276) 2016-04-23 00:06:49 +08:00
remitamine 237a41108a [eagleplatform] extract all http formats 2016-04-22 14:32:38 +01:00
remitamine e962ae15d3 [newstube] extract http formats(closes #9253) 2016-04-22 11:26:43 +01:00
remitamine 7c36ea7d54 [rtbf] improve extraction(fixes #9267) 2016-04-21 22:52:49 +01:00
remitamine 9260cf1d97 [tubitv] fix extraction(closes #8741) 2016-04-21 20:30:19 +01:00
Sergey M․ bdbb8530c7
[vimeo] Pass Referer for check-password request 2016-04-22 00:02:39 +06:00
Sergey M․ 09a9fadb84
[dump] Remove extractor 2016-04-21 23:31:34 +06:00
Sergey M․ bf09af3acb
Add --hls-prefer-ffmpeg 2016-04-21 23:02:17 +06:00
Sergey M․ 88296ac326
[planetaplay] Remove remainings of extractor 2016-04-21 22:57:38 +06:00
Sergey M․ 870d525848
[options] Remove experimental mark for --hls-prefer-native 2016-04-21 22:44:01 +06:00
Sergey M․ 6577112890
[planetaplay] Remove extractor (Closes #9256) 2016-04-21 22:33:54 +06:00
Sergey M․ 1988647dda
[tvigle] Skip hls completely (#9259) 2016-04-21 22:15:20 +06:00
Yen Chi Hsuan a292cba256
[mgtv] Fix _VALID_URL and add localized name 2016-04-22 00:07:43 +08:00
Yen Chi Hsuan 982e518a96
[dispeak] Rename DigitalSpeaking to DigitallySpeaking 2016-04-22 00:07:43 +08:00
Yen Chi Hsuan 748e730099
[dispeak] Several fixes 2016-04-22 00:07:43 +08:00
Sergey M b6c0d4f431 Merge pull request #9110 from remitamine/parse_duration
[utils] imporove parse_duration to handle more formats
2016-04-21 22:53:16 +07:00
remitamine acaff49575 [utils] imporove parse_duration to handle more formats 2016-04-21 16:34:54 +01:00
Yen Chi Hsuan 1da19488f9
[mgtv] Add new extractor (closes #9212) 2016-04-21 23:29:51 +08:00
Yen Chi Hsuan 442c4d361f
[dispeak/gdcvault] Add the test case from #5784 2016-04-21 19:47:10 +08:00
Yen Chi Hsuan ec59d657e7
[dispeak] Add new extractor
Both GDCVault and GPUTechConf uses the service of DigitalSpeaking.
2016-04-21 19:36:33 +08:00
Yen Chi Hsuan 99ef96f84c
[gdcvault] Fix for videos with hard-coded hostnames
Fixes #9248
2016-04-21 18:07:03 +08:00
Yen Chi Hsuan 4dccea8ad0
[streetvoice] Fix extraction
The old API results in URLs with HTTP 403 from time to time.

Hopefully fixes #9219.
2016-04-21 13:07:53 +08:00
Yen Chi Hsuan 2c0d9c6217
[extractor/common] Allow empty post data 2016-04-21 13:06:06 +08:00
Sergey M․ 12a5134596
[tvigle] Fix extraction (Closes #9259) 2016-04-20 23:52:41 +06:00
Sergey M․ 16e633a5d7
[quickvid] Remove extractor (Closes #9258) 2016-04-20 23:29:02 +06:00
Sergey M․ 494ab6db73
[youtube] Capture and output login error message 2016-04-20 22:14:32 +06:00
Sergey M․ 107701fcfc
[people] Remove bogus comment 2016-04-20 03:40:02 +06:00
Sergey M․ f77970765a
[people] Add extractor 2016-04-20 03:37:23 +06:00
Philipp Hagemeister 81215d5652 release 2016.04.19 2016-04-19 03:03:52 +02:00
Sergey M․ 241a318f27
[vimeo] Improve _VALID_URL (Closes #9229) 2016-04-18 21:40:28 +06:00
Sergey M․ 4fdf082375
[theonion] Remove extractor (Closes #9220)
It now uses generic onionstudios embed
2016-04-17 23:12:23 +06:00
Jaime Marquínez Ferrándiz 1b6182d8f7 [youtube:playlist] Fetch all the videos in a mix (fixes #3837)
Since there doesn't seem to be any indication, it stops when there aren't new videos in the webpage.
2016-04-17 17:07:57 +02:00
remitamine 7bab22a402 [vice] remove unused import and variable 2016-04-17 14:06:19 +01:00
Yen Chi Hsuan 0f97fb4d00
[musicplayon] Relax _VALID_URL and improve metadata extraction
In r'pl=\d+&play=\d+' pages, several metadata items are missing

Closes #9222.
2016-04-17 17:24:33 +08:00
Yen Chi Hsuan b1cf58f48f
[musicplayon] Fix extraction (closes #9222) 2016-04-17 15:08:51 +08:00
remitamine 3014b0ae83 Merge pull request #9195 from remitamine/ffmpeg-pipe
[downloader/external] enable piping for FFmpegFD(closes #2124)
2016-04-16 22:00:49 +01:00
remitamine b9f2fdd37f [ffmpeg] Clarify rationale for pipe(-) exclusion in _ffmpeg_filename_argument 2016-04-16 21:50:13 +01:00
remitamine bbb3f730bb [onionstudios] extract m3u8 formats 2016-04-16 20:53:13 +01:00
remitamine d868f43c58 [ffmpeg] check for - file name in _ffmpeg_filename_argument 2016-04-16 19:45:56 +01:00
Yen Chi Hsuan 21525bb8ca
[kuwo:category] Update the test
Now the webpage says there are 24 songs.
2016-04-17 02:38:05 +08:00
Sergey M․ d8f103159f
[nerdist] Remove extractor
It now uses brightcove
2016-04-17 00:16:31 +06:00
remitamine 663ee5f0a9 [vice] extract youtube embed 2016-04-16 17:49:39 +01:00
Sergey M․ b6b950bf58
[cbs] Remove unused import 2016-04-16 22:47:10 +06:00
Sergey M․ 11e60fcad8
[extractor/generic] Improve instagram embeds (Closes #9213) 2016-04-16 22:39:20 +06:00
Sergey M․ c23533a100
[instagram] Add support for iframe embeds 2016-04-16 22:31:05 +06:00
Sergey M․ 0dafea02e6
[instagram] Add support for embed URLs 2016-04-16 22:23:08 +06:00