Commit graph

7997 commits

Author SHA1 Message Date
Kagami Hiiragi 25bcd3550e [vlive] Address site update
Changes:
* Fix video params extraction
* Don't make status request since status info now available on the page
* Remove unneeded code
* Fix test
2016-05-24 12:54:28 +03:00
remitamine a4760d204f [ooyala] use api v2 to reduce requests for format extraction 2016-05-24 00:22:29 +01:00
remitamine e8593f346a [ooyala] extract subtitles 2016-05-23 23:58:16 +01:00
remitamine 05b651e3a5 [washingtonpost] reduce requests for m3u8 manifests 2016-05-23 13:04:50 +01:00
remitamine 42a7439717 [cbs] allow to pass content id to the extractor(closes #9589) 2016-05-23 09:31:37 +01:00
remitamine b1e9ebd080 [washingtonpost] remove unnecessary code 2016-05-23 02:30:12 +01:00
remitamine 0c50eeb987 [reuters] Add new extractor 2016-05-23 02:27:31 +01:00
remitamine 4b464a6a78 [washingtonpost] improve format extraction and add support for video pages extraction 2016-05-23 00:48:11 +01:00
Sergey M․ 5db9df622f
[life:embed] Use native hls 2016-05-23 04:22:09 +06:00
Sergey M․ 5181759c0d
[life] Update _VALID_URL 2016-05-23 04:00:08 +06:00
Sergey M․ e54373204a
[lifenews] Fix metadata extraction 2016-05-23 03:44:04 +06:00
remitamine 102810ef04 [voxmedia] fix volume embed extraction 2016-05-22 20:37:35 +01:00
Yen Chi Hsuan 78d3b3e213
[generic] Improve Livestream detection (closes #2234) 2016-05-23 01:40:11 +08:00
Yen Chi Hsuan 7a46542f97
[livestream] Video IDs should always be strings (#2234) 2016-05-23 01:40:11 +08:00
remitamine db3b8b2103 [tf1] add support for more related web sites 2016-05-22 17:03:17 +01:00
remitamine c5f5155100 [wat] extract all formats 2016-05-22 17:03:17 +01:00
Yen Chi Hsuan 4a12077855
[genric] Eliminate duplicated video URLs (closes #6562) 2016-05-22 22:23:20 +08:00
Thor77 70346165fe [bandcamp] raise ExtractorError when track not streamable (#9465)
* [bandcamp] raise ExtractorError when track not streamable

* [bandcamp] update md5 for second test

* don't rely on json-data, but just check for 'file'

* don't rely on presence of 'file'
2016-05-22 14:15:39 +08:00
Sergey M․ 4d8819d249
[extractor/generic] Add support for theplatform embeds (Closes #8636, closes #9476) 2016-05-22 06:52:39 +06:00
Sergey M․ 898f4b49cc
[theplatform] Add _extract_urls 2016-05-22 06:47:22 +06:00
Sergey M․ 0150a00f33
[cc] Add test for tosh.cc (Closes #9566) 2016-05-22 02:58:41 +06:00
TRox1972 c8831015f4
[ComedyCentral] Add support for tosh.cc.com and cc.com/video-clips 2016-05-22 02:55:10 +06:00
Sergey M․ 92d221ad48
[periscope] Update uploader_id (Closes #9565) 2016-05-22 02:39:15 +06:00
Sergey M․ 0db9a05f88
[periscope:user] Adapt to layout changes (Closes #9563) 2016-05-22 02:15:56 +06:00
Jaime Marquínez Ferrándiz 4c718d3c50 [rtve] Recognize 'filmoteca' URLs 2016-05-21 17:37:35 +02:00
Yen Chi Hsuan 115c65793a
[jwplatform] Don't fail with RTMP URLs without mp4:, mp3: or flv: 2016-05-21 13:50:38 +08:00
Yen Chi Hsuan 661d46b28f
[cbslocal] Add new extractor (closes #9522) 2016-05-21 13:40:45 +08:00
Yen Chi Hsuan 5ce3d5bd1b
[sendtonews] Add new extractor
Used in CBSLocal. Part of #9522
2016-05-21 13:39:42 +08:00
Yen Chi Hsuan 612b5f403e
[jwplatform] Improved m3u8 and rtmp support
Changes made for SendtoNewsIE. Part of #9522
2016-05-21 13:38:01 +08:00
Yen Chi Hsuan 9f54e692d2
[anvato] Add new extractor
Used in CBSLocal (#9522)
2016-05-21 13:18:29 +08:00
Yen Chi Hsuan 7b2fcbfd4e
[common] Skip TYPE=CLOSED-CAPTIONS lines in m3u8 manifests
According to [1], valid values for TYPE are AUDIO, VIDEO, SUBTITLES
and CLOSED-CAPTIONS. Such a value is found in Anvato master playlists,
though I don't use _extract_m3u8_formats() in the end.

Part of #9522.

[1] https://tools.ietf.org/html/draft-pantos-http-live-streaming-19#section-4.3.4.1
2016-05-21 13:16:28 +08:00
Yen Chi Hsuan 16da9bbc29
[common] Add _m3u8_meta_format() template
For extractors who handle m3u8 manifests by themselves. (eg., AnvatoIE)

Part of #9522
2016-05-21 13:15:28 +08:00
Sergey M․ c8602b2f9b
[nrk] Unquote subtitles' URLs 2016-05-21 05:09:16 +06:00
Sergey M․ b219f5e51b
[brightcove:new] Improve error reporting 2016-05-21 00:59:06 +06:00
Sergey M․ 1846e9ade0
[localnews8] Fix extractor (Closes #9539) 2016-05-20 22:31:08 +06:00
TRox1972 6756602be6
[LocalNews8] add extractor (Closes #9200) 2016-05-20 22:10:13 +06:00
Sergey M․ 6c114b1210
[extractor/generic] Remove generic id and title from wistia extractionand update tests 2016-05-20 21:55:35 +06:00
Sergey M․ 7ded6545ed
[extractor/generic] Add test for wistia standard embed 2016-05-20 21:43:36 +06:00
Sergey M․ aa5957ac49
[extractor/generic] Add support for async wistia embeds (Closes #9549) 2016-05-20 21:33:31 +06:00
remitamine 64413f7563 [cbc] fix extraction for flv only videos(fixes #5309) 2016-05-20 16:21:23 +01:00
Sergey M․ 45f160a43c
[wistia] Improve hls support 2016-05-20 21:16:08 +06:00
Sergey M․ 36ca2c55db
[wistia] Skip storyboard and improve extraction 2016-05-20 21:04:01 +06:00
Sergey M․ f0c96af9cb
[wistia] Add alias and modernize 2016-05-20 20:55:10 +06:00
Yen Chi Hsuan 31a70191e7
[cbc] Add the test case from #5156 2016-05-20 19:04:50 +08:00
Yen Chi Hsuan ad96b4c8f5
[common] Extract audio formats in SMIL
Found in http://www.cbc.ca/player/play/2657631896

Closes #5156
2016-05-20 19:02:53 +08:00
Yen Chi Hsuan 043dc9d36f
[cbc] Fix for old-styled URLs
The URL http://www.cbc.ca/player/News/ID/2672225049/ (#6342) redirects
to http://www.cbc.ca/player/play/2672224672, while youtube-dl wasn't
able to handle it correctly.
2016-05-20 18:39:54 +08:00
remitamine 52f7c75cff [cbc] extract http formats and update tests 2016-05-20 06:58:46 +01:00
Sergey M․ f6e588afc0
[24video] Fix description extraction 2016-05-20 08:53:04 +06:00
remitamine a001296703 [learnr] Add new extractor(closes #4284) 2016-05-19 18:18:03 +01:00
Sergey M․ dd81769c62
[ndtv] Fix extraction 2016-05-19 04:34:19 +06:00
remitamine b78531a36a [formula1] Add new extractor(closes #3617) 2016-05-18 22:24:46 +01:00
Sergey M․ 11e6a0b641
[nfb] Modernize and extract subtitles 2016-05-18 00:25:15 +06:00
Sergey M․ 15cda1ef77
[nfb] Fix uploader extraction 2016-05-17 23:46:47 +06:00
Yen Chi Hsuan 055f0d3d06
[abcnews] Added a new extractor (closes #3992)
Related: #6108, #8664, #9459
2016-05-17 15:38:57 +08:00
Sergey M․ f7199423e5
[groupon] Add support for Youtube embeds (Closes #9508) 2016-05-16 00:30:13 +06:00
Sergey M․ a0a81918f1
[collegehumor] Remove extractor
It now uses brightcove
2016-05-15 22:07:51 +06:00
Yen Chi Hsuan 5572d598a5
[hearthisat] Update the first test 2016-05-15 15:44:04 +08:00
Yen Chi Hsuan cec9727c7f
[hearthisat] Detect invalid download links (fixes #9440) 2016-05-15 15:35:31 +08:00
Sergey M․ 69c9cc2716
[xvideos] Extract html5 player formats (Closes #9495) 2016-05-15 03:38:04 +06:00
Sergey M․ ed56f26039
[extractor/common] Improve name extraction for m3u8 formats 2016-05-15 03:34:35 +06:00
Sergey M․ 6f41b2bcf1
[extractor/generic] Improve 3qsdn embeds support (Closes #9453) 2016-05-14 23:58:25 +06:00
Sergey M․ 5d39176f6d
[extractor/generic:3qsdn] Add support for embeds 2016-05-14 23:40:34 +06:00
Sergey M․ 5c86bfe70f
[3qsdn] Add extractor 2016-05-14 23:35:03 +06:00
Sergey M․ ca950f49e9
[ora] Revert extraction to regexes
It's less fragile than using js_to_json with ora js
2016-05-14 20:45:18 +06:00
felix 640eea0a0c
[ora] minimise fragile regex shenanigans; recognise unsafespeech.com URLs 2016-05-14 20:13:06 +06:00
Jakub Wilk 791ff52f75 [teamcoco] Fix base64 regexp 2016-05-14 13:19:54 +02:00
Sergey M․ b5abf86148
[cinemassacre] Remove extractor (Closes #9457)
It now uses jwplatform
2016-05-14 04:53:14 +06:00
remitamine 0730be9022 [sina] fix extraction(fixes #1146) 2016-05-13 20:25:01 +01:00
Sergey M․ 96c2e3e909
[imdb] Improve extraction 2016-05-13 23:25:05 +06:00
Sergey M․ f196508f7b
[imdb] Relax _VALID_URL (Closes #9481) 2016-05-13 22:19:00 +06:00
Yen Chi Hsuan cc1028aa6d
[openload] Fix extraction (closes #9472) 2016-05-13 18:11:08 +08:00
remitamine ad55e10165 [brightcove] change the protocol for m3u8 formats to m3u8_native 2016-05-13 08:35:38 +01:00
remitamine 18cf6381f6 [nrk] extract m3u8 formats 2016-05-13 08:05:28 +01:00
remitamine cdf32ff15d [extractors] add import for UstudioEmbedIE 2016-05-13 05:25:32 +01:00
remitamine 99d79b8692 [ustudio] add support ustudio app/embed urls 2016-05-13 05:21:45 +01:00
remitamine b9e7bc55da [mgtv] extract http formats 2016-05-12 22:46:23 +01:00
Sergey M․ d8d540cf0d
[nrk] Rework extractor (Closes #9470) 2016-05-13 02:07:12 +06:00
Sergey M․ 0df79d552a
[twitch:bookmarks] Remove extractor
Bookmarks no longer available
2016-05-13 00:14:30 +06:00
Sergey M․ 0db3a66162
[twitch] Skip dead tests 2016-05-12 23:57:52 +06:00
Yen Chi Hsuan a3fa6024d6
[bloomberg] Fix test_Bloomberg
In this test case, sometimes HLS is the best format while sometimes HDS
is. To prevent occasional test failures, force HDS to be the best
format. In the past, testing against HDS formats causes the same error
as #9214, which is fixed as #9377 landed.
2016-05-12 20:08:42 +08:00
Yen Chi Hsuan 7e8ddca1bb
[vevo] Delay the georestriction check to prevent false alerts
Fixes #9408
2016-05-12 19:56:58 +08:00
Yen Chi Hsuan 4540515cb3
[iqiyi] Fix 1080P extraction (closes #9446) 2016-05-12 18:48:27 +08:00
Yen Chi Hsuan edaa23f822
[compat] Rename struct_(un)pack to compat_struct_(un)pack 2016-05-10 14:51:38 +08:00
Yen Chi Hsuan dab0daeeb0
[utils,compat] Move struct_pack and struct_unpack to compat.py 2016-05-10 14:51:38 +08:00
Sergey M․ 3951e7eb93
[ceskatelevize] Simplify, restore bonus video test and skip georestricted test (Closes #9431) 2016-05-09 20:37:20 +06:00
Tomáš Čech f1f6f5aa5e
[ceskatelevize] Add support for live streams
Live streams has no playlist title, use title of the stream containing
TV channel name. Internal m3u8 handler doesn't seem to handle well
continuous streams. Add test for live stream. Remove no longer
reachable test.
2016-05-09 18:58:15 +06:00
Sergey M․ c52f4efaee
[mva] Improve _VALID_URLs 2016-05-08 20:10:20 +06:00
Sergey M․ f23a92a0ce
[mva] Add extractor (Closes #6667) 2016-05-08 20:02:54 +06:00
Yen Chi Hsuan 3b01a9fbb6
[litv] Add new extractor
LiTV is a streaming platform providing free and paid legal contents in
Taiwan.
2016-05-08 14:34:38 +08:00
Peter Rowlands 93fdb14177 don't use selection by attribute 2016-05-08 10:33:17 +09:00
Peter Rowlands 370d4eb8ad use stricter file selector
in case of empty in case of empty ./track/video/file entries
2016-05-08 10:02:48 +09:00
Peter Rowlands 3452c3a27c update tests 2016-05-08 10:02:19 +09:00
Sergey M․ 9c072d38c6
[arte] Improve language preference (Closes #9401, closes #9162) 2016-05-08 06:52:42 +06:00
Peter Rowlands 81f35fee2f fix extractors.py import order 2016-05-08 08:57:16 +09:00
Peter Rowlands 0fdbe3146c use dict.get in case upload_date does not exist 2016-05-08 08:56:22 +09:00
Yen Chi Hsuan a0904c5d80
[telegraaf] Fix extractor (closes #9318) 2016-05-08 00:56:31 +08:00
Sergey M․ cb1fa58813
[flickr] Extract uploader URL (Closes #9426) 2016-05-07 20:15:40 +06:00
remitamine 3fd6332c05 [flickr] extract license field(closes #9425) 2016-05-07 15:13:14 +01:00
Sergey M 401d147893 Merge pull request #9400 from inondle/master
[liveleak] Adds support for thumbnails and updates tests
2016-05-06 19:23:31 +00:00
inondle e2ee97dcd5 [liveleak] Adds support for thumbnails, updates tests 2016-05-06 12:05:37 -07:00
Sergey M․ f745403b5b
[vevo] Revert videoplayer.vevo.com to api.vevo.com 2016-05-06 23:37:17 +06:00
Sergey M․ 3e80e6f40d
[vevo] Allow request to api.vevo.com to fail (Closes #9417)
I don't know whether this it's tempopary or api has just gone
2016-05-06 23:35:58 +06:00
Sergey M․ 25cb7a0eeb
[youtube] Allow empty attribute values in description regex 2016-05-06 22:11:18 +06:00
remitamine 04e88ca2ca [vk] improve extraction(fixes #7976) 2016-05-06 15:02:40 +01:00
Peter Rowlands 8d93c21466 add multi_video test case 2016-05-06 12:08:43 +09:00
Peter Rowlands 1dbfd78754 fix multi_video part naming, add upload_date field 2016-05-06 12:07:29 +09:00
Peter Rowlands 22e35adefd use url instead of single formats entry 2016-05-06 10:41:30 +09:00
Yen Chi Hsuan 6f59aa934b
[periscope:user] Add new extractor for user pages
Closes #9388
2016-05-06 02:14:39 +08:00
Peter Rowlands 833b644fff use xpath_text 2016-05-06 01:24:02 +09:00
Sergey M․ 915620fd68
[redtube] PEP 8 2016-05-05 21:34:06 +06:00
Sergey M․ ac12e888f9
[redtube] Extract all formats, duration, upload date and view count (Closes #9397) 2016-05-05 21:02:54 +06:00
Yen Chi Hsuan 7d08f6073d
[kuwo:category] Update test 2016-05-05 20:20:26 +08:00
remitamine 758a059241 [dailymail] Add new extractor(closes #2667) 2016-05-05 13:13:22 +01:00
Yen Chi Hsuan 4f8c56eb4e
[fczenit] Fix extraction and update test
Closes #9359
2016-05-05 17:55:37 +08:00
Peter Rowlands 57cf9b7f06 [afreecatv] Add new extractor for afreecatv.com VODs 2016-05-05 03:59:23 +09:00
Sergey M․ 9da526aae7
[yandexmusic:playlist] Update test 2016-05-04 23:18:48 +06:00
Sergey M․ 75b81df3af
[udemy] Modernize 2016-05-04 23:14:12 +06:00
Sergey M․ aabdc83d6e
[udemy] Fix course enroll (Closes #9393) 2016-05-04 23:03:44 +06:00
Sergey M․ 2a48e6f01a
[yandexmusic:playlist] Respect track order for long (>150) playlists 2016-05-04 22:45:01 +06:00
Sergey M․ 203a3c0e6a
[yandexmusic:playlist] Make title optional 2016-05-04 22:35:28 +06:00
Sergey M․ d36724cca4
[yandexmusic:playlist] Remove unused imports 2016-05-04 22:34:37 +06:00
Sergey M․ 15fc0658f7
[yandexmusic:playlist] Modernize 2016-05-04 22:33:29 +06:00
Sergey M․ e960c3c223
[yandexmusic:playlist] Improve extraction (Closes #6801) 2016-05-04 22:25:39 +06:00
Sergey M․ bc7e77a04b
[vevo] Use raise_geo_restricted 2016-05-03 23:18:36 +06:00
Sergey M․ 964f49336f
[aol] Improve _VALID_URL (Closes #9381) 2016-05-03 21:24:51 +06:00
Sergey M․ 57d8e32a3e
[xfileshare] Add support for streamin.to 2016-05-03 16:58:11 +06:00
Sergey M․ 4174552391
[xfileshare] Refactor _VALID_URL and remove ded sites 2016-05-03 15:35:32 +06:00
Sergey M․ 80bc4106af
[xfileshare] Add support for thevideobee.to (Closes #9374) 2016-05-03 15:09:23 +06:00
Yen Chi Hsuan 7759be38da
[xiami] Detect georestriction and skip tests 2016-05-03 16:19:43 +08:00
Yen Chi Hsuan a0a309b973
[kuwo:category] Fix description and update test 2016-05-03 16:06:28 +08:00
Sergey M․ 8a92e51c60
[extractor/common] Relax wording for creator metafield 2016-05-02 21:31:35 +06:00
Sergey M․ df5f4e8888
[vevo] Remove superfluous code 2016-05-02 18:47:35 +06:00
Sergey M․ 5c9ced9504
[vevo] Improve genre extraction 2016-05-02 18:19:00 +06:00
Sergey M․ 686cc89634
[discovery] Fix typo 2016-05-02 07:07:35 +06:00
Sergey M․ 9508738f9a
[vevo] Extract featured artist 2016-05-02 03:36:40 +06:00
Sergey M․ 78a3ff33ab
[vevo:playlist] Add fallback for playlist id 2016-05-02 03:29:48 +06:00
Sergey M․ 881dbc86c4
[vevo] Extract track related metafields and add artists to title (Closes #1684) 2016-05-02 03:28:58 +06:00
Sergey M․ 8e7d004888
[vevo] Add test for video only available via webpage 2016-05-02 03:06:48 +06:00
Sergey M․ 9618c44824
[vevo] Extract video versions from webpage as a last resort (Closes #8426, closes #9366) 2016-05-02 02:58:20 +06:00
Sergey M․ 516ea41a7d
[vevo] Fix _call_api 2016-05-02 02:54:50 +06:00
Sergey M․ e2bd301ce7
[vevo:playlist] Fix genre playlists 2016-05-02 01:00:42 +06:00
Sergey M․ 0c9d288ba0
[vevo:playlist] Remove debug params 2016-05-02 00:50:31 +06:00
Sergey M․ e0da32df6e
[vevo:playlist] Add extractor (Closes #9334, closes #9364) 2016-05-02 00:48:26 +06:00
Sergey M․ 68bb2fef95
[tagesschau] Restrict playlist entry regex 2016-05-01 07:15:23 +06:00
Sergey M․ 854cc54bc1
[tagesschau] Expand video id 2016-05-01 07:01:55 +06:00
Sergey M․ 651ad35ce0
[tagesschau] Relax _VALID_URL 2016-05-01 06:57:19 +06:00
Sergey M․ 6a0f9a24d0
[tagesschau] Separate player extractor 2016-05-01 06:45:44 +06:00
remitamine 9cf79e8f4b [ccc] improve extraction 2016-05-01 01:45:17 +01:00
Sergey M․ 2844b09336
[tagesschau] Fix article media ids 2016-05-01 04:42:05 +06:00
Sergey M․ 1a2b377cc2
[tagesschau] Fix audio support 2016-05-01 04:38:46 +06:00