Commit Graph

8871 Commits

Author SHA1 Message Date
Sergey M? 5c2266df4b Switch codebase to use sanitized_Request instead of
compat_urllib_request.Request

[downloader/dash] Use sanitized_Request

[downloader/http] Use sanitized_Request

[atresplayer] Use sanitized_Request

[bambuser] Use sanitized_Request

[bliptv] Use sanitized_Request

[brightcove] Use sanitized_Request

[cbs] Use sanitized_Request

[ceskatelevize] Use sanitized_Request

[collegerama] Use sanitized_Request

[extractor/common] Use sanitized_Request

[crunchyroll] Use sanitized_Request

[dailymotion] Use sanitized_Request

[dcn] Use sanitized_Request

[dramafever] Use sanitized_Request

[dumpert] Use sanitized_Request

[eitb] Use sanitized_Request

[escapist] Use sanitized_Request

[everyonesmixtape] Use sanitized_Request

[extremetube] Use sanitized_Request

[facebook] Use sanitized_Request

[fc2] Use sanitized_Request

[flickr] Use sanitized_Request

[4tube] Use sanitized_Request

[gdcvault] Use sanitized_Request

[extractor/generic] Use sanitized_Request

[hearthisat] Use sanitized_Request

[hotnewhiphop] Use sanitized_Request

[hypem] Use sanitized_Request

[iprima] Use sanitized_Request

[ivi] Use sanitized_Request

[keezmovies] Use sanitized_Request

[letv] Use sanitized_Request

[lynda] Use sanitized_Request

[metacafe] Use sanitized_Request

[minhateca] Use sanitized_Request

[miomio] Use sanitized_Request

[meovideo] Use sanitized_Request

[mofosex] Use sanitized_Request

[moniker] Use sanitized_Request

[mooshare] Use sanitized_Request

[movieclips] Use sanitized_Request

[mtv] Use sanitized_Request

[myvideo] Use sanitized_Request

[neteasemusic] Use sanitized_Request

[nfb] Use sanitized_Request

[niconico] Use sanitized_Request

[noco] Use sanitized_Request

[nosvideo] Use sanitized_Request

[novamov] Use sanitized_Request

[nowness] Use sanitized_Request

[nuvid] Use sanitized_Request

[played] Use sanitized_Request

[pluralsight] Use sanitized_Request

[pornhub] Use sanitized_Request

[pornotube] Use sanitized_Request

[primesharetv] Use sanitized_Request

[promptfile] Use sanitized_Request

[qqmusic] Use sanitized_Request

[rtve] Use sanitized_Request

[safari] Use sanitized_Request

[sandia] Use sanitized_Request

[shared] Use sanitized_Request

[sharesix] Use sanitized_Request

[sina] Use sanitized_Request

[smotri] Use sanitized_Request

[sohu] Use sanitized_Request

[spankwire] Use sanitized_Request

[sportdeutschland] Use sanitized_Request

[streamcloud] Use sanitized_Request

[streamcz] Use sanitized_Request

[tapely] Use sanitized_Request

[tube8] Use sanitized_Request

[tubitv] Use sanitized_Request

[twitch] Use sanitized_Request

[twitter] Use sanitized_Request

[udemy] Use sanitized_Request

[vbox7] Use sanitized_Request

[veoh] Use sanitized_Request

[vessel] Use sanitized_Request

[vevo] Use sanitized_Request

[viddler] Use sanitized_Request

[videomega] Use sanitized_Request

[viewvster] Use sanitized_Request

[viki] Use sanitized_Request

[vk] Use sanitized_Request

[vodlocker] Use sanitized_Request

[voicerepublic] Use sanitized_Request

[wistia] Use sanitized_Request

[xfileshare] Use sanitized_Request

[xtube] Use sanitized_Request

[xvideos] Use sanitized_Request

[yandexmusic] Use sanitized_Request

[youku] Use sanitized_Request

[youporn] Use sanitized_Request

[youtube] Use sanitized_Request

[patreon] Use sanitized_Request

[extractor/common] Remove unused import

[nfb] PEP 8
2015-11-23 21:56:23 +06:00
Sergey M․ 67dda51722 Rename compat_urllib_request_Request to sanitized_Request and move to utils 2015-11-23 21:55:15 +06:00
Sergey M․ e4c4bcf36f [vimeo] Use compat_urllib_request_Request 2015-11-23 21:55:14 +06:00
Sergey M․ 82d8a8b6e2 [YoutubeDL] Wrap plain-text URL requests in compat_urllib_request_Request 2015-11-23 21:55:13 +06:00
Sergey M․ 13a10d5aa3 [compat] Add compat_urllib_request_Request
This is actually not a compatibility routine but rather a workaround for URLs without protocol specified.
The protocol-less URL is treated as HTTP one since it's most probable scenario and it will most likely to
redirect to HTTPS if HTTPS was actually expected. This routine could also be useful for any Request
preprocessing that may be added in future.
2015-11-23 21:55:12 +06:00
Sergey M․ 9022726446 [youtube] Fix test 2015-11-23 21:37:21 +06:00
Sergey M․ 94bfcd23b7 [youtube] Fix test 2015-11-23 21:35:23 +06:00
Sergey M․ 526b3b0716 [youtube] Clarify ytplayer.config extraction rationale 2015-11-23 21:14:03 +06:00
Sergey M․ 61f92af1cf [youtube] Add test with '};' in tags 2015-11-23 21:02:37 +06:00
Sergey M․ a72778d364 [youtube] Improve ytplayer.config extraction 2015-11-23 21:00:06 +06:00
Sergey M 5ae17037a3 Merge pull request #7599 from lalinsky/fix-youtube
[youtube] More explicit player config JSON extraction (fixes #7468)
2015-11-23 20:52:23 +06:00
Sergey M․ 02f0da20b0 [pluralsight] Add support for alternative webpage layout (Closes #7607) 2015-11-23 03:08:38 +06:00
Lukáš Lalinský b41631c4e6 [youtube] Send the list of patterns directly to _search_regex 2015-11-22 13:53:26 +01:00
Lukáš Lalinský 0e49d9a6b0 [youtube] Fall back to the original regex for ytplayer.config 2015-11-22 13:49:33 +01:00
Sergey M․ 4a7d108ab3 [rutube] Remove unnecessary print 2015-11-22 18:24:17 +06:00
Lukáš Lalinský 3cfd000849 [youtube] More explicit player config JSON extraction (fixes #7468) 2015-11-22 13:14:35 +01:00
Sergey M․ 1b38185361 [pornhd] Fix title extraction (Closes #7596) 2015-11-22 18:08:30 +06:00
Sergey M․ 9cb9a5df77 [utils] Check ext with trailing slash against the list of known extensions 2015-11-22 17:27:13 +06:00
Sergey M․ 3e12bc583a [utils] Improve determine_ext (Closes #7593) 2015-11-22 06:29:39 +06:00
Sergey M․ e568c2233e [youtube] Add test for multi page list of playlists 2015-11-22 05:03:23 +06:00
Sergey M․ 061a75edd6 [youtube] Extract base for entry list extractors and support multi page lists of playlists 2015-11-22 05:01:01 +06:00
Philipp Hagemeister 82c4d7b0ce release 2015.11.21 2015-11-21 23:36:27 +01:00
Sergey M․ 136dadde95 [youtube:show] Rework in terms of playlists base extractor 2015-11-22 04:18:20 +06:00
Sergey M․ 0c14841585 [youtube:user:playlists] Add extractor (Closes #3817) 2015-11-22 04:17:07 +06:00
Sergey M․ 0eebf34d9d [pluralsight] Rephrase 2015-11-22 00:58:25 +06:00
Sergey M․ cf186b77a7 [pluralsight] Clarify allowed qualities guessing rationale 2015-11-22 00:56:40 +06:00
Sergey M․ a3372437bf [soundcloud] Remove unused variable 2015-11-22 00:49:58 +06:00
Sergey M․ 4c57b4853d [pluralsight] Until listing formats request only single format 2015-11-22 00:42:58 +06:00
Sergey M․ 38eb2968ab [pluralsight] Clarify and randomize ViewClip sleep interval 2015-11-22 00:07:09 +06:00
Andrzej Lichnerowicz bea56c9569 [pluralsight] prevent error 429 when sensing video formats 2015-11-21 23:49:58 +06:00
Sergey M․ 7e508ff2cf [pluralsight] Improve login detection 2015-11-21 21:49:37 +06:00
Sergey M․ 563772eda4 [pluralsight] Extract base class 2015-11-21 21:37:29 +06:00
Sergey M․ 0533915aad [pluralsight] Update some more URLs 2015-11-21 21:35:08 +06:00
Sergey M․ c3a227d1c4 [pluralsight] Update _LOGIN_URL 2015-11-21 21:25:48 +06:00
Sergey M․ f6c903e708 [soundcloud:search] Simplify (Closes #7213) 2015-11-21 21:21:21 +06:00
Sergey M․ 7dc011c063 [soundcloud:search] Remove no track results message 2015-11-21 21:00:42 +06:00
Sergey M․ 4e3b303016 [soundcloud:search] Fix non-ASCII searches 2015-11-21 20:55:48 +06:00
Sergey M․ 7e1f5447e7 [utils] Improve encode_dict 2015-11-21 20:46:33 +06:00
Sergey M․ 7e3472758b [soundcloud:search] PEP 8 2015-11-21 20:04:35 +06:00
reiv 328a22e175 [soundcloud] Remove limit on search results 2015-11-21 19:41:36 +06:00
reiv 417b453699 [soundcloud] Use correct error message conventions 2015-11-21 19:41:31 +06:00
reiv 6ea7190a3e Rewrite as list comprehension. 2015-11-21 19:41:26 +06:00
reiv b54b08c91b Simplify with itertools.islice(). 2015-11-21 19:41:19 +06:00
reiv c30943b1c0 Fix some compatibility issues, cleanup. 2015-11-21 19:41:15 +06:00
reiv 2abf7cab80 [soundcloud] Add Soundcloud search extractor 2015-11-21 19:41:08 +06:00
Sergey M․ 4137196899 [rutube] Extract all formats 2015-11-21 18:02:52 +06:00
Sergey M․ 019839faaa [extractor/common] Use baseURL from f4m manifest for recursive manifest extraction 2015-11-21 18:01:39 +06:00
Sergey M․ f52183a878 [rutube:embed] Extend _VALID_URL (Closes #7588) 2015-11-21 17:39:24 +06:00
Yen Chi Hsuan 750b9ff032 [generic] Extract M3U8 formats (closes #7582) 2015-11-21 16:43:01 +08:00
Yen Chi Hsuan 28602e747c [generic] Refactor 2015-11-21 16:08:54 +08:00
Yen Chi Hsuan 6cc37c69e2 [generic] Unescape URLs from JWPlayer (#7582) 2015-11-21 14:12:34 +08:00
Sergey M․ a5cd0eb8a4 [pluralsight:course] Improve _VALID_URL 2015-11-21 08:32:48 +06:00
Sergey M․ c23e266427 [pluralsight] Do not require pluralsight account
Looks like some courses are available without pluralsight account
2015-11-21 08:25:52 +06:00
Sergey M․ 651acffbe5 [pluralsight] Update ViewClip URL 2015-11-21 08:21:33 +06:00
Sergey M․ 71bd93b89c [pluralsight] Do not rely on argument order in query (Closes #7583) 2015-11-21 08:08:34 +06:00
Sergey M․ 6da620de58 [kaltura] Add test for referrer protected video (#7409) 2015-11-21 01:40:28 +06:00
Sergey M․ bdceea7afd [kaltura] Clean description 2015-11-21 01:39:29 +06:00
Sergey M․ d80a39cec8 [kaltura] Improve 2015-11-21 01:38:08 +06:00
Sergey M․ 5b5fae5f20 [generic] Use referrer from source kaltura embed URLs (#7409) 2015-11-21 01:35:58 +06:00
Sergey M․ 01b06aedcf [kaltura] Add support for referrer protected videos (#7409) 2015-11-21 01:34:02 +06:00
ashutosh-mishra 17cc153435 Typo fix, found while going through the code. 2015-11-20 22:51:46 +05:30
Sergey M․ 67446fd49b [instagram] Improve _VALID_URL (Closes #7568) 2015-11-20 04:07:39 +06:00
Sergey M․ 325bb615a7 [theplatform] Style 2015-11-19 22:58:43 +06:00
Sergey M․ ee5cd8418e [theplatform] Handle protocolless feed URLs (Closes #7532) 2015-11-19 22:58:29 +06:00
Sergey M․ 342609a1b4 [bloomberg] Reax _VALID_URL (Closes #7546) 2015-11-19 22:55:06 +06:00
Sergey M f270cf1a26 Merge pull request #7519 from barlik/master
Clarify that automatic subtitles are generated.
2015-11-19 22:44:08 +06:00
hedii 371c3b796c [YoutubeDL] Add playlist finished downloading message (Closes #7517)
Conflicts:
	youtube_dl/YoutubeDL.py
2015-11-19 22:39:02 +06:00
Sergey M․ 6b7ceee1b9 [vimeo] Add test for #7552 2015-11-19 22:31:16 +06:00
Sergey M․ fdb20a27a3 [vimeo:group] Improve _VALID_URL (Closes #7552) 2015-11-19 22:30:58 +06:00
Sergey M․ 2c94198eb6 [vimeo] Improve playlists extraction 2015-11-19 21:29:32 +06:00
Philipp Hagemeister e8110b8125 release 2015.11.19 2015-11-19 15:35:13 +01:00
Yen Chi Hsuan c39fd7b1ca [UDNEmbed] Fix generic UDN pages
Closes #7547
2015-11-19 22:32:56 +08:00
Sergey M․ a9c09a7c62 [pbs] Update API URL (Closes #7565) 2015-11-19 20:25:28 +06:00
Philipp Hagemeister 82beaabb41 release 2015.11.18 2015-11-18 19:23:04 +01:00
Jaime Marquínez Ferrándiz 63b4295d20 [youtube:playlist] fix title extraction (fixes #7544 and #7545) 2015-11-18 18:28:05 +01:00
Sergey M․ 312a3f389b [pbs] Extend _VALID_URL 2015-11-18 00:46:41 +06:00
Jaime Marquínez Ferrándiz 609af1ae1c [dplay] Add 'encoding: utf-8' line 2015-11-17 17:58:16 +01:00
Jaime Marquínez Ferrándiz 4cd759f73d [dplay] Add extractor (closes #7515)
Since I haven't figured out how to download the hds stream, we use the hls one instead.
2015-11-17 17:52:29 +01:00
Jaime Marquínez Ferrándiz e156e70281 [rtve] Remove unused import 2015-11-17 16:23:29 +01:00
Sergey M․ 9b464929fe [rtve.es:alacarta] Fix extraction 2015-11-17 21:11:42 +06:00
Sergey M 0c176d7bde Merge pull request #7514 from ping/patch-7301
[neteasemusic] Fixes #7301
2015-11-16 14:25:29 +00:00
Sergey M․ 7a3f0c00ad [utils] Style 2015-11-16 20:24:09 +06:00
Sergey M․ 7aefc49c40 [utils] Skip invalid/non HTML entities (Closes #7518) 2015-11-16 20:20:16 +06:00
Rastislav Barlik 741dd8ea65 Clarify that automatic subtitles are generated.
It wasn't clear what automatic word mean.
2015-11-16 14:15:25 +00:00
ping 76adc82068 [neteasemusic] Fixes #7301 2015-11-16 11:39:18 +08:00
Philipp Hagemeister bd1512d196 release 2015.11.15 2015-11-15 22:16:08 +01:00
Sergey M․ 9a4acbfaf5 [theplatform] Add test for #7385 2015-11-16 00:28:04 +06:00
Sergey M․ ad1f4e7902 [theplatform] Handle explicitly specified SMIL (#7385) 2015-11-15 23:43:23 +06:00
Sergey M b328295910 Merge pull request #7436 from davidbz/add_proxy_to_update_procedure
Add proxy support for update_self
2015-11-15 11:13:22 +00:00
David Ben Zakai 828b2a5cd9 Removing an unnecessary import 2015-11-15 09:40:32 +02:00
Sergey M․ 2ff7cbeaaa [nowtv:list] Add extrator (Closes #7147) 2015-11-15 08:30:13 +06:00
Sergey M․ b2f7738830 [dumpert] Use original protocol 2015-11-15 02:25:00 +06:00
Sergey M․ dc0279532a [dumpert] Disable SSL (Closes #7504) 2015-11-15 02:21:24 +06:00
Sergey M․ 0c59d02bdc [periscope] Relax _VALID_URL (Closes #7503) 2015-11-15 00:20:17 +06:00
Jaime Marquínez Ferrándiz 0f72beb515 [periscope] Remove unused imports 2015-11-14 18:31:33 +01:00
Sergey M․ d781e29316 [bbc] Allow selectionunavailable errors (Closes #7502) 2015-11-14 23:08:13 +06:00
Sergey M․ 3b3e8ed332 [quickscope] Remove extractor (2) 2015-11-14 22:34:30 +06:00
Sergey M․ dcdfeb33d2 [quickscope] Remove extractor 2015-11-14 22:32:54 +06:00
Sergey M․ 0d85c3a732 [lynda] Style 2015-11-14 16:44:24 +06:00
Sergey M․ 903d136942 [lynda] Logout only when login info present (Closes #7500) 2015-11-14 16:43:58 +06:00
Yen Chi Hsuan 9d584da7d0 [xfileshare] Correct _VALID_URL 2015-11-14 17:27:32 +08:00
Yen Chi Hsuan 31752f76f7 [twitter:card] Add add_ie for the external test 2015-11-14 17:03:26 +08:00
Yen Chi Hsuan 5f1b2aea80 [twitter:card] Support vine.co embeds (closes #7496) 2015-11-14 17:02:07 +08:00
Sergey M․ 4479600d57 [instagram] Add test for #7497 2015-11-14 07:21:20 +06:00
Sergey M․ a90189c3ad [instagram] Relax _VALID_URL (Closes #7497) 2015-11-14 07:20:33 +06:00
Sergey M․ d8a1caf04f [brightcove:new] Style 2015-11-14 06:22:12 +06:00
Sergey M․ cb33d389ed [brightcove:new] Add test with rtmp streams 2015-11-14 06:20:09 +06:00
Sergey M․ e01b432ad3 [brightcove:new] Fix test 2015-11-14 06:11:17 +06:00
Sergey M․ fd91257c40 [brightcove] Order imports alphabetically 2015-11-14 06:08:36 +06:00
Sergey M․ c7b959ce38 [utils] Remove unused function 2015-11-14 06:07:44 +06:00
Sergey M․ 75eac8961e [brightcove] Remove unused import 2015-11-14 06:07:24 +06:00
Sergey M․ 3b7d9aa487 Rename all references to legacy studio Brightcove extractor 2015-11-14 06:05:46 +06:00
Sergey M․ 1f4b722b00 [generic] Clarify Brightcove Legacy Studio comment 2015-11-14 06:03:32 +06:00
Sergey M․ f6519f89b0 [generic] Extract Brightcove New Studio embeds 2015-11-14 06:03:07 +06:00
Sergey M․ 24af85298e [brightcove] Fix _extract_urls 2015-11-14 06:01:56 +06:00
Sergey M․ e721d857c2 [brightcove] Clarify IE_NAMEs 2015-11-14 05:56:51 +06:00
Sergey M․ 5c17f0a67a [brightcove:embedinpage] Rename extractor to brightcove new
It's not actually embed_in_page but "New Studio" and allows both iframe and embed_in_page embeds
2015-11-14 05:55:59 +06:00
Sergey M․ 4fcaa4f4a5 [brightcove] Rename extractor to brightcove legacy
Old embedding approaches are now "Legacy Studio"
2015-11-14 05:54:16 +06:00
Sergey M․ 536f819eda [brightcove] Imrove extraction of new embeds 2015-11-14 05:51:05 +06:00
Sergey M․ a662489877 [brightcove:embedinpage] Make more robust and extract rtmp streams 2015-11-14 05:09:50 +06:00
Sergey M․ a2973eb597 Merge branch 'brightcove_in_page_embed' of https://github.com/remitamine/youtube-dl into remitamine-brightcove_in_page_embed 2015-11-14 01:23:15 +06:00
Sergey M․ 4e21b3a94f [cbs] Use android UA for higher quality streams (Closes #7490) 2015-11-14 00:25:28 +06:00
Jaime Marquínez Ferrándiz b703ebeeaf [twitter] Don't fail if the description doesn't contain an URL (fixes #7489) 2015-11-13 19:09:42 +01:00
Jaime Marquínez Ferrándiz b84a5f0337 [twitter] Update tests checksums 2015-11-13 18:55:07 +01:00
Philipp Hagemeister a1ec9a7553 release 2015.11.13 2015-11-13 11:07:30 +01:00
Sergey M․ 91d644b5ba [ruutu] Relax formats extraction 2015-11-13 02:43:27 +06:00
Sergey M․ 5d6c3d6a66 [ruutu] Skip NOT-USED URLs(Closes #7478) 2015-11-13 02:41:38 +06:00
Jaime Marquínez Ferrándiz 1ebb4717df [cbsnews] Fix construction of 'play_path' in some videos (fixes #7394) 2015-11-12 21:02:56 +01:00
Yen Chi Hsuan cf5881fc4d Credit @ferama
For providing idea for vidto.me (#7167) and extending nowvideo support (#6760)
2015-11-12 21:33:46 +08:00
Sergey M․ fcd817a326 [vimeo] Fix extraction (Closes #7460) 2015-11-12 03:56:11 +06:00
Sergey M․ 031ec536f0 [gorillavid] Rename to xfileshare 2015-11-11 23:00:53 +06:00
Sergey M․ 668db403f9 [gorillavid] Add test for vidto.me and strip title 2015-11-11 22:47:28 +06:00
Sergey M․ b9ad101926 [gorillavid] Add support for vidto.me 2015-11-11 22:44:03 +06:00
Sergey M․ 435911029f [vidto] Remove extractor 2015-11-11 22:43:17 +06:00
Sergey M․ 699ed30cee [novamov] Modernize 2015-11-11 22:34:49 +06:00
Sergey M․ 9eab37dca0 [vimeo] Simplify set cookie 2015-11-11 22:32:13 +06:00
Sergey M․ 9a8a12b7d8 [vimeo] Append cookies instead of overriding 2015-11-11 22:23:23 +06:00
Yen Chi Hsuan a4c2ab35c1 Merge remote-tracking branch 'upstream/master' 2015-11-12 00:08:42 +08:00
Sergey M․ 3d9c4bf09a [vimeo] Fix password protected videos (Closes #7451) 2015-11-11 21:21:21 +06:00
Yen Chi Hsuan 8b8a39e279 [vidto] Several simplifications and improvements
1. Use InfoExtractor._hidden_inputs
2. Fetch title from <title> tag
3. Cookies are preserved automatically
4. Use single quotes everywhere
5. Do not declare variables for one-time use only
2015-11-11 23:17:59 +08:00
Sergey M․ 82393e2bb2 [novamov] Follow continue-to-the-video button if any (Closes #7330) 2015-11-11 21:02:05 +06:00
Sergey M․ 2eb99a4b98 [nowvideo] Replace main host to resolvable one 2015-11-11 21:00:23 +06:00
Yen Chi Hsuan 990e6e8fa3 [vidto] Minor fixes
1. import order
2. fatal is already True in helper functions
2015-11-11 20:13:03 +08:00
Rémy Léone d8b7e80d29 Remove duplicate key 2015-11-11 12:00:31 +01:00
Yen Chi Hsuan 37120974dc [vidto] PEP8 2015-11-11 02:02:46 +08:00
Marco Ferragina 42fc93c709 vidto extractor: code cleanup 2015-11-11 01:58:47 +08:00
Marco Ferragina a625e56543 [vidto] Add extractor 2015-11-11 01:52:43 +08:00
Sergey M․ 9b738b2caa [funnyordie] Fix extraction and extract m3u8 formats 2015-11-10 21:32:54 +06:00
David Ben Zakai 90bb5667bf Using internal opener 2015-11-10 17:15:23 +02:00
David Ben Zakai d3d3e2e3aa Adding proxy to update procedure 2015-11-10 16:31:42 +02:00