Commit Graph

54 Commits (9688f237163b6aa546fde00bb3fd1e3445dd4c31)

Author SHA1 Message Date
Sergey M․ 6c907eb33f
[downloader/http] Request last data block of exact remaining size
Always request last data block of exact size remaining to download if possible not the current block size.
5 years ago
Sergey M․ f7b42518dc
[downloader/http] Finish downloading once received data length matches expected
Always do this if possible, i.e. if Content-Length or expected length is known, not only in test.
This will save unnecessary last extra loop trying to read 0 bytes.
5 years ago
Sergey M․ 3089bc748c
Fix W504 and disable W503 (closes #20863) 6 years ago
Sergey M․ 067aa17edf
Start moving to ytdl-org 6 years ago
Sergey M․ 5d6c81b63f
[downloader/http] Fix resume when writing ot stdout (closes #16699) 7 years ago
Sergey M․ b91a7a4e5e
[downloader/http] Randomize HTTP chunk size 7 years ago
Sergey M․ 00c97e3e7a
[downloader/http] Add ability to pass downloader options via info dict 7 years ago
Sergey M․ cf7259bc93
[downloader/http] Fix 302 infinite loops by not reusing requests 7 years ago
Sergey M․ ba515388b8
Introduce --http-chunk-size 7 years ago
Sergey M․ bec49996c6
[downloader/http] Return actual download result (closes #14971) 7 years ago
Sergey M․ a3c3a1e128
[http] Rework HTTP downloader
* Simplify code and split into separate routines to facilitate maintaining
* Make retry mechanism work on errors during actual download not only during connection establishment phase
* Retry on ECONNRESET and ETIMEDOUT during reading data from network
* Retry on content too short and various timeout errors
* Show error description on retry
* Closes #506, closes #809, closes #2849, closes #4240, closes #6023, closes #8625, closes #9483
7 years ago
Sergey M․ d3f0687cf7
[downloader/fragment] Use temp file for current fragment 8 years ago
Remita Amine 75a2485407 [fragment,hls,f4m,dash,ism] improve fragment downloading
- resume immediately
- no need to concatenate segments and decrypt them on every resume
- no need to save temp files for segments

and for hls downloader:
- no need to download keys for segments that already downloaded
8 years ago
Yen Chi Hsuan d54739a2e6
[downloader/http] xattr values should be bytes 8 years ago
Yen Chi Hsuan d7753d1948
[downloader/http] Use write_xattr function for --xattr-set-filesize 8 years ago
Sergey M․ 611c1dd96e [refactor] Single quotes consistency 9 years ago
Sergey M․ d800609c62 [refactor] Do not specify redundant None as second argument in dict.get() 9 years ago
Sergey M? 5c2266df4b Switch codebase to use sanitized_Request instead of
compat_urllib_request.Request

[downloader/dash] Use sanitized_Request

[downloader/http] Use sanitized_Request

[atresplayer] Use sanitized_Request

[bambuser] Use sanitized_Request

[bliptv] Use sanitized_Request

[brightcove] Use sanitized_Request

[cbs] Use sanitized_Request

[ceskatelevize] Use sanitized_Request

[collegerama] Use sanitized_Request

[extractor/common] Use sanitized_Request

[crunchyroll] Use sanitized_Request

[dailymotion] Use sanitized_Request

[dcn] Use sanitized_Request

[dramafever] Use sanitized_Request

[dumpert] Use sanitized_Request

[eitb] Use sanitized_Request

[escapist] Use sanitized_Request

[everyonesmixtape] Use sanitized_Request

[extremetube] Use sanitized_Request

[facebook] Use sanitized_Request

[fc2] Use sanitized_Request

[flickr] Use sanitized_Request

[4tube] Use sanitized_Request

[gdcvault] Use sanitized_Request

[extractor/generic] Use sanitized_Request

[hearthisat] Use sanitized_Request

[hotnewhiphop] Use sanitized_Request

[hypem] Use sanitized_Request

[iprima] Use sanitized_Request

[ivi] Use sanitized_Request

[keezmovies] Use sanitized_Request

[letv] Use sanitized_Request

[lynda] Use sanitized_Request

[metacafe] Use sanitized_Request

[minhateca] Use sanitized_Request

[miomio] Use sanitized_Request

[meovideo] Use sanitized_Request

[mofosex] Use sanitized_Request

[moniker] Use sanitized_Request

[mooshare] Use sanitized_Request

[movieclips] Use sanitized_Request

[mtv] Use sanitized_Request

[myvideo] Use sanitized_Request

[neteasemusic] Use sanitized_Request

[nfb] Use sanitized_Request

[niconico] Use sanitized_Request

[noco] Use sanitized_Request

[nosvideo] Use sanitized_Request

[novamov] Use sanitized_Request

[nowness] Use sanitized_Request

[nuvid] Use sanitized_Request

[played] Use sanitized_Request

[pluralsight] Use sanitized_Request

[pornhub] Use sanitized_Request

[pornotube] Use sanitized_Request

[primesharetv] Use sanitized_Request

[promptfile] Use sanitized_Request

[qqmusic] Use sanitized_Request

[rtve] Use sanitized_Request

[safari] Use sanitized_Request

[sandia] Use sanitized_Request

[shared] Use sanitized_Request

[sharesix] Use sanitized_Request

[sina] Use sanitized_Request

[smotri] Use sanitized_Request

[sohu] Use sanitized_Request

[spankwire] Use sanitized_Request

[sportdeutschland] Use sanitized_Request

[streamcloud] Use sanitized_Request

[streamcz] Use sanitized_Request

[tapely] Use sanitized_Request

[tube8] Use sanitized_Request

[tubitv] Use sanitized_Request

[twitch] Use sanitized_Request

[twitter] Use sanitized_Request

[udemy] Use sanitized_Request

[vbox7] Use sanitized_Request

[veoh] Use sanitized_Request

[vessel] Use sanitized_Request

[vevo] Use sanitized_Request

[viddler] Use sanitized_Request

[videomega] Use sanitized_Request

[viewvster] Use sanitized_Request

[viki] Use sanitized_Request

[vk] Use sanitized_Request

[vodlocker] Use sanitized_Request

[voicerepublic] Use sanitized_Request

[wistia] Use sanitized_Request

[xfileshare] Use sanitized_Request

[xtube] Use sanitized_Request

[xvideos] Use sanitized_Request

[yandexmusic] Use sanitized_Request

[youku] Use sanitized_Request

[youporn] Use sanitized_Request

[youtube] Use sanitized_Request

[patreon] Use sanitized_Request

[extractor/common] Remove unused import

[nfb] PEP 8
9 years ago
Sergey M․ 84bc4dcb0f [downloader/http] Clarify rationale for Content-Range check (#6426) 9 years ago
Sergey M․ 10eaa8ef1d [downloader/http] Report unable to resume 9 years ago
Sergey M․ c3124c3085 [downloader/http] Simplify 9 years ago
remitamine 8d5b8b477e [utils] import re 9 years ago
remitamine d7d2a9a3db [utils] restart download if server does not support byte ranges 9 years ago
Jaime Marquínez Ferrándiz 2ddcd88129 Remove code that was only used by the Grooveshark extractor 10 years ago
Philipp Hagemeister c59e701e35 Default to continuedl=True
We already do this in the CLI interface, so it should be just fine.
10 years ago
Jaime Marquínez Ferrándiz f28fe66970 [downloader/http] Add missing fields for _hook_progress call
It would fail if you run 'youtube-dl --no-part URL' a second time when the file has already been downloaded.

(Reported in Fedora: https://bugzilla.redhat.com/show_bug.cgi?id=1195779)
10 years ago
Jaime Marquínez Ferrándiz 123397317c [downloader/http] Remove wrong '_hook_progress' call (fixes #5117) 10 years ago
Philipp Hagemeister 09ab40b7d1 Merge branch 'progress-as-hook2' 10 years ago
Philipp Hagemeister a91a2c1a83 [downloader] Remove various unneeded assignments and imports 10 years ago
Philipp Hagemeister 16e7711e22 [downloader/http] Remove gruesome import 10 years ago
Philipp Hagemeister 5cda4eda72 [YoutubeDL] Use a progress hook for progress reporting
Instead of every downloader calling two helper functions, let our progress report be an ordinary progress hook like everyone else's.
Closes #4875.
10 years ago
Philipp Hagemeister 1283204917 [http] PEP8 (#4831) 10 years ago
light94 e77d2975af Handling Connection Reset by Peer Error 10 years ago
Philipp Hagemeister 881e6a1f5c Add --xattr-set-filesize option (Fixes #1348) 10 years ago
Jaime Marquínez Ferrándiz e1554a407d [extractors] Use http_headers for setting the User-Agent and the Referer 10 years ago
Jaime Marquínez Ferrándiz 3fcfb8e9fa [utils] YoutubeDLHandler: don't use 'Youtubedl-user-agent' for overriding the default user agent
Setting the 'User-Agent' header is enough
10 years ago
Philipp Hagemeister 1cc79574fc Fix imports and general cleanup
· Import from compat what comes from compat. Yes, some names are available in utils too, but that's an implementation detail.
· Use _match_id consistently whenever possible
· Fix some outdated tests
· Use consistent valid URL (always match the whole protocol, no ^ at start required)
· Use modern test definitions
10 years ago
Philipp Hagemeister cf372f0778 Merge remote-tracking branch 'SyxbEaEQ2/rate-limit' 10 years ago
Philipp Hagemeister 4d2d638df4 [http] Modernize 10 years ago
Philipp Hagemeister b686fc18da [hlsnative] Support test parameter 10 years ago
Sergey M․ 88a23aef5a [http] Avoid closing stdout (Fixes #3686) 10 years ago
Philipp Hagemeister d769be6c96 [grooveshark,http] Make HTTP POST downloads work 10 years ago
SyxbEaEQ2 c7667c2d7f [downloader/(common/http)] Changes calculation of the rate-limit. (Fix #2297, fix #2140, fix #595, fix #2370) 10 years ago
Sergey M․ 32fd27ec98 [http] Fix string/None comparison with int while in test 11 years ago
Sergey M․ df297c8794 [http] Improve download mechanism when Range HTTP header is ignored 11 years ago
Sergey M․ a59e40a1ea Replace 'referer' with 'http_referer' 11 years ago
Sergey M․ 1d0e49e1c7 Use explicitly set Referer header for downloading 11 years ago
Jaime Marquínez Ferrándiz 22ff5d2105 [http] Use the `YoutubeDL.urlopen` method 11 years ago
rzhxeo 60daf7f0bb Set resume_len to 0 if download is restarted 11 years ago
Jaime Marquínez Ferrándiz f89197d73e Some pep8 style fixes 11 years ago