Commit Graph

216 Commits (5b28cef72db3b531680d89c121631c73ae05354f)

Author SHA1 Message Date
coletdjnz 5faf6528fb
[http] Fix bug in retrying on read timeout in py < 3.10
socket.timeout is not an alias of TimeoutError in py < 3.10
Fixes bug in a2e77303e3
Authored-by: coletdjnz
2 years ago
pukkandan 3a408f9d19
Show name of downloader in verbose log
Closes #3703
2 years ago
pukkandan 0f06bcd759
[cleanup] Minor fixes (See desc)
* [youtube] Fix `--youtube-skip-dash-manifest`
* [build] Use `$()` in `Makefile`. Closes #3684
* Fix bug in 385ffb467b
* Fix bug in 43d7f5a5d0
* [cleanup] Remove unnecessary `utf-8` from `str.encode`/`bytes.decode`
* [utils] LazyList: Expose unnecessarily "protected" attributes
and other minor cleanup
2 years ago
felix e4fa34a13e
[hls] Fix unapplied byte_range for EXT-X-MAP fragment
Cherry-picked from #3302
Authored by: fstirlitz
2 years ago
pukkandan 3fe75fdc80
[cleanup] Misc fixes (see desc)
* Do not warn when fixup is skipped for existing file
* [fragment] Fix `--skip-unavailable-fragments` for HTTP Errors
* [utils] write_string: Fix bug in 59f943cd50
* [utils] parse_codecs: Subtitle codec is generally referred to as `scodec`. https://github.com/yt-dlp/yt-dlp/pull/2174#discussion_r790156048
* [docs] Remove note about permissions. Closes #3597
2 years ago
pukkandan 1d485a1a79
[cleanup] Misc fixes
Closes #3565, https://github.com/yt-dlp/yt-dlp/issues/3514#issuecomment-1105944364
2 years ago
pukkandan 0a5a191a2a
Improve `--clean-infojson`
It should not removes fields that may be needed for `--load-infojson`.
Eg: `_ffmpeg_args`, `_has_drm`
2 years ago
Elyse 00828e2c93
[downloader/ffmpeg] Specify headers for each URL (#3553)
Closes #2696
Authored by: elyse0
2 years ago
Lesmiscore 69b59b4b4b
[downloader/fc2] Stop heatbeating once FFmpeg finishes
Authored by: Lesmiscore
2 years ago
pukkandan 9b8ee23b99
[dependencies] Create module with all dependency imports 2 years ago
pukkandan 43cc91ad75
bugfix for 19a0394044, 3d3bb1688b 2 years ago
felix 77f9033095
[compat] Split into sub-modules (#2173)
Authored by: fstirlitz, pukkandan
2 years ago
pukkandan 19a0394044
[cleanup] Misc cleanup and refactor (#2173) 2 years ago
Lesmiscore (Naoya Ozaki) c854208ccf
[downloader/fragment] Make single thread download work for --live-from-start (#3446)
Authored by: Lesmiscore
2 years ago
pukkandan e06bd8800f
Fix `--skip-unavailable-fragments`
Bug in d71fd41249
Closes #3437
2 years ago
pukkandan 3b9d9f4374
Do not change fragment chunk-size when `--test`
Closes #3434
2 years ago
pukkandan f82711587c
[cleanup] Sort imports
Using https://github.com/PyCQA/isort

    isort -m VERTICAL_HANGING_INDENT --py 36 -l 80 --rr -n --tc .
2 years ago
pukkandan 86e5f3ed2e
[cleanup] Upgrade syntax
Using https://github.com/asottile/pyupgrade

1. `__future__` imports and `coding: utf-8` were removed
2. Files were rewritten with `pyupgrade --py36-plus --keep-percent-format`
3. f-strings were cherry-picked from `pyupgrade --py36-plus`

Extractors are left untouched (except removing header) to avoid unnecessary merge conflicts
2 years ago
pukkandan f9934b9614
[cleanup] Mark some compat variables for removal (#2173)
Authored by fstirlitz, pukkandan
2 years ago
pukkandan 0edb3e336c
Do not prevent download if locking is unsupported
Closes #3022

Failure to lock download-archive is still fatal.
This is consistent with youtube-dl's behavior
2 years ago
pukkandan ce0593ef61
[http] Fix #3215 2 years ago
pukkandan a44ca5a470
[cleanup] Misc fixes
Closes https://github.com/yt-dlp/yt-dlp/pull/3213, Closes https://github.com/yt-dlp/yt-dlp/pull/3117

Related: https://github.com/yt-dlp/yt-dlp/issues/3146#issuecomment-1077323114, https://github.com/yt-dlp/yt-dlp/pull/3277#discussion_r841019671, a825ffbffa (commitcomment-68538986), https://github.com/yt-dlp/yt-dlp/issues/2360, 5fa3c9a88f (r70393519), 5fa3c9a88f (r70393254)
2 years ago
pukkandan c4f60dd7cd
[utils] Add `try_call` 2 years ago
pukkandan 28787f16c6
[downloader] Fix invocation of `HttpieFD`
Closes #3154
2 years ago
coletdev af14914baa
Remove `Accept-Encoding` header from `std_headers` (#3153)
This should be set by each downloader to what it supports.
Fixes https://github.com/yt-dlp/yt-dlp/issues/3142
Authored-by: coletdjnz
2 years ago
coletdev 8a23db9519
[wget] Fix proxy (#3152)
Upstream PR: https://github.com/ytdl-org/youtube-dl/pull/29343
Authored-by: kikuyan, coletdjnz
2 years ago
Lesmiscore (Naoya Ozaki) be4685ab7b
[http] Reject broken range before request (#3079)
* And fix filesize estimate for byterange downloads

Closes #2001
Authored by: Lesmiscore, Jules-A, pukkandan
2 years ago
coletdev a2e77303e3
[downloader/http] Retry on more errors (#3065)
Closes #3056, #2071
Related: #3034, #2969
Authored-by: coletdjnz
2 years ago
Lesmiscore (Naoya Ozaki) d71fd41249
[fragment] Read downloaded fragments only when needed (#3069)
Authored by: Lesmiscore
2 years ago
coletdev b3edc8068e
[downloader/mhtml] Fix fragments with absolute urls (#3044)
Authored-by: coletdjnz
2 years ago
pukkandan 592b748582
[cleanup] Minor cleanup
Closes #3006
2 years ago
pukkandan cf4f42cb97
Protect stdout from unexpected progress and console-title
Closes #3023
2 years ago
pukkandan 08d30158ec
[cleanup, docs] Misc cleanup
Closes #2828, closes #2734, closes #2802, closes #2937
2 years ago
Emanuel Hoogeveen 45806d44a7
[downloader] Obey `--file-access-retries` when deleting/renaming (#2224)
Authored by: ehoogeveen-medweb
2 years ago
Lesmiscore (Naoya Ozaki) 93c8410d33
[downloader/fragment] Fix bugs around resuming with Range (#2901)
Authored by: Lesmiscore
2 years ago
Lesmiscore 195c22840c
[downloader/fragment] Ignore `FileNotFoundError` when downloading livestreams
when `--live-from-start` is used for YouTube and the live ends, request for the last segment prematurely ends (or 404, 403).
this is causing lack of the file and `FileNotFoundError`
lacking segment doesn't have any data, so it's safe to ignore
2 years ago
Lesmiscore f0734e1190
[downloader/fragment] Fix problem where multiple threads can share one iterator
which causes "ValueError: generator already executing" error

Closes #2881
2 years ago
Lesmiscore (Naoya Ozaki) 15dfb3929c
[fc2:live] Add extractor (#2418)
Authored by: Lesmiscore
2 years ago
Lesmiscore (Naoya Ozaki) a539f06570
[downloader/fragment] Improve `--live-from-start` for YouTube livestreams (#2870) 2 years ago
pukkandan dcd55f766d
[aria2c] Add `--http-accept-gzip=true`
Closes #1936, #1236
2 years ago
pukkandan 079a7cfc71
[downloader] Do not use aria2c for non-native `m3u8`
Closes #2718
2 years ago
pukkandan 1e5d87beee
[websocket] Make syntax error in `websockets` module non-fatal
Closes #2633
2 years ago
pukkandan 1d3586d0d5
[aes] Add unpad_pkcs7 2 years ago
pukkandan af6793f804
[downloader/ffmpeg] Handle unknown formats better 2 years ago
pukkandan d57576b9d9
[httpie] Fix `available` method
Closes #2330
2 years ago
pukkandan f44afb54ef
[aria2c] Don't show progress when `--no-progress` 2 years ago
pukkandan 185bf31070
[youtube] End `live-from-start` properly when stream ends with 403
Closes #2089
2 years ago
Emanuel Hoogeveen 205a0654c0
Add option `--file-access-retries` (#2066)
Closes #517
Authored by: ehoogeveen-medweb
2 years ago
pukkandan b69fd25c25
[cleanup] Misc cleanup
Closes #1942 #1976 #2020 #2058 #1984
2 years ago
pukkandan 4ac5b94807
[dash] Fix `--test`
Bug in adbc4ec4bb
2 years ago
pukkandan 4273cc776d
[dash] Fix aria2c dash downloads
Bug in adbc4ec4bb
2 years ago
The Hatsune Daishi adbc4ec4bb
[dash,youtube] Download live from start to end (#888)
* Add option `--live-from-start` to enable downloading live videos from start
* Add key `is_from_start` in formats to identify formats (of live videos) that downloads from start
* [dash] Create protocol `http_dash_segments_generator` that allows a function to be passed instead of fragments
* [fragment] Allow multiple live dash formats to download simultaneously
* [youtube] Implement fragment re-fetching for the live dash formats
* [youtube] Re-extract dash manifest every 5 hours (manifest expires in 6hrs)
* [postprocessor/ffmpeg] Add `FFmpegFixupDuplicateMoovPP` to fixup duplicated moov atoms

Known issue: Ctrl+C doesn't work on Windows when downloading multiple formats

Closes #1521
Authored by: nao20010128nao, pukkandan
2 years ago
pukkandan f304da8a29
[cleanup] Misc cleanup
Closes #1805, closes #1800
3 years ago
pukkandan 7578d77d8c
[downloader] Add colors to download progress 3 years ago
pukkandan 234416e4bf
[downloader/ffmpeg] Fix for direct videos inside mpd manifests
Closes #1751
3 years ago
pukkandan d76991ab07
Fix `--check-formats` for `mhtml`
Closes #1709
3 years ago
nyuszika7h 6b993ca765
[hls] Better FairPlay DRM detection (#1661)
Authored by: nyuszika7h
3 years ago
pukkandan 59a7a13ef9
[docs] Minor documentation improvements
Closes #1583, #1599
3 years ago
pukkandan a4211baff5
[cleanup] Minor cleanup 3 years ago
pukkandan 8913ef74d7
[ffmpeg] Detect libavformat version for `aac_adtstoasc`
and print available features in verbose head
Based on https://github.com/ytdl-org/youtube-dl/pull/29581
3 years ago
pukkandan bd93fd5d45
[fragment] Fix progress display in fragmented downloads
Closes #1517
3 years ago
pukkandan 49a57e70a9
[cleanup] misc 3 years ago
pukkandan d3c93ec2b7
Don't create console for subprocesses on Windows (#1261)
Closes #1251
3 years ago
pukkandan b4b855ebc7
[fragment] Print error message when skipping fragment 3 years ago
pukkandan 2cda6b401d
Revert "[fragments] Pad fragments before decrypting (#1298)"
This reverts commit 373475f035.
3 years ago
pukkandan aa7785f860
[utils] Standardize timestamp formatting code
Closes #1285
3 years ago
pukkandan 9fab498fbf
[http] Retry on socket timeout
Closes #1222
3 years ago
shirt 373475f035
[fragments] Pad fragments before decrypting (#1298)
Closes #197, #1297, #1007
Authored by: shirt-dev
3 years ago
pukkandan 580d3274e5 [youtube] Expose different formats with same itag 3 years ago
pukkandan 03b4de722a
[downloader] Fix slow progress hooks
Closes #1301
3 years ago
pukkandan d5a39f0bad
[http] Show the last encountered error
Closes #1262
3 years ago
pukkandan fc5c8b6492
[eria2c] Fix --skip-unavailable fragment 3 years ago
pukkandan c111cefa5d
[downloader/ffmpeg] Improve simultaneous download and merge 3 years ago
pukkandan 21186af70a
[downloader] Fix throttledratelimit
The timer should not reset at start of each block
3 years ago
pukkandan d1d5c08f29
[minicurses] Fix when printing to file
Closes #1215
3 years ago
pukkandan b5ae35ee6d
[cleanup] Misc cleanup 3 years ago
pukkandan b922db9fe5
[http] Respect user-provided chunk size over extractor's 3 years ago
pukkandan 819e05319b
Improved progress reporting (See desc) (#1125)
* Separate `--console-title` and `--no-progress`
* Add option `--progress` to show progress-bar even in quiet mode
* Fix and refactor `minicurses`
* Use `minicurses` for all progress reporting
* Standardize use of terminal sequences and enable color support for windows 10
* Add option `--progress-template` to customize progress-bar and console-title
* Add postprocessor hooks and progress reporting

Closes: #906, #901, #1085, #1170
3 years ago
pukkandan 49e7e9c3ce
[docs,build] Change all pycryptodome references to pycryptodomex 3 years ago
shirt 7687c8ac6e
[HLS] Fix decryption issues (#1117)
* Unpad HLS fragments with PKCS#7 according to datatracker.ietf.org/doc/html/rfc8216
* media_sequence should only be incremented in for media fragments
* The native decryption should only be used if ffmpeg is unavailable since it is significantly slower. Closes #1086

Authored by: shirt-dev, pukkandan
3 years ago
coletdjnz 99e9e001de
[youtube] Cleanup authentication code (#786)
Authored by: coletdjnz
3 years ago
pukkandan be867b03f5
bugfix for bd50a52b0d 3 years ago
The Hatsune Daishi ee2b3563f3
[downloader/niconico] Pass custom headers (#1063)
Closes #1057
Authored by: nao20010128nao
3 years ago
The Hatsune Daishi bd50a52b0d
Basic framework for simultaneous download of multiple formats (#1036)
Authored by: nao20010128nao
3 years ago
pukkandan d806c9fd97
[docs,cleanup] Add deprecation warning in docs
for some counter intuitive behaviour that may be removed in future.

and fix linter
3 years ago
pukkandan 1009f67c2a
[fragment,aria2c] Generalize and refactor some code 3 years ago
pukkandan d9d8b85747
[fragment] Fix range header when using `-N` and media sequence (#1048)
Authored by: shirt
3 years ago
pukkandan daf7ac2b92
[fragment] Avoid repeated request for AES key 3 years ago
pukkandan 96933fc1b6
[aria2c] Fix IV for some AES-128 streams
Authored by: shirt
3 years ago
pukkandan 57aa7b8511
[hls] Byterange + AES128 is supported by native downloader 3 years ago
pukkandan edf65256aa
[hls,aes] Fallback to native implementation for AES-CBC
and detect `Cryptodome` in addition to `Crypto`

Closes #935
Related: #938
3 years ago
pukkandan d5fe04f5c7
Fix `--compat-option no-direct-merge` 3 years ago
Paul Wrubel d75201a873
Use `os.replace` where applicable (#793)
When using 
```py
os.remove(encodeFilename(filename))
os.rename(encodeFilename(temp_filename), encodeFilename(filename))
```
the `os.remove` need not be atomic and so can be executed arbitrarily compared to the immediately following rename call. It is better to use `os.replace` instead

Authored by: paulwrubel
3 years ago
pukkandan 691d5823d6
[aria2c] Obey `--rate-limit` 3 years ago
pukkandan 45842107b9
fix bug in 6251555f1c
:ci skip
3 years ago
pukkandan 6251555f1c
[downloader/ffmpeg] Support for DASH manifests (experimental)
Closes #159
3 years ago
pukkandan 330690a214
[downloader/ffmpeg] Allow passing custom arguments before -i
Closes #686
3 years ago
Felix S 7a6742b5f9
[webvtt] Fix timestamp overflow adjustment (#698)
In some streams, empty segments may appear with a bogus, non-monotone MPEG timestamp.
This should not be considered as an overflow

Authored by: fstirlitz
3 years ago
pukkandan f775c83110
Fix `--force-overwrites` when using `-k`
For formats that need merge, the `.fxxx` files are not removed before
downloading the corresponding `.part` files. This causes the rename to fail
3 years ago
Felix S 0715f7e19b
Revert erroneous use of the `Content-Length` header (#637)
This reverts commit 6c907eb33f

The use of the Content-Length value here is erroneous and may lead
to truncated downloads if a compression scheme is specified in the
Content-Encoding header, as the Content-Length header refers to the
size of encoded data, not of the raw bytestream. This has been noticed
in the wild with WebVTT subtitle segments.

Authored by: fstirlitz
3 years ago