240 Commits (6ba3085616652cbf05d1858efc321fdbfc4c6119)

Author	SHA1	Message	Date
Lesmiscore (Naoya Ozaki)	be4685ab7b	[http] Reject broken range before request (#3079) ... * And fix filesize estimate for byterange downloads Closes #2001 Authored by: Lesmiscore, Jules-A, pukkandan	3 years ago
coletdev	a2e77303e3	[downloader/http] Retry on more errors (#3065) ... Closes #3056, #2071 Related: #3034, #2969 Authored-by: coletdjnz	3 years ago
Lesmiscore (Naoya Ozaki)	d71fd41249	[fragment] Read downloaded fragments only when needed (#3069) ... Authored by: Lesmiscore	3 years ago
coletdev	b3edc8068e	[downloader/mhtml] Fix fragments with absolute urls (#3044) ... Authored-by: coletdjnz	3 years ago
pukkandan	592b748582	[cleanup] Minor cleanup ... Closes #3006	3 years ago
pukkandan	cf4f42cb97	Protect stdout from unexpected progress and console-title ... Closes #3023	3 years ago
pukkandan	08d30158ec	[cleanup, docs] Misc cleanup ... Closes #2828, closes #2734, closes #2802, closes #2937	3 years ago
Emanuel Hoogeveen	45806d44a7	[downloader] Obey `--file-access-retries` when deleting/renaming (#2224) ... Authored by: ehoogeveen-medweb	3 years ago
Lesmiscore (Naoya Ozaki)	93c8410d33	[downloader/fragment] Fix bugs around resuming with Range (#2901) ... Authored by: Lesmiscore	3 years ago
Lesmiscore	195c22840c	[downloader/fragment] Ignore `FileNotFoundError` when downloading livestreams ... when `--live-from-start` is used for YouTube and the live ends, request for the last segment prematurely ends (or 404, 403). this is causing lack of the file and `FileNotFoundError` lacking segment doesn't have any data, so it's safe to ignore	3 years ago
Lesmiscore	f0734e1190	[downloader/fragment] Fix problem where multiple threads can share one iterator ... which causes "ValueError: generator already executing" error Closes #2881	3 years ago
Lesmiscore (Naoya Ozaki)	15dfb3929c	[fc2:live] Add extractor (#2418) ... Authored by: Lesmiscore	3 years ago
Lesmiscore (Naoya Ozaki)	a539f06570	[downloader/fragment] Improve `--live-from-start` for YouTube livestreams (#2870)	3 years ago
pukkandan	dcd55f766d	[aria2c] Add `--http-accept-gzip=true` ... Closes #1936, #1236	3 years ago
pukkandan	079a7cfc71	[downloader] Do not use aria2c for non-native `m3u8` ... Closes #2718	3 years ago
pukkandan	1e5d87beee	[websocket] Make syntax error in `websockets` module non-fatal ... Closes #2633	3 years ago
pukkandan	1d3586d0d5	[aes] Add unpad_pkcs7	3 years ago
pukkandan	af6793f804	[downloader/ffmpeg] Handle unknown formats better	3 years ago
pukkandan	d57576b9d9	[httpie] Fix `available` method ... Closes #2330	3 years ago
pukkandan	f44afb54ef	[aria2c] Don't show progress when `--no-progress`	3 years ago
pukkandan	185bf31070	[youtube] End `live-from-start` properly when stream ends with 403 ... Closes #2089	3 years ago
Emanuel Hoogeveen	205a0654c0	Add option `--file-access-retries` (#2066) ... Closes #517 Authored by: ehoogeveen-medweb	3 years ago
pukkandan	b69fd25c25	[cleanup] Misc cleanup ... Closes #1942 #1976 #2020 #2058 #1984	3 years ago
pukkandan	4ac5b94807	[dash] Fix `--test` ... Bug in adbc4ec4bb	3 years ago
pukkandan	4273cc776d	[dash] Fix aria2c dash downloads ... Bug in adbc4ec4bb	3 years ago
The Hatsune Daishi	adbc4ec4bb	[dash,youtube] Download live from start to end (#888) ... * Add option `--live-from-start` to enable downloading live videos from start * Add key `is_from_start` in formats to identify formats (of live videos) that downloads from start * [dash] Create protocol `http_dash_segments_generator` that allows a function to be passed instead of fragments * [fragment] Allow multiple live dash formats to download simultaneously * [youtube] Implement fragment re-fetching for the live dash formats * [youtube] Re-extract dash manifest every 5 hours (manifest expires in 6hrs) * [postprocessor/ffmpeg] Add `FFmpegFixupDuplicateMoovPP` to fixup duplicated moov atoms Known issue: Ctrl+C doesn't work on Windows when downloading multiple formats Closes #1521 Authored by: nao20010128nao, pukkandan	3 years ago
pukkandan	f304da8a29	[cleanup] Misc cleanup ... Closes #1805, closes #1800	3 years ago
pukkandan	7578d77d8c	[downloader] Add colors to download progress	3 years ago
pukkandan	234416e4bf	[downloader/ffmpeg] Fix for direct videos inside mpd manifests ... Closes #1751	3 years ago
pukkandan	d76991ab07	Fix `--check-formats` for `mhtml` ... Closes #1709	3 years ago
nyuszika7h	6b993ca765	[hls] Better FairPlay DRM detection (#1661) ... Authored by: nyuszika7h	3 years ago
pukkandan	59a7a13ef9	[docs] Minor documentation improvements ... Closes #1583, #1599	3 years ago
pukkandan	a4211baff5	[cleanup] Minor cleanup	3 years ago
pukkandan	8913ef74d7	[ffmpeg] Detect libavformat version for `aac_adtstoasc` ... and print available features in verbose head Based on https://github.com/ytdl-org/youtube-dl/pull/29581	3 years ago
pukkandan	bd93fd5d45	[fragment] Fix progress display in fragmented downloads ... Closes #1517	3 years ago
pukkandan	49a57e70a9	[cleanup] misc	3 years ago
pukkandan	d3c93ec2b7	Don't create console for subprocesses on Windows (#1261) ... Closes #1251	3 years ago
pukkandan	b4b855ebc7	[fragment] Print error message when skipping fragment	3 years ago
pukkandan	2cda6b401d	Revert "[fragments] Pad fragments before decrypting (#1298)" ... This reverts commit 373475f035.	3 years ago
pukkandan	aa7785f860	[utils] Standardize timestamp formatting code ... Closes #1285	3 years ago
pukkandan	9fab498fbf	[http] Retry on socket timeout ... Closes #1222	3 years ago
shirt	373475f035	[fragments] Pad fragments before decrypting (#1298) ... Closes #197, #1297, #1007 Authored by: shirt-dev	3 years ago
pukkandan	580d3274e5	[youtube] Expose different formats with same itag	3 years ago
pukkandan	03b4de722a	[downloader] Fix slow progress hooks ... Closes #1301	3 years ago
pukkandan	d5a39f0bad	[http] Show the last encountered error ... Closes #1262	3 years ago
pukkandan	fc5c8b6492	[eria2c] Fix --skip-unavailable fragment	3 years ago
pukkandan	c111cefa5d	[downloader/ffmpeg] Improve simultaneous download and merge	3 years ago
pukkandan	21186af70a	[downloader] Fix throttledratelimit ... The timer should not reset at start of each block	3 years ago
pukkandan	d1d5c08f29	[minicurses] Fix when printing to file ... Closes #1215	3 years ago
pukkandan	b5ae35ee6d	[cleanup] Misc cleanup	3 years ago
pukkandan	b922db9fe5	[http] Respect user-provided chunk size over extractor's	3 years ago
pukkandan	819e05319b	Improved progress reporting (See desc) (#1125) ... * Separate `--console-title` and `--no-progress` * Add option `--progress` to show progress-bar even in quiet mode * Fix and refactor `minicurses` * Use `minicurses` for all progress reporting * Standardize use of terminal sequences and enable color support for windows 10 * Add option `--progress-template` to customize progress-bar and console-title * Add postprocessor hooks and progress reporting Closes: #906, #901, #1085, #1170	3 years ago
pukkandan	49e7e9c3ce	[docs,build] Change all pycryptodome references to pycryptodomex	3 years ago
shirt	7687c8ac6e	[HLS] Fix decryption issues (#1117) ... * Unpad HLS fragments with PKCS#7 according to datatracker.ietf.org/doc/html/rfc8216 * media_sequence should only be incremented in for media fragments * The native decryption should only be used if ffmpeg is unavailable since it is significantly slower. Closes #1086 Authored by: shirt-dev, pukkandan	3 years ago
coletdjnz	99e9e001de	[youtube] Cleanup authentication code (#786) ... Authored by: coletdjnz	3 years ago
pukkandan	be867b03f5	bugfix for bd50a52b0d	3 years ago
The Hatsune Daishi	ee2b3563f3	[downloader/niconico] Pass custom headers (#1063) ... Closes #1057 Authored by: nao20010128nao	3 years ago
The Hatsune Daishi	bd50a52b0d	Basic framework for simultaneous download of multiple formats (#1036) ... Authored by: nao20010128nao	3 years ago
pukkandan	d806c9fd97	[docs,cleanup] Add deprecation warning in docs ... for some counter intuitive behaviour that may be removed in future. and fix linter	3 years ago
pukkandan	1009f67c2a	[fragment,aria2c] Generalize and refactor some code	3 years ago
pukkandan	d9d8b85747	[fragment] Fix range header when using `-N` and media sequence (#1048) ... Authored by: shirt	3 years ago
pukkandan	daf7ac2b92	[fragment] Avoid repeated request for AES key	3 years ago
pukkandan	96933fc1b6	[aria2c] Fix IV for some AES-128 streams ... Authored by: shirt	3 years ago
pukkandan	57aa7b8511	[hls] Byterange + AES128 is supported by native downloader	3 years ago
pukkandan	edf65256aa	[hls,aes] Fallback to native implementation for AES-CBC ... and detect `Cryptodome` in addition to `Crypto` Closes #935 Related: #938	3 years ago
pukkandan	d5fe04f5c7	Fix `--compat-option no-direct-merge`	3 years ago
Paul Wrubel	d75201a873	Use `os.replace` where applicable (#793) ... When using ```py os.remove(encodeFilename(filename)) os.rename(encodeFilename(temp_filename), encodeFilename(filename)) ``` the `os.remove` need not be atomic and so can be executed arbitrarily compared to the immediately following rename call. It is better to use `os.replace` instead Authored by: paulwrubel	3 years ago
pukkandan	691d5823d6	[aria2c] Obey `--rate-limit`	3 years ago
pukkandan	45842107b9	fix bug in 6251555f1c ... :ci skip	3 years ago
pukkandan	6251555f1c	[downloader/ffmpeg] Support for DASH manifests (experimental) ... Closes #159	3 years ago
pukkandan	330690a214	[downloader/ffmpeg] Allow passing custom arguments before -i ... Closes #686	3 years ago
Felix S	7a6742b5f9	[webvtt] Fix timestamp overflow adjustment (#698) ... In some streams, empty segments may appear with a bogus, non-monotone MPEG timestamp. This should not be considered as an overflow Authored by: fstirlitz	3 years ago
pukkandan	f775c83110	Fix `--force-overwrites` when using `-k` ... For formats that need merge, the `.fxxx` files are not removed before downloading the corresponding `.part` files. This causes the rename to fail	3 years ago
Felix S	0715f7e19b	Revert erroneous use of the `Content-Length` header (#637) ... This reverts commit 6c907eb33f The use of the Content-Length value here is erroneous and may lead to truncated downloads if a compression scheme is specified in the Content-Encoding header, as the Content-Length header refers to the size of encoded data, not of the raw bytestream. This has been noticed in the wild with WebVTT subtitle segments. Authored by: fstirlitz	3 years ago
Felix S	25a3f4f5d6	[webvtt] Merge daisy-chained duplicate cues (#638) ... Fixes: https://github.com/yt-dlp/yt-dlp/issues/631#issuecomment-893338552 Previous deduplication algorithm only removed duplicate cues with identical text, styles and timestamps. This change also merges cues that come in ‘daisy chains’, where sequences of cues with identical text and styles appear in which the ending timestamp of one equals the starting timestamp of the next. This deduplication algorithm has the somewhat unfortunate side effect that NOTE blocks between cues, if found, will be emitted in a different order relative to their original cues. This may be unwanted if perfect fidelity is desired, but then so is daisy-chain deduplication itself. NOTE blocks ought to be ignored by WebVTT players in any case. Authored by: fstirlitz	3 years ago
pukkandan	ad3dc496bb	Misc fixes - See desc ... * Remove unnecessary uses of _list_from_options_callback * Fix download tests - Bug from 6e84b21559 * Rename ExecAfterDownloadPP to ExecPP and refactor its tests * Ensure _write_ytdl_file closes file handle on error - Potential fix for #517	3 years ago
pukkandan	3448870205	[docs] Fix some mistakes and improve doc	3 years ago
pukkandan	b868936cd6	[cleanup] Misc	3 years ago
pukkandan	9cc1a3130a	Fix resuming when using `--no-part` ... Closes #576	3 years ago
nikhil	bb36a55c41	[nbcolympics:stream] Fix extractor ... PR: https://github.com/ytdl-org/youtube-dl/pull/29688 Closes: #617, https://github.com/ytdl-org/youtube-dl/issues/29665 * Livestreams are untested * If using ffmpeg as downloader, v4.3+ is needed since `-http_seekable` option is necessary * Instead of making a seperate key for each arg that needs to be passed to ffmpeg, I made `_ffmpeg_args` * This deprecates `_seekable`, but the option is kept for compatibility Authored by: nchilada, pukkandan	3 years ago
pukkandan	e5611e8eda	[ffmpeg] Fix streaming `mp4` to `stdout`	3 years ago
pukkandan	a46a815b05	[cleanup] Fix linter in 96fccc101f	3 years ago
pukkandan	96fccc101f	[downloader] Allow streaming unmerged formats to stdout using ffmpeg ... For this to work: 1. The downloader must be ffmpeg 2. The selected formats must have the same protocol 3. The formats must be downloadable by ffmpeg to stdout Partial solution for: https://github.com/ytdl-org/youtube-dl/issues/28146, https://github.com/ytdl-org/youtube-dl/issues/27265	3 years ago
pukkandan	dbf5416a20	[cleanup] Refactor some code	3 years ago
pukkandan	f45e6c1126	[downloader] Pass same status object to all `progress_hooks`	3 years ago
pukkandan	3ba7740dd8	[downloader] Pass `info_dict` to `progress_hook`s	3 years ago
pukkandan	11f9be0912	[youtube] Extract data from multiple clients (#536) ... * `player_client` accepts multiple clients * default `player_client` = `android,web` * music clients can be specifically requested * Add IOS `player_client` * Hide live dash since they can't be downloaded Closes #501 Authored-by: pukkandan, colethedj	3 years ago
pukkandan	723d44b92b	[fragment] Handle errors in threads correctly	3 years ago
pukkandan	6b6c16ca6c	[downloader/ffmpeg] Fix `--ppa` when using simultaneous download	3 years ago
siikamiika	c2603313b1	[youtube_live_chat] use `clickTrackingParams` (#449) ... Authored by: siikamiika	3 years ago
pukkandan	bd4d1ea398	[cleanup] Minor refactoring of `fragment`	3 years ago
pukkandan	8e897ed283	[fragment] Return status of download correctly	3 years ago
siikamiika	d534c4520b	[youtube_live_chat] Fix download with cookies (#437) ... Closes #417 Authored by: siikamiika	3 years ago
siikamiika	c60ee3a218	[youtube_live_chat] Support ongoing live chat (#422) ... Authored by: siikamiika	3 years ago
pukkandan	8a77e5e6bc	[cleanup] Revert unnecessary changes in 51d9739f80	3 years ago
pukkandan	51d9739f80	Add option `--throttled-rate` below which video data is re-extracted ... Currently only for HTTP downloads Closes #430, workaround for https://github.com/ytdl-org/youtube-dl/issues/29326	3 years ago
pukkandan	4c7853de14	[fragment] Merge during download for `-N`, and refactor `hls`/`dash` (#364)	3 years ago
pukkandan	e36d50c5dd	[websockets] Add `WebSocketFragmentFD` (#399) ... Necessary for #392 Co-authored by: nao20010128nao, pukkandan	3 years ago
pukkandan	ff0f78e1fe	[aria2c] Lower `--min-split-size` for HTTP downloads ... This makes downloading smaller files much faster	3 years ago
felix	cdb19aa4c2	[downloader/mhtml] Add new downloader (#343) ... This downloader is intended to be used for streams that consist of a timed sequence of stand-alone images, such as slideshows or thumbnail streams This can be used for implementing: https://github.com/ytdl-org/youtube-dl/issues/4974#issue-58006762 https://github.com/ytdl-org/youtube-dl/issues/4540#issuecomment-69574231 https://github.com/ytdl-org/youtube-dl/pull/11185#issuecomment-335554239 https://github.com/ytdl-org/youtube-dl/issues/9868 https://github.com/ytdl-org/youtube-dl/pull/14951 Authored by: fstirlitz	3 years ago
felix	2ec1759f9d	[downloader/ffmpeg] Hide FFmpeg banner unless in verbose mode (#372) ... Authored by: fstirlitz	3 years ago
pukkandan	e06ca6ddac	[hls] Decrypt fragment when reading from disk ... Closes #373	3 years ago
pukkandan	d89da64b1d	[hls,dash] When using `concurrent_fragment_downloads`, do not keep the fragment content in memory ... Partial fix for #359 This is a temporary solution until #364 can be implemented	3 years ago
pukkandan	5dcd8e1d88	[hls] Disable external downloader for `webtt`	3 years ago
pukkandan	18e674b4f6	[ffmpeg] Download and merge in a single step if possible	4 years ago
pukkandan	856bb8f99d	[downloader] Fix `write_debug`	4 years ago
pukkandan	0760b0a7e2	Standardize `write_debug`	4 years ago
pukkandan	000ee7ef34	[fragment] Make sure first segment is not skipped	4 years ago
pukkandan	e8e738406a	Add experimental option `--check-formats` to test the URLs before format selection	4 years ago
pukkandan	848887eb7a	[downloader] Fix `quiet` and `to_stderr`	4 years ago
pukkandan	6ef6bcbd6b	[fragment] Ensure the file is closed on error	4 years ago
The Hatsune Daishi	0fa9a1e236	[whowatch] Add extractor #292 ... closes #223 Authored by: nao20010128nao Modified from: 9e4a0e061a/youtube_dl/extractor/whowatch.py	4 years ago
pukkandan	eb55bad5a0	[aria2c] Fix whitespace being stripped off ... Closes #276	4 years ago
pukkandan	2291dbce2a	[niconico] Fix HLS formats ... Closes #171 * The structure of the API JSON was changed * Smile Video seems to be no longer available. So remove the warning * Move ping to downloader * Change heartbeat interval to 40sec * Remove unnecessary API headers Authored-by: CXwudi, tsukumijima, nao20010128nao, pukkandan Tested by: tsukumijima	4 years ago
Felix S	66a1b8643a	[downloader/ism] Support muxing TTML subtitles	4 years ago
Felix S	15828bcf25	[downloader/hls] Handle MPEG-2 PES timestamp overflow	4 years ago
Felix S	333217f43e	[downloader/hls] Remove duplicate cues using a sliding window of candidates	4 years ago
Felix S	4a2f19abbd	[downloader/hls] Assemble single-file WebVTT subtitles from HLS segments	4 years ago
Felix S	fd76a14259	[extractor/common, downloader/ism] Extract SSTR subtitle tracks ... _parse_ism_formats was extended into _parse_ism_formats_and_subtitles; all direct users were updated, though _extract_ism_formats was left as a compatibility wrapper. The SSTR downloader was also modified in order to prepare for muxing subtitle streams, although no support for any subtitle codecs was added in this commit.	4 years ago
Felix S	d4553567d2	[downloader/ism] Prevent writing the header again when resuming an interrupted download	4 years ago
Felix S	4d49884c58	[downloader/fragment] Allow persisting extra state when a download is interrupted	4 years ago
pukkandan	88728713c8	Py2 compatibility for `FileNotFoundError`	4 years ago
pukkandan	33245766ab	[downloader] Fix ffmpeg selection for m3u8_native	4 years ago
pukkandan	72e1fe969f	[downloader] Fix downloader selection for m3u8 ... Bug introduced by: 52a8a1e1b9 and a31953b0e6	4 years ago
pukkandan	a31953b0e6	[downloader] Fix external downloader selection for m3u8 ... Closes #239	4 years ago
pukkandan	68379de561	[downloaders] Fix API access ... Bug introduced by: 52a8a1e1b9	4 years ago
pukkandan	52a8a1e1b9	Option to choose different downloader for different protocols ... * Renamed `--external-downloader-args` to `--downloader-args` * Added `native` as an option for the downloader * Use similar syntax to `--downloader-args` etc. Eg: `--downloader dash:native --downloader aria2c` * Deprecated `--hls-prefer-native` and `--hls-prefer-ffmpeg` since the same can now be done with `--downloader "m3u8:native"` and `m3u8:ffmpeg` respectively * Split `frag_urls` protocol into `m3u8_frag_urls` and `dash_frag_urls` * Standardize shortening of protocol names with `downloader.shorten_protocol_name`	4 years ago
pukkandan	83b20a970d	[youtube_live_chat] fix bug when trying to set cookies ... Closes #194	4 years ago
pukkandan	beb4b92a66	More consistent warning messages (#173) ... Co-authored by: Damiano Amatruda <damiano.amatruda@outlook.com>	4 years ago
pukkandan	fe845284c4	[aria2c] Support retry/abort unavailable fragments (#173) ... Co-authored by: Damiano Amatruda <damiano.amatruda@outlook.com>	4 years ago
pukkandan	2b3bf01c90	[aria2c] Better default arguments	4 years ago
pukkandan	e92caff5d5	Refactor (See desc) ... * Create `FFmpegPostProcessor.real_run_ffmpeg` that can accept multiple input/output files along with switches for each * Rewrite `cli_configuration_args` and related functions * Create `YoutubeDL._ensure_dir_exists` - this was previously defined in multiple places	4 years ago
shirt	4cf1e5d2f9	Native concurrent downloading of fragments (#166) ... * Option `--concurrent-fragments` (`-N`) to set the number of threads Related: #165 Known issues: * When receiving Ctrl+C, the process will exit only after finishing the currently downloading fragments * The download progress shows the speed of only one thread Authored by shirt-dev	4 years ago
pukkandan	0a473f2f0f	More improvements to HLS/DASH external downloader code ... * Fix error when there is no `protocol` in `info_dict` * Move HLS byte range detection to `Aria2cFD` so that the download will fall back to the native downloader instead of ffmpeg * Fix bug with getting no fragments in DASH * Convert `check_results` in `can_download` to a generator	4 years ago
shirt-dev	d7009caa03	Improve HLS/DASH external downloader code (#162) ... Authored by: shirt	4 years ago
pukkandan	9e631877f8	[downloader] Fix bug for ffmpeg/httpie ... Caused by: 7f7de7f94d	4 years ago
pukkandan	5ef7d9bdd8	Release 2021.03.01	4 years ago
pukkandan	7f7de7f94d	Allow specifying path in `--external-downloader`	4 years ago
pukkandan	9bd2020476	[hls] Enable `--hls-use-mpegts` by default when downloading live-streams ... * Also added option `--no-hls-use-mpegts` to disable this Related: #96	4 years ago
Pccode66	7a5c1cfe93	Completely change project name to yt-dlp (#85) ... * All modules and binary names are changed * All documentation references changed * yt-dlp no longer loads youtube-dlc config files * All URLs changed to point to organization account Co-authored-by: Pccode66 Co-authored-by: pukkandan	4 years ago