Commit Graph

554 Commits (ad9a8115aa29a1a95c961b16fcf129a228d98f50)

Author SHA1 Message Date
coletdev d5820461e8
Use certificates from `certifi` if installed (#3115)
Fixes #3102 and most `CERTIFICATE_VERIFY_FAILED` issues

Authored by: coletdjnz
3 years ago
pukkandan 0c14d66ad9
Fix `autonumber`
Bug in 09b49e1f68
3 years ago
pukkandan 16c620bc55
Handle float in `--wait-for-video`
Closes #3082
3 years ago
pukkandan 592b748582
[cleanup] Minor cleanup
Closes #3006
3 years ago
pukkandan cf4f42cb97
Protect stdout from unexpected progress and console-title
Closes #3023
3 years ago
MMM 10331a2672
Fix `--print` with `--ignore-no-formats` when url is `None` (#3000)
Authored by: flashdagger
3 years ago
pukkandan 08d30158ec
[cleanup, docs] Misc cleanup
Closes #2828, closes #2734, closes #2802, closes #2937
3 years ago
coletdev 4390d5ec12
Add brotli content-encoding support (#2433)
Authored by: coletdjnz
3 years ago
pukkandan 50e93e03a7
Update to ytdl-commit-6508688
Make default upload_/release_date a compat_str
6508688e88

Except:
* "[NDR] Overhaul NDR and NJoy extractors" https://github.com/ytdl-org/youtube-dl/pull/30531
    - 01824d275b
    - 39a98b09a2
    - f0a05a55c2
    - 4186e81777
3 years ago
pukkandan 8b7539d27c
Implement `--add-header` without modifying `std_headers`
Closes #2526, #1614
3 years ago
pukkandan 2a938746f3
Fix verbose log when stdout/stderr encoding is `None`
See: 5c10453827
3 years ago
pukkandan 319b6059d2
Better error message when no --live-from-start format 3 years ago
pukkandan 4c3f8c3fb6
Handle negative duration from extractor
Closes #2921
3 years ago
Lesmiscore (Naoya Ozaki) 03f830040a
[YoutubeDL] Fill more fields for playlists (#2824) 3 years ago
pukkandan 09b49e1f68
Add pre-processor stage `after_filter`
* Move `_match_entry` and `post_extract` to `process_video_result`. It is also left in `process_info` for API compat
* `--list-...` options and `--force-write-archive` now obey filtering options
* Move `SponsorBlockPP` to `after_filter`. Closes https://github.com/yt-dlp/yt-dlp/issues/2536
* Reverts 4ec82a72bb since this commit addresses the issue it was solving
3 years ago
pukkandan 60f3e99592
Tolerate failure to `--write-link` due to unknown URL
Closes #2724
3 years ago
pukkandan 8d93e69d67
Create necessary directories for `--print-to-file`
Closes #2721
3 years ago
pukkandan 3aa915400d
Fix `-all` for `--sub-langs`
Closes #2703
3 years ago
pukkandan 5c10453827
Fix for when stdout/stderr encoding is None
Closes #2711
3 years ago
pukkandan e8969bda94
Obey `--abort-on-error` for "ffmpeg not installed"
Closes #1840
3 years ago
pukkandan 57ebfca39b
Set `webpage_url_...` from `webpage_url` and not input URL
Closes #2756
3 years ago
Lukas Fink 1ce9a3cb49
Add regex operator and quoting to format filters (#2698)
Closes #2681 
Authored by: lukasfink1
3 years ago
pukkandan 297e9952b6
[extractor] Allow `http_headers` to be specified for `thumbnails` 3 years ago
pukkandan 455a15e2dc
[cleanup,docs] Minor fixes
Closes #2541, #2484
3 years ago
pukkandan 6a0546e313
[outtmpl] Handle hard-coded file extension better
When we know that the user-provided extension is the correct final one,
replace it with intermediate extension during download
3 years ago
pukkandan fc08bdd6ab
[extractor] Allow non-fatal `title` extraction 3 years ago
pukkandan 88f23a18e0
[docs,cleanup] Fix linter and misc cleanup
Closes #2419
3 years ago
pukkandan bb66c24797
Add option `--print-to-file`
Closes #2372
3 years ago
pukkandan 6a5a30f9e2
Ensure `_type` is present in `info.json`
Closes #2447
3 years ago
pukkandan d37707bda4
Fix/improve `InAdvancePagedList` 3 years ago
pukkandan 6970b6005e
[cleanup] Minor fixes
Closes #2334
3 years ago
pukkandan 0bfc53d05c
List playlist thumbnails in `--list-thumbnails` 3 years ago
xtkoba f81c62a6a4
Add option --legacy-server-connect (#778)
to allow HTTPS connection to servers that do not support RFC 5746 secure renegotiation

Authored by: xtkoba
3 years ago
MinePlayersPE 426764371f
[iq.com] Add extractors (#2354)
Closes #704
Authored by: MinePlayersPE
3 years ago
pukkandan 3b603dbdf1
Add option `--concat-playlist`
Closes #1855, related: #382
3 years ago
pukkandan ed8d87f911
[cleanup, docs] Minor fixes
Closes #2230
3 years ago
pukkandan e04938ab88
Check for existing thumbnail/subtitle in final directory
Closes #2275
3 years ago
pukkandan dc5f409cdc
Fix typo in ed5835b451 3 years ago
pukkandan 8130779db6
Allow listing formats, thumbnails, subtitles using `--print` (#2238)
Closes #2083
Authored by: pukkandan, Zirro
3 years ago
pukkandan ed5835b451
Allow `--print` to be run at any post-processing stage 3 years ago
coletdjnz f0d785d3ed
[youtube:tab] Extract more playlist metadata (#2069)
* Add fields modified_date, modified_timestamp
* Add field playlist_count
* [youtube:tab] Extract view_count, playlist_count, modified_date

Authored by: coletdjnz, pukkandan
3 years ago
pukkandan 9e907ebddf
[cleanup] Misc cleanup 3 years ago
pukkandan b62fa6d75f
Fix `-s --ignore-no-formats --force-write-archive`
Bug in a13e684813
3 years ago
pukkandan be72c62480
Fix recursion error in f46e2f9d92
Closes #2216
3 years ago
pukkandan a13e684813
Write `download_archive` only after all formats are downloaded
Closes #1470
3 years ago
pukkandan f46e2f9d92
Add key `requested_downloads` in the root `info_dict` 3 years ago
pukkandan 9c906919ae
Add field `video_autonumber`
Closes #662
3 years ago
pukkandan 6020e05d23
Raise error if subtitle download fails
Closes #2212
3 years ago
pukkandan ebed8b3732
Add more post-processing stages
playlist = After entire playlist
after_video = After downloading all formats of a video
3 years ago
pukkandan 1e43a6f733
Allow `--exec` to be run at any post-processing stage
Deprecates `--exec-before-download`
3 years ago
pukkandan ca30f449a1
Add `--print playlist:` to print fields per playlist 3 years ago
pukkandan 4c922dd3fc
Fix live title for multiple formats 3 years ago
pukkandan abbeeebc4c
[outtmpl] Alternate form for `D` and fix suffix's case
Fixes: https://github.com/yt-dlp/yt-dlp/issues/2085#issuecomment-1002247689, https://github.com/yt-dlp/yt-dlp/pull/2132/files#r775729811
3 years ago
pukkandan ceb98323f2
Don't treat empty containers as `None` in `sanitize_info` 3 years ago
Matt Broadway f59f5ef8b6
[cookies] Support other keyrings (#2032)
Authored by: mbway
3 years ago
pukkandan 6da22e7d4f
Avoid recursion error when re-extracting info 3 years ago
Benedikt Wildenhain 4b4b7f746c
[OpenCast] Add extractors (#1905)
Original PR: https://github.com/ytdl-org/youtube-dl/pull/26934
Authored by: bwildenhain, C0D3D3V
3 years ago
pukkandan 37893bb0c9
[outtmpl] Change filename sanitization type to `S`
`F` is already used for float!
Bug in e0fd95737d
3 years ago
Emanuel Hoogeveen 205a0654c0
Add option `--file-access-retries` (#2066)
Closes #517
Authored by: ehoogeveen-medweb
3 years ago
pukkandan b69fd25c25
[cleanup] Misc cleanup
Closes #1942 #1976 #2020 #2058 #1984
3 years ago
pukkandan e0fd95737d
[outtmpl] Add alternate forms `F`, `D`
and improve `id` detection

F = sanitize as filename (# = restricted)
D = add Decimal suffixes

Closes #2085, 2081
3 years ago
pukkandan fa9f30b802
Add interactive format selection with `-f -`
Closes #2065
3 years ago
The Hatsune Daishi adbc4ec4bb
[dash,youtube] Download live from start to end (#888)
* Add option `--live-from-start` to enable downloading live videos from start
* Add key `is_from_start` in formats to identify formats (of live videos) that downloads from start
* [dash] Create protocol `http_dash_segments_generator` that allows a function to be passed instead of fragments
* [fragment] Allow multiple live dash formats to download simultaneously
* [youtube] Implement fragment re-fetching for the live dash formats
* [youtube] Re-extract dash manifest every 5 hours (manifest expires in 6hrs)
* [postprocessor/ffmpeg] Add `FFmpegFixupDuplicateMoovPP` to fixup duplicated moov atoms

Known issue: Ctrl+C doesn't work on Windows when downloading multiple formats

Closes #1521
Authored by: nao20010128nao, pukkandan
3 years ago
Unit 193 e75bb0d6c3
[cleanup] Fix some typos (#2033)
Authored by: unit193
3 years ago
pukkandan dd0228ce1f
Remove known invalid thumbnails from `info_dict`
Related: https://github.com/yt-dlp/yt-dlp/issues/980#issuecomment-997396821
3 years ago
PilzAdam e978789f0f
[outtmpl] Add operator `&` for replacement text (#2012)
Authored by: PilzAdam
3 years ago
pukkandan 39ca3b5c7f
[extractor] Standardize `_live_title` 3 years ago
pukkandan 0bb322b9c0
Add field `webpage_url_domain`
Closes #1311
3 years ago
pukkandan 5bfc8bee5a
Fix PostProcessor hooks not registered for some PPs
Closes #1993
3 years ago
pukkandan e3c7d49571
[compat] Suppress errors in enabling VT mode
Closes #1932
3 years ago
pukkandan b5475f1145
Pre-process when using `--flat-playlist` 3 years ago
pukkandan 38d79fd16c
Use `parse_duration` for `--wait-for-video`
and some minor fix
3 years ago
pukkandan 3efb96a6d1
Fix control characters being printed to `--console-title`
Closes #1859
3 years ago
pukkandan 91f071af60
Release 2021.12.01 3 years ago
pukkandan 5c22c63da3
Fix `--trim-filename` when filename has `.`
Closes #1837
3 years ago
pukkandan ee8dd27a73
[cleanup] Add deprecation warnings 3 years ago
pukkandan f304da8a29
[cleanup] Misc cleanup
Closes #1805, closes #1800
3 years ago
pukkandan 36eaf3039a
[build] Save Git HEAD at release alongside version info 3 years ago
pukkandan f2ebc5c7be
Option `--wait-for-video` to wait for scheduled streams 3 years ago
pukkandan b222c27145
Option `--break-per-input` to apply --break-on... to each input URL 3 years ago
pukkandan 5e5be0c0b2
Fix --break-on-archive when pre-checking 3 years ago
pukkandan 7578d77d8c
[downloader] Add colors to download progress 3 years ago
pukkandan 75689fe59b
Ensure directory exists when checking formats 3 years ago
pukkandan e04b003e64
[FixupM3u8] Fixup MPEG-TS in MP4 container
Closes #1701, https://github.com/ytdl-org/youtube-dl/issues/26410
3 years ago
pukkandan ff51ed588f
Clarify video/audio-only formats in -F
Related: #1759
3 years ago
Zirro 0e6b018a10
Ensure path for link files exists (#1755)
Authored by: Zirro
3 years ago
pukkandan 282f570918
[utils] Fix error when copying `LazyList` 3 years ago
pukkandan c07a39ae8e
[utils] Fix `PagedList`
Bug in d8cf8d97a8
3 years ago
pukkandan c5e3f84972
[utils] Allow alignment in `render_table`
and add tests
3 years ago
pukkandan dd2a987d3f
[tests] Fix tests 3 years ago
pukkandan 9222c38182
[cleanup] Minor cleanup
Closes #1696, Closes #1673
3 years ago
pukkandan 013b50b794
Fix 'postprocessor_hooks`
Closes #1650
3 years ago
pukkandan dac5df5a98
Add option `--embed-info-json` to embed info-json in mkv
Closes #1644
3 years ago
pukkandan 7c7f7161fc
Fix `--load-info-json` of playlists with failed entries 3 years ago
pukkandan bf5f605e76
bugfix for e08a85d865 3 years ago
pukkandan e08a85d865
Fix writing playlist infojson with `--no-clean-infojson` 3 years ago
pukkandan 093a17107e
Allow using a custom format selector through API
Closes #1619, #1464
3 years ago
pukkandan c586f9e8de
[cleanup] minor fixes 3 years ago
pukkandan 59a7a13ef9
[docs] Minor documentation improvements
Closes #1583, #1599
3 years ago
pukkandan 4476d2c764
[outtmpl] Add alternate forms for `q` and `j` 3 years ago
pukkandan aa9369a2d8
[cleanup] Minor improvements to error and debug messages 3 years ago
pukkandan a49891c761
Fix bug in `--load-infojson` of playlists
Fixes: https://github.com/yt-dlp/yt-dlp/issues/1514#issuecomment-962659529
3 years ago
pukkandan 34921b4345
[utils] Add `join_nonempty` 3 years ago
pukkandan a4211baff5
[cleanup] Minor cleanup 3 years ago
pukkandan 8913ef74d7
[ffmpeg] Detect libavformat version for `aac_adtstoasc`
and print available features in verbose head
Based on https://github.com/ytdl-org/youtube-dl/pull/29581
3 years ago
pukkandan 0930b11fda
[docs,cleanup] Improve docs and minor cleanup
Closes #1387, #1404, #1408, #1485, #1415, #1450, #1492
3 years ago
pukkandan 48f796874d
[utils] Create `DownloadCancelled` exception
as super-class of ExistingVideoReached, RejectedVideoReached, MaxDownloadsReached

Third parties can also sub-class this to cancel the download queue from a hook
3 years ago
pukkandan 08438d2ca5
[outtmpl] Add type `link` for internet shortcut files
and refactor related code
Closes #1405
3 years ago
pukkandan f2fe69c7b0
Approximate filesize from bitrate
Closes #1400
3 years ago
pukkandan 9f1a1c36e6
Separate `--check-all-formats` from `--check-formats`
Previously, `--check-formats` tested only the selected video formats, but ALL thumbnails
3 years ago
pukkandan 96565c7e55
[cleanup] Add keyword automatically to SearchIE descriptions
and some minor cleanup of docs
3 years ago
pukkandan ec11a9f4a2
[minicurses] Add more colors 3 years ago
pukkandan 49a57e70a9
[cleanup] misc 3 years ago
pukkandan 6e21fdd279
[build] Enable lazy-extractors in releases
Set the environment variable `YTDLP_NO_LAZY_EXTRACTORS`
to forcefully disable lazy extractor loading
3 years ago
pukkandan d3c93ec2b7
Don't create console for subprocesses on Windows (#1261)
Closes #1251
3 years ago
pukkandan 176f1866cb
Add HDR information to formats 3 years ago
pukkandan 17bddf3e95
Reduce default `--socket-timeout` 3 years ago
pukkandan e820fbaa6f
Do not verify thumbnail URLs by default
Partially reverts cca80fe611 and 0ba692acc8

Unless `--check-formats` is specified, this causes yt-dlp to return incorrect thumbnail urls.
See https://github.com/yt-dlp/yt-dlp/issues/340#issuecomment-877909966, #402

But the overhead in general use is not worth it

Closes #694, #725
3 years ago
pukkandan 239df02103
Make `duration_string` and `resolution` available in --match-filter
Related: #1309
3 years ago
pukkandan 71ce444a3f
Fix --restrict-filename when used with default template 3 years ago
pukkandan 03b4de722a
[downloader] Fix slow progress hooks
Closes #1301
3 years ago
pukkandan 48ee10ee8a
Fix conflict b/w id and ext in format selection
Closes #1282
3 years ago
pukkandan 883d4b1eec
[YoutubeDL] Write verbose header to logger 3 years ago
pukkandan 993191c0d5
Fix bug in c111cefa5d 3 years ago
pukkandan b836dc94f2
[outtmpl] Fix bug in expanding environment variables 3 years ago
pukkandan c111cefa5d
[downloader/ffmpeg] Improve simultaneous download and merge 3 years ago
pukkandan 975a0d0df9
Calculate more fields for merged formats
Closes #947
3 years ago
pukkandan a903d8285c
Fix bug in storyboards
Caused by 9359f3d4f0
3 years ago
pukkandan ed39cac53d Load archive only after printing verbose head
If there is some issue in loading archive, the verbose head should be visible in the logs
3 years ago
pukkandan a169858f24 Fix `check_formats` output being written to stdout when `-qv`
Closes #1229
3 years ago
pukkandan d1d5c08f29
[minicurses] Fix when printing to file
Closes #1215
3 years ago
pukkandan b5ae35ee6d
[cleanup] Misc cleanup 3 years ago
pukkandan 4e3b637d5b
Merge webm formats into mkv if thumbnails are to be embedded
This was originally implemented in 4d971a16b8 (#173) by @damianoamatruda
but was reverted in 3b297919e0
since it was unintentionally being triggered for `write_thumbnail` (See #500)
3 years ago
pukkandan 819e05319b
Improved progress reporting (See desc) (#1125)
* Separate `--console-title` and `--no-progress`
* Add option `--progress` to show progress-bar even in quiet mode
* Fix and refactor `minicurses`
* Use `minicurses` for all progress reporting
* Standardize use of terminal sequences and enable color support for windows 10
* Add option `--progress-template` to customize progress-bar and console-title
* Add postprocessor hooks and progress reporting

Closes: #906, #901, #1085, #1170
3 years ago
pukkandan 8472674399
[FixupM3u8] Do not run if merge is needed
We pass the relevant arguments to the merger, so separate fixup in redundant
3 years ago
pukkandan 519804a92f
bugfix for 80c03fa98f 3 years ago
pukkandan b11c04a8ae
Fix `-f mp4` behaving differently from youtube-dl 3 years ago
Felix S 9359f3d4f0
[extractor] Extract storyboards from SMIL manifests (#1128)
Authored by: fstirlitz
3 years ago
pukkandan e6f21b3d92
[docs,cleanup] Some minor refactoring and improve docs 3 years ago
pukkandan 3ae5e79774
[postprocessor] Add plugin support
Adds option `--use-postprocessor` to enable them
3 years ago
pukkandan 80c03fa98f
Allow empty output template to skip a type of file
Closes #760, #1111
3 years ago
pukkandan 360167b9fc Fix `--flat-playlist` when neither IE nor id is known 3 years ago
pukkandan 91dd88b90f
[outtmpl] Alternate form of format type `l` for `\n` delimited list 3 years ago
pukkandan 524e2e4fda
[outtmpl] Format type `U` for unicode normalization 3 years ago
pukkandan ff1c7fc9d3
Allow `0` in `--playlist-items` 3 years ago
pukkandan 4c88ff87fc
[build] Improve release process (#880)
* Automate more of the release process by animelover1984, pukkandan - closes #823
* Fix sha256 by nihil-admirari - closes #385
* Bring back brew taps by nao20010128nao #865
* Provide `--onedir` zip for windows by pukkandan - Closes #1024, #661, #705 and #890

Authored by: pukkandan, animelover1984, nihil-admirari, nao20010128nao
3 years ago
pukkandan b19404591a
Separate the options `--ignore-errors` and `--no-abort-on-error`
In youtube-dl, `-i` ignores both download and post-processing error, and
treats the download as successful even if the post-processor fails.

yt-dlp used to skip the entire video on either error and there was no
option to ignore the post-processing errors like youtube-dl does.

By splitting the option into two, now either just the download errors
(--no-abort-on-error, default on CLI) or all errors (--ignore-errors)
can be ignored as per the users' needs

Closes #893
3 years ago
pukkandan 77c4a9ef68
Download subtitles in order of `--sub-langs`
Closes #1041
3 years ago
Ashish Gupta 388bc4a640
[Hotstar] Add referer for subs (#1062)
Authored by: Ashish0804
3 years ago
pukkandan a21e0ab1a1
[ffmpeg] Add `aac_adtstoasc` when merging if needed
Related: #1039
3 years ago
pukkandan 7c37ff97d3
Allow alternate fields in outtmpl
Closes #899, #1004
3 years ago
pukkandan edf65256aa
[hls,aes] Fallback to native implementation for AES-CBC
and detect `Cryptodome` in addition to `Crypto`

Closes #935
Related: #938
3 years ago
pukkandan f5aa5cfbff
Add format type `B` for outtmpl to treat the value as bytes
This is useful to limit the filename to a certain number of bytes rather than characters
Closes #1003
3 years ago
pukkandan d5fe04f5c7
Fix `--compat-option no-direct-merge` 3 years ago
pukkandan 4614bc22c1 Allow `--force-write-archive` to work with `--flat-playlist`
Related: #876
3 years ago
pukkandan 8e5fecc88c
Handle more playlist errors with `-i` 3 years ago
pukkandan 409e18286e
Fix `extra_info` being reused across runs
58adec4677 was supposed to solve this, but ended up being an incomplete fix
Closes #727
3 years ago
pukkandan 8113999995
Fix `--compat-option playlist-index` 3 years ago
pukkandan e0493e90fc
fix bug in 88acdbc269 3 years ago
pukkandan 8b7491c8d1
Fix `add_info_extractor` when used via API
Bug from: 251ae04e6a
3 years ago
pukkandan 251ae04e6a
[lazy_extractor] Create instance only after pre-checking archive 3 years ago
pukkandan 1151c4079a
[extractor] Show video id in error messages if possible 3 years ago
pukkandan 88acdbc269
[extractor] Better error message for DRM (#729)
Closes #636
3 years ago
pukkandan da33e35b05
Don't try to merge with final extension
The formats may not be directly mergable into the final extension
3 years ago
pukkandan 58adec4677
Fix `extra_info` being reused across runs
Closes #727
3 years ago
pukkandan 9e598870dd
Fix `playlist_index` not obeying `playlist_start`
and add tests
Closes #720
3 years ago
pukkandan 8f18aca871
Let `--match-filter` reject entries early
Makes redundant: `--match-title`, `--reject-title`, `--min-views`, `--max-views`
3 years ago
pukkandan 3ad56b4236
Fix `-J` when there are failed videos 3 years ago
pukkandan 86c66b2d3e
Fix `-F` for extractors that directly return url
Related: #693
3 years ago
pukkandan 6c7274ecd2
Fix resuming of single formats when using --no-part
Closes #576
3 years ago
pukkandan ad3dc496bb Misc fixes - See desc
* Remove unnecessary uses of _list_from_options_callback
* Fix download tests - Bug from 6e84b21559
* Rename ExecAfterDownloadPP to ExecPP and refactor its tests
* Ensure _write_ytdl_file closes file handle on error - Potential fix for #517
3 years ago
pukkandan 2831b4686c Show libraries present in verbose head 3 years ago
pukkandan e9f4ccd19e
Add option `--replace-in-metadata` 3 years ago
pukkandan 3448870205
[docs] Fix some mistakes and improve doc 3 years ago
pukkandan b868936cd6 [cleanup] Misc 3 years ago
pukkandan 2b8a2973bd Allow entire infodict to be printed using `%()s`
Makes `--dump-json` redundant
3 years ago
pukkandan b7b04c782e Add option `--no-simulate` to not simulate even when `--print` or `--list...` are used
* Deprecates `--print-json`
* Some listings like `--list-extractors` are handled by `yt_dlp` and so are not affected by this. These have been documented as such

Addresses: https://github.com/ytdl-org/youtube-dl/issues/29675, https://github.com/ytdl-org/youtube-dl/issues/29580#issuecomment-882046305
3 years ago
pukkandan 6e84b21559
Fix bugs related to `sanitize_info`
Related: 8012d892bd (r54555230)
3 years ago
pukkandan b51d2ae3ca
Add compat-option `no-keep-subs`
Closes #630
3 years ago
pukkandan 124bc071ee
Fix wrong extension for intermediate files
Closes #632
3 years ago
pukkandan 8012d892bd
Ensure sanitization of infodict before printing to stdout
* `filter_requested_info` is renamed to a more appropriate name `sanitize_info`
3 years ago
pukkandan a46a815b05
[cleanup] Fix linter in 96fccc101f 3 years ago
pukkandan 96fccc101f [downloader] Allow streaming unmerged formats to stdout using ffmpeg
For this to work:
1. The downloader must be ffmpeg
2. The selected formats must have the same protocol
3. The formats must be downloadable by ffmpeg to stdout

Partial solution for: https://github.com/ytdl-org/youtube-dl/issues/28146, https://github.com/ytdl-org/youtube-dl/issues/27265
3 years ago
pukkandan dbf5416a20 [cleanup] Refactor some code 3 years ago
pukkandan f5510afef0
[FormatSort] Fix bug for audio with unknown codec 3 years ago
pukkandan e4f0275711
Add compat-option `no-clean-infojson` 3 years ago
pukkandan 7d1eb38af1
Add format types `j`, `l`, `q` for outtmpl
Closes #345
3 years ago
pukkandan 901130bbcf Expand and escape environment variables correctly in outtmpl
Fixes: https://www.reddit.com/r/youtubedl/comments/otfmq3/ytdlp_same_parameters_different_results
3 years ago
pukkandan ea05b3020d
Remove `asr` appearing twice in `-F` 3 years ago
pukkandan 3ba7740dd8
[downloader] Pass `info_dict` to `progress_hook`s 3 years ago
pukkandan 245524e6a3
Release 2021.07.21
and fix some typos
Closes #538
3 years ago
pukkandan e37d0efbd9
Fix bug where `original_url` was not propagated when `_type`=`url` 3 years ago
Matt Broadway 982ee69a74
Add option `--cookies-from-browser` to load cookies from a browser (#488)
* also adds `--no-cookies-from-browser`

Original PR: https://github.com/ytdl-org/youtube-dl/pull/29201
Authored by: mbway
3 years ago
pukkandan ae30b84072
Add field `live_status` 3 years ago
pukkandan cc9d1493c6
bugfix for 50fed816dd 3 years ago
pukkandan b35496d825
Add `only_once` param for `write_debug` 3 years ago
pukkandan c84aeac6b5
Add `only_once` param for `report_warning`
Related: https://github.com/yt-dlp/yt-dlp/pull/488#discussion_r667527297
3 years ago
pukkandan 50fed816dd
Errors in playlist extraction should obey `--ignore-errors`
Related: https://github.com/yt-dlp/yt-dlp/issues/535#issuecomment-883277272, https://github.com/yt-dlp/yt-dlp/issues/518#issuecomment-881794754
3 years ago
pukkandan 3f698246b2
Rename `NOTE` in `-F` to `MORE INFO`
since it's often confused to be the same as `format_note`
3 years ago
pukkandan cca80fe611 [youtube] Extract even more thumbnails and reduce testing
* Also fix bug where `_test_url` was being ignored

Ref: https://stackoverflow.com/a/20542029
Related: #340
3 years ago
pukkandan 0ba692acc8
[youtube] Extract more thumbnails
* The thumbnail URLs are hard-coded and their actual existence is tested lazily
* Added option `--no-check-formats` to not test them

Closes #340, Related: #402, #337, https://github.com/ytdl-org/youtube-dl/issues/29049
3 years ago
pukkandan 3b297919e0
Revert "Merge webm formats into mkv if thumbnails are to be embedded (#173)"
This reverts commit 4d971a16b8 by @damianoamatruda
Closes #500

This was wrongly checking for `write_thumbnail`
3 years ago
pukkandan b5ac45b197
Fix selectors `all`, `mergeall` and add tests
Bug from: 981052c9c6
3 years ago
pukkandan 3acf6d3856
[Funimation] Rewrite extractor (See desc) (#444)
* Support direct `/player/` URL
* Treat the different versions of an episode as different formats of a single video. So `experience_id` can no longer be used as the video `id` and the `episode_id` is used instead. This means that all existing archives will break
* Extractor options `language` and `version` to pre-select them
* Compat option `seperate-video-versions` to fall back to old behavior (including using the old video IDs)

Closes #428
3 years ago
pukkandan bc97cdae67
[cleanup] Fix linter and some typos
Related: https://github.com/ytdl-org/youtube-dl/pull/29398
3 years ago
pukkandan 169dbde946
Fixes for `--list` options (See desc)
1. Fix `--list-formats-old`
2. Allow listing with `--quiet`
3. Allow various listings to work together
4. Allow `--print` to work with listing
3 years ago
pukkandan 981052c9c6 Some minor fixes and refactoring (see desc)
* [utils] Fix issues with reversal
* check_formats should catch `DownloadError`, not `ExtractorError`
* Simplify format selectors with `LazyList` and `yield from`
3 years ago
pukkandan 49c258e18d
[youtube] Fix subtitle names for age-gated videos
Related: https://github.com/iv-org/invidious/pull/2205#issuecomment-868680486
3 years ago
pukkandan d3f62c1967
Fix `--throttled-rate` when using `--load-info-json` 3 years ago
pukkandan 5d3a0e794b
Add `--extractor-args` to pass extractor-specific arguments 3 years ago
Adrik 4513a41a72
Process videos when using `--ignore-no-formats-error` (#441)
Authored by: krichbanana
3 years ago
pukkandan 6033d9808d
Fix `--flat-playlist` when entry has no `ie_key` 3 years ago
pukkandan 51d9739f80
Add option `--throttled-rate` below which video data is re-extracted
Currently only for HTTP downloads

Closes #430, workaround for https://github.com/ytdl-org/youtube-dl/issues/29326
3 years ago
pukkandan e36d50c5dd
[websockets] Add `WebSocketFragmentFD` (#399)
Necessary for #392

Co-authored by: nao20010128nao, pukkandan
3 years ago
pukkandan f89b3e2d7a
Skip fixup of existing files and add `--fixup force` to force it 3 years ago
pukkandan fd7cfb6444
[cleanup] Refactor fixup 3 years ago
pukkandan 9fea350f0d
Fix id sanitization in filenames
Closes #415
4 years ago
pukkandan e858a9d6d3
[EmbedThumbnail] Add compat-option `embed-thumbnail-atomicparsley`
to force use of atomicparsley for embedding thumbnails in mp4

Related: #411
4 years ago
pukkandan 551f93885e
Ignore `images` formats from merge 4 years ago
pukkandan 8326b00aab
Allow `images` formats
Necessary for #343.

* They are identified by `vcodec=acodec='none'`
* These formats show as the worst in `-F`
* Any postprocessor that expects audio/video will be skipped
* `b*` and all related selectors will skip such formats
* This commit also does not add any selector for downloading such formats. They have to be explicitly requested by the `format_id`. Implementation of a selector is left for when #389 is resolved
4 years ago
pukkandan b0249bcaf0
Expand `--check-formats` to thumbnails
Closes #402
4 years ago
pukkandan 21cd8fae49
Use `NamedTemporaryFile` for `--check-formats` 4 years ago
pukkandan 8ba8714880
[EmbedThumbnail] Fix for already downloaded thumbnail 4 years ago
pukkandan 187986a857
Better error handling of syntax errors in `-f` 4 years ago
pukkandan 0181adefc6
[build] Build Windows x86 version with py3.7
and remove redundant tests
Closes #390

:ci skip

Co-authored by: pukkandan, shirt-dev
4 years ago
pukkandan 385a27fad1
Improve offset parsing in outtmpl 4 years ago
pukkandan 639f1cea92
Fix `%d` and empty default in outtmpl
Closes #388
4 years ago
pukkandan b5c5d84f60
Revert "[build] Build Windows x86 version with py3.8"
This reverts commit aa75e51f99.

See #390

This is being reverted instead of modified due to #388
4 years ago
pukkandan aa75e51f99
[build] Build Windows x86 version with py3.8
and remove redundant tests
:ci skip

Ao-authored by: pukkandan, shirt-dev
4 years ago
pukkandan 76a264ac9e
Make outtmpl more robust and catch errors early 4 years ago
pukkandan 324ad82006
[utils] Generalize `traverse_dict` to `traverse_obj` 4 years ago
pukkandan bd99f6e648
Add field `original_url` with the user-inputted URL
So that they can be processed by `--parse-metadata` for example

`webpage_url` is the same, but may be modified by the extractor
4 years ago
pukkandan ecb5419149
Make more fields available for `--print` when used with `--flat-playlist` 4 years ago
pukkandan 87ea7dfc04
Fix filename sanitization
Bug from 752cda3880
4 years ago
pukkandan 752cda3880
Fix and refactor `prepare_outtmpl`
The following tests would have failed previously:
%(id)d %(id)r
%(ext)s-%(ext|def)d
%(width|)d
%(id)r %(height)r
%(formats.0)r
%s
4 years ago
felix cc52de4356
[cleanup] Point all shebang to `python3` (#372)
Authored by: fstirlitz
4 years ago
pukkandan 885cc0b75c
[embedthumbnail] Embed if any thumbnail was downloaded, not just the best 4 years ago
pukkandan ae8f99e648
Remove `None` values from `info.json` 4 years ago
pukkandan 835a1478b4
Write messages to `stderr` when both `quiet` and `verbose` 4 years ago
pukkandan 120fe5134a
Pre-check archive and filters during playlist extraction
This makes `--break-on-existing` much faster.
It also helps `--break-on-reject` if the playlist extractor can extract the relevant fields
4 years ago
pukkandan 56a8fb4f77
Refactor `__process_playlist` using `LazyList` 4 years ago
pukkandan c77495e3a4
[cleanup] `_match_entry` 4 years ago
pukkandan c19bc311cb
[cleanup] Refactor updater
The updater now uses `.update.run_update` and not `.update.update_self`.
Although I don't expect anyone to be using the updater via API, a wrapper `update_self` is provided for compatibility just in case
4 years ago
pukkandan bc516a3f3c
Sanitize and sort playlist thumbnails
Closes #341
4 years ago
pukkandan 18e674b4f6
[ffmpeg] Download and merge in a single step if possible 4 years ago
pukkandan 28b0eb0f65
[cleanup] See desc
* Remove struct from `embedthumbnail`
* Use bullet lists in readme where numbered list don't make sense
* Fix error introduced in 9c2b75b561 when `ie_result` is `None`
4 years ago
pukkandan 9c2b75b561
Field `additional_urls` to download additional videos from metadata 4 years ago
pukkandan 4ec82a72bb
Ensure `post_extract` and `pre_process` only run once
Previously, they ran once for each format requested
4 years ago
pukkandan 07cce701de
[cleanup] linter, code formatting and readme 4 years ago
pukkandan ff2751ac9c
[youtube] Always extract `maxresdefault` thumbnail
Fixes: https://github.com/ytdl-org/youtube-dl/issues/29049
4 years ago
pukkandan a06916d98e
[extractor] Add `write_debug` and `get_param` 4 years ago
pukkandan 681de68e9d
Write thumbnail of playlist
Related: https://github.com/ytdl-org/youtube-dl/pull/28872, https://github.com/ytdl-org/youtube-dl/pull/28860
This is slightly different from the above PRs in that this downloads the playlist's thumbnail instead of the uploader's profile picture. But for youtube channel URLs these are the same
4 years ago
pukkandan 7aee40c13c
Fix bug in listing subtitles
Bug introduced by: 2412044c90
4 years ago
pukkandan 2412044c90
Add field `name` for subtitles
Co-authored by: pukkandan, tpikonen

Based on: #310, https://github.com/ytdl-org/youtube-dl/pull/26112
4 years ago
pukkandan fe346461ff
Fix `--check-formats` when there is network error 4 years ago
pukkandan 53c18592d3
Add option `--print`
Deprecates: `--get-description`, `--get-duration`, `--get-filename`, `--get-format`, `--get-id`, `--get-thumbnail`, `--get-title`, `--get-url`
Closes #295
4 years ago
pukkandan e632bce2e4
[options] Refactor callbacks 4 years ago
pukkandan 0760b0a7e2
Standardize `write_debug` 4 years ago
pukkandan d908aa636a
[cleanup] Fix typos 4 years ago
pukkandan 4171221823
Add compat-option `no-attach-infojson` 4 years ago
pukkandan 53ed7066ab
Option `--compat-options` to revert some of yt-dlp's changes
* Deprecates `--list-formats-as-table`, `--list-formats-old`
4 years ago
pukkandan a61f4b287b
Deprecate support for python versions < 3.6
Closes #267
4 years ago
pukkandan 41d1cca328
Update to ytdl-commit-a726009
[blinkx] Remove extractor
a726009987
4 years ago
pukkandan 717297545b
Fix `playlist_index` and add `playlist_autonumber` (#302)
Now `playlist_index` is always the position of the video in the actual playlist and `playlist_autonumber` is the position of the item in the playlist queue
4 years ago
pukkandan e8e738406a
Add experimental option `--check-formats` to test the URLs before format selection 4 years ago
pukkandan e625be0d10
Improve output template internal formatting
* Allow slicing lists/strings using `field.start🔚step`
* A field can also be used as offset like `field1+num+field2`
* A default value can be given using `field|default`
* Capture all format strings and set it to `None` if invalid. This prevents invalid fields from causing errors
4 years ago
pukkandan 848887eb7a
[downloader] Fix `quiet` and `to_stderr` 4 years ago
pukkandan 3158150cb7
[utils] Add `network_exceptions` 4 years ago
pukkandan f59ae58163
Fix number of digits in `%(playlist_index)s`
When used with `--playlist-(items|start|end)`, the number of digits should depend on the last index in the playlist, not number of items
4 years ago
pukkandan 0d1bb027aa
Move option warnings to `YoutubeDL`
Previously, these warnings did not obey `--no-warnings` and did not output colors
4 years ago
pukkandan 4cd0a709aa
Fix `preload_download_archive` writing verbose message to `stdout`
* And move it after all deprecated warnings
4 years ago
pukkandan 58f197b76c
Revert "[core] be able to hand over id and title using url_result"
This reverts commit 0704d2224b.

This is a commit from `youtube-dlc`. It is not clear what the original purpose of this was. It seems to be a way for extractors to pass `title` and `id` through when the entry is processed by another extractor

* But `title` can already be passed through using `url_transparent`
* `id` is never supposed to be passed through since it could cause issues with archiving
4 years ago
pukkandan 598d185db1
Fix case sensitivity of format selector
Bug introduced in f8d4ad9ab0
4 years ago
pukkandan a439a3a45c
Improve output template (see desc)
* Objects can be traversed like `%(field.key1.key2)s`
* A number can be added to the field as `%(field+n)s`
* Deprecates `--autonumber-start`
4 years ago
pukkandan 26e2805c3f
Add option `--skip-playlist-after-errors`
Allows to skip the rest of a playlist after a given number of errors are encountered
4 years ago
pukkandan c32b0aab8a
Improve --sub-langs (see desc)
* Treat `--sub-langs` entries as regex
* `all` can be used to refer to all the subtitles
* the language code can be prefixed with `-` to exclude it
* Deprecates `--all-subs`
Closes #253
4 years ago
pukkandan b7da73eb19
Add option `--ignore-no-formats-error`
* Ignores the "no video format" and similar errors
* Experimental - Some extractors may still throw these errors
4 years ago
pukkandan 921b76cab8
Ensure `mergeall` selects best format when multistreams are disabled 4 years ago
pukkandan 56d868dbb7
Allow running some `postprocessors` before actual download 4 years ago
pukkandan e01d6aa435
Fix `mergeall` when requested formats are unavailable 4 years ago
pukkandan 52a8a1e1b9
Option to choose different downloader for different protocols
* Renamed `--external-downloader-args` to `--downloader-args`
* Added `native` as an option for the downloader
* Use similar syntax to `--downloader-args` etc. Eg: `--downloader dash:native --downloader aria2c`
* Deprecated `--hls-prefer-native` and `--hls-prefer-ffmpeg` since the same can now be done with `--downloader "m3u8:native"` and `m3u8:ffmpeg` respectively
* Split `frag_urls` protocol into `m3u8_frag_urls` and `dash_frag_urls`
* Standardize shortening of protocol names with `downloader.shorten_protocol_name`
4 years ago
pukkandan f8d4ad9ab0
Format selector `mergeall` to download and merge all formats 4 years ago
pukkandan eff635394a
Ability to select nth best format
Eg: -f `bv*.2`

* Also cleaned up the related format selection code
4 years ago
pukkandan b050d210df
Fix some typos and linter 4 years ago
pukkandan 143db31d48
Parse metadata from multiple fields
Closes #196
4 years ago
pukkandan 498f560638
Ability to load playlist infojson
* If `--no-clean-infojson` is given, the video ids are saved/loaded from in the infojson along with their playlist index
* If a video entry that was not saved is requested, we fallback to using `webpage_url` to re-extract the entries

Related: https://github.com/yt-dlp/yt-dlp/issues/190#issuecomment-804921024
4 years ago
pukkandan 394dcd4486
Write current epoch to infojson when using `--no-clean-infojson`
Closes #191
4 years ago
pukkandan 4d971a16b8
Merge webm formats into mkv if thumbnails are to be embedded (#173)
Co-authored by: Damiano Amatruda <damiano.amatruda@outlook.com>
4 years ago
Damiano Amatruda 3561530776
Parse resolution in info dictionary (#173) 4 years ago
Damiano Amatruda 4690688658
Use headers and cookies when downloading subtitles (#173) 4 years ago
pukkandan 23c1a66730 Make sure `post_hook` gets the final filename 4 years ago
pukkandan a515a78dd3
fix some typos and linter 4 years ago
pukkandan 75d43ca080
Option to keep private keys in the infojson
Options: --clean-infojson, --no-clean-infojson

Related: https://github.com/yt-dlp/yt-dlp/issues/42#issuecomment-800778391
4 years ago
pukkandan 5226731e2d
Recursively remove private keys from infojson
Related: https://github.com/yt-dlp/yt-dlp/issues/42
4 years ago
pukkandan dcf64d43e0
[movefiles] Fix bugs and make more robust 4 years ago
pukkandan 10db0d2f57
Update to ytdl-commit-3be0980
3be098010f
4 years ago
pukkandan e92caff5d5
Refactor (See desc)
* Create `FFmpegPostProcessor.real_run_ffmpeg` that can accept multiple input/output files along with switches for each
* Rewrite `cli_configuration_args` and related functions
* Create `YoutubeDL._ensure_dir_exists` - this was previously defined in multiple places
4 years ago
pukkandan 0a473f2f0f
More improvements to HLS/DASH external downloader code
* Fix error when there is no `protocol` in `info_dict`
* Move HLS byte range detection to `Aria2cFD` so that the download will fall back to the native downloader instead of ffmpeg
* Fix bug with getting no fragments in DASH
* Convert `check_results` in `can_download` to a generator
4 years ago
shirt-dev d7009caa03
Improve HLS/DASH external downloader code (#162)
Authored by: shirt
4 years ago
pukkandan 5ef7d9bdd8
Release 2021.03.01 4 years ago