christoph-heinrich
e121e3cee7
[cleanup] Minor fixes ( #4096 )
...
Authored by: christoph-heinrich
2 years ago
Evan Spensley
145c5a83a8
[extractor/GoogleDrive] Add folder extractor ( #4009 )
...
Closes #3388
Authored by: evansp, pukkandan
2 years ago
pukkandan
f3c0c77304
[extractor] Handle `json_ld` with multiple `@type`s
...
Closes : #4022
3 years ago
pukkandan
56ba69e4c9
[cleanup] Misc fixes
...
Closes #4027
3 years ago
Aurélien Grosdidier
d05460e5fe
[extractor/FranceCulture] Fix extractor ( #3874 )
...
Closes #3742
Authored by: aurelg, pukkandan
3 years ago
pukkandan
b7c47b7438
[extractor] Add `_search_json`
...
All fetching of JSON objects should eventually be done with this function
but only `youtube` is being refactored for now
3 years ago
Lesmiscore
0bea4fd807
[extractor/0000studio] Add extractors ( #3959 )
...
Authored by: Lesmiscore
3 years ago
pukkandan
1890fc6389
[cleanup] Misc fixes
...
Cherry-picks from: #3498 , #3947
Related: #3949 , https://github.com/yt-dlp/yt-dlp/issues/1839#issuecomment-1140313836
Authored by: pukkandan, flashdagger, gamer191
3 years ago
pukkandan
c4910024f3
[extractor] Fix bug in 617f658b7e
...
While the function signature don't enforce it, some IEs that override
`_download_webpage_handle` assume all optional arguments to be keyword-only
Closes #3954
3 years ago
siddharth ravikumar
e50c3500b4
[extractor/npr] Use stream url from json-ld ( #3455 )
...
Closes #1934
Authored by: r5d
3 years ago
pukkandan
09d02ea429
[extractor] Fix bug in f95b9dee45
...
Closes #3951
3 years ago
pukkandan
f95b9dee45
[extractor] Add dev option `--load-pages`
3 years ago
pukkandan
617f658b7e
[extractor, cleanup] Refactor `_download_...` methods
3 years ago
Lesmiscore
222a230871
[extractor/common] Recognize `src` attribute from HTML5 media elements ( #3899 )
...
Authored by: Lesmiscore
3 years ago
coletdjnz
ee27297f82
[extractor/youtube] Fix initial player response extraction
...
Authored by: pukkandan, coletdjnz
3 years ago
pukkandan
8a82af3511
[cleanup] Misc fixes and cleanup
...
Closes #3780 , Closes #3853 , Closes #3850
3 years ago
pukkandan
c487cf0010
[cleanup] Misc
3 years ago
pukkandan
2762dbb17e
[compat] Add `functools.cached_property`
3 years ago
pukkandan
21633673c3
[cleanup] Minor fixes
3 years ago
pukkandan
2414649192
[cleanup] Misc cleanup
3 years ago
pukkandan
82d020804d
[extractor] Use classmethod/property where possible
...
and refactor lazy extractors accordingly.
This reduces the need to create extractor instances
3 years ago
pukkandan
8dcce6a89c
[extractor] Document netrc machines
...
Closes #3169
3 years ago
pukkandan
d4736fdb43
Remove warning for videos with an empty title
3 years ago
pukkandan
3fe75fdc80
[cleanup] Misc fixes (see desc)
...
* Do not warn when fixup is skipped for existing file
* [fragment] Fix `--skip-unavailable-fragments` for HTTP Errors
* [utils] write_string: Fix bug in 59f943cd50
* [utils] parse_codecs: Subtitle codec is generally referred to as `scodec`. https://github.com/yt-dlp/yt-dlp/pull/2174#discussion_r790156048
* [docs] Remove note about permissions. Closes #3597
3 years ago
pukkandan
1d485a1a79
[cleanup] Misc fixes
...
Closes #3565 , https://github.com/yt-dlp/yt-dlp/issues/3514#issuecomment-1105944364
3 years ago
pukkandan
0a5a191a2a
Improve `--clean-infojson`
...
It should not removes fields that may be needed for `--load-infojson`.
Eg: `_ffmpeg_args`, `_has_drm`
3 years ago
Elyse
a076c1f97a
[extractor] Update `manifest_url`s after redirect ( #3575 )
...
Authored by: elyse0
3 years ago
Elyse
779da8e31b
[extractor] Update dash `manifest_url` after redirects ( #3563 )
...
Closes #2696
Authored by: elyse0
3 years ago
felix
77f9033095
[compat] Split into sub-modules ( #2173 )
...
Authored by: fstirlitz, pukkandan
3 years ago
pukkandan
f82711587c
[cleanup] Sort imports
...
Using https://github.com/PyCQA/isort
isort -m VERTICAL_HANGING_INDENT --py 36 -l 80 --rr -n --tc .
3 years ago
pukkandan
86e5f3ed2e
[cleanup] Upgrade syntax
...
Using https://github.com/asottile/pyupgrade
1. `__future__` imports and `coding: utf-8` were removed
2. Files were rewritten with `pyupgrade --py36-plus --keep-percent-format`
3. f-strings were cherry-picked from `pyupgrade --py36-plus`
Extractors are left untouched (except removing header) to avoid unnecessary merge conflicts
3 years ago
pukkandan
f9934b9614
[cleanup] Mark some compat variables for removal ( #2173 )
...
Authored by fstirlitz, pukkandan
3 years ago
felix
cfb0511d82
[cleanup] Remove unused code paths ( #2173 )
...
Notes:
* `_windows_write_string`: Fixed in 3.6
* https://bugs.python.org/issue1602
* PEP: https://www.python.org/dev/peps/pep-0528
* Windows UTF-8 fix: Fixed in 3.3
* https://bugs.python.org/issue13216
* `__loader__`: is always present in 3.3+
* https://bugs.python.org/issue14646
* `workaround_optparse_bug9161`: Fixed in 2.7
* https://bugs.python.org/issue9161
Authored by: fstirlitz
3 years ago
coletdev
61d3665d9d
[youtube] Fix uploader for collaborative playlists ( #3332 )
...
Authored by: coletdjnz
3 years ago
pukkandan
a44ca5a470
[cleanup] Misc fixes
...
Closes https://github.com/yt-dlp/yt-dlp/pull/3213 , Closes https://github.com/yt-dlp/yt-dlp/pull/3117
Related: https://github.com/yt-dlp/yt-dlp/issues/3146#issuecomment-1077323114 , https://github.com/yt-dlp/yt-dlp/pull/3277#discussion_r841019671 , a825ffbffa (commitcomment-68538986)
, https://github.com/yt-dlp/yt-dlp/issues/2360 , 5fa3c9a88f (r70393519)
, 5fa3c9a88f (r70393254)
3 years ago
pukkandan
04f3fd2c89
[cleanup] Use `_html_extract_title`
3 years ago
Fam0r
fbfde1c3e6
[elonet] Rewrite extractor ( #3277 )
...
Closes #2911
Authored by: Fam0r, pukkandan
3 years ago
pukkandan
90137ca4be
[utils] Add `filter_dict`
3 years ago
pukkandan
ae6a1b9585
[docs] Minor improvements
...
Closes #3127 , Closes #3081 , Closes #3177
3 years ago
pukkandan
231025c463
Fix bug in 52efa4b312
...
Closes #3173
3 years ago
pukkandan
700ccbe3f1
[extractor] Allow control characters inside json
...
Closes #3174
3 years ago
pukkandan
52efa4b312
[extractor] Add `_perform_login` function ( #2943 )
...
* Adds new functions `_initialize_pre_login` and `_perform_login` as part of the extractor API
* Adds `ie.supports_login` to the public API
3 years ago
coletdjnz
a825ffbffa
[extractor] Support merging subtitles with data
...
Authored-by: coletdjnz
3 years ago
pukkandan
08d30158ec
[cleanup, docs] Misc cleanup
...
Closes #2828 , closes #2734 , closes #2802 , closes #2937
3 years ago
pukkandan
497d2fab6c
[utils] Better traceback for `ExtractorError`
3 years ago
Zenon Mousmoulas
27231526ae
[ant1newsgr] Add extractor ( #1982 )
...
Authored by: zmousm
3 years ago
pukkandan
a10aa588b0
[FormatSort] Consider `acodec`=`ogg` as `vorbis`
3 years ago
pukkandan
7265a2190c
Fix doubling of `video_id` in `ExtractorError`
3 years ago
coletdev
ded9f32667
[extractor] Support `--mark-watched` without `_NETRC_MACHINE` ( #2939 )
...
Authored by: coletdjnz
3 years ago
Lesmiscore (Naoya Ozaki)
77cc7c6e60
[nhk] Add support for NHK for School ( #2850 )
...
Authored by: Lesmiscore
3 years ago
pukkandan
ffa89477ea
[extractor] Fix for manifests without period duration
...
Closes #2705
Authored by: dirkf, pukkandan
3 years ago
pukkandan
297e9952b6
[extractor] Allow `http_headers` to be specified for `thumbnails`
3 years ago
pukkandan
079a7cfc71
[downloader] Do not use aria2c for non-native `m3u8`
...
Closes #2718
3 years ago
pukkandan
455a15e2dc
[cleanup,docs] Minor fixes
...
Closes #2541 , #2484
3 years ago
pukkandan
85553414ae
[generic] Allow further processing of json_ld URL
...
Closes #2578
3 years ago
pukkandan
fc08bdd6ab
[extractor] Allow non-fatal `title` extraction
3 years ago
pukkandan
2edb38e8ca
[extractor] Extract video inside `Article` json_ld
...
Closes #2448
3 years ago
pukkandan
f40ee5e9a0
[extractor] Add convinience function `_yes_playlist`
3 years ago
pukkandan
6970b6005e
[cleanup] Minor fixes
...
Closes #2334
3 years ago
coletdjnz
6c73052c0a
[youtube] Extract channel subscriber count ( #2399 )
...
Closes #2350
* Adds `channel_follower_count` field
Authored-by: coletdjnz
3 years ago
pukkandan
311b6615d8
[extractor] Improve `url_result` and related
3 years ago
pukkandan
135dfa2c7e
[extractor,cleanup] Use `_search_nextjs_data`
3 years ago
MinePlayersPE
426764371f
[iq.com] Add extractors ( #2354 )
...
Closes #704
Authored by: MinePlayersPE
3 years ago
teridon
8bcd404818
[digitalconcerthall] Add extractor ( #1931 )
...
Authored by: teridon
3 years ago
coletdjnz
f0d785d3ed
[youtube:tab] Extract more playlist metadata ( #2069 )
...
* Add fields modified_date, modified_timestamp
* Add field playlist_count
* [youtube:tab] Extract view_count, playlist_count, modified_date
Authored by: coletdjnz, pukkandan
3 years ago
chris
f522573787
[extractor] Extract chapters from JSON-LD ( #2031 )
...
Authored by: iw0nderhow, pukkandan
3 years ago
nixxo
7592749cbe
[extractor] Extract thumbnails from JSON-LD ( #2195 )
...
Authored by: nixxo
3 years ago
Felix S
4afa3ec4b6
[extractor] Detect more subtitle codecs in MPD manifests ( #2174 )
...
Authored by: fstirlitz
3 years ago
MinePlayersPE
d43de6821c
[GameJolt] Add extractors ( #2036 )
...
Authored by: MinePlayersPE
3 years ago
siddharth
774a46c53d
[npr] Make SMIL extraction non-fatal ( #2099 )
...
Closes #1934
Authored by: r5d
3 years ago
pukkandan
b69fd25c25
[cleanup] Misc cleanup
...
Closes #1942 #1976 #2020 #2058 #1984
3 years ago
The Hatsune Daishi
adbc4ec4bb
[dash,youtube] Download live from start to end ( #888 )
...
* Add option `--live-from-start` to enable downloading live videos from start
* Add key `is_from_start` in formats to identify formats (of live videos) that downloads from start
* [dash] Create protocol `http_dash_segments_generator` that allows a function to be passed instead of fragments
* [fragment] Allow multiple live dash formats to download simultaneously
* [youtube] Implement fragment re-fetching for the live dash formats
* [youtube] Re-extract dash manifest every 5 hours (manifest expires in 6hrs)
* [postprocessor/ffmpeg] Add `FFmpegFixupDuplicateMoovPP` to fixup duplicated moov atoms
Known issue: Ctrl+C doesn't work on Windows when downloading multiple formats
Closes #1521
Authored by: nao20010128nao, pukkandan
3 years ago
Zenon Mousmoulas
d5c3254889
[extractor] Support default implicit graph in JSON-LD ( #1983 )
...
Original PR: https://github.com/ytdl-org/youtube-dl/pull/30229
Per W3C JSON-LD v1.1 §4.9 (non-normative ref):
When a JSON-LD document's top-level structure is a map that contains
no other keys than @graph and optionally @context (properties that
are not mapped to an IRI or a keyword are ignored), @graph is
considered to express the otherwise implicit default graph.
Authored by: zmousm
3 years ago
pukkandan
a8549f19e7
[tiktok] Fix 53dad39e30
3 years ago
pukkandan
39ca3b5c7f
[extractor] Standardize `_live_title`
3 years ago
pukkandan
19188702ef
[FormatSort] Prevent incorrect deprecation warning
...
Closes #1981
3 years ago
The Hatsune Daishi
66f4c04e50
[extractor] Add `_search_nuxt_data` ( #1921 )
...
Authored by: nao20010128nao
3 years ago
pukkandan
d2b2fca53f
[extractor] Ignore errors in comment extraction when `-i` is given
...
Closes #1787
3 years ago
pukkandan
2c4aaaddc9
[lazy_extractors] Fix for search IEs
...
Closes #1851
3 years ago
pukkandan
ee8dd27a73
[cleanup] Add deprecation warnings
3 years ago
pukkandan
f304da8a29
[cleanup] Misc cleanup
...
Closes #1805 , closes #1800
3 years ago
pukkandan
f2ebc5c7be
Option `--wait-for-video` to wait for scheduled streams
3 years ago
pukkandan
8f122fa070
[extractor] Extract `average_rating` from JSON-LD
...
Eg: Crunchyroll
3 years ago
pukkandan
234416e4bf
[downloader/ffmpeg] Fix for direct videos inside mpd manifests
...
Closes #1751
3 years ago
nyuszika7h
6b993ca765
[hls] Better FairPlay DRM detection ( #1661 )
...
Authored by: nyuszika7h
3 years ago
pukkandan
9ac24e235e
[curiositystream] Add more metadata
...
Closes #1568
3 years ago
Lauren Liberda
f98709af31
[extractor] Add `_search_nextjs_data` ( #1386 )
...
Authored by: selfisekai
3 years ago
pukkandan
34921b4345
[utils] Add `join_nonempty`
3 years ago
pukkandan
aeb2a9ad27
[FormatSort] `eac3` is better than `ac3`
3 years ago
pukkandan
0930b11fda
[docs,cleanup] Improve docs and minor cleanup
...
Closes #1387 , #1404 , #1408 , #1485 , #1415 , #1450 , #1492
3 years ago
pukkandan
10beccc980
[FormatSort] Fix some fields' defaults
...
Closes #1479
3 years ago
pukkandan
0db3bae879
[extractor] Fix some errors being converted to `ExtractorError`
3 years ago
pukkandan
96565c7e55
[cleanup] Add keyword automatically to SearchIE descriptions
...
and some minor cleanup of docs
3 years ago
pukkandan
ec11a9f4a2
[minicurses] Add more colors
3 years ago
pukkandan
49a57e70a9
[cleanup] misc
3 years ago
pukkandan
176f1866cb
Add HDR information to formats
3 years ago
pukkandan
e820fbaa6f
Do not verify thumbnail URLs by default
...
Partially reverts cca80fe611
and 0ba692acc8
Unless `--check-formats` is specified, this causes yt-dlp to return incorrect thumbnail urls.
See https://github.com/yt-dlp/yt-dlp/issues/340#issuecomment-877909966 , #402
But the overhead in general use is not worth it
Closes #694 , #725
3 years ago
pukkandan
ec3f6640c1
[crunchyroll] Add season to flat-playlist
...
Closes #1319
3 years ago
pukkandan
a0c716bb61
[instagram] Show appropriate error when login is needed
...
Closes #1264
3 years ago
pukkandan
6993f78d1b
[extractor,utils] Detect more codecs/mimetypes
...
Fixes: https://github.com/ytdl-org/youtube-dl/issues/29943
3 years ago
pukkandan
a2160aa45f
[extractor] Generalize `getcomments` implementation
3 years ago
pukkandan
cc16383ff3
[extractor] Simplify search extractors
3 years ago
pukkandan
ba10757412
[extractor] Detect `EXT-X-KEY` Apple FairPlay
3 years ago
pukkandan
b5ae35ee6d
[cleanup] Misc cleanup
3 years ago
pukkandan
819e05319b
Improved progress reporting (See desc) ( #1125 )
...
* Separate `--console-title` and `--no-progress`
* Add option `--progress` to show progress-bar even in quiet mode
* Fix and refactor `minicurses`
* Use `minicurses` for all progress reporting
* Standardize use of terminal sequences and enable color support for windows 10
* Add option `--progress-template` to customize progress-bar and console-title
* Add postprocessor hooks and progress reporting
Closes : #906 , #901 , #1085 , #1170
3 years ago
Felix S
9359f3d4f0
[extractor] Extract storyboards from SMIL manifests ( #1128 )
...
Authored by: fstirlitz
3 years ago
pukkandan
e6f21b3d92
[docs,cleanup] Some minor refactoring and improve docs
3 years ago
ChillingPepper
3cf4b91dc5
[SovietsCloset] Add duration from m3u8 ( #908 )
...
Authored by: ChillingPepper
3 years ago
pukkandan
1813a6ccd4
[youtube] Fix `--mark-watched` with `--cookies-from-browser`
...
Closes #1019
3 years ago
pukkandan
bd6f722de8
dump files should obey `--trim-filename` ( #1043 )
...
Authored by: sulyi
3 years ago
DigitalDJ
f9cc0161e6
[extractor] Fix root-relative URLs in MPD ( #1006 )
...
Authored by: DigitalDJ
3 years ago
pukkandan
0001fcb586
Add option `--netrc-location`
...
Closes #792 , #963
3 years ago
pukkandan
198e3a04c9
[FormatSort] Remove priority of `lang`
3 years ago
pukkandan
6251555f1c
[downloader/ffmpeg] Support for DASH manifests (experimental)
...
Closes #159
3 years ago
pukkandan
1151c4079a
[extractor] Show video id in error messages if possible
3 years ago
pukkandan
88acdbc269
[extractor] Better error message for DRM ( #729 )
...
Closes #636
3 years ago
pukkandan
3fb4e21b38
[lazy_extractors] Fix `suitable` and add flake8 test
3 years ago
pukkandan
5ad28e7ffd
[extractor] Common function `_match_valid_url`
3 years ago
pukkandan
a8731fcc1d
minor bugfixes
...
bugs due to be2fc5b212
, e9f4ccd19e
3 years ago
pukkandan
be2fc5b212
[extractor] Detect `sttp` as subtitles in MPD
...
Closes #656
Solution by: fstirlitz
3 years ago
pukkandan
b868936cd6
[cleanup] Misc
3 years ago
pukkandan
28f436bad0
[extractor] Reset non-repeating warnings per video
3 years ago
MinePlayersPE
3dbb2a9dcb
[RCTIPlus] Support events and TV ( #625 )
...
Authored by: MinePlayersPE
3 years ago
LE
68f5867cf0
[CBS] Add fallback ( #579 )
...
Related: https://github.com/ytdl-org/youtube-dl/issues/29564
Authored-by: llacb47, pukkandan
3 years ago
pukkandan
dbf5416a20
[cleanup] Refactor some code
3 years ago
pukkandan
f5510afef0
[FormatSort] Fix bug for audio with unknown codec
3 years ago
pukkandan
ae30b84072
Add field `live_status`
3 years ago
pukkandan
0ba692acc8
[youtube] Extract more thumbnails
...
* The thumbnail URLs are hard-coded and their actual existence is tested lazily
* Added option `--no-check-formats` to not test them
Closes #340 , Related: #402 , #337 , https://github.com/ytdl-org/youtube-dl/issues/29049
3 years ago
Felix S
da1c94ee45
[generic] Extract previously missed subtitles ( #515 )
...
* [generic] Extract subtitles in cases missed previously
* [common] Detect discarded subtitles in SMIL manifests
* [generic] Extract everything in the SMIL manifest
Authored by: fstirlitz
3 years ago
felix
198f7ea89e
[extractor] Allow extracting multiple groups in `_search_regex`
...
From #497 , Authored by: fstirlitz
3 years ago
pukkandan
6606817a86
[utils] Add `variadic`
3 years ago
pukkandan
4bb6b02f93
Improve `extractor_args` parsing
3 years ago
pukkandan
46890374f7
[extractor] Minor improvements (See desc)
...
1. Allow removal of login hint - extractors can set their own login hint as part of `msg`
2. Cleanup `_merge_subtitles` signature
3 years ago
pukkandan
60755938b3
[extractor] Prevent unnecessary download of hls manifests
...
and refactor `hls_split_discontinuity` code
3 years ago
pukkandan
5d3a0e794b
Add `--extractor-args` to pass extractor-specific arguments
3 years ago
pukkandan
e36d50c5dd
[websockets] Add `WebSocketFragmentFD` ( #399 )
...
Necessary for #392
Co-authored by: nao20010128nao, pukkandan
3 years ago
pukkandan
d0fb4bd16f
[pornhub] Extract `cast`
...
Closes #406 , https://github.com/ytdl-org/youtube-dl/pull/27384
4 years ago
felix
cdb19aa4c2
[downloader/mhtml] Add new downloader ( #343 )
...
This downloader is intended to be used for streams that consist of a
timed sequence of stand-alone images, such as slideshows or thumbnail
streams
This can be used for implementing:
https://github.com/ytdl-org/youtube-dl/issues/4974#issue-58006762
https://github.com/ytdl-org/youtube-dl/issues/4540#issuecomment-69574231
https://github.com/ytdl-org/youtube-dl/pull/11185#issuecomment-335554239
https://github.com/ytdl-org/youtube-dl/issues/9868
https://github.com/ytdl-org/youtube-dl/pull/14951
Authored by: fstirlitz
4 years ago
pukkandan
8326b00aab
Allow `images` formats
...
Necessary for #343 .
* They are identified by `vcodec=acodec='none'`
* These formats show as the worst in `-F`
* Any postprocessor that expects audio/video will be skipped
* `b*` and all related selectors will skip such formats
* This commit also does not add any selector for downloading such formats. They have to be explicitly requested by the `format_id`. Implementation of a selector is left for when #389 is resolved
4 years ago
pukkandan
3b1fe47d84
Release 2021.06.08
4 years ago
felix
e2efe599aa
[common] Fix FourCC fallback when parsing ISM ( #372 )
...
In some DASH manifests, the FourCC attribute is actually present,
but empty. We thus apply the same fallback to 'AACL' that we do
when the attribute is entirely absent.
Authored by: fstirlitz
4 years ago
pukkandan
b0089e8992
[fancode] Add extractor (#316,#354)
...
Closes #269 , #363
Authored by: rmsmachine
4 years ago
pukkandan
adddc50cbf
[extractor] Functions to parse socket.io response as json
...
Authored by: pukkandan, llacb47
4 years ago
pukkandan
37a3bb66a7
[extractor] Allow `note=False` when extracting manifests
4 years ago
pukkandan
177877c544
[extractor] Always prefer native hls downloader by default
...
When the manifest is not downloadable by native downloader, it already is able to detect it and switch to `ffmpeg`. So there doesn't seem to be a reason anymore to use ffmpeg as the preferred downloader
4 years ago
pukkandan
3907333c5d
[extractor] Skip subtitles without URI in m3u8 manifests
...
Closes #339
Authored by: hheimbuerger
4 years ago
pukkandan
9d5d4d64f8
[youtube] Better message when login required
4 years ago
pukkandan
07cce701de
[cleanup] linter, code formatting and readme
4 years ago
pukkandan
a06916d98e
[extractor] Add `write_debug` and `get_param`
4 years ago
pukkandan
2412044c90
Add field `name` for subtitles
...
Co-authored by: pukkandan, tpikonen
Based on: #310 , https://github.com/ytdl-org/youtube-dl/pull/26112
4 years ago