Commit Graph

240 Commits (83cc7b8aae1328b0d148b631357f753c61c38a29)

Author SHA1 Message Date
pukkandan 3975b4d2e8
Allow extractors to specify section_start/end for clips 2 years ago
pukkandan 612f2be5d3
Bugfix for 7b2c3f47c6 2 years ago
pukkandan 7b2c3f47c6
[cleanup] Misc 2 years ago
pukkandan 8072ef2bbd
[extractor/BiliIntl] Fix metadata extraction
Closes #4116
2 years ago
coletdev f0bc6e2019
[extractor] Add `default` parameter to `_search_json` (#4057)
Authored by: pukkandan, coletdjnz
2 years ago
pukkandan a70635b8a1
[cleanup, utils] Don't use kwargs for `format_field` 2 years ago
christoph-heinrich e121e3cee7
[cleanup] Minor fixes (#4096)
Authored by: christoph-heinrich
2 years ago
Evan Spensley 145c5a83a8
[extractor/GoogleDrive] Add folder extractor (#4009)
Closes #3388
Authored by: evansp, pukkandan
2 years ago
pukkandan f3c0c77304
[extractor] Handle `json_ld` with multiple `@type`s
Closes: #4022
2 years ago
pukkandan 56ba69e4c9
[cleanup] Misc fixes
Closes #4027
2 years ago
Aurélien Grosdidier d05460e5fe
[extractor/FranceCulture] Fix extractor (#3874)
Closes #3742
Authored by: aurelg, pukkandan
2 years ago
pukkandan b7c47b7438
[extractor] Add `_search_json`
All fetching of JSON objects should eventually be done with this function
but only `youtube` is being refactored for now
2 years ago
Lesmiscore 0bea4fd807
[extractor/0000studio] Add extractors (#3959)
Authored by: Lesmiscore
2 years ago
pukkandan 1890fc6389
[cleanup] Misc fixes
Cherry-picks from: #3498, #3947
Related: #3949, https://github.com/yt-dlp/yt-dlp/issues/1839#issuecomment-1140313836
Authored by: pukkandan, flashdagger, gamer191
2 years ago
pukkandan c4910024f3
[extractor] Fix bug in 617f658b7e
While the function signature don't enforce it, some IEs that override
`_download_webpage_handle` assume all optional arguments to be keyword-only

Closes #3954
2 years ago
siddharth ravikumar e50c3500b4
[extractor/npr] Use stream url from json-ld (#3455)
Closes #1934
Authored by: r5d
2 years ago
pukkandan 09d02ea429
[extractor] Fix bug in f95b9dee45
Closes #3951
2 years ago
pukkandan f95b9dee45
[extractor] Add dev option `--load-pages` 2 years ago
pukkandan 617f658b7e
[extractor, cleanup] Refactor `_download_...` methods 2 years ago
Lesmiscore 222a230871
[extractor/common] Recognize `src` attribute from HTML5 media elements (#3899)
Authored by: Lesmiscore
3 years ago
coletdjnz ee27297f82
[extractor/youtube] Fix initial player response extraction
Authored by: pukkandan, coletdjnz
3 years ago
pukkandan 8a82af3511
[cleanup] Misc fixes and cleanup
Closes #3780, Closes #3853, Closes #3850
3 years ago
pukkandan c487cf0010
[cleanup] Misc 3 years ago
pukkandan 2762dbb17e
[compat] Add `functools.cached_property` 3 years ago
pukkandan 21633673c3
[cleanup] Minor fixes 3 years ago
pukkandan 2414649192
[cleanup] Misc cleanup 3 years ago
pukkandan 82d020804d
[extractor] Use classmethod/property where possible
and refactor lazy extractors accordingly.

This reduces the need to create extractor instances
3 years ago
pukkandan 8dcce6a89c
[extractor] Document netrc machines
Closes #3169
3 years ago
pukkandan d4736fdb43
Remove warning for videos with an empty title 3 years ago
pukkandan 3fe75fdc80
[cleanup] Misc fixes (see desc)
* Do not warn when fixup is skipped for existing file
* [fragment] Fix `--skip-unavailable-fragments` for HTTP Errors
* [utils] write_string: Fix bug in 59f943cd50
* [utils] parse_codecs: Subtitle codec is generally referred to as `scodec`. https://github.com/yt-dlp/yt-dlp/pull/2174#discussion_r790156048
* [docs] Remove note about permissions. Closes #3597
3 years ago
pukkandan 1d485a1a79
[cleanup] Misc fixes
Closes #3565, https://github.com/yt-dlp/yt-dlp/issues/3514#issuecomment-1105944364
3 years ago
pukkandan 0a5a191a2a
Improve `--clean-infojson`
It should not removes fields that may be needed for `--load-infojson`.
Eg: `_ffmpeg_args`, `_has_drm`
3 years ago
Elyse a076c1f97a
[extractor] Update `manifest_url`s after redirect (#3575)
Authored by: elyse0
3 years ago
Elyse 779da8e31b
[extractor] Update dash `manifest_url` after redirects (#3563)
Closes #2696 
Authored by: elyse0
3 years ago
felix 77f9033095
[compat] Split into sub-modules (#2173)
Authored by: fstirlitz, pukkandan
3 years ago
pukkandan f82711587c
[cleanup] Sort imports
Using https://github.com/PyCQA/isort

    isort -m VERTICAL_HANGING_INDENT --py 36 -l 80 --rr -n --tc .
3 years ago
pukkandan 86e5f3ed2e
[cleanup] Upgrade syntax
Using https://github.com/asottile/pyupgrade

1. `__future__` imports and `coding: utf-8` were removed
2. Files were rewritten with `pyupgrade --py36-plus --keep-percent-format`
3. f-strings were cherry-picked from `pyupgrade --py36-plus`

Extractors are left untouched (except removing header) to avoid unnecessary merge conflicts
3 years ago
pukkandan f9934b9614
[cleanup] Mark some compat variables for removal (#2173)
Authored by fstirlitz, pukkandan
3 years ago
felix cfb0511d82
[cleanup] Remove unused code paths (#2173)
Notes:

* `_windows_write_string`: Fixed in 3.6
  * https://bugs.python.org/issue1602
  * PEP: https://www.python.org/dev/peps/pep-0528

* Windows UTF-8 fix: Fixed in 3.3
  * https://bugs.python.org/issue13216

* `__loader__`: is always present in 3.3+
  * https://bugs.python.org/issue14646

* `workaround_optparse_bug9161`: Fixed in 2.7
  * https://bugs.python.org/issue9161

Authored by: fstirlitz
3 years ago
coletdev 61d3665d9d
[youtube] Fix uploader for collaborative playlists (#3332)
Authored by: coletdjnz
3 years ago
pukkandan a44ca5a470
[cleanup] Misc fixes
Closes https://github.com/yt-dlp/yt-dlp/pull/3213, Closes https://github.com/yt-dlp/yt-dlp/pull/3117

Related: https://github.com/yt-dlp/yt-dlp/issues/3146#issuecomment-1077323114, https://github.com/yt-dlp/yt-dlp/pull/3277#discussion_r841019671, a825ffbffa (commitcomment-68538986), https://github.com/yt-dlp/yt-dlp/issues/2360, 5fa3c9a88f (r70393519), 5fa3c9a88f (r70393254)
3 years ago
pukkandan 04f3fd2c89
[cleanup] Use `_html_extract_title` 3 years ago
Fam0r fbfde1c3e6
[elonet] Rewrite extractor (#3277)
Closes #2911
Authored by: Fam0r, pukkandan
3 years ago
pukkandan 90137ca4be
[utils] Add `filter_dict` 3 years ago
pukkandan ae6a1b9585
[docs] Minor improvements
Closes #3127, Closes #3081, Closes #3177
3 years ago
pukkandan 231025c463
Fix bug in 52efa4b312
Closes #3173
3 years ago
pukkandan 700ccbe3f1
[extractor] Allow control characters inside json
Closes #3174
3 years ago
pukkandan 52efa4b312
[extractor] Add `_perform_login` function (#2943)
* Adds new functions `_initialize_pre_login` and `_perform_login` as part of the extractor API
* Adds `ie.supports_login` to the public API
3 years ago
coletdjnz a825ffbffa
[extractor] Support merging subtitles with data
Authored-by: coletdjnz
3 years ago
pukkandan 08d30158ec
[cleanup, docs] Misc cleanup
Closes #2828, closes #2734, closes #2802, closes #2937
3 years ago
pukkandan 497d2fab6c
[utils] Better traceback for `ExtractorError` 3 years ago
Zenon Mousmoulas 27231526ae
[ant1newsgr] Add extractor (#1982)
Authored by: zmousm
3 years ago
pukkandan a10aa588b0
[FormatSort] Consider `acodec`=`ogg` as `vorbis` 3 years ago
pukkandan 7265a2190c
Fix doubling of `video_id` in `ExtractorError` 3 years ago
coletdev ded9f32667
[extractor] Support `--mark-watched` without `_NETRC_MACHINE` (#2939)
Authored by: coletdjnz
3 years ago
Lesmiscore (Naoya Ozaki) 77cc7c6e60
[nhk] Add support for NHK for School (#2850)
Authored by: Lesmiscore
3 years ago
pukkandan ffa89477ea
[extractor] Fix for manifests without period duration
Closes #2705
Authored by: dirkf, pukkandan
3 years ago
pukkandan 297e9952b6
[extractor] Allow `http_headers` to be specified for `thumbnails` 3 years ago
pukkandan 079a7cfc71
[downloader] Do not use aria2c for non-native `m3u8`
Closes #2718
3 years ago
pukkandan 455a15e2dc
[cleanup,docs] Minor fixes
Closes #2541, #2484
3 years ago
pukkandan 85553414ae
[generic] Allow further processing of json_ld URL
Closes #2578
3 years ago
pukkandan fc08bdd6ab
[extractor] Allow non-fatal `title` extraction 3 years ago
pukkandan 2edb38e8ca
[extractor] Extract video inside `Article` json_ld
Closes #2448
3 years ago
pukkandan f40ee5e9a0
[extractor] Add convinience function `_yes_playlist` 3 years ago
pukkandan 6970b6005e
[cleanup] Minor fixes
Closes #2334
3 years ago
coletdjnz 6c73052c0a
[youtube] Extract channel subscriber count (#2399)
Closes #2350
* Adds `channel_follower_count` field
Authored-by: coletdjnz
3 years ago
pukkandan 311b6615d8
[extractor] Improve `url_result` and related 3 years ago
pukkandan 135dfa2c7e
[extractor,cleanup] Use `_search_nextjs_data` 3 years ago
MinePlayersPE 426764371f
[iq.com] Add extractors (#2354)
Closes #704
Authored by: MinePlayersPE
3 years ago
teridon 8bcd404818
[digitalconcerthall] Add extractor (#1931)
Authored by: teridon
3 years ago
coletdjnz f0d785d3ed
[youtube:tab] Extract more playlist metadata (#2069)
* Add fields modified_date, modified_timestamp
* Add field playlist_count
* [youtube:tab] Extract view_count, playlist_count, modified_date

Authored by: coletdjnz, pukkandan
3 years ago
chris f522573787
[extractor] Extract chapters from JSON-LD (#2031)
Authored by: iw0nderhow, pukkandan
3 years ago
nixxo 7592749cbe
[extractor] Extract thumbnails from JSON-LD (#2195)
Authored by: nixxo
3 years ago
Felix S 4afa3ec4b6
[extractor] Detect more subtitle codecs in MPD manifests (#2174)
Authored by: fstirlitz
3 years ago
MinePlayersPE d43de6821c
[GameJolt] Add extractors (#2036)
Authored by: MinePlayersPE
3 years ago
siddharth 774a46c53d
[npr] Make SMIL extraction non-fatal (#2099)
Closes #1934
Authored by: r5d
3 years ago
pukkandan b69fd25c25
[cleanup] Misc cleanup
Closes #1942 #1976 #2020 #2058 #1984
3 years ago
The Hatsune Daishi adbc4ec4bb
[dash,youtube] Download live from start to end (#888)
* Add option `--live-from-start` to enable downloading live videos from start
* Add key `is_from_start` in formats to identify formats (of live videos) that downloads from start
* [dash] Create protocol `http_dash_segments_generator` that allows a function to be passed instead of fragments
* [fragment] Allow multiple live dash formats to download simultaneously
* [youtube] Implement fragment re-fetching for the live dash formats
* [youtube] Re-extract dash manifest every 5 hours (manifest expires in 6hrs)
* [postprocessor/ffmpeg] Add `FFmpegFixupDuplicateMoovPP` to fixup duplicated moov atoms

Known issue: Ctrl+C doesn't work on Windows when downloading multiple formats

Closes #1521
Authored by: nao20010128nao, pukkandan
3 years ago
Zenon Mousmoulas d5c3254889
[extractor] Support default implicit graph in JSON-LD (#1983)
Original PR: https://github.com/ytdl-org/youtube-dl/pull/30229

Per W3C JSON-LD v1.1 §4.9 (non-normative ref):

    When a JSON-LD document's top-level structure is a map that contains
    no other keys than @graph and optionally @context (properties that
    are not mapped to an IRI or a keyword are ignored), @graph is
    considered to express the otherwise implicit default graph.

Authored by: zmousm
3 years ago
pukkandan a8549f19e7
[tiktok] Fix 53dad39e30 3 years ago
pukkandan 39ca3b5c7f
[extractor] Standardize `_live_title` 3 years ago
pukkandan 19188702ef
[FormatSort] Prevent incorrect deprecation warning
Closes #1981
3 years ago
The Hatsune Daishi 66f4c04e50
[extractor] Add `_search_nuxt_data` (#1921)
Authored by: nao20010128nao
3 years ago
pukkandan d2b2fca53f
[extractor] Ignore errors in comment extraction when `-i` is given
Closes #1787
3 years ago
pukkandan 2c4aaaddc9
[lazy_extractors] Fix for search IEs
Closes #1851
3 years ago
pukkandan ee8dd27a73
[cleanup] Add deprecation warnings 3 years ago
pukkandan f304da8a29
[cleanup] Misc cleanup
Closes #1805, closes #1800
3 years ago
pukkandan f2ebc5c7be
Option `--wait-for-video` to wait for scheduled streams 3 years ago
pukkandan 8f122fa070
[extractor] Extract `average_rating` from JSON-LD
Eg: Crunchyroll
3 years ago
pukkandan 234416e4bf
[downloader/ffmpeg] Fix for direct videos inside mpd manifests
Closes #1751
3 years ago
nyuszika7h 6b993ca765
[hls] Better FairPlay DRM detection (#1661)
Authored by: nyuszika7h
3 years ago
pukkandan 9ac24e235e
[curiositystream] Add more metadata
Closes #1568
3 years ago
Lauren Liberda f98709af31
[extractor] Add `_search_nextjs_data` (#1386)
Authored by: selfisekai
3 years ago
pukkandan 34921b4345
[utils] Add `join_nonempty` 3 years ago
pukkandan aeb2a9ad27
[FormatSort] `eac3` is better than `ac3` 3 years ago
pukkandan 0930b11fda
[docs,cleanup] Improve docs and minor cleanup
Closes #1387, #1404, #1408, #1485, #1415, #1450, #1492
3 years ago
pukkandan 10beccc980
[FormatSort] Fix some fields' defaults
Closes #1479
3 years ago
pukkandan 0db3bae879
[extractor] Fix some errors being converted to `ExtractorError` 3 years ago
pukkandan 96565c7e55
[cleanup] Add keyword automatically to SearchIE descriptions
and some minor cleanup of docs
3 years ago
pukkandan ec11a9f4a2
[minicurses] Add more colors 3 years ago