Commit Graph

275 Commits (227bf1a33be7b89cd7d44ad046844c4ccba104f4)

Author SHA1 Message Date
coletdjnz 227bf1a33b
[networking] Rewrite architecture (#2861)
New networking interface consists of a `RequestDirector` that directs
each `Request` to appropriate `RequestHandler` and returns the
`Response` or raises `RequestError`. The handlers define adapters to
transform its internal Request/Response/Errors to our interfaces.

User-facing changes:
- Fix issues with per request proxies on redirects for urllib
- Support for `ALL_PROXY` environment variable for proxy setting
- Support for `socks5h` proxy
   - Closes https://github.com/yt-dlp/yt-dlp/issues/6325, https://github.com/ytdl-org/youtube-dl/issues/22618, https://github.com/ytdl-org/youtube-dl/pull/28093
- Raise error when using `https` proxy instead of silently converting it to `http`

Authored by: coletdjnz
1 year ago
pukkandan bc344cd456
[core] Allow extractors to mark formats as potentially DRM (#7396)
This is useful for HLS where detecting whether the format is
actually DRM requires the child manifest to be downloaded.

Makes the error message when using `--test` inconsistent,
but doesn't really matter.
1 year ago
pukkandan 5fd8367496
[extractor] Support multiple `_VALID_URL`s (#5812)
Authored by: nixxo
1 year ago
pukkandan d7cd97e8d8
Fix bug in db3ad8a676
Closes #7367
1 year ago
pukkandan ad54c9130e
[cleanup] Misc
Closes #6288, Closes #7197, Closes #7265, Closes #7353, Closes #5773
Authored by: mikf, freezboltz, pukkandan
1 year ago
Nicolai Dagestad db3ad8a676
Add option `--netrc-cmd` (#6682)
Authored by: NDagestad, pukkandan
Closes #1706
1 year ago
coletdjnz 8213ce28a4
[extractor/youtube] Extract `channel_is_verified` (#7213)
Authored by: coletdjnz
1 year ago
coletdjnz c35448b7b1
[extractor/youtube] Extract more metadata for comments (#7179)
Adds new comment fields:
* `author_url` - The url to the comment author's page
* `author_is_verified` - Whether the author is verified on the platform
* `is_pinned` - Whether the comment is pinned to the top of the comments

Closes https://github.com/yt-dlp/yt-dlp/issues/5411

Authored by: coletdjnz
1 year ago
coletdjnz b87e01c123
[cookies] Move `YoutubeDLCookieJar` to cookies module (#7091)
Authored by: coletdjnz
2 years ago
Audrey 5caf30dbc3
[extractor/youtube] Extract `heatmap` data (#7100)
Closes #3888
Authored by: tntmod54321
2 years ago
pukkandan 84ffeb7d5e
[extractor] Do not warn for invalid chapter data in description
Fixes https://github.com/yt-dlp/yt-dlp/issues/6811#issuecomment-1509876209
2 years ago
pukkandan f68434cc74
[extractor] Extract more metadata from ISM
Fixes 81b6102d20 (r105892531)
2 years ago
pukkandan baa922b5c7
[extractor] Do not exit early for unsuitable `url_result` 2 years ago
mushbite 22ccd5420b
[extractor/rutube] Extract chapters from description (#6345)
Authored by: mushbite
2 years ago
pukkandan 5b28cef72d
[cleanup] Misc 2 years ago
pukkandan f34804b2f9
[extractor/youtube] Fix 5038f6d713
* [fragment] Fix `request_data`
* [youtube] Don't use POST for now. It may be easier to break in future

Authored by: bashonly, coletdjnz
2 years ago
pukkandan 43a3eaf963
[extractor] Fix DRM detection in m3u8
Fixes https://github.com/ytdl-org/youtube-dl/issues/31693#issuecomment-1445202857
2 years ago
pukkandan 45b2ee6f4f
Update to ytdl-commit-2dd6c6e
[YouTube] Avoid crash if uploader_id extraction fails
2dd6c6edd8

Except:
    * 295736c9cba714fb5de7d1c3dd31d86e50091cf8 [jsinterp] Improve parsing
    * 384f632e8a9b61e864a26678d85b2b39933b9bae [ITV] Overhaul ITV extractor
    * 33db85c571304bbd6863e3407ad8d08764c9e53b [feat]: Add support to external downloader aria2p
2 years ago
LowSuggestion912 b23167e754
[extractor/common] Fix `_search_nuxt_data` (#6062)
Authored by: LowSuggestion912
2 years ago
bashonly 7e68567e50
[downloader/hls] Allow extractors to provide AES key (#6158)
and related cleanup

Authored by: bashonly, Grub4K

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
2 years ago
bashonly 5ab3534d44
[extractor/slideslive] Fix slides and chapters/duration (#6024)
* Fix slides/thumbnails extraction
* Extract duration to fix issues w/ `--embed-chapters`, `--split-chapters`
* Add `InfoExtractor._extract_mpd_vod_duration` method
* Expand applicability of `InfoExtractor._parse_m3u8_vod_duration` method
Authored by: bashonly
2 years ago
bashonly cb73b8460c
[extractor/nbc] Fix `NBC` and `NBCStations` extractors (#6033)
Improve `InfoExtractor._parse_smil_formats` extension detection
Closes #6019
Authored by: bashonly
2 years ago
pukkandan edfc7725b1
[cleanup] Misc 2 years ago
pukkandan 08e29b9f1f
[cleanup] Misc
Closes #5576, closes #5887
2 years ago
Simon Sawicki 32a84bcf4e
Update to ytdl-commit-195f22f6
[generic] Improve KVS (etc) extraction
195f22f679

Closes #3716
Authored by: Grub4k, pukkandan
2 years ago
Matthew e756f45ba0
Improve handling for overriding extractors with plugins (#5916)
* Extractors replaced with plugin extractors now show in debug output
* Better testcase handling
* Added documentation
Authored by: coletdjnz, pukkandan
2 years ago
pukkandan 0b5546c723
[extractor] Let `_extract_format` functions obey `--ignore-no-formats` 2 years ago
pukkandan 71df9b7fd5
[cleanup] Misc 2 years ago
pukkandan 9bcfe33be7
[utils] Make `ExtractorError` mutable 2 years ago
pukkandan 9f14daf22b
[extractor] Deprecate `_sort_formats` 2 years ago
pukkandan 784320c98c
Implement universal format sorting
Closes #5566
2 years ago
pukkandan d0d74b7197
[utils] Move format sorting code into `utils` 2 years ago
pukkandan 105bfd90f5
Add new field `aspect_ratio`
Closes #5402
2 years ago
pukkandan 6368e2e639
[cleanup] Misc
Closes #5541
2 years ago
pukkandan 171a31dbe8
[extractor] Add a way to distinguish IEs that returns only videos 2 years ago
pukkandan f7fc8d39e9
[extractor] Fix `fatal=False` for `_search_nuxt_data`
Closes #5423
2 years ago
Matthew 0cf643b234
[extractor/youtube] Differentiate between no and disabled comments (#5491)
`comments` and `comment_count` will be set to None, as opposed to 
an empty list and 0, respectively.

Fixes https://github.com/yt-dlp/yt-dlp/issues/5068

Authored by: coletdjnz, pukkandan
2 years ago
pukkandan 46d09f8707
[cleanup] Lint and misc cleanup 2 years ago
pukkandan 62b8dac490
[extractor] Improve `_generic_title` 2 years ago
m4tu4g c13a301a94
[extractor/zeenews] Add extractor (#5289)
Closes #4967 
Authored by: m4tu4g, pukkandan
2 years ago
pukkandan d5d1df8afd
[cleanup Misc
Closes #5162
2 years ago
pukkandan 5225df50cf
[extractor/youtube:tab] Let `approximate_date` return timestamp 2 years ago
pukkandan ade1fa70cb
[extractor/generic] Separate embed extraction into own function (#5176) 2 years ago
Simon Sawicki 1d55ebabc9
[extractor/common] Fix `json_ld` type checks (#5145)
Closes #5144, #5143
Authored by: Grub4K
2 years ago
Matthew 867c66ff97
[extractor/youtube] Extract concurrent view count for livestreams (#5152)
Adds new field `concurrent_view_count`
Closes https://github.com/yt-dlp/yt-dlp/issues/4843

Authored by: coletdjnz
2 years ago
pukkandan a057779d5e
[cleanup] Minor fixes
Closes #5129, Closes #4982
2 years ago
coletdjnz d3a3d7f0cc
[extractor/JWPlatform] Fix extractor (#5112)
Fix bitrate and filesize extraction and support embeds with unquoted urls.

Related: #5106 

Authored by: coletdjnz
2 years ago
pukkandan 8b7fb8b60d
[extractor] Make search_json able to parse lists
Now `contains_pattern` can be set to `\[.+\]`
2 years ago
nixxo 81b6102d20
[downloader/ism] Support ec-3 codec (#5004)
Closes #296
Authored by: nixxo
2 years ago
pukkandan 0f60ba6e65
[extractor] Improve json+ld extraction
Related #5035
2 years ago