coletdjnz
c35448b7b1
[extractor/youtube] Extract more metadata for comments ( #7179 )
...
Adds new comment fields:
* `author_url` - The url to the comment author's page
* `author_is_verified` - Whether the author is verified on the platform
* `is_pinned` - Whether the comment is pinned to the top of the comments
Closes https://github.com/yt-dlp/yt-dlp/issues/5411
Authored by: coletdjnz
1 year ago
coletdjnz
18f8fba7c8
[extractor/youtube] Fix continuation loop with no comments ( #7148 )
...
Deep check the response for incomplete data.
Authored by: coletdjnz
1 year ago
coletdjnz
93e12ed76e
[extractor/youtube] Extract uploader metadata for feed/playlist items
...
Fixes https://github.com/yt-dlp/yt-dlp/issues/7104
Authored by: coletdjnz
1 year ago
Audrey
5caf30dbc3
[extractor/youtube] Extract `heatmap` data ( #7100 )
...
Closes #3888
Authored by: tntmod54321
1 year ago
pukkandan
4823ec9f46
Update to ytdl-commit-d1c6c5
...
[YouTube] [core] Improve platform debug log, based on yt-dlp
d1c6c5c4d6
Except:
* 6ed34338285f722d0da312ce0af3a15a077a3e2a [jsinterp] Add short-cut evaluation for common expression
* There was no performance improvement when tested with https://github.com/ytdl-org/youtube-dl/issues/30641
* e8de54bce50f6f77a4d7e8e80675f7003d5bf630 [core] Handle `/../` sequences in HTTP URLs
* We plan to implement this differently
1 year ago
kangalio
69a40e4a7f
[extractor/youtube:music:search_url] Extract title ( #7102 )
...
Authored by: kangalio
Closes #7095
1 year ago
coletdjnz
447afb9eaa
[extractor/youtube] Support podcasts and releases tabs
...
Closes https://github.com/yt-dlp/yt-dlp/issues/6893
Authored by: coletdjnz
1 year ago
coletdjnz
7666b93604
[extractor/youtube] Define strict uploader metadata mapping ( #6384 )
...
New mapping:
```
channel -> channel name
channel_id -> UCID
channel_url -> UCID channel url
uploader -> channel name (same as channel field)
uploader_id -> @handle
uploader_url -> @handle channel url
```
Authored by: coletdjnz
2 years ago
pukkandan
c6786ff3ba
[extractor/youtube] Revert default formats to `https`
2 years ago
coletdjnz
141a8dff98
[extractor/youtube] Fix comment loop detection for pinned comments ( #6714 )
...
Pinned comments may repeat a second time - this is expected.
Fixes https://github.com/yt-dlp/yt-dlp/issues/6712
Authored by: coletdjnz
2 years ago
Nicholas Defranco
071670cbea
[extractor/youtube] Fix parsing `comment_count` ( #6523 )
...
Closes #5849
Authored by: nick-cd
2 years ago
coletdjnz
607510b9f2
[extractor/youtube] Handle incomplete initial data from watch page ( #6510 )
...
Authored by: coletdjnz
2 years ago
pukkandan
e389d172b6
Fix 2a23d92d9e
...
Closes #6517
2 years ago
pukkandan
2a23d92d9e
[extractor/youtube] Construct fragment list lazily
...
Building fragment list for all formats take significant time for large videos
2 years ago
pukkandan
86cb922118
[extractor/youtube] Add extractor-arg `include_duplicate_formats`
2 years ago
Lesmiscore
c795c39f27
[extractor/youtube] Add client name to `format_note` when `-v` ( #6254 )
...
Authored by: Lesmiscore, pukkandan
2 years ago
pukkandan
c9abebb851
[extractor/youtube] Bypass throttling for `-f17`
...
and related cleanup
Thanks @AudricV for the finding
2 years ago
pukkandan
392389b7df
[cleanup] Misc
2 years ago
mushbite
22ccd5420b
[extractor/rutube] Extract chapters from description ( #6345 )
...
Authored by: mushbite
2 years ago
coletdjnz
7f51861b18
[extractor/youtube] Detect and break on looping comments ( #6301 )
...
Fixes https://github.com/yt-dlp/yt-dlp/issues/6290
Authored by: coletdjnz
2 years ago
pukkandan
5b28cef72d
[cleanup] Misc
2 years ago
pukkandan
31e183557f
[extractor/youtube] Extract channel `view_count` when `/about` tab is passed
2 years ago
pukkandan
f34804b2f9
[extractor/youtube] Fix 5038f6d713
...
* [fragment] Fix `request_data`
* [youtube] Don't use POST for now. It may be easier to break in future
Authored by: bashonly, coletdjnz
2 years ago
pukkandan
5038f6d713
[extractor/youtube] Construct dash formats with `range` query
...
Closes #6369
2 years ago
pukkandan
a538772969
[cleanup] Misc
...
Closes #5897
2 years ago
bashonly
c61cf091a5
[extractor/youtube] `uploader_id` includes `@` with handle
...
Authored by: bashonly
2 years ago
bashonly
149eb0bbf3
[extractor/youtube] Fix `uploader_id` extraction
...
Closes #6247
Authored by: bashonly
2 years ago
Bruno Guerreiro
78a78fa74d
[extractor/youtube] Add hyperpipe instances ( #6020 )
...
Authored by: Generator
2 years ago
Roland Hieber
05799a48c7
[extractor/youtube] Update invidious and piped instances ( #6030 )
...
Authored by: rohieb
2 years ago
Simon Sawicki
6839ae1f6d
[utils] `traverse_obj`: Fix more bugs
...
and cleanup uses of `default=[]`
Continued from b1bde57bef
2 years ago
pukkandan
b032ff0f03
[extractor/youtube] Handle `consent.youtube`
2 years ago
pukkandan
dad2210c0c
[extractor/youtube] Support `/live/` URL
2 years ago
mzhou
253ac4ba6a
[extractor/youtube] Retry manifest refresh for live-from-start ( #5670 )
...
Avoids ending download early when live stream is temporarily offline.
Best used with somewhat large `--retry-sleep extractor:` and `--extractor-retries`
Authored by: mzhou
2 years ago
pukkandan
08e29b9f1f
[cleanup] Misc
...
Closes #5576 , closes #5887
2 years ago
pukkandan
9bb856998b
[extractor/youtube] Extract DRC formats
2 years ago
Matthew
c733555106
[extractor/youtube:tab] Extract metadata from channel items ( #5569 )
...
Authored by: coletdjnz
2 years ago
pukkandan
71eb82d1b2
[extractor/youtube] Subtitles cannot be translated to `und`
...
Closes #5674
2 years ago
Bnyro
bc87dac75f
[extractor/youtube] Add `piped.video` ( #5571 )
...
Closes #5518
Authored by: Bnyro
2 years ago
pukkandan
9f14daf22b
[extractor] Deprecate `_sort_formats`
2 years ago
pukkandan
6368e2e639
[cleanup] Misc
...
Closes #5541
2 years ago
pukkandan
a4894d3e25
[extractor/youtube] Consider language in format de-duplication
2 years ago
pukkandan
171a31dbe8
[extractor] Add a way to distinguish IEs that returns only videos
2 years ago
pukkandan
a8c754cc00
[extractor/youtube] Fix bug in handling of music URLs
...
Bug in bd7e919a75
Closes #5502
2 years ago
pukkandan
08270da5c3
[extractor/youtube] Fix `ytuser:`
2 years ago
pukkandan
bd7e919a75
[extractor/youtube:tab] Improvements to tab handling ( #5487 )
...
* Better handling of direct channel URLs - See https://github.com/yt-dlp/yt-dlp/pull/5439#issuecomment-1309322019
* Prioritize tab id from URL slug - Closes #5486
* Add metadata for the wrapping playlist
* Simplify redirect for music playlists
2 years ago
Matthew
e72e48c53f
[extractor/youtube] Ignore incomplete data error for comment replies ( #5490 )
...
When --ignore-errors is used.
Closes https://github.com/yt-dlp/yt-dlp/issues/4669
Authored by: coletdjnz
2 years ago
Matthew
0cf643b234
[extractor/youtube] Differentiate between no and disabled comments ( #5491 )
...
`comments` and `comment_count` will be set to None, as opposed to
an empty list and 0, respectively.
Fixes https://github.com/yt-dlp/yt-dlp/issues/5068
Authored by: coletdjnz, pukkandan
2 years ago
Matthew
4dc23a8051
[extractor/youtube:tab] Fix video metadata from tabs ( #5489 )
...
Closes #5488
Authored by: coletdjnz
2 years ago
Matthew
86973308cd
[extractor/youtube:tab] Update tab handling for redesign ( #5439 )
...
Closes #5432 , #5430 , #5419
Authored by: coletdjnz, pukkandan
2 years ago
Bruno Guerreiro
e14ea7fbd9
[extractor/youtube] Update piped instances ( #5441 )
...
Closes #5286
Authored by: Generator
2 years ago
Matthew
6141346d18
[extractor/youtube] Update playlist metadata extraction for new layout ( #5376 )
...
Fixes https://github.com/yt-dlp/yt-dlp/issues/5373
Authored by: coletdjnz
2 years ago
pukkandan
2e30b46fe4
[extractor/youtube] Improve chapter parsing from description
...
Closes #5448
2 years ago
nosoop
9da6612b0f
[extractor/youtube] Fix `duration` for premieres ( #5382 )
...
Closes #5378
Authored by: nosoop
2 years ago
coletdjnz
e63faa101c
[extractor/youtube] Fix `live_status` extraction for playlist videos
...
Regression in 867c66ff97
Authored by: coletdjnz
2 years ago
bsun0000
5318156f1c
[extractor/youtube] Mark videos as fully watched
...
Closes #2555
Authored by: bsun0000
2 years ago
pukkandan
d5d1df8afd
[cleanup Misc
...
Closes #5162
2 years ago
pukkandan
6678a4f0b3
[extractor/youtube] Fix live_status
...
Bug in 4d37720a0c
2 years ago
pukkandan
5225df50cf
[extractor/youtube:tab] Let `approximate_date` return timestamp
2 years ago
pukkandan
0468a3b325
[jsinterp] Improve separating regex
...
Fixes https://github.com/yt-dlp/yt-dlp/issues/4635#issuecomment-1273974909
2 years ago
Matthew
867c66ff97
[extractor/youtube] Extract concurrent view count for livestreams ( #5152 )
...
Adds new field `concurrent_view_count`
Closes https://github.com/yt-dlp/yt-dlp/issues/4843
Authored by: coletdjnz
2 years ago
Lesmiscore
4d37720a0c
[extractor/youtube] Download `post_live` videos from start ( #5091 )
...
* The fragments are generated as a `LazyList`. So only the required formats are expanded during download, but all fragment lists are printed/written in infojson.
* The m3u8 formats which cannot be downloaded from start are not extracted by default, but can be enabled with an extractor-arg. The extractor-arg `include_live_dash` is renamed to `include_incomplete_formats` to account for this new use-case.
Closes #1564
Authored by: Lesmiscore, pukkandan
2 years ago
pukkandan
a057779d5e
[cleanup] Minor fixes
...
Closes #5129 , Closes #4982
2 years ago
pukkandan
7a32c70d13
[cleanup] Fix flake8 and minor refactor
...
Issues from ab029d7e92
, 1fb53b946c
2 years ago
pukkandan
709ee21417
[extractor/youtube] Do not warn on duplicate chapters
...
Eg: vYbaM8w8yzw
2 years ago
pukkandan
1fb53b946c
[extractor/youtube:tab] Improve continuation items extraction
2 years ago
pukkandan
1dd18a8808
[extractor/YoutubeShortsAudioPivot] Support `source` URLs
...
`ytshortsap:` is no longer needed
2 years ago
pukkandan
0a5095fe8d
[extractor/youtube:tab] Support `reporthistory` page
...
Closes #4929
2 years ago
coletdjnz
0ca0f88121
[extractor/heise] Fix extractor ( #5029 )
...
Fixes https://github.com/yt-dlp/yt-dlp/issues/1520
Authored by: coletdjnz
2 years ago
coletdjnz
80eb0bd9b9
[extractor/youtube] Add support for Shorts audio pivot feed ( #4932 )
...
This feed shows Shorts using the audio of a given video.
ytshortsap: prefix can be used as a shortcut until YouTube
implements an official view.
Closes #4911
Authored by: coletdjnz
2 years ago
coletdjnz
c26f9b991a
[extractor/youtube] Support changing extraction language ( #4470 )
...
Adds `--extractor-args youtube:lang=<supported lang code>` extractor arg to prefer translated fields (e.g. title and description) of that language, if available, for all YouTube extractors. See README or error message for list of supported language codes.
Closes https://github.com/yt-dlp/yt-dlp/issues/387
Authored by: coletdjnz
2 years ago
coletdjnz
3ffb2f5bea
[extractor/youtube] Fix video like count extraction
...
Support new combined button layout
Authored by: coletdjnz
2 years ago
pukkandan
17ffed1842
[docs] Improvements
...
* Move detailed installation instructions to https://github.com/yt-dlp/yt-dlp/wiki/Installation
* Link to wiki where applicable
* Fix some mistakes. Closes #4853 , Closes #4855 , Closes #4852
* Improve some error messages
2 years ago
pukkandan
7c6eb424d3
[extractor/youtube] Detect `lazy-load-for-videos` embeds
...
Closes #4812
2 years ago
pukkandan
05deb747bb
[jsinterp] Fix escape in regex
2 years ago
pukkandan
b505e8517a
[extractor/youtube] Fallback regex for nsig code extraction
2 years ago
coletdjnz
1ff88b7aec
[extractor/youtube] Add `no-youtube-prefer-utc-upload-date` compat option ( #4771 )
...
This option reverts 992f9a730b
and 17322130a9
to prefer the non-UTC upload date in microformats.
Authored by: coletdjnz, pukkandan
2 years ago
pukkandan
da4db748fa
[utils] Add `deprecation_warning`
...
See https://github.com/yt-dlp/yt-dlp/pull/2173#issuecomment-1097021515
2 years ago
pukkandan
d81ba7d491
[jsinterp, extractor/youtube] Minor fixes
2 years ago
pukkandan
c4b2df872d
[jsinterp] Fix `_separate`
...
Ref: https://github.com/yt-dlp/yt-dlp/issues/4635#issuecomment-1231126941
2 years ago
Samantaz Fox
224b5a35f7
[extractor/youtube] Update iOS Innertube clients ( #4792 )
...
Authored by: SamantazFox
2 years ago
coletdjnz
50ac0e5416
[extractor/youtube] Use device-specific user agent ( #4770 )
...
Thwart latest fingerprinting attempt (see https://github.com/iv-org/invidious/issues/3230#issuecomment-1226887639 )
Authored by: coletdjnz
2 years ago
pukkandan
5e01315aa1
[cache, extractor/youtube] Invalidate old cache
2 years ago
pukkandan
992dc6b486
[jsinterp] Implement timeout
...
Workaround for #4716
2 years ago
pukkandan
b25cac650f
[extractor/youtube] Fix bug in format sorting
2 years ago
pukkandan
90a1df305b
[test] Fix test_youtube_signature
2 years ago
pukkandan
a831c2ea90
[cleanup] Misc
2 years ago
pukkandan
25836db6be
[extractor/youtube] Add fallback to phantomjs
...
Related #4635
2 years ago
pukkandan
580ce00782
[youtube] Improve signature caching
...
and refactor related functions
2 years ago
pukkandan
f6ca640b12
[jsinterp] Fix for youtube player 1f7d5369
...
Closes #4635 again
2 years ago
pukkandan
3ce2933693
[youtube] Fix error reporting of "Incomplete data"
...
Related: #4669
2 years ago
pukkandan
5c6d2ef9d1
[youtube] Improve format sorting for IOS formats
...
When no itag/resolution is available for reference, use the closest resolution
2 years ago
Lesmiscore
62b58c0936
[docs] Consistent use of `e.g.` ( #4643 )
...
Authored by: Lesmiscore
2 years ago
pukkandan
8f53dc44a0
[jsinterp] Handle new youtube signature functions
...
Closes #4635
2 years ago
pukkandan
7e798d725e
[extractor] Fix format sorting of `channels`
2 years ago
coletdjnz
c7dcf0b31e
[extractor/youtube] Add `androidSdkVersion` parameter to Android Innertube clients
...
Required to prevent YouTube returning a bad player response in some cases.
See: https://github.com/yt-dlp/yt-dlp/pull/4593 , https://github.com/TeamNewPipe/NewPipe/issues/8713 , https://github.com/iv-org/invidious/issues/3230 , https://github.com/Tyrrrz/YoutubeExplode/issues/647
Authored by: coletdjnz
2 years ago
pukkandan
a416623436
[extractor/youtube] Extract more format info
2 years ago
coletdjnz
a3e9642116
[extractor/youtube] Prevent redirect to unwanted videos ( #4593 )
...
Example: https://www.youtube.com/watch?v=aQvGIIdgFDM
Authored by: coletdjnz
2 years ago
coletdjnz
a0c830f488
[extractor/youtube] Bump Innertube client versions
...
YouTube may be requiring new versions soon. See https://github.com/iv-org/invidious/issues/3230 , https://github.com/TeamNewPipe/NewPipe/issues/8713
Authored by: coletdjnz
2 years ago
pukkandan
be5c1ae862
Standardize retry mechanism ( #1649 )
...
* [utils] Create `RetryManager`
* Migrate all retries to use the manager
* [extractor] Add wrapper methods for convenience
* Standardize console messages for retries
* Add `--retry-sleep` for extractors
2 years ago
pukkandan
bfd973ece3
[extractors] Use new framework for existing embeds ( #4307 )
...
`Brightcove` is difficult to migrate because it's subclasses may depend
on the signature of the current functions. So it is left as-is for now
Note: Tests have not been migrated
2 years ago