Commit Graph

731 Commits (1722099ded6d2d3568197c412e740fda82d188d4)

Author SHA1 Message Date
mahanstreamer aca5774e68
[bitchute] Fix test (#758)
Authored by: mahanstreamer
3 years ago
pukkandan 3fb4e21b38
[lazy_extractors] Fix `suitable` and add flake8 test 3 years ago
pukkandan 4dfbf8696b
[utils] Add `parse_qs` 3 years ago
pukkandan 8fc54b1230
[youtube] Add `shorts` to `_VALID_URL`
Normally the generic extractor will redirect the URL,
but the cookies consent screen may sometimes appear instead

Closes #752
3 years ago
pukkandan da33e35b05
Don't try to merge with final extension
The formats may not be directly mergable into the final extension
3 years ago
pukkandan 5ad28e7ffd
[extractor] Common function `_match_valid_url` 3 years ago
Jérôme Duval f79ec47d71
[tv5mondeplus] Fix extractor (#739)
Authored by: korli
3 years ago
Ashish 45b0596290
[HearThisAtIE] Fix extractor (#742)
Closes: #740 
Authored by: Ashish0804
3 years ago
Ashish 96c23f3be8
[Zee5] Fix extractor and add subtitles (#733)
Closes #728
Authored by Ashish0804
3 years ago
CHJ85 6e7dfe4959
[BannedVideo] Add Extractor (#717)
Closes: #669
Original PR: https://github.com/ytdl-org/youtube-dl/pull/24572
Authored by: smege1001, blackjack4494, pukkandan
3 years ago
animelover1984 c34f505b04
[bilibili] Add category extractor (#695)
Authored by: animelover1984
3 years ago
Ashish 14183d1f80
[Hungama] Fix `HungamaSongIE` and add `HungamaAlbumPlaylistIE` (#744)
Authored by: Ashish0804
3 years ago
pukkandan 58adec4677
Fix `extra_info` being reused across runs
Closes #727
3 years ago
pukkandan 9e598870dd
Fix `playlist_index` not obeying `playlist_start`
and add tests
Closes #720
3 years ago
pukkandan 8f18aca871
Let `--match-filter` reject entries early
Makes redundant: `--match-title`, `--reject-title`, `--min-views`, `--max-views`
3 years ago
pukkandan 3ad56b4236
Fix `-J` when there are failed videos 3 years ago
Glenn Slayden 5d62709bc7
[cleanup] Replace improper use of tab in trovo (#719)
:ci skip

Authored by: glenn-slayden
3 years ago
shirt 5fa206fb54
[ParamountPlus] Fix geo verification (#711)
Closes #681 
Authored by: shirt
3 years ago
mzbaulhaque df2a5633da
[pornhub] Separate and fix playlist extractor (#700)
Closes #680
Authored by: mzbaulhaque
3 years ago
Felix S 7a6742b5f9
[webvtt] Fix timestamp overflow adjustment (#698)
In some streams, empty segments may appear with a bogus, non-monotone MPEG timestamp.
This should not be considered as an overflow

Authored by: fstirlitz
3 years ago
The Hatsune Daishi e040bb0a41
[voicy] Add extractor (#667)
Authored by: nao20010128nao
3 years ago
pukkandan f8fabc9930
[kakao] Fix extractor
Closes #699
3 years ago
jhwgh1968 d967c68e4c
[eroprofile] Fix page skipping in albums (#701)
Bug from #658 
Authored by: jhwgh1968
3 years ago
SsSsS 3dd39c5f9a
[instagram] Add referrer to prevent throttling (#676)
Code from: https://github.com/ytdl-org/youtube-dl/pull/29751
Fixes: https://github.com/ytdl-org/youtube-dl/issues/29736

Authored by: u-spec-png, kikuyan
3 years ago
mzbaulhaque be44eefd5e
[filmmodu] Add extractor (#690)
Closes #288
Authored by: mzbaulhaque
3 years ago
pukkandan f775c83110
Fix `--force-overwrites` when using `-k`
For formats that need merge, the `.fxxx` files are not removed before
downloading the corresponding `.part` files. This causes the rename to fail
3 years ago
pukkandan b714b41f81
[soundcloud] Refetch `client_id` on 403
Closes #673
3 years ago
pukkandan 31654882e9
[options] Add `_set_from_options_callback` 3 years ago
pukkandan 86c66b2d3e
Fix `-F` for extractors that directly return url
Related: #693
3 years ago
pukkandan 37242e56f2
Fix bug during subtitle conversion 3 years ago
pukkandan 6c7274ecd2
Fix resuming of single formats when using --no-part
Closes #576
3 years ago
coletdjnz 641ad5d813
[youtube] Extract error messages from HTTPError response (#644)
Authored by: coletdjnz
3 years ago
Felix S 0715f7e19b
Revert erroneous use of the `Content-Length` header (#637)
This reverts commit 6c907eb33f

The use of the Content-Length value here is erroneous and may lead
to truncated downloads if a compression scheme is specified in the
Content-Encoding header, as the Content-Length header refers to the
size of encoded data, not of the raw bytestream. This has been noticed
in the wild with WebVTT subtitle segments.

Authored by: fstirlitz
3 years ago
pukkandan a8731fcc1d minor bugfixes
bugs due to be2fc5b212, e9f4ccd19e
3 years ago
pukkandan ade6dc5e9e
[version] update
:ci skip all
3 years ago
jhwgh1968 c196640ff1
[eroprofile] Add album downloader (#658)
Authored by: jhwgh1968
3 years ago
SsSsS 60c8fc73c6
[instagram] Fix comments extraction (#660)
Authored-by: u-spec-png <miloradkalabasdt@gmail.com>
3 years ago
Ashish bc8745480e
[BandCamp] Add BandcampMusicIE (#668)
Authored by Ashish0804
3 years ago
The Hatsune Daishi ff5e16f2f6
[mirrativ] Add extractors (#657)
Authored by: nao20010128nao
3 years ago
pukkandan be2fc5b212
[extractor] Detect `sttp` as subtitles in MPD
Closes #656
Solution by: fstirlitz
3 years ago
pukkandan 7be9ccff0b [utils] Fix `InAdvancePagedList.__getitem__`
Since it didn't have any cache, the page was re-fetched for each video.
* Also generalized the cache code
3 years ago
funniray 245d43cacf
[crunchyroll] Fix thumbnail (#650)
Authored by: funniray
3 years ago
mzbaulhaque 246fb276e0
[blackboardcollaborate] Add new extractor (#646)
Authored by: Ashish0804
3 years ago
shirt 6e6e0d95b3
[paramountplus] Separate extractor and fix some titles (#652)
Co-authored-by: shirt, pukkandan
3 years ago
Felix S 25a3f4f5d6
[webvtt] Merge daisy-chained duplicate cues (#638)
Fixes: https://github.com/yt-dlp/yt-dlp/issues/631#issuecomment-893338552

Previous deduplication algorithm only removed duplicate cues with
identical text, styles and timestamps.  This change also merges
cues that come in ‘daisy chains’, where sequences of cues with
identical text and styles appear in which the ending timestamp of
one equals the starting timestamp of the next.

This deduplication algorithm has the somewhat unfortunate side effect
that NOTE blocks between cues, if found, will be emitted in a different
order relative to their original cues.  This may be unwanted if perfect
fidelity is desired, but then so is daisy-chain deduplication itself.
NOTE blocks ought to be ignored by WebVTT players in any case.

Authored by: fstirlitz
3 years ago
pukkandan ad3dc496bb Misc fixes - See desc
* Remove unnecessary uses of _list_from_options_callback
* Fix download tests - Bug from 6e84b21559
* Rename ExecAfterDownloadPP to ExecPP and refactor its tests
* Ensure _write_ytdl_file closes file handle on error - Potential fix for #517
3 years ago
pukkandan 2831b4686c Show libraries present in verbose head 3 years ago
pukkandan 8c0ae192a4 [ffmpeg] Fix `--ffmpeg-location` when directory is given
Bug introduced in 89efdc15dd
Closes #654
3 years ago
pukkandan e9f4ccd19e
Add option `--replace-in-metadata` 3 years ago
pukkandan a38bd1defa
[viki] Print error message from API request
Closes #651
3 years ago
Ashish b6a35ad83b
[HotStar] Use API for metadata and extract subtitles (#640)
The API is not rate-limited unlike the webpage

Authored by: Ashish0804
3 years ago
SsSsS bfd56b74b9
[peertube] Fix videos without description (#639)
Authored by: u-spec-png
3 years ago
PSlava 858a65ecc1
[youtube] Improve signature function detection (#641)
Authored by: PSlava (Slava <slash@i-slash.com>)
3 years ago
Wes 3b34e38813
[aenetworks] Update `_THEPLATFORM_KEY` and `_THEPLATFORM_SECRET` (#643)
Original PR: https://github.com/ytdl-org/youtube-dl/pull/29749
Fixes: https://github.com/ytdl-org/youtube-dl/issues/29300

Authored by: wesnm
3 years ago
pukkandan 3448870205
[docs] Fix some mistakes and improve doc 3 years ago
pukkandan b868936cd6 [cleanup] Misc 3 years ago
pukkandan c681cb5d93 Allow multiple `--exec` and `--exec-before-download` 3 years ago
pukkandan 379e44ed3c [youtube] Raise appropriate error when API pages can't be downloaded 3 years ago
pukkandan 28f436bad0 [extractor] Reset non-repeating warnings per video 3 years ago
pukkandan 2b8a2973bd Allow entire infodict to be printed using `%()s`
Makes `--dump-json` redundant
3 years ago
pukkandan b7b04c782e Add option `--no-simulate` to not simulate even when `--print` or `--list...` are used
* Deprecates `--print-json`
* Some listings like `--list-extractors` are handled by `yt_dlp` and so are not affected by this. These have been documented as such

Addresses: https://github.com/ytdl-org/youtube-dl/issues/29675, https://github.com/ytdl-org/youtube-dl/issues/29580#issuecomment-882046305
3 years ago
pukkandan 6e84b21559
Fix bugs related to `sanitize_info`
Related: 8012d892bd (r54555230)
3 years ago
pukkandan 575e17a1b9
[utils] Fix `traverse_obj` depth when is_user_input 3 years ago
pukkandan 57015a4a3f
[youtube] extractor-arg to show live dash formats
If replay is enabled, these formats can be used to download the last 4 hours
3 years ago
pukkandan 9cc1a3130a
Fix resuming when using `--no-part`
Closes #576
3 years ago
pukkandan b51d2ae3ca
Add compat-option `no-keep-subs`
Closes #630
3 years ago
Jesse fee5f0c909
[adobepass] Add MSO Cablevision (#635)
Authored by: Jessecar96
3 years ago
funniray 7bb6434767
[vrv] Fix thumbnail extraction (#634)
Authored by: funniray
3 years ago
pukkandan 124bc071ee
Fix wrong extension for intermediate files
Closes #632
3 years ago
pukkandan a047eeb6d2
Add regex to `--match-filter`
This does not fully deprecate `--match-title`/`--reject-title`
since `--match-filter` is only checked after the extraction is complete,
while `--match-title` can often be checked from the flat playlist.

Fixes: https://github.com/ytdl-org/youtube-dl/issues/9092, https://github.com/ytdl-org/youtube-dl/issues/23035
3 years ago
Max Teegen 77b87f0519 Add all format filtering operators also to `--match-filter`
PR: https://github.com/ytdl-org/youtube-dl/pull/27361

Authored by: max-te
3 years ago
pukkandan 678da2f21b [twitch:clips] Extract `display_id`
PR: https://github.com/ytdl-org/youtube-dl/pull/29684
Fixes: https://github.com/ytdl-org/youtube-dl/issues/29666

Authored by: dirkf
3 years ago
pukkandan cc3fa8d39d Handle `BrokenPipeError`
PR: https://github.com/ytdl-org/youtube-dl/pull/29505
Fixes: https://github.com/ytdl-org/youtube-dl/issues/29082

Authored by: kikuyan
3 years ago
pukkandan 89efdc15dd [ffpmeg] Allow `--ffmpeg-location` to be a file with different name 3 years ago
pukkandan 8012d892bd
Ensure sanitization of infodict before printing to stdout
* `filter_requested_info` is renamed to a more appropriate name `sanitize_info`
3 years ago
Stavros Ntentos 9d65e7bd6d
Fix `--compat-options filename` (#629)
The correct default filename is `%(title)s-%(id)s.%(ext)s`

Authored by: stdedos
3 years ago
SsSsS 36576d7c4c
[Newgrounds] Improve extractor and fix playlist (#627)
Authored by: u-spec-png
3 years ago
nikhil bb36a55c41
[nbcolympics:stream] Fix extractor
PR: https://github.com/ytdl-org/youtube-dl/pull/29688
Closes: #617, https://github.com/ytdl-org/youtube-dl/issues/29665

* Livestreams are untested
* If using ffmpeg as downloader, v4.3+ is needed since `-http_seekable` option is necessary
* Instead of making a seperate key for each arg that needs to be passed to ffmpeg, I made `_ffmpeg_args`
* This deprecates `_seekable`, but the option is kept for compatibility

Authored by: nchilada, pukkandan
3 years ago
MinePlayersPE 3dbb2a9dcb
[RCTIPlus] Support events and TV (#625)
Authored by: MinePlayersPE
3 years ago
The Hatsune Daishi 9997eee4af
[openrec] Add extractors (#624)
Authored by: nao20010128nao
3 years ago
Wes 3e376d183e
[nbcolympics] Update extractor for 2020 olympics (#621)
Fixes: https://github.com/yt-dlp/yt-dlp/issues/617#issuecomment-891834323

Authored by: wesnm
3 years ago
Sam 888299e6ca
[VrtNU] Fix XSRF token (#588)
PR: https://github.com/ytdl-org/youtube-dl/pull/29614
Authored-by: pgaig
3 years ago
pukkandan e5611e8eda
[ffmpeg] Fix streaming `mp4` to `stdout` 3 years ago
SsSsS 8e6cc12c80
[Vine] Remove invalid formats (#614)
Authored by: u-spec-png
3 years ago
pukkandan e9d9efc0f2
[version] update
:ci skip all
3 years ago
pukkandan cf87314d4e
[youtube] Extract SAPISID only once 3 years ago
pukkandan 1bd3639f69
[tenplay] Add MA15+ age limit (#606)
Authored by: pento
3 years ago
LE 68f5867cf0
[CBS] Add fallback (#579)
Related: https://github.com/ytdl-org/youtube-dl/issues/29564
Authored-by: llacb47, pukkandan
3 years ago
Ashish 605cad0be7
[Vimeo] Better extraction of original file (#599)
Authored by: Ashish0804
3 years ago
Ashish e8384376c0
[CBS] Add ParamountPlusSeriesIE (#603)
Authored by: Ashish0804
3 years ago
David e7e94f2a5c
[youtube] Add age-gate bypass for unverified accounts (#600)
Adds `_creator` variants for each client

Authored by: zerodytrash, colethedj, pukkandan
3 years ago
pukkandan a46a815b05
[cleanup] Fix linter in 96fccc101f 3 years ago
pukkandan 96fccc101f [downloader] Allow streaming unmerged formats to stdout using ffmpeg
For this to work:
1. The downloader must be ffmpeg
2. The selected formats must have the same protocol
3. The formats must be downloadable by ffmpeg to stdout

Partial solution for: https://github.com/ytdl-org/youtube-dl/issues/28146, https://github.com/ytdl-org/youtube-dl/issues/27265
3 years ago
pukkandan dbf5416a20 [cleanup] Refactor some code 3 years ago
pukkandan d74a58a186
Set `home:` as the default key for `-P` 3 years ago
pukkandan f5510afef0
[FormatSort] Fix bug for audio with unknown codec 3 years ago
pukkandan e4f0275711
Add compat-option `no-clean-infojson` 3 years ago
pukkandan e0f2b4b47d
[utils] Fix slicing of reversed `LazyList`
Closes #589
3 years ago
coletdjnz eca330cb88
[youtube] Fix default global API key
bug introduced in 000c15a4ca
3 years ago
Wes d24734daea
[adobepass] Add MSO Sling TV (#596)
Original PR: ytdl-org/youtube-dl#29686
Closes: #300, ytdl-org/youtube-dl#18132

Authored by: wesnm
3 years ago
MinePlayersPE d9e6e9481e
[RCTIPlus] Remove PhantomJS dependency (#595)
Authored by: MinePlayersPE
3 years ago
pukkandan 3619f78d2c
[youtube] Misc cleanup (#577)
Authored by: pukkandan, colethedj
3 years ago
pukkandan 65c2fde23f
[youtube] Add `thirdParty` to agegate clients (#577)
* This allows more videos like `tf2U5Vyj0oU` to become embeddable
    See https://github.com/yt-dlp/yt-dlp/pull/575#issuecomment-888837000
* Also added tests for all types of age-gate

Closes #581
3 years ago
pukkandan 000c15a4ca
[youtube] simplify and de-duplicate client definitions (#577) 3 years ago
colethedj 9275f62cf8
[youtube] Improve age-gate detection (#577)
Authored by: colethedj
3 years ago
coletdjnz 6552469433
[youtube] Force `hl=en` for comments (#594)
Closes #532
3 years ago
MinePlayersPE 11cc45718c
[vidio] Fix login error detection (#582)
Authored by: MinePlayersPE
3 years ago
Ashish fe07e2c69f
[Hotstar] Support cookies (#584)
Closes #583 
Authored by: Ashish0804
3 years ago
Ashish 89ce723edd
[Mxplayer] Add h265 formats (#572)
Authored by: Ashish0804
3 years ago
Sipherdrakon 45d1f15725
[dplay] Add `ScienceChannelIE` (#567)
Authored by: Sipherdrakon
3 years ago
rigstot a318f59d14
[generic] Support KVS player (#549)
* Replaces the extractor for thisvid

Fixes: https://github.com/ytdl-org/youtube-dl/issues/2077
Authored-by: rigstot
3 years ago
pukkandan 7d1eb38af1
Add format types `j`, `l`, `q` for outtmpl
Closes #345
3 years ago
pukkandan 901130bbcf Expand and escape environment variables correctly in outtmpl
Fixes: https://www.reddit.com/r/youtubedl/comments/otfmq3/ytdlp_same_parameters_different_results
3 years ago
MinePlayersPE c0bc527bca
[YouTube] Age-gate bypass implementation (#575)
* Calling the API with `clientScreen=EMBED` allows access to most age-gated videos - discovered by @ccdffddfddfdsfedeee (https://github.com/yt-dlp/yt-dlp/issues/574#issuecomment-887171136)
* Adds clients: (web/android/ios)_(embedded/agegate), mweb_embedded
* Renamed mobile_web to mweb

Closes #574

Authored by pukkandan, MinePlayersPE
3 years ago
pukkandan 2a9c6dcd22
[youtube] Fix format sorting when using alternate clients 3 years ago
coletdjnz 5a1fc62b41
[youtube] Add `mobile_web` client (#557)
Authored by: colethedj
3 years ago
pukkandan b4c055bac2
[youtube] Add `player_client=all` 3 years ago
pukkandan ea05b3020d
Remove `asr` appearing twice in `-F` 3 years ago
pukkandan 9536bc072d
[bilibili] Improve `_VALID_URL` 3 years ago
Ashish 8242bf220d
[HotStarSeriesIE] Fix regex (#569)
Authored by: Ashish0804
3 years ago
Ashish 4bfa401d40
[UtreonIE] Add extractor (#562)
Authored by: Ashish0804
3 years ago
nixxo 0222620725
[mediaset] Fix extraction (#564)
Closes #365
Authored by: nixxo
3 years ago
pukkandan 1fe3c4c27e
[version] update
:ci skip all
3 years ago
pukkandan f703a88055
Release 2021.07.24 3 years ago
pukkandan a353beba83
[youtube:tab] Extract video duration early
Based on: https://github.com/ytdl-org/youtube-dl/pull/29487 by glenn-slayden
3 years ago
pukkandan 052e135029
[youtube] Simplify `_get_text` early 3 years ago
xtkoba cb89cfc14b
[test] Add Python 3.10 (#480)
Authored-by: pukkandan, xtkoba
3 years ago
pukkandan 063c409dfb
[cookies] Handle errors when importing `keyring`
Workaround for #551
3 years ago
Matt Broadway 767b02a99b
[cookies] Handle `sqlite` `ImportError` gracefully (#554)
Closes #544
Authored by: mbway
3 years ago
pukkandan f45e6c1126
[downloader] Pass same status object to all `progress_hooks` 3 years ago
pukkandan 3944e7af92
[youtube] Fix subtitles only being extracted from the first client
Closes #547
3 years ago
pukkandan ad34b2951e
Try all clients even if age-gated
Reverts: 892e31ce7c

If some API calls have any issue, saving the state will cause unnecessary errors
3 years ago
pukkandan c8fa48fd94
[youtube] Disable `get_video_info` age-gate workaround
This now seems to be completely dead
Closes: #553
3 years ago
coletdjnz 2fd226f6a7
[youtube] Fix age-gated videos for API clients when cookies are supplied (#545)
Fixes #543
Authored by: colethedj
3 years ago
pukkandan 3ba7740dd8
[downloader] Pass `info_dict` to `progress_hook`s 3 years ago
pukkandan 29b208f6f9
[cookies] bugfix
Fixes: https://github.com/yt-dlp/yt-dlp/pull/488#discussion_r674352059
3 years ago
pukkandan e4d666d27b
[version] update
:ci skip all
3 years ago
pukkandan 245524e6a3
Release 2021.07.21
and fix some typos
Closes #538
3 years ago
pukkandan 9c0d7f4951
[youtube] Make `--extractor-retries` work for more errors
Closes #507
3 years ago
pukkandan e37d0efbd9
Fix bug where `original_url` was not propagated when `_type`=`url` 3 years ago
coletdjnz c926c9541f
[youtube] Add debug message for SAPISID cookie extraction (#540)
Authored by: colethedj
3 years ago
Matt Broadway 982ee69a74
Add option `--cookies-from-browser` to load cookies from a browser (#488)
* also adds `--no-cookies-from-browser`

Original PR: https://github.com/ytdl-org/youtube-dl/pull/29201
Authored by: mbway
3 years ago
pukkandan 7ea6541124
[youtube] Improve extraction of livestream metadata
Modified from and closes #441
Authored by: pukkandan, krichbanana
3 years ago
pukkandan ae30b84072
Add field `live_status` 3 years ago
pukkandan cc9d1493c6
bugfix for 50fed816dd 3 years ago
Philip Xu f6755419d1
[douyin] Add extractor (#513)
Authored-by: pukkandan, pyx
3 years ago
Henrik Heimbuerger 145bd631c5
[nebula] Authentication via tokens from cookie jar (#537)
Closes #496
Co-authored-by: hheimbuerger, TpmKranz
3 years ago
pukkandan b35496d825
Add `only_once` param for `write_debug` 3 years ago
pukkandan 352d63fdb5
[utils] Improve `traverse_obj` 3 years ago
pukkandan 11f9be0912
[youtube] Extract data from multiple clients (#536)
* `player_client` accepts multiple clients
* default `player_client` = `android,web`
* music clients can be specifically requested
* Add IOS `player_client`
* Hide live dash since they can't be downloaded

Closes #501

Authored-by: pukkandan, colethedj
3 years ago