Commit Graph

636 Commits (582fad70f547b7008495400b38c8b32c3412cd27)

Author SHA1 Message Date
Philip Xu f6755419d1
[douyin] Add extractor (#513)
Authored-by: pukkandan, pyx
3 years ago
Henrik Heimbuerger 145bd631c5
[nebula] Authentication via tokens from cookie jar (#537)
Closes #496
Co-authored-by: hheimbuerger, TpmKranz
3 years ago
pukkandan 352d63fdb5
[utils] Improve `traverse_obj` 3 years ago
pukkandan 11f9be0912
[youtube] Extract data from multiple clients (#536)
* `player_client` accepts multiple clients
* default `player_client` = `android,web`
* music clients can be specifically requested
* Add IOS `player_client`
* Hide live dash since they can't be downloaded

Closes #501

Authored-by: pukkandan, colethedj
3 years ago
coletdjnz a1a7907bc0
[youtube] Fix controversial videos when requested via API (#533)
Closes: https://github.com/yt-dlp/yt-dlp/issues/511#issuecomment-883024350
Authored by: colethedj
3 years ago
pukkandan d61fc64618
[youtube:tab] Fix channels tab 3 years ago
pukkandan da503b7a52
[youtube] Make `parse_time_text` and `_extract_chapters` non-fatal
Related: #532, 7c365c2109
3 years ago
pukkandan 7c365c2109
[youtube] Sanity check `chapters` (and refactor related code)
Closes #520
3 years ago
pukkandan cca80fe611 [youtube] Extract even more thumbnails and reduce testing
* Also fix bug where `_test_url` was being ignored

Ref: https://stackoverflow.com/a/20542029
Related: #340
3 years ago
pukkandan 0ba692acc8
[youtube] Extract more thumbnails
* The thumbnail URLs are hard-coded and their actual existence is tested lazily
* Added option `--no-check-formats` to not test them

Closes #340, Related: #402, #337, https://github.com/ytdl-org/youtube-dl/issues/29049
3 years ago
pukkandan d9488f69c1
[crunchyroll:playlist] Force http
Closes #495
3 years ago
mzbaulhaque 8d9b902243
[pornflip] Add new extractor (#523)
Authored-by: mzbaulhaque
3 years ago
coletdjnz fe93e2c4cf
[youtube] misc cleanup and bug fixes (#505)
* Update some `_extract_response` calls to keep them consistent
* Cleanup continuation extraction related code using new API format
* Improve `_extract_account_syncid` to support multiple parameters
* Generalize `get_text` and related functions into one
* Update `INNERTUBE_CONTEXT_CLIENT_NAME` with integer values

Authored by: colethedj
3 years ago
coletdjnz 314ee30548
[youtube] Fix session index extraction and headers for non-web player clients (#526)
Fixes #522
3 years ago
coletdjnz 34917076ad
[youtube] Fix authentication when using multiple accounts
`SESSION_INDEX` in `ytcfg` is the index of the active account and should be sent as `X-Goog-AuthUser` header

Closes #518
Authored by @colethedj
3 years ago
The Hatsune Daishi ccc7795ca3
[yahoo:gyao:player] Relax `_VALID_URL` (#503)
Authored by: nao20010128nao
3 years ago
Felix S da1c94ee45
[generic] Extract previously missed subtitles (#515)
* [generic] Extract subtitles in cases missed previously
* [common] Detect discarded subtitles in SMIL manifests
* [generic] Extract everything in the SMIL manifest

Authored by: fstirlitz
3 years ago
coletdjnz 47193e0298
[youtube:tab] Extract playlist availability (#504)
Authored by: colethedj
3 years ago
coletdjnz 49bd8c66d3
[youtube:comments] Improve comment vote count parsing (fixes #506) (#508)
Authored by: colethedj
3 years ago
Felix S 182b6ae8a6
[RTP] Fix extraction and add subtitles (#497)
Authored by: fstirlitz
3 years ago
felix 198f7ea89e
[extractor] Allow extracting multiple groups in `_search_regex`
From #497, Authored by: fstirlitz
3 years ago
coletdjnz c888ffb95a
[youtube] Use android client as default and add age-gate bypass for it (#492)
Authored by: colethedj
3 years ago
coletdjnz 9752433221
[youtube:comments] Fix `is_favorited` (#491)
Authored by colethedj
3 years ago
pukkandan f0ff9979c6
[vlive] Extract thumbnail directly in addition to the one from Naver
Closes #477
3 years ago
coletdjnz 2d6659b9ea
[youtube:comments] Move comment extraction to new API (#466)
Closes #438, #481, #485 

Authored by: colethedj
3 years ago
Kevin O'Connor c5370857b3
[BravoTV] Improve metadata extraction (#483)
Authored by: kevinoconnor7
3 years ago
pukkandan 6606817a86
[utils] Add `variadic` 3 years ago
zackmark29 73d829c144
[VIKI] Rewrite extractors (#475)
Closes #462
Also added extractor-arg `video_types` to `vikichannel`

Co-authored-by: zackmark29, pukkandan
3 years ago
pukkandan 60bdb7bd9e
[youtube] Fix sorting of 3gp format 3 years ago
pukkandan 4bb6b02f93
Improve `extractor_args` parsing 3 years ago
pukkandan 51f8a31d65
Update to ytdl-commit-a803582
[peertube] only call description endpoint if necessary
a803582717
3 years ago
Tom-Oliver Heidel be05d5cff1
[soundcloud] Allow login using oauth token (#469)
Authored by: blackjack4494
3 years ago
zenerdi0de 30d569d2ac
[fancode] Fix extraction, support live and allow login with refresh token (#471)
Authored-by: zenerdi0de
3 years ago
OhMyBahGosh 08625e4125
[AdobePass] Add Spectrum MSO (#470)
From: https://github.com/ytdl-org/youtube-dl/pull/26792

Co-authored by: kevinoconnor7, ohmybahgosh
3 years ago
pukkandan 3acf6d3856
[Funimation] Rewrite extractor (See desc) (#444)
* Support direct `/player/` URL
* Treat the different versions of an episode as different formats of a single video. So `experience_id` can no longer be used as the video `id` and the `episode_id` is used instead. This means that all existing archives will break
* Extractor options `language` and `version` to pre-select them
* Compat option `seperate-video-versions` to fall back to old behavior (including using the old video IDs)

Closes #428
3 years ago
pukkandan 46890374f7
[extractor] Minor improvements (See desc)
1. Allow removal of login hint - extractors can set their own login hint as part of `msg`
2. Cleanup `_merge_subtitles` signature
3 years ago
pukkandan 60755938b3
[extractor] Prevent unnecessary download of hls manifests
and refactor `hls_split_discontinuity` code
3 years ago
pukkandan bc97cdae67
[cleanup] Fix linter and some typos
Related: https://github.com/ytdl-org/youtube-dl/pull/29398
3 years ago
nyuszika7h e010672ab5
[videa] Fix extraction (#463)
Authored by: nyuszika7h
3 years ago
MinePlayersPE 17f0eb66b8
[RCTIPlus] Add extractor (#443)
Authored by: MinePlayersPE
3 years ago
pukkandan b1e60d1806 [facebook] Extract description and fix title
Partially fixes: #453
3 years ago
krichbanana f6745c4980
[Youtube] Choose correct Live chat API for upcoming streams (#460)
Authored by: krichbanana
3 years ago
coletdjnz 109dd3b237
[youtube] Use new API for additional video extraction requests (#328)
Co-authored-by: colethedj, pukkandan
Closes https://github.com/yt-dlp/yt-dlp/issues/427
Workarounds for https://github.com/ytdl-org/youtube-dl/issues/29326, https://github.com/yt-dlp/yt-dlp/issues/319, https://github.com/ytdl-org/youtube-dl/issues/29086
3 years ago
LE 1e79316e20
[TBS] Support livestreams (#448)
Authored by: llacb47
3 years ago
coletdjnz 45261e063b
[youtube:comments] Fix error handling and add `itct` to params (#446)
Should close #439 (untested)

Authored by: colethedj
3 years ago
pukkandan 49c258e18d
[youtube] Fix subtitle names for age-gated videos
Related: https://github.com/iv-org/invidious/pull/2205#issuecomment-868680486
3 years ago
pukkandan 5d3a0e794b
Add `--extractor-args` to pass extractor-specific arguments 3 years ago
Mevious 125728b038
[funimation] Add `FunimationShowIE` (#442)
Closes #436

Authored by: Mevious
3 years ago
LE 412cce82b0
[yahoo] Fix extraction (#435)
Fixes: https://github.com/ytdl-org/youtube-dl/issues/28290

Co-authored-by: llacb47, pukkandan
3 years ago
pukkandan 2b18a8c590
[plutotv] Improve `_VALID_URL`
Closes #431
3 years ago
pukkandan ed807c1837
Update to ytdl-commit-379f52a
[liveleak] Remove extractor
379f52a495
3 years ago
Mevious 29f63c9672
[funimation] Extract subtitles (#434)
Closes #420, https://github.com/ytdl-org/youtube-dl/issues/25645
Related: https://github.com/ytdl-org/youtube-dl/pull/24906

Authored by: Mevious
3 years ago
pukkandan 9fc0de5796
[hotstar] Use server time for authentication instead of local time
Closes #396
3 years ago
siikamiika c60ee3a218
[youtube_live_chat] Support ongoing live chat (#422)
Authored by: siikamiika
3 years ago
pukkandan e6779b9400
[twitcasting] Websocket support (#399)
Closes #392
Authored by: nao20010128nao
3 years ago
pukkandan e36d50c5dd
[websockets] Add `WebSocketFragmentFD` (#399)
Necessary for #392

Co-authored by: nao20010128nao, pukkandan
3 years ago
pukkandan 4e6767b5f2
[youtube] Temporary fix for age-gate
Related:
https://stackoverflow.com/a/67629882
https://github.com/yt-dlp/yt-dlp/issues/319
https://github.com/ytdl-org/youtube-dl/issues/29333
https://github.com/ytdl-org/youtube-dl/issues/29086
3 years ago
pukkandan d0fb4bd16f
[pornhub] Extract `cast`
Closes #406, https://github.com/ytdl-org/youtube-dl/pull/27384
3 years ago
felix 3fd4c2a543
[mediasite] Extract slides (#343)
Fixes:
https://github.com/ytdl-org/youtube-dl/issues/4974#issue-58006762
https://github.com/ytdl-org/youtube-dl/issues/4540#issuecomment-69574231
https://github.com/ytdl-org/youtube-dl/pull/11185#issuecomment-335554239

Authored by: fstirlitz
3 years ago
felix cdb19aa4c2
[downloader/mhtml] Add new downloader (#343)
This downloader is intended to be used for streams that consist of a
timed sequence of stand-alone images, such as slideshows or thumbnail
streams

This can be used for implementing:

https://github.com/ytdl-org/youtube-dl/issues/4974#issue-58006762
https://github.com/ytdl-org/youtube-dl/issues/4540#issuecomment-69574231
https://github.com/ytdl-org/youtube-dl/pull/11185#issuecomment-335554239

https://github.com/ytdl-org/youtube-dl/issues/9868
https://github.com/ytdl-org/youtube-dl/pull/14951


Authored by: fstirlitz
3 years ago
pukkandan 8326b00aab
Allow `images` formats
Necessary for #343.

* They are identified by `vcodec=acodec='none'`
* These formats show as the worst in `-F`
* Any postprocessor that expects audio/video will be skipped
* `b*` and all related selectors will skip such formats
* This commit also does not add any selector for downloading such formats. They have to be explicitly requested by the `format_id`. Implementation of a selector is left for when #389 is resolved
3 years ago
pukkandan 45db527fa6
[youtube] Login is not needed for `:ytrec` 3 years ago
coletdjnz 4ba001080f
[youtube] Non-fatal alert reporting for unavailable videos page (#401)
Co-Authored by: colethedj, pukkandan
4 years ago
coletdjnz 1974e99f4b
[youtube] Improve SAPISID cookie handling (closes #393) (#395)
Author: colethedj
4 years ago
pukkandan 3b1fe47d84
Release 2021.06.08 4 years ago
pukkandan 46358f647d
Update to ytdl-commit-c2350ca
Update MSVC 2010 redist URL
c2350cac24
4 years ago
pukkandan 89ee4cf8ae
[viki] Fix extraction
Closes #381
Code from: 59e583f7e8
4 years ago
pukkandan eb0f9d6838
[zoom] Extract transcripts as subtitles 4 years ago
pukkandan d3d8d8184a
[extractor] Fix pre-checking archive for some extractors
The `id` regex group must be present for `_match_id` and pre-checking archive to work correctly
4 years ago
pukkandan e85a39717a
[twitcasting] Add TwitCastingUserIE, TwitCastingLiveIE
Closes #374

Code adapted from: f1fb9222bb/youtube_dl/extractor/twitcasting.py
Authored by: pukkandan, nao20010128nao
4 years ago
MinePlayersPE f2cd7060fc
[vidio] Add VidioPremierIE and VidioLiveIE (#371)
Authored-by: MinePlayersPE
4 years ago
felix e2efe599aa
[common] Fix FourCC fallback when parsing ISM (#372)
In some DASH manifests, the FourCC attribute is actually present,
but empty.  We thus apply the same fallback to 'AACL' that we do
when the attribute is entirely absent.

Authored by: fstirlitz
4 years ago
pukkandan 5e1dba8ed6
Remove duplicate file `trovolive.py` 4 years ago
pukkandan bea742222f
[youtube] Support shorts URL
Closes #375
4 years ago
Ashish 6e6390321c
[Hotstar] Add HotStarSeriesIE (#366)
Authored by: Ashish0804
4 years ago
pukkandan 14eb1ee1cb
Update to ytdl-commit-d495292
[ard] Relax _VALID_URL and fix video ids
d495292852

Closes #357
4 years ago
coletdjnz 879e7199bb
[archiveorg] Add YoutubeWebArchiveIE (#356)
Co-authored by: colethedj, pukkandan, alex-gedeon
4 years ago
MinePlayersPE 10bb7e51e8
[vidio] Add login support (#362)
Authored by: MinePlayersPE
4 years ago
pukkandan b0089e8992
[fancode] Add extractor (#316,#354)
Closes #269, #363

Authored by: rmsmachine
4 years ago
Ashish a3ed14cbaf
[Voot] Add VootSeriesIE (#351)
Authored by: Ashish0804
4 years ago
LE 9dee4df559
[Saitosan] Add new extractor (#350)
Closes #224
Authored by: llacb47
4 years ago
pukkandan adddc50cbf
[extractor] Functions to parse socket.io response as json
Authored by: pukkandan, llacb47
4 years ago
MinePlayersPE 46c43ffc9d
[vidio] Support premium videos (#358)
Authored by: MinePlayersPE
4 years ago
pukkandan 37a3bb66a7
[extractor] Allow `note=False` when extracting manifests 4 years ago
pukkandan 46953e7e6e
[youtube:playlist] fix bug 4 years ago
pukkandan 077c476276
[zee5] Fix m3u8 formats extension 4 years ago
rhsmachine 65af1839c6
[patreon] Support vimeo embeds (#349)
Authored by: rhsmachine
4 years ago
pukkandan 177877c544
[extractor] Always prefer native hls downloader by default
When the manifest is not downloadable by native downloader, it already is able to detect it and switch to `ffmpeg`. So there doesn't seem to be a reason anymore to use ffmpeg as the preferred downloader
4 years ago
Oliver Freyermuth f17c702270
[ard] Allow URLs without `-` before id
https://github.com/ytdl-org/youtube-dl/pull/29091

Authored by: olifre
4 years ago
pukkandan 3907333c5d
[extractor] Skip subtitles without URI in m3u8 manifests
Closes #339

Authored by: hheimbuerger
4 years ago
Ashish 09d18ad07e
[Sonyliv] Add subtitle support (#342)
Authored by: Ashish0804
4 years ago
Ashish 9572eaaa11
[ShemarooMe] Add extractor (#332)
Closes #307
Co-authored-by: Ashish0804, pukkandan
4 years ago
pukkandan 8d68ab98a7
[youtube] Fix bug where not all hls formats were extracted
Bug introduced in 9297939ec3
4 years ago
Ashish 135e6b93f4
[SonyLIV] Add SonyLIVSeriesIE (#331)
Authored by: Ashish0804
4 years ago
king-millez 13a49340ed
[telemundo] add extractor (#327)
Closes #284
Authored by: king-millez
4 years ago
pukkandan 09f1580e2d
[youtube] `/live` URLs should raise error if channel is not live
Fixes: https://github.com/ytdl-org/youtube-dl/issues/29090
4 years ago
pukkandan b73612a254
Update to ytdl-commit-dfbbe29
[redbulltv] fix embed data extraction
dfbbe2902f
4 years ago
king-millez 5014558ab9
[parlview] Add extractor (#322)
Authored by: king-millez
4 years ago
coletdjnz 00ae27690d
[youtube] Add `html5=1` param to `get_video_info` page requests (#329)
Workaround for #319, https://github.com/ytdl-org/youtube-dl/issues/29086
Authored by: colethedj
4 years ago
pukkandan 9d5d4d64f8
[youtube] Better message when login required 4 years ago
king-millez 98784ef8d6
[audius:artist] Add extractor (#323)
Authored by: king-millez
4 years ago
pukkandan d3fc8074a4
[youtube] Sort audio-only formats correctly
Closes #317
4 years ago
pukkandan 07cce701de
[cleanup] linter, code formatting and readme 4 years ago
king-millez 74e001af1d
[tenplay] Fix extractor (#314)
Authored by: king-millez
4 years ago
pukkandan ff2751ac9c
[youtube] Always extract `maxresdefault` thumbnail
Fixes: https://github.com/ytdl-org/youtube-dl/issues/29049
4 years ago
pukkandan abcdd12b26
[youtube:tab] Support youtube music `MP` pages 4 years ago
pukkandan 18db754858
[youtube:tab] Redirect `UC` channels that doesn't have a `videos` tab
Many topic URLs don't have a videos tab, but has an equivalent `UU` playlist.
If there is no playlist, fallback to using channel page
4 years ago
pukkandan fe03a6cdc8
[youtube:tab] Support youtube music `VL` and `browse` pages 4 years ago
pukkandan cd684175ad
[youtube:tab] Support channel search
Fixes: https://github.com/ytdl-org/youtube-dl/issues/29071
4 years ago
pukkandan da692b7920
[cleanup] youtube tests 4 years ago
pukkandan 95c01b6c16
[youtube:tab] Show alerts only from the final webpage 4 years ago
pukkandan a06916d98e
[extractor] Add `write_debug` and `get_param` 4 years ago
coletdjnz 9297939ec3
[Youtube] Extract more formats for `music.youtube` URLs (#311)
Based on: https://github.com/ytdl-org/youtube-dl/pull/28778, https://github.com/ytdl-org/youtube-dl/pull/26160

Co-authored-by: craftingmod, colethedj, pukkandan
4 years ago
pukkandan 774d79cc4c
[youtube] Add language names
Co-authored by: nixxo, tpikonen
Based on: https://github.com/ytdl-org/youtube-dl/pull/26112
Closes: #310
4 years ago
pukkandan 2412044c90
Add field `name` for subtitles
Co-authored by: pukkandan, tpikonen

Based on: #310, https://github.com/ytdl-org/youtube-dl/pull/26112
4 years ago
pukkandan 120916dac2
[youtube] multiple subtitles in same language
Fixes: https://github.com/ytdl-org/youtube-dl/issues/21164
Related: #310, https://github.com/ytdl-org/youtube-dl/pull/26112
4 years ago
pukkandan 0fb983f62d
[youtube] Extract audio language 4 years ago
pukkandan 0760b0a7e2
Standardize `write_debug` 4 years ago
pukkandan d908aa636a
[cleanup] Fix typos 4 years ago
pukkandan 3d89341b47
[common] bugfix for when `compat_opts` is not given 4 years ago
pukkandan d8ec40b39f
[rmcdecouverte] Generalize `_VALID_URL`
Closes #291
4 years ago
pukkandan fac988053f
Release 2021.05.11
* and some documentation improvements
4 years ago
pukkandan 61241abbb0
[generic] Respect the encoding in manifest 4 years ago
pukkandan 53ed7066ab
Option `--compat-options` to revert some of yt-dlp's changes
* Deprecates `--list-formats-as-table`, `--list-formats-old`
4 years ago
pukkandan 2f567473c6
[Plugins] Prioritize plugins over standard extractors
and prevent plugins from overwriting the standard extractor classes

Closes #304
4 years ago
pukkandan 41d1cca328
Update to ytdl-commit-a726009
[blinkx] Remove extractor
a726009987
4 years ago
pukkandan 12e73423f1
[plutotv] Fix format extraction for some urls
* And fallback to the first urls if ad-free urls can't be found
Closes #299
4 years ago
pukkandan 7700b37f39
[plutotv] Extract subtitles from manifests 4 years ago
Ashish c28cfda81f
[SonyLiv] Fix `title` and `series` extraction (#301)
Authored by: Ashish0804
4 years ago
pukkandan 3158150cb7
[utils] Add `network_exceptions` 4 years ago
pukkandan 06425e9621
[blinkx] Minor fix
Fixes: https://github.com/ytdl-org/youtube-dl/issues/28941
4 years ago
pukkandan 1815d1028b
[zee5] Fix py2 compatibility 4 years ago
The Hatsune Daishi 0fa9a1e236
[whowatch] Add extractor #292
closes #223

Authored by: nao20010128nao 
Modified from: 9e4a0e061a/youtube_dl/extractor/whowatch.py
4 years ago
pukkandan 80185155a1
[ukcolumn] Add Extractor
Closes #287
4 years ago
pukkandan c755f1901f
[CBS] Improve `_VALID_URL` to support movies
Closes #290
Tested by: BeeMuffins
4 years ago
pukkandan 68b91dc905
[youtube] Add `oembed` to reserved names 4 years ago
pukkandan 88f06afc0c
[rmcdecouverte] Improve `_VALID_URL`
Closes #291
4 years ago
CXwudi 40078a55e2
[niconico] Fix bug in thumbnail extraction #289
Bug from: 6b1d8c1e30
Authored by: CXwudi
4 years ago
pukkandan 07e4a40a9a
[crackle] Improve extraction (See desc)
Closes #282

* Refactor authorization as an extension to `_download_json`
* Better error messages and warnings
* Respect `--ignore-no-formats-error`
* Extract subtitles from manifests
* Try with crackle's geo-location service if all hard-coded countries fail
4 years ago
pukkandan e28f1c0ae8
[cleanup] Fix linter and some typos
* Also remove inconsistent use of `"` in setup.py
4 years ago
pukkandan ef39f8600a
[curiositystream] Fix collections
Closes #277

* A bug with authentication was reported in <https://github.com/yt-dlp/yt-dlp/issues/277#issuecomment-828254721> but cannot be tested without an account
4 years ago
pukkandan 2291dbce2a
[niconico] Fix HLS formats
Closes #171

* The structure of the API JSON was changed
* Smile Video seems to be no longer available. So remove the warning
* Move ping to downloader
* Change heartbeat interval to 40sec
* Remove unnecessary API headers

Authored-by: CXwudi, tsukumijima, nao20010128nao, pukkandan
Tested by: tsukumijima
4 years ago
pukkandan 895b0931e5
[youtube:tab] Detect playlists inside community posts 4 years ago
pukkandan 1ad047d0f7
[nebula] Move to nebula.app
Closes #272
Tested by: Lamieur
4 years ago
pukkandan be6202f12b
Subtitle extraction from streaming media manifests #247
Authored by fstirlitz
Modified from: https://github.com/ytdl-org/youtube-dl/pull/6144

Closes: #73
Fixes:
https://github.com/ytdl-org/youtube-dl/issues/6106
https://github.com/ytdl-org/youtube-dl/issues/14977
https://github.com/ytdl-org/youtube-dl/issues/21438
https://github.com/ytdl-org/youtube-dl/issues/23609
https://github.com/ytdl-org/youtube-dl/issues/28132

Might also fix (untested):
https://github.com/ytdl-org/youtube-dl/issues/15424
https://github.com/ytdl-org/youtube-dl/issues/18267
https://github.com/ytdl-org/youtube-dl/issues/23899
https://github.com/ytdl-org/youtube-dl/issues/24375
https://github.com/ytdl-org/youtube-dl/issues/24595
https://github.com/ytdl-org/youtube-dl/issues/27899

Related:
https://github.com/ytdl-org/youtube-dl/issues/22379
https://github.com/ytdl-org/youtube-dl/pull/24517
https://github.com/ytdl-org/youtube-dl/pull/24886
https://github.com/ytdl-org/youtube-dl/pull/27215

Notes:
* The functions `extractor.common._extract_..._formats` are still kept for compatibility
* Only some extractors have currently been moved to using `_extract_..._formats_and_subtitles`
* Direct subtitle manifests (without a master) are not supported and are wrongly identified as containing video formats
* AES support is untested
* The fragmented TTML subtitles extracted from DASH/ISM are valid, but are unsupported by `ffmpeg` and most video players
    * Their XML fragments can be dumped using `ffmpeg -i in.mp4 -f data -map 0 -c copy out.ttml`.
        Once the unnecessary headers are stripped out of this, it becomes a valid self-contained ttml file
    * The ttml subs downloaded from DASH manifests can also be directly opened with <https://github.com/SubtitleEdit>
* Fragmented WebVTT files extracted from DASH/ISM are also unsupported by most tools
    * Unlike the ttml files, the XML fragments of these cannot be dumped using `ffmpeg`
    * The webtt subs extracted from DASH can be parsed by <https://github.com/gpac/gpac>
    * But validity of the those extracted from ISM are untested
4 years ago
Felix S e8f834cd8d [threeqsdn] Extract subtitles from streaming manifests 4 years ago
Felix S e0e624ca7f [canvas] Extract subtitles from streaming manifests 4 years ago
Felix S ec4f374c05 [wat] Extract subtitles from streaming manifests 4 years ago
Felix S c811e8d8bd [atresplayer] Extract subtitles from streaming manifests 4 years ago
Felix S b2cd5da460 [francetv] Extract subtitles from the HLS manifest 4 years ago
Felix S 2de3b21e05 [uplynk] Extract subtitles from HLS manifests 4 years ago
Felix S 4bed436371 [twitter] Extract subtitles from HLS manifests 4 years ago
Felix S efe9dba595 [srgssr] Extract subtitles from HLS manifests 4 years ago
Felix S 47f4203dd3 [nytimes] Extract subtitles from HLS manifests 4 years ago
Felix S 015c10aeec [roosterteeth] Use common code for subtitle extraction 4 years ago
Felix S a00d781b73 [elonet] Use common code for subtitle extraction 4 years ago
Felix S 0c541b563f [tv4] Extract subtitles from streaming manifests 4 years ago
Felix S 64a5cf7929 [byutv] Extract subtitles from streaming manifests 4 years ago
Felix S 7a450a3b1c [generic] Extract subtitles from direct SSTR manifest links 4 years ago
Felix S 7de27caf16 [generic] Extract subtitles from direct DASH manifest links 4 years ago
Felix S c26326c1be [generic] Extract subtitles from direct HLS manifest links 4 years ago
Felix S 66a1b8643a [downloader/ism] Support muxing TTML subtitles 4 years ago
Felix S 4a2f19abbd [downloader/hls] Assemble single-file WebVTT subtitles from HLS segments 4 years ago
Felix S f6a1d69a87 [extractor/common] Extend _extract_akamai_formats to also extract subtitle tracks 4 years ago
Felix S fd76a14259 [extractor/common, downloader/ism] Extract SSTR subtitle tracks
_parse_ism_formats was extended into _parse_ism_formats_and_subtitles;
all direct users were updated, though _extract_ism_formats was left
as a compatibility wrapper.

The SSTR downloader was also modified in order to prepare for muxing
subtitle streams, although no support for any subtitle codecs was
added in this commit.
4 years ago
Felix S 171e59edd4 [extractor/common] Extract DASH subtitle tracks
_extract_mpd_formats and _parse_mpd_formats were extended into
_…_formats_and_subtitles; wrappers with old names are provided
for compatibility.
4 years ago
Felix S a0c3b2d5cf [extractor/common] Extract HLS subtitle tracks
_extract_m3u8_formats is renamed to _extract_m3u8_formats_and_subtitles
and extended to handle subtitle tracks instead of skipping them;
a wrapper with the old name is provided for compatibility.

_parse_m3u8_formats is likewise renamed and extended, but without adding
the compatibility wrapper; the test suite is adjusted to test the enhanced
method instead.
4 years ago
Felix S 19bb39202d [extractor/common] Generalise _merge_subtitles
This allows modifying a subtitles dictionary in-place.
4 years ago
Hadi0609 db9a564b6a
[zee5] Fix extraction for some URLs (#279)
Closes: #278
4 years ago
Felix S c72967d5de
[mediasite] Generalize URL pattern (#275)
Authored by: fstirlitz
4 years ago
pukkandan b982cbdd0e
[limelight] Obey `allow_unplayable_formats` 4 years ago
pukkandan 6a04a74e8b
[FormatSort] Fix for when some formats have quality and others don't 4 years ago
CXwudi 6b1d8c1e30
[niconico] Fix title and thumbnail extraction (#273)
Authored by: CXwudi
4 years ago
Ashish 87c3d06271
[Mxplayer] Add MxplayerShowIE (#270)
Authored by: Ashish0804
4 years ago
pukkandan cf9d6cfb0c
[tubi] Raise "no video formats" error when video url is empty
Related: #266
4 years ago
pukkandan bbed5763f1
[francetvinfo] Improve video id extraction
Closes #261
4 years ago
pukkandan 1bdae7d312
Update to ytdl-commit-7e8b3f9
[youtube] Remove unused code
7e8b3f9439
4 years ago
Felix S a471f21da6
[mildom] Remove proxy (#260)
Closes #251
Makes 2cff495997, ab406a1c0e, #252 obsolete

Authored by: fstirlitz
4 years ago
pukkandan 6efb071135
[BilibiliChannel] Fix pagination
Closes #222

ccca21d7f5
Coauthored by: nao20010128nao, pukkandan
4 years ago
pukkandan 3b4775e021
[go] Fix `_VALID_URL`
Closes #255
4 years ago
pukkandan ab406a1c0e
[mildom] Warn user of proxy 4 years ago
pukkandan 8c54a3051d
[youtube] Bugfix in `_extract_ytcfg` 4 years ago
pukkandan 3097d9e512
[mildom:user:vod] Download only necessary amount of pages 4 years ago
pukkandan c1df120eda
[mildom:vod] Remove proxy
* Proxy is needed only for live videos
4 years ago
pukkandan 2cff495997
[mildom] Change proxy
Related: #251
Closes #252
4 years ago
pukkandan d0491a1ebe
[twitcasting] Fix extractor
* `Origin: https://twitcasting.tv` must be sent when requesting the webpage. Otherwise the extracted `m3u8` will always give a `502`
* Fix regex for when `data-movie-playlist` is a dict containing the needed list
* media initialization is fully supported; so change downloader to native

Closes #220
4 years ago
nixxo b9d68c199b
[rai] Add support for http formats (#208)
Authored by: nixxo
4 years ago
pukkandan 201c145953
Update to ytdl-commit-9f6c03
[cbsnews] Fix extraction for python <3.6
9f6c03a006
4 years ago
pukkandan 5d34200268
[youtube:tab] Reload with unavailable videos for all playlists
If the unavailable video is in a later page, the warning and button are not shown in the initial webpage
So we force all playlists' initial page to reload with the correct params
4 years ago
pukkandan b7da73eb19
Add option `--ignore-no-formats-error`
* Ignores the "no video format" and similar errors
* Experimental - Some extractors may still throw these errors
4 years ago
pukkandan 6a39ee13f7
Fix inconsistent use of `report_warning` 4 years ago
coletdjnz 358de58c4d
[youtube:tab] Show unavailable videos in playlists (#242)
Closes #231

Authored by: colethedj
4 years ago
pukkandan a7191c6f57
Fix some linter and typos 4 years ago
lkho baa5873942
[viu:ott] Fix extractor (see desc)
* add language_flag_id query param
* add support for premium account (untested since I dont have a premium account)
* support entire series

Code from:
https://github.com/blackjack4494/youtube-dlc/pull/211
https://github.com/ytdl-org/youtube-dl/pull/15182
https://github.com/ytdl-org/youtube-dl/pull/26775

Fixes:
https://github.com/yt-dlp/yt-dlp/issues/219
https://github.com/ytdl-org/youtube-dl/issues/27946
https://github.com/ytdl-org/youtube-dl/issues/27863
https://github.com/ytdl-org/youtube-dl/issues/27812
https://github.com/ytdl-org/youtube-dl/issues/27464
https://github.com/ytdl-org/youtube-dl/issues/26788
https://github.com/blackjack4494/yt-dlc/issues/136

Possibly also fixes (untested):
https://github.com/ytdl-org/youtube-dl/issues/16992
https://github.com/ytdl-org/youtube-dl/issues/26701

Co-authored by: lkho, pukkandan
4 years ago
coletdjnz 79360d99d3
[youtube] Standardize API calls for tabs, mixes and search (#245)
Authored by: colethedj
4 years ago
pukkandan 46fff7105e
[youtube] Ignore invalid stretch ratio
Closes #244
4 years ago
Ashish b5be6dd504
[TubiTv] Add TubiTvShowIE (#243)
Authored by: Ashish0804
4 years ago
coletdjnz 8ea3f7b909
[youtube] Improve channel syncid extraction to support ytcfg (#241)
Authored by: colethedj
4 years ago
pukkandan b31fdeedfd
[documentation] Improvements 4 years ago
coletdjnz f4f751af40
[youtube] Parse API parameters from initial webpage (#230)
* Obtain innertube_context, api_key and x-goog-visitor-id from webpage
* Generalize the header & Innertube_context extraction across YouTube extractors

Related: 1b0a13f33c

Authored by: colethedj
4 years ago
pukkandan 1988fab7e3
[youtube] Fix thumbnail URL
Closes: https://github.com/yt-dlp/yt-dlp/issues/233 https://github.com/ytdl-org/youtube-dl/issues/28023
4 years ago
pukkandan f7ad71607d
Update to ytdl-commit-4fb25ff
[maoritv] Add new extractor
4fb25ff5a3

Except:
[vimeo] improve extraction 3ae9c0f410
[youtube:tab] Pass innertube context... 1b0a13f33c
4 years ago
pukkandan f37468c41f
Fix default of `dynamic_mpd` 4 years ago
pukkandan 3ffc7c89b0
[youtube] Fix _extract_alerts
* Sometimes one warning is split between multiple runs
* Also simplified code
4 years ago
Henrik Heimbuerger f1823403b0
[nebula] Add extractor (watchnebula.com) (#122)
Authored by: hheimbuerger
4 years ago
LE 384fb069ec
[NFHSNetwork] Add extractor (#217)
Authored by: llacb47
4 years ago
bopol a4ddaf231e
[nitter] Fix extraction of reply tweets (#218)
And update instance list

Closes #215 

Authored by: B0pol
4 years ago
pukkandan 7e60c06925
[bilibili] Fix uploader
Related: https://github.com/yt-dlp/yt-dlp/issues/222
4 years ago
coletdjnz d92f5d5a90
[youtube] Extract comments' approximate timestamp (#221)
Authored by: colethedj
4 years ago
bopol c24ce07a84
[nitter] Fix thumbnails (#216)
Authored by: B0pol
4 years ago
Ashish de6758128e
[DiscoveryPlusIndia] Add DiscoveryPlusIndiaShowIE (#213)
Co-authored-by: Ashish <ashish@pop-os.localdomain>
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
4 years ago
pukkandan 73d4343e39
Fix some `m3u8` not obeying `--allow-unplayable-formats` 4 years ago
pukkandan 39ed931e53
Update to ytdl-commit-654b4f4
[youtube] prioritize information from YoutubeIE for playlist entries
654b4f4ff2
4 years ago
Matthew b28f8d244a
[YouTube] Show premium state in `availability` (#209)
Authored by colethedj
4 years ago
pukkandan 54df8fc5b2
[ViewSource] Add extractor to handle `view-source:` 4 years ago
The Hatsune Daishi 5d39972ed0
[mildom] Update extractor to comply with current proxy (#212)
Authored by nao20010128nao
4 years ago
pukkandan 0481374e1d
[DiscoveryPlus] Add discoveryplus.in (#203)
Closes #201
4 years ago
pukkandan df0c81513e
[generic] Detect Invidious' link element (Closes #195) 4 years ago
pukkandan 3f6a90eb63
[ard:mediathek] Fix video id extraction (Closes #202) 4 years ago
pukkandan b050d210df
Fix some typos and linter 4 years ago
pukkandan cce889b900 Update to ytdl v2021-04-01
14f29f087e

Closes #205
4 years ago
nixxo a6ae61a4c2
[la7] Add podcasts and podcast playlists (#198)
Authored by: nixxo
4 years ago
pukkandan b704fc1a68
Revert commit 8562218
[ard] improve clip id extraction
8562218350

This was merged incorrectly
4 years ago
pukkandan 1418a0437f
Update to ytdl-commit-8562218
[ard] improve clip id extraction
8562218350
4 years ago
pukkandan 3700c7ef10
[linuxacadamy] Fix login 4 years ago
pukkandan 394dcd4486
Write current epoch to infojson when using `--no-clean-infojson`
Closes #191
4 years ago
pukkandan 83b20a970d
[youtube_live_chat] fix bug when trying to set cookies
Closes #194
4 years ago
pukkandan e1feb88fdf
[niconico] Fix for when logged in
Related: https://github.com/yt-dlp/yt-dlp/issues/171#issuecomment-803692716
Co-authored by: CXwudi, xtkoba
4 years ago
pukkandan 421a459573
Update to ytdl-commit-7e79ba7
7e79ba7dd6
[vimeo:album] Fix extraction for albums with number of videos multiple to page size
4 years ago
Matthew c224251aad
[youtube] Show if video is `private`, `unlisted` etc in new field `availability` (#188)
Closes: #185, https://github.com/ytdl-org/youtube-dl/issues/25631

Authored by: colethedj, pukkandan
4 years ago
pukkandan 037cc66ec8
[linuxacadamy] Improve regex
TODO: We need to make a more robust standard regex for fetching js objects from html
4 years ago
Two Sheds" Jackson 9160a0c6a2
[nbc] Improve metadata extraction (#187)
Authored by: 2ShedsJackson
4 years ago
2ShedsJackson 5c5fae6d2f
[amcnetworks] Fix extractor (#179)
* Prefer use of manifest based on `releasePid` since the one based on `videoPid` may have Fairplay
* Additional thumbnail images were added
* Don't add `season_number` and `series` to `title`
* `series` is now set to `None` rather than "_" when empty
* fix bug with age limit

Authored by: 2ShedsJackson
4 years ago
pukkandan c1d3a4a8f0
[niconico] Add user extractor
Co-authored by: animelover1984
4 years ago
pukkandan adc74b3c6d
[bilibili] Add anthology support
Closes: #118

Co-authored by: animelover1984
4 years ago
pukkandan beb4b92a66
More consistent warning messages (#173)
Co-authored by: Damiano Amatruda <damiano.amatruda@outlook.com>
4 years ago
pukkandan e167860ce7
[documentation] Add deprecated options and aliases in readme 4 years ago
pukkandan e3c076970e
[youtube] bugfix for channel playlist extraction
Bug introduced in cd7c66cf01
4 years ago
pukkandan 7978e172f3
[niconico] Partial fix
TODO:
Smile formats don't work
Extraction code that no longer works needs to be removed

Adapted from:
b5eff52dd9
by animelover1984

Related:
https://github.com/yt-dlp/yt-dlp/issues/171
https://github.com/animelover1984/youtube-dl/issues/29
https://github.com/ytdl-org/youtube-dl/issues/28470
4 years ago
nixxo 605d299f83
[stitcher] Merge from youtube-dl (#175)
This cherry-picks the commits:
a563c97c5c (diff-cc37718b18689c1c0588c4c74518853ef43ab020bbd674f3c04d4eae037798d2)
8522bcd97c (diff-cc37718b18689c1c0588c4c74518853ef43ab020bbd674f3c04d4eae037798d2)

Authored by: nixxo
4 years ago
nixxo 18c1f04362
[rcs] Improved extraction (See desc) (#170)
* improved `video_data` extraction
* added an extra fallback value for `description`
* improved regex in `RCSVariousIE`

Authored by: nixxo
4 years ago
pukkandan e4beae703d
[formatSort] Fix `quality` being ignored
Closes #172
4 years ago
pukkandan 10db0d2f57
Update to ytdl-commit-3be0980
3be098010f
4 years ago
Matthew a1c5d2ca64
[Youtube] Rewrite comment extraction (#167)
Closes #121

TODO:
* Add an option for the user to specify newest/popular and max number of comments
* Refactor the download code and generalize with TabIE
* Parse time_text to timestamp
4 years ago
CHJ85 ea3a012d2a
[pluto.tv] Add extractor (#163)
https://github.com/ytdl-org/youtube-dl/pull/27621

Authored by: kevinoconnor7
4 years ago
pukkandan 5b8917fb52
[zee5] Support zee5originals 4 years ago
nixxo 8eec0120a2
[rai] fix drm check (#168)
Bug introduced by #150
Authored by: nixxo
4 years ago
nixxo e4edeb6226
[wimtv] Add extractor (#161)
Added support for VODs, live and embeds

Authored by: nixxo
4 years ago
Ashish d488e254d9
[Zee5] Add Show Extractor (#160)
Co-authored-by: Ashish <ashish@pop-os.localdomain>
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
4 years ago
pukkandan 54759df586
[zee5] Improve regex 4 years ago
nixxo 605b684c2d
[mtv] Add mtv.it and extract series metadata (#156)
* New extractors: MTVItalia, MTVItaliaProgramma
* Extract fields: series, season_number, episode_number

Authored-by: nixxo
4 years ago
teesid 1f52a09e2e
[vimeo] Fix videos with password
https://github.com/ytdl-org/youtube-dl/pull/27992

Fixes: https://github.com/ytdl-org/youtube-dl/issues/28354

Authored by teesid
4 years ago
pukkandan fc21af505c
Fix some videos downloading with m3u8 extension 4 years ago
pukkandan 015f3b3120
[bilibili] Change `Accept` header (Closes #145)
This is a temporary fix. Ideally we should find a more reasonable accept string that just "*/*"

Fixes: https://github.com/ytdl-org/youtube-dl/issues/28363 https://github.com/ytdl-org/youtube-dl/issues/28341

Thanks to animelover1984 for identifying the problem
4 years ago
nixxo 0852947fcc
[rai] Check for DRM (#150)
Authored by: nixxo <nixxo@protonmail.com>
4 years ago
pukkandan 2be71994c0
[youtube] Detect when Mixes end or wrap around 4 years ago
pukkandan 26fe8ffed0
[youtube] Fix community page continuation (Closes #152) 4 years ago
nixxo feee67ae88
[gedi] Improvements from youtube-dl (#149)
Authored-by: nixxo <c.nixxo@gmail.com>
4 years ago
Ashish 1caaf92d47
[MXPlayer] Rewrite extractor with show support (#141)
Co-authored-by: Ashish <ashish@pop-os.localdomain>
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
4 years ago
Matthew d069eca7a3
[Youtube] Fix private feeds/playlists on multi-channel accounts (#143)
Authored by: colethedj
4 years ago
Matthew f3eaa8dd1c
[Youtube] Extract alerts from continuation (#144)
Related: #143

Authored by: colethedj
4 years ago
pukkandan 36147a63e3
[trovo] Pass origin header (Closes #139)
Fixes: https://github.com/ytdl-org/youtube-dl/issues/28346
4 years ago
pukkandan 57db6a87ef
[lbry] Support lbry:// url
https://github.com/ytdl-org/youtube-dl/pull/28207

Fixes: https://github.com/ytdl-org/youtube-dl/issues/28084

Authored by: nixxo <nixxo@protonmail.com>
4 years ago
pukkandan cd7c66cf01
[youtube] Fix history, trending and mix playlists (#136)
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
Co-authored-by: Matthew <colethedj@protonmail.com>
4 years ago
shirt-dev 2c736b4f61
[cbs] Add support for ParamountPlus (#138)
Related: https://github.com/ytdl-org/youtube-dl/issues/28342

Authored-by: shirtjs <2660574+shirtjs@users.noreply.github.com>
4 years ago
pukkandan 3721515bde
Update to ytdl-2021.03.03 4 years ago
Matthew a5c5623470
[YouTube] Use new browse API for continuation page extraction. (#131)
Known issues (these issues existed in previous API as well)
* Mix playlists only give 1 page (25 vids)
* Trending only gives 1 video
* History gives 5 pages (200 vids)

Co-authored-by: colethedj, pukkandan
4 years ago
pukkandan c705177da2
[youtube] Throw error when `--extractor-retries` are exhausted (Closes #130) 4 years ago
Ashutosh Chaudhary 097b056c5a
[mxplayer] Add new extractor
https://github.com/ytdl-org/youtube-dl/pull/27325
Authored by: codeasashu
4 years ago
Han Dai f3b737ed19
[nick] fix extraction
https://github.com/ytdl-org/youtube-dl/pull/27900
Authored by: DennyDai
4 years ago
pukkandan ee1e05581e
[mtv] Fix extractor by reverting changes made in youtube-dlc
youtube-dl has since fixed the extractor and the changes from the two sources are incompatible
4 years ago
pukkandan ec5e77c558
Update to ytdl-2021.03.02 4 years ago
shirt-dev b3b30a4bca
Fix HLS playlist downloading (#127)
Co-authored-by: shirtjs <2660574+shirtjs@users.noreply.github.com>
4 years ago
pukkandan 5ef7d9bdd8
Release 2021.03.01 4 years ago
pukkandan 62bff2c170
Add option `--extractor-retries` to retry on known extractor errors
* Currently only used by youtube

Fixes https://github.com/ytdl-org/youtube-dl/issues/28194
Possibly also fixes: https://github.com/ytdl-org/youtube-dl/issues/28289 (can not confirm since the issue isn't reliably reproducible)
4 years ago
pukkandan f0884c8b3f Cleanup some code (see desc)
* `--get-comments` doesn't imply `--write-info-json` if `-J`, `-j` or `--print-json` are used
* Don't pass `config_location` to `YoutubeDL` (it is unused)
* [bilibiliaudio] Recognize the file as audio-only
* Update gitignore
* Fix typos
4 years ago
pukkandan 277d6ff5f2
Extract comments only when needed #95 (Closes #94) 4 years ago
pukkandan 1cf376f55a Add option `--sleep-requests` to sleep b/w requests (Closes #106)
* Also fix documentation of `sleep_interval_subtitles`

Related issues:
https://github.com/blackjack4494/yt-dlc/issues/158
https://github.com/blackjack4494/youtube-dlc/issues/195
https://github.com/ytdl-org/youtube-dl/pull/28270
https://github.com/ytdl-org/youtube-dl/pull/28144
https://github.com/ytdl-org/youtube-dl/issues/27767
https://github.com/ytdl-org/youtube-dl/issues/23638
https://github.com/ytdl-org/youtube-dl/issues/26287
https://github.com/ytdl-org/youtube-dl/issues/26319
4 years ago
pukkandan 86878b6cd9 [hrfensehen] Fix wrong import 4 years ago
pukkandan b3d1242534 [youtube] Fix inconsistent `webpage_url` (closes #119) 4 years ago
pukkandan ed9b7e3dd3 Fix bug with m3u8 format extraction 4 years ago
Robin Dunn 31a5e037a7
[viki] Fix viki play pass authentication (#111)
Authored by: RobinD42
4 years ago
pukkandan 14fdfea973 [youtube] Retry on incomplete `ytInitialData`
Related: #116
4 years ago
pukkandan bce8cbb089 [tennistv] Fix format sorting 4 years ago
pukkandan 7c60c33efe Update to ytdl-commit-4460329
44603290e5

Except: gedi, tmz
7422a2194f
8cb4b71909
d81421af4b
4 years ago
Pccode66 7a5c1cfe93
Completely change project name to yt-dlp (#85)
* All modules and binary names are changed
* All documentation references changed
* yt-dlp no longer loads youtube-dlc config files
* All URLs changed to point to organization account

Co-authored-by: Pccode66
Co-authored-by: pukkandan
4 years ago