95e42d7336
[extractor/common] Ensure response handle is not prematurely closed before it can be read if it matches expected_status ( resolves #17195 , closes #17846 , resolves #17447 )
6 years ago
bebef10909
[extractor/common] Add validation for JSON-LD URLs
6 years ago
0e7b8d3eac
[extractor/common] Fix typos
6 years ago
6f1f59f39c
[extractor/common] Introduce channel meta fields
6 years ago
bd21ead2a2
[extractor/common] add support for DASH and MSS formats extraction in SMIL manifests
6 years ago
0685d9727b
[utils] Share JSON-LD regex
6 years ago
eca1f0d115
[extractor/common] Properly escape % in MPD templates ( closes #16867 )
6 years ago
5e8e2fa51f
[extractor/common] Use source URL as Referer for HTML5 entries ( closes #16849 )
6 years ago
d391b7e23d
[extractor/common] Introduce expected_status for convenient accept of failed HTTP requests
...
Useful when some non-success (2xx) HTTP status codes should be considered normal. Previously this required to manually catch corresponding exceptions and read the response.
7 years ago
504f20dd30
Remove experimental mark for some options
7 years ago
5f95927a62
Improve geo bypass mechanism
...
* Introduce geo bypass context
* Add ability to bypass based on IP blocks in CIDR notation
* Introduce --geo-bypass-ip-block
7 years ago
e7e4a6e0f9
[extractor/common] Extract interaction statistic
7 years ago
0fe7783ece
[extractor/common] Add _download_json_handle
7 years ago
66b686727b
[extractor/common] Relax JSON-LD context check ( closes #16006 )
7 years ago
6780154e6b
[extractor/common] Improve thumbnail extraction for HTML5 entries
7 years ago
47a5cb7734
Generalize XML manifest processing code and improve XSPF parsing ( closes #15794 )
7 years ago
e0d198c18d
[extractor/common] Add _download_xml_handle
7 years ago
96b8b9abae
[extractor/generic] Support relative URIs in _parse_xspf
...
<location> can have relative URIs, not just absolute.
7 years ago
f4b7427279
[extractor/common] Improve jwplayer subtitles extraction ( closes #15695 )
7 years ago
240f26229d
[extractor/common] Respect secure schemes in _extract_wowza_formats
7 years ago
00c97e3e7a
[downloader/http] Add ability to pass downloader options via info dict
7 years ago
3931b84597
[extractor/common] Improve _json_ld for articles
7 years ago
0d9c48de4f
[extractor/common] Improve DASH formats extraction for jwplayer ( #9242 , #15187 )
7 years ago
126f225bcf
[extractor/common] Add container meta field for formats extracted in _parse_mpd_formats
7 years ago
2501d41ef4
[common] use AACL as the default fourcc when AudioTag is 255
7 years ago
9d6ac71c27
[extractor/common] Fix extraction of DASH formats with the same representation id ( closes #15111 )
7 years ago
2132edaa03
[extractor/common] Move X-Forwarded-For setup code into _request_webpage
7 years ago
c10c93238e
[extractor/common] Introduce uploader, uploader_id and uploader_url meta fields for playlists ( #11427 , #15018 )
7 years ago
78593e294c
Add references for #14844
7 years ago
603fc4e0ea
[extractor/common] Add durations for DASH fragments with bare SegmentURLs
7 years ago
41bf647e89
[extractor/common] Add support for DASH manifests with SegmentLists with bare SegmentURLs
7 years ago
f610dbb05f
[extractor/common] Use final URL when dumping request ( closes #14769 )
7 years ago
ea2295842f
[common] skip Apple FairPlay m3u8 manifests( closes #14741 )
7 years ago
187ee66c94
[extractor/common] Add protocol for f4m formats
7 years ago
48107c198b
[f4m] Prefer baseURL for relative URLs ( closes #14660 )
7 years ago
044eeb1455
[extractor/common] Respect URL query in _extract_wowza_formats ( closes #14645 )
7 years ago
9211e3319e
[extractor/common] Prefix format id for audio only HLS formats
7 years ago
50d808f5c9
[common] add support for jwplayer youtube embeds
7 years ago
c110944fa2
[extractor/common] Fix typo in _parse_mpd_formats
7 years ago
4ed2d7b7d1
Fix flake8 issues after #14225
7 years ago
a88d461dff
Merge pull request #14225 from Tithen-Firion/openload-phantomjs-method
...
Openload phantomjs method
7 years ago
1ed4549942
[extractor/common] Extract format id from label attribute of source tag for HTML5 videos ( #14034 )
7 years ago
dd121cc1ca
[extractor/common] Extract height from res attribute of source tag for HTML5 videos ( closes #14034 )
7 years ago
e01c3d2ef7
[extractor/common] Introduce _parse_xml
7 years ago
b359e977b9
[extractor/common] Make HLS and DASH extraction non fatal in _parse_html5_media_entries ( closes #13970 )
7 years ago
4850478543
[extractor/common] Add support for float durations in _parse_mpd_formats ( closes #13919 )
7 years ago
868f79db41
[extractor/common] Fix _media_formats
7 years ago
ac8491fcca
[extractor/common] Make _family_friendly_search optional
7 years ago
82889d4ae5
[extractor/common] Respect source's type attribute for HTML5 media ( closes #13892 )
7 years ago
1141e9104b
Use relative paths for DASH fragments ( closes #12990 )
...
10x reduced JSON size
refs #13810
7 years ago
749ca5eced
[extractor/common] Fix playlist_from_matches
7 years ago
4328ddf82b
[extractor/common] Add support for AMP tags in _parse_html5_media_entries
7 years ago
c69701c6ab
[extractor/common] Improve _json_ld
7 years ago
96a2daa1ee
[extractor/common] Improve jwplayer subtitles extraction
8 years ago
6a9cb29509
[extractor/common] Fix json dumping with --geo-bypass
...
The line "[debug] Using fake IP %s (%s) as X-Forwarded-For." was printed
to stdout even with -j/-J, which breaks the resultant JSON.
8 years ago
0a268c6e11
[extractor/common] Improve jwplayer formats extraction ( closes #13379 )
8 years ago
1afd0b0da7
[extractor/common] Return unicode string from _match_id
8 years ago
f2e2f0c777
[extractor/common] Fix rtmp and rtsp formats' URLs in _extract_wowza_formats
8 years ago
6f76679804
[extractor/common] Add support for schemeless URLs in _extract_wowza_formats ( closes #13088 , closes #13092 )
8 years ago
76d5a36391
[extractor/common] Respect Width and Height attributes in ISM manifests
8 years ago
ff6f9a6704
[extractor/common] fix typo in _extract_akamai_formats
8 years ago
c89267d31a
Merge branch 'master' into openload-phantomjs-method
8 years ago
55949fede6
[common] introduce chapters field
8 years ago
33a81c2c6f
[extractor/common] Extract view count from JSON-LD
8 years ago
c89b49f743
[extractor/common] Add manifest_url for explicit group rendition formats
8 years ago
ff99fe529e
Don't list master m3u8 playlists in format list ( closes #12832 )
8 years ago
ac9c69ace7
[extractor/common] Improve jwplayer regex
8 years ago
40e41780f1
[phantomjs] add cookie support
8 years ago
3019cb0c99
[extractor/common] Rephrase comment
8 years ago
ddd258f922
[test_InfoExtractor] Add m3u8 parsing test for NAME attribute in EXT-X-STREAM-INF tag
8 years ago
9c99bef704
[extractor/common] Use float for scaled tbr
8 years ago
cb2520802d
[extractor/common] Improve m3u8 extraction ( closes #12211 )
...
* Extract m3u8 parsing to separate method
* Improve rendition groups extraction
* Build stream name according stream GROUP-ID
* Ignore reference to AUDIO group without URI when stream has no CODECS
+ Add test coverage for parsing m3u8 from #11507 , #11995 , #12211 and twitch vod
8 years ago
bae1404893
[extractor/common] Add support for video of WebPage context in _json_ld ( closes #12778 )
8 years ago
bf1b87cd91
[common] Relax JWPlayer regex and remove duplicate urls( #12768 )
8 years ago
40fcba5edb
improve coding style
8 years ago
fd47550885
[extractor/common] Add coding cookie
8 years ago
4457823dda
[extractor/common] Move censorship checks to a separate method and add check for just another ISP
8 years ago
4f06c1c9fc
Merge branch 'master' of github.com-rndusr:rg3/youtube-dl into fix/str-item-assignment
8 years ago
c73e330e7a
_find_jwplayer_data() returns dict or None
...
This simplifies code for callers of `_find_jwplayer_data()` which no longer have
to run `_parse_json()` on the return value.
It also makes sure that `_find_jwplayer_data()` returns either a `dict` or
`None` and nothing else.
8 years ago
46b18f2349
[BostonGlobe] New. Nonstandard version of Brightcove.
...
Has a "data-brightcove-video-id" instead of a "data-video-id," otherwise
pretty much just Brightcove. Except the Globe isn't all Brightcove
videos, so fallback to Generic, too.
Also, abstract playlist_from_matches() from generic.py to common.py, and use
it here.
History of these changes can be found in
51170427d4b1143572a498dedaee61863a5b2c5b.
8 years ago
b51dc9db0e
[extractor/common] Extract SMIL formats from jwplayer
8 years ago
1a2192cb90
[extractor/common] Pass arguments to _parse_jwplayer_formats and PEP8
8 years ago
0236cd0dfd
[extractor/common] Improve height extraction and extract bitrate
8 years ago
ed0cf9b383
[extractor/common] Move jwplayer formats extraction in separate method
8 years ago
eeb0a95684
[extractor/common] Add 'preference' to _parse_html5_media_entries
...
Some websites, like NJPWorld, put different qualities on different
player pages.
8 years ago
eea0716cae
[extractor/common] Print origin country for fake IP
8 years ago
336a76551b
[extractor/common] Do not quit _initialize_geo_bypass on empty countries
8 years ago
dc0a869e5e
[extractor/common] Fix typo
8 years ago
e39b5d4ab8
[extractor/common] Allow calling _initialize_geo_bypass from extractors ( #11970 )
8 years ago
3ccdde8cb7
[extractor/common] Emphasize geo bypass APIs are experimental
8 years ago
4248dad92b
Improve geo bypass mechanism
...
* Rename options to preffixly match with --geo-verification-proxy
* Introduce _GEO_COUNTRIES for extractors
* Implement faking IP right away for sites with known geo restriction
8 years ago
0a840f584c
Rename bypass geo restriction options
8 years ago
0016b84e16
Add faked X-Forwarded-For to formats' HTTP headers
8 years ago
773f291dcb
Add experimental geo restriction bypass mechanism
...
Based on faking X-Forwarded-For HTTP header
8 years ago
a4a554a793
[generic] Try parsing JWPlayer embedded videos ( closes #12030 )
8 years ago
08a00eef79
[extractor/common] skip m3u8 manifests protected with Adobe Flash Access
8 years ago
242a14a1f6
[extractor/common] Fix audio only with audio group in m3u8 ( closes #11995 )
8 years ago
2aec7256ae
[extractor/common] Speed-up media tags regex ( closes #11979 )
8 years ago
45024183ae
[infoq] Add audio only format if available ( #11565 )
...
* [infoq] Add audio only format if available
Refactor cookie code into a function.
Renamed formats to http_video, http_audio, rtmp_video
Renamed extract functions to video instead of videos as they return
one or no video.
* [infoq] Rename to _extract_cookies as it more than one
* [infoq] Remove redundant determine_ext
* [infoq] Add comment about hardcoded URL
* [infoq] Use _hidden_inputs instead of messy regex
* [infoq] Probe if audio URL is valid
Make it possible to pass headers to _is_valid_url
* [infoq] Add audio only test
8 years ago
e228616c6e
[extractor/common] Fix initialization template ( closes #11605 , closes #11825 )
8 years ago
Xiao Di Guan