Remita Amine
f39ffc5877
[common] extract formats from #EXT-X-MEDIA tags
8 years ago
Remita Amine
c7c43a93ba
[common] add helper method to extract akamai m3u8 and f4m formats
8 years ago
Yen Chi Hsuan
ad120ae1c5
[extractor/common] Change the default m3u8 protocol in HTML5
...
Helper functions should have consistent default values
8 years ago
Yen Chi Hsuan
520251c093
[extractor/common] Recognize m3u8 manifests in HTML5 multimedia tags
8 years ago
Remita Amine
2118fdd1a9
[common] add separate method for getting netrc ligin info
8 years ago
Sergey M․
321b5e082a
[extractor/common] Respect default in _search_json_ld
8 years ago
Remita Amine
37768f9242
[common] correctly lower the preference of m3u8 master manifest format
8 years ago
Remita Amine
d16b3c6677
[common] extract partOfTVSeries info in json-ld
8 years ago
Remita Amine
c57244cdb1
[common] lower the preference of m3u8 master manifest format
8 years ago
Sergey M․
46933a15d6
[extractor/common] Support root JSON-LD lists ( Closes #10203 )
8 years ago
Remita Amine
8b40854529
[common] lower proto_preference of rtsp formats
...
Most of the time the RtspFD fail to download videos but it report
success of the download with this output:
[mpv] 0 bytes
[download] 100% of 0.00B
8 years ago
Sergey M․
b070564efb
[extractor/common] Support multiple properties in _og_search_property
8 years ago
Sergey M․
40090e8d51
[extractor/common] Improve is_suitable
...
In order to fix breakage introduced by a3aa814b77
8 years ago
Sergey M․
f094834857
[extractor/common] Add support for $ in SegmentTemplate in MPD manifests
8 years ago
Remita Amine
00f4764cb7
[common] extract vbr, abr and fps for Unified Streaming Platform m3u8 manifests
8 years ago
Yen Chi Hsuan
0b68de3cc1
Merge pull request #8876 from remitamine/html5_media
...
[extractor/common] add helper method to extract html5 media entries
8 years ago
Sergey M․
95b31e266b
[extractor/common] Add expected_type in json ld routines
8 years ago
Sergey M․
6b3a3098b5
[extractor/common] Extract more metadata for VideoObject in _json_ld
8 years ago
Sergey M․
2de624fdd5
[extractor/common] Introduce filesize metafield for thumbnails
8 years ago
Déstin Reed
3fee7f636c
[flipagram] Add extractor
8 years ago
Yen Chi Hsuan
38cce791c7
Rename --cn-verfication-proxy to --geo-verification-proxy
...
And deprecate the former one
Since commit f138873900
, this option is
not limited to China websites, so rename it.
9 years ago
Sergey M․
bf8dd79045
[extractor/common] Fix sorting with custom field preference
9 years ago
remitamine
59bbe4911a
[extractor/common] add helper method to extract html5 media entries
9 years ago
Sergey M․
88d9f6c0c4
[utils] Add support for name list in _html_search_meta
9 years ago
remitamine
e154c65128
[downloader/hls] Add support for AES-128 encrypted segments in hlsnative downloader
9 years ago
Yen Chi Hsuan
a6571f1073
[common] Fix <bootstrapInfo> detection in F4M manifests
...
Regression since 0a5685b26f
9 years ago
Sergey M․
77b8b4e696
[extractor/common] Borrow quality metadata from parent set-level manifest for f4m
9 years ago
Yen Chi Hsuan
fac2af3c51
[common] Fix m3u8 extraction in f4m manifests
9 years ago
Yen Chi Hsuan
448bb5f333
[common] Fix non-bootstrapped support in f4m
9 years ago
Yen Chi Hsuan
240b60453e
[common] Support m3u8 in f4m manifests
...
Related: #9531
9 years ago
Yen Chi Hsuan
0a5685b26f
[common] Support non-bootstraped streams in f4m manifests
...
Related: #9531
9 years ago
Yen Chi Hsuan
7b2fcbfd4e
[common] Skip TYPE=CLOSED-CAPTIONS lines in m3u8 manifests
...
According to [1], valid values for TYPE are AUDIO, VIDEO, SUBTITLES
and CLOSED-CAPTIONS. Such a value is found in Anvato master playlists,
though I don't use _extract_m3u8_formats() in the end.
Part of #9522 .
[1] https://tools.ietf.org/html/draft-pantos-http-live-streaming-19#section-4.3.4.1
9 years ago
Yen Chi Hsuan
16da9bbc29
[common] Add _m3u8_meta_format() template
...
For extractors who handle m3u8 manifests by themselves. (eg., AnvatoIE)
Part of #9522
9 years ago
Yen Chi Hsuan
ad96b4c8f5
[common] Extract audio formats in SMIL
...
Found in http://www.cbc.ca/player/play/2657631896
Closes #5156
9 years ago
Sergey M․
ed56f26039
[extractor/common] Improve name extraction for m3u8 formats
9 years ago
Sergey M․
8a92e51c60
[extractor/common] Relax wording for creator metafield
9 years ago
Yen Chi Hsuan
e9c6cdf4a1
[common] Fix format_id construction for HLS
9 years ago
Kagami Hiiragi
b24d6336a7
[vlive] Add support for live videos
9 years ago
Yen Chi Hsuan
d6712378e7
Merge branch 'akamai_pv' of https://github.com/remitamine/youtube-dl into remitamine-akamai_pv
9 years ago
remitamine
fb72ec58ae
[extractor/common] do not process f4m manifest that contain akamai playerVerificationChallenge
9 years ago
Yen Chi Hsuan
2c0d9c6217
[extractor/common] Allow empty post data
9 years ago
Sergey M․
49caf3307f
[extractor/common] Remove irrelevant comment
9 years ago
Sergey M․
bacec0397f
[extractor/common] Relax _hidden_inputs
9 years ago
Sergey M․
fb38aa8b53
[extractor/common] Support arbitrary format strings for template based identifiers in mpd manifests ( Closes #9119 , closes #9120 )
9 years ago
Sergey M․
7a93ab5f3f
[extractor/common] Introduce music album metafields
9 years ago
Sergey M․
b507cc925b
[extractor/common] Carry long line
9 years ago
Sergey M․
db8ee7ec05
[extractor/common] Fix numeric identifiers conversion in DASH URL templates
9 years ago
remitamine
df634be2ed
[common] prefer using mime type over ext for smil subtitle extraction
...
the subtitle ext for http://www.cnet.com/videos/download-amazon-prime-movies-and-tv/
is adb_xml while using the mime type it get tt(application/smptett+xml)
9 years ago
Sergey M․
41d06b0424
[extractor/common] Improve _request_webpage
...
* Do not ignore data, headers and query for Requests
* Default values for headers and query switched to dicts since these are used by urllib itself
9 years ago
Sergey M․
b22ca76204
[extractor/common] Filter out unsupported encrypted media for f4m formats ( Closes #8573 )
9 years ago
Sergey M․
19dbaeece3
Remove _sort_formats from _extract_*_formats methods
...
Now _sort_formats should be called explicitly.
_sort_formats has been added to all the necessary places in code.
Closes #8051
9 years ago
Sergey M․
15707c7e02
[compat] Add compat_urllib_parse_urlencode and eliminate encode_dict
...
encode_dict functionality has been improved and moved directly into compat_urllib_parse_urlencode
All occurrences of compat_urllib_parse.urlencode throughout the codebase have been replaced by compat_urllib_parse_urlencode
Closes #8974
9 years ago
remitamine
49dea4913b
Merge pull request #8513 from remitamine/dash-sort
...
[extractor/common] fix dash formats sorting
9 years ago
Sergey M․
0fdbb3322b
[extractor/common] Add _parse_f4m_formats routine
9 years ago
remitamine
09f572fbc0
[extractor/common] add transform_source to _download_smil and _extract_smil_formats
9 years ago
remitamine
15bf934de5
Merge pull request #8819 from remitamine/simple-webpage-requests
...
[extractor/common] simplify using data, headers and query params with _download_* methods
9 years ago
remitamine
cdfee16818
[extractor/common] add data, headers and query params to _request_webpage
9 years ago
Yen Chi Hsuan
a6c8b75904
[common] Use mimeType to determine file extensions ( #8766 )
9 years ago
Yen Chi Hsuan
64f08d4ff2
Merge pull request #8766 from yan12125/dash-detect-ext
...
Detect file extensions of DASH formats from their codecs
9 years ago
Yen Chi Hsuan
af7d5a63b2
[common] Document protocol http_dash_segments
9 years ago
Yen Chi Hsuan
2def60c5f3
[common] Use codec2ext for DASH formats ( #8764 )
9 years ago
Yen Chi Hsuan
e9c0cdd389
[jython] Introduce compat_os_name
...
os.name is always 'java' on Jython
9 years ago
Sergey M․
7bcd2830dd
[extractor/common] Document uploader_url
9 years ago
Sergey M․
2bc0c46f98
[extractor/common] Document license metafield
9 years ago
Sergey M․
d77ab8e255
Add --mark-watched feature ( Closes #5054 )
9 years ago
Sergey M․
9cdffeeb3f
[extractor/common] Clarify rationale on media playlist detection
9 years ago
Sergey M․
fbb6edd298
[extractor/common] Properly extract audio only formats in master m3u8 playlists
9 years ago
Sergey M․
f5bdb44443
[extractor/common] Add _remove_duplicate_formats
9 years ago
remitamine
cafcf657a4
add more subtitles mime types to mimetype2ext and fix the platform subtitle extraction
9 years ago
Sergey M․
611c1dd96e
[refactor] Single quotes consistency
9 years ago
Sergey M․
d800609c62
[refactor] Do not specify redundant None as second argument in dict.get()
9 years ago
Sergey M․
bb20526b64
[extractor/common] Improve base url construction
9 years ago
remitamine
c349456ef6
[extractor/common] strip http urls in smil manifest
9 years ago
remitamine
81e1c4e2fc
[extractor/common] remove duplicate rtmp formats in smil manifest
9 years ago
remitamine
dd86780596
[extractor/common] fix dash formats sorting
9 years ago
remitamine
154c209e2d
[extractor/common] improve dash format ids
9 years ago
remitamine
51e9094f4a
[extractor/common] extract youtube dash formats filesize( fixes #8480 )
9 years ago
remitamine
d413095f7e
[extractor/common] remove duplicated formats and subtiles in smil manifests
9 years ago
remitamine
6a3828fddd
[common] use float conversion instead of using division from __future__
9 years ago
remitamine
91cb6b5065
rename _parse_mpd to _parse_mpd_formats and add default value for mpd namespace
9 years ago
remitamine
0826a0b555
[common] sort dash formats
9 years ago
remitamine
255732f0d3
[common] fix segment duration calculation
9 years ago
remitamine
53c269c6fd
[common] fix media_template string formating
9 years ago
remitamine
675d001633
[common] skip drm protected dash formats
9 years ago
remitamine
d577c79632
[common] ignore ISO 639-2 generic codes
9 years ago
remitamine
f14be22816
[common] remove duplicate reference to namespace
9 years ago
remitamine
9c74423510
[common] fix media template regex
9 years ago
remitamine
1bac34556f
[common] add a generic support for mpd manifests
9 years ago
Yen Chi Hsuan
2d2fa82d17
[common] Add _extract_dash_manifest_formats
9 years ago
Yen Chi Hsuan
c94678957f
[common] Remove unused arguments
9 years ago
Yen Chi Hsuan
16f38a699f
[common] Rename to namespace
...
For consistency with _parse_smil_*
9 years ago
Yen Chi Hsuan
df374b5222
[common] Prefer the manifest than formats_dict in determining codecs
9 years ago
Yen Chi Hsuan
5ea1eb78f5
[common] Fix for youtube
9 years ago
Yen Chi Hsuan
b323e1707d
[common] Modify _parse_dash_manifest for use in Facebook
9 years ago
Yen Chi Hsuan
17b598d30c
[common] _parse_dash_manifest() from youtube.py
9 years ago
Sergey M․
350cf045d8
[extractor/common] Restrict checks when auto calculating tbr
9 years ago
remitamine
a9d5f12fec
Merge pull request #8328 from remitamine/hls-master-detect
...
[extractor/common] detect media playlist in _extract_m3u8_formats
9 years ago
remitamine
7f32e5dc35
[extractor/common] detect media playlist in _extract_m3u8_formats
9 years ago
Sergey M․
b0d21deda9
[extractor/common] Auto calculate tbr when missing
9 years ago
Yen Chi Hsuan
77f785076f
[common] Keep full codec name from m3u8 manifests
...
See #8293 . This is for consistency between YouTube and HLS formats.
9 years ago
Yen Chi Hsuan
0b26ba3fc8
[extractor/common] Allow passing more parameters to _search_json_ld
9 years ago
Sergey M․
4ca2a3cf3c
[extractor/common] Add initial support for JSON-LD metadata extraction into info_dict
9 years ago
Jakub Wilk
dfb1b1468c
Fix typos
...
Closes #8200 .
9 years ago
Sergey M
3f3343cd3e
Merge pull request #8061 from dstftw/introduce-chapter-and-series-fields
...
Introduce chapter and series fields
9 years ago
Sergey M․
27bfd4e526
[extractor/common] Introduce number fields for chapters and series
9 years ago
Philipp Hagemeister
32f9036447
[ccc] Add language information to formats
9 years ago
Sergey M․
7109903e61
[extractor/common] Document chapter and series fields
9 years ago
Sergey M․
7e5edcfd33
Simplify formats accumulation for f4m/m3u8/smil formats
...
Now all _extract_*_formats routines return a list
9 years ago
remitamine
39d60b715a
Merge pull request #7769 from remitamine/sort
...
[common] lower (m3u8,rtmp,rtsp) format preference only if required program is not available
9 years ago
remitamine
d497a201ca
[common] use specific variable for protocol preference in _sort_formats
9 years ago
remitamine
8d29e47f54
[common] simplify the use of _extract_m3u8_formats and _extract_f4m_formats
9 years ago
Sergey M․
9b9c5355e4
Rename error_to_str to error_to_compat_str
9 years ago
Sergey M․
7f8b271465
Properly convert errors to strings
9 years ago
Sergey M․
dd85e4d707
[extractor/common] Properly decode error string on python 2 ( Closes #1354 , closes #3957 , closes #4037 , closes #6449 )
9 years ago
Sergey M․
62d231c004
[extractor/common] Clarify duration can be float
9 years ago
Sergey M?
5c2266df4b
Switch codebase to use sanitized_Request instead of
...
compat_urllib_request.Request
[downloader/dash] Use sanitized_Request
[downloader/http] Use sanitized_Request
[atresplayer] Use sanitized_Request
[bambuser] Use sanitized_Request
[bliptv] Use sanitized_Request
[brightcove] Use sanitized_Request
[cbs] Use sanitized_Request
[ceskatelevize] Use sanitized_Request
[collegerama] Use sanitized_Request
[extractor/common] Use sanitized_Request
[crunchyroll] Use sanitized_Request
[dailymotion] Use sanitized_Request
[dcn] Use sanitized_Request
[dramafever] Use sanitized_Request
[dumpert] Use sanitized_Request
[eitb] Use sanitized_Request
[escapist] Use sanitized_Request
[everyonesmixtape] Use sanitized_Request
[extremetube] Use sanitized_Request
[facebook] Use sanitized_Request
[fc2] Use sanitized_Request
[flickr] Use sanitized_Request
[4tube] Use sanitized_Request
[gdcvault] Use sanitized_Request
[extractor/generic] Use sanitized_Request
[hearthisat] Use sanitized_Request
[hotnewhiphop] Use sanitized_Request
[hypem] Use sanitized_Request
[iprima] Use sanitized_Request
[ivi] Use sanitized_Request
[keezmovies] Use sanitized_Request
[letv] Use sanitized_Request
[lynda] Use sanitized_Request
[metacafe] Use sanitized_Request
[minhateca] Use sanitized_Request
[miomio] Use sanitized_Request
[meovideo] Use sanitized_Request
[mofosex] Use sanitized_Request
[moniker] Use sanitized_Request
[mooshare] Use sanitized_Request
[movieclips] Use sanitized_Request
[mtv] Use sanitized_Request
[myvideo] Use sanitized_Request
[neteasemusic] Use sanitized_Request
[nfb] Use sanitized_Request
[niconico] Use sanitized_Request
[noco] Use sanitized_Request
[nosvideo] Use sanitized_Request
[novamov] Use sanitized_Request
[nowness] Use sanitized_Request
[nuvid] Use sanitized_Request
[played] Use sanitized_Request
[pluralsight] Use sanitized_Request
[pornhub] Use sanitized_Request
[pornotube] Use sanitized_Request
[primesharetv] Use sanitized_Request
[promptfile] Use sanitized_Request
[qqmusic] Use sanitized_Request
[rtve] Use sanitized_Request
[safari] Use sanitized_Request
[sandia] Use sanitized_Request
[shared] Use sanitized_Request
[sharesix] Use sanitized_Request
[sina] Use sanitized_Request
[smotri] Use sanitized_Request
[sohu] Use sanitized_Request
[spankwire] Use sanitized_Request
[sportdeutschland] Use sanitized_Request
[streamcloud] Use sanitized_Request
[streamcz] Use sanitized_Request
[tapely] Use sanitized_Request
[tube8] Use sanitized_Request
[tubitv] Use sanitized_Request
[twitch] Use sanitized_Request
[twitter] Use sanitized_Request
[udemy] Use sanitized_Request
[vbox7] Use sanitized_Request
[veoh] Use sanitized_Request
[vessel] Use sanitized_Request
[vevo] Use sanitized_Request
[viddler] Use sanitized_Request
[videomega] Use sanitized_Request
[viewvster] Use sanitized_Request
[viki] Use sanitized_Request
[vk] Use sanitized_Request
[vodlocker] Use sanitized_Request
[voicerepublic] Use sanitized_Request
[wistia] Use sanitized_Request
[xfileshare] Use sanitized_Request
[xtube] Use sanitized_Request
[xvideos] Use sanitized_Request
[yandexmusic] Use sanitized_Request
[youku] Use sanitized_Request
[youporn] Use sanitized_Request
[youtube] Use sanitized_Request
[patreon] Use sanitized_Request
[extractor/common] Remove unused import
[nfb] PEP 8
9 years ago
Sergey M․
019839faaa
[extractor/common] Use baseURL from f4m manifest for recursive manifest extraction
9 years ago
Sergey M
30eecc6a04
Merge pull request #7296 from jaimeMF/xml_attrib_unicode
...
Use a wrapper around xml.etree.ElementTree.fromstring in python 2.x (…
9 years ago
Sergey M․
dbd82a1d4f
[extractor/common] Fix m3u8 extraction on failure
9 years ago
Sergey M․
dc519b5421
[extractor/common] Make ie_key and IE_NAME return unicode string
9 years ago
Jaime Marquínez Ferrándiz
36e6f62cd0
Use a wrapper around xml.etree.ElementTree.fromstring in python 2.x ( #7178 )
...
Attributes aren't unicode objects, so they couldn't be directly used in info_dict fields (for example '--write-description' doesn't work with bytes).
9 years ago
remitamine
3711304510
[extractor/common] get the redirected m3u8_url in _extract_m3u8_formats
9 years ago
Jaime Marquínez Ferrándiz
865d1fbafc
[extractor/common] Remove unused import
9 years ago
Sergey M․
943a1e24b8
[extractor/common] Use more generic URLError in _is_valid_url
9 years ago
Sergey M․
02835c6bf4
[extractor/common] Document repost_count
9 years ago
Sergey M․
448ef1f31c
[extractor/common] Allow angle brackets in attributes in _og_regexes ( #7215 )
9 years ago
Sergey M․
7a6d76a64d
[extractor/common] Require closing quote in _og_regexes ( Closes #7174 )
...
E.g. do not match `property='og:video:type'` when `og:video` is requested.
9 years ago
Sergey M․
4180a3d8b7
[extractor/common] Allow quoteless content attribute in og regexes ( Closes #7115 )
9 years ago
Yen Chi Hsuan
57935b2564
[extractor/common] Allow HTML5 unquoted attribute values
...
Fixes #7108
HTML5 allows unquoted attribute values. See the "Unquoted attribute value
syntax" section [1] for more information
[1] http://www.w3.org/TR/html5/syntax.html
9 years ago
Sergey M․
4bba371644
[YoutubeDL] Autocalculate ext for subtitles when missing
9 years ago
Sergey M․
e5851b963a
[extractor/common] Make f4m extraction for SMIL non fatal
9 years ago
Sergey M․
4de6131090
[extractor/common] Add fatal to _extract_f4m_formats
9 years ago
Sergey M․
3a1341a7bc
[extractor/common] Make m3u8 extraction for SMIL non fatal
9 years ago
Sergey M․
c78e48177c
[extractor/common] Check validity of direct URLs
9 years ago
Sergey M․
647eab4541
[extractor/common] Extract upload date from SMIL
9 years ago
Sergey M․
1e5bcdec02
[extractor/common] Extract images from SMIL
9 years ago
Sergey M․
e7d8e98a9f
[extractor/common] Allow float bitrates
9 years ago
Sergey M․
8aab976bbd
[extractor/common] Document release_date field
9 years ago
Sergey M․
c430802e32
[extractor/common] Add raise_geo_restricted
9 years ago
Sergey M․
586f1cc532
[extractor/common] Skip html comment tags ( Closes #6822 )
9 years ago
Sergey M․
73eb13dfc7
[extractor/common] Case insensitive inputs extraction
9 years ago
Sergey M․
be0e5dbd83
[extractor/common] Extract submit inputs
9 years ago
Sergey M․
43e7d3c945
[extractor/common] Add raise_login_required
9 years ago
Jaime Marquínez Ferrándiz
8c97f81943
[common] Follow convention of using 'cls' in classmethods
9 years ago
Yen Chi Hsuan
f738dd7b7c
[common] Remove debugging codes
9 years ago
Yen Chi Hsuan
912e0b7e46
[common] Add _merge_subtitles()
9 years ago
Yen Chi Hsuan
03bc7237ad
[common] _parse_smil_subtitles: accept `lang` as the subtitle language
9 years ago
Sergey M․
5cdefc4625
[extractor/common] Add more subtitle mime types for guess when ext is missing
9 years ago
Sergey M․
ce00af8767
[extractor/common] Add default subtitles lang
9 years ago
Yen Chi Hsuan
f877c6ae5a
[theplatform] Use InfoExtractor._parse_smil_formats()
9 years ago