Thomas Christlieb
2af12ad9d2
Introduce get_elements_by_class and get_elements_by_attribute utility functions
8 years ago
Sergey M․
4195096ea8
[utils] Improve comments processing in js_to_json ( closes #11947 )
8 years ago
Michal Čihař
b3ee552e4b
[utils] Handle single-line comments in js_to_json
8 years ago
Sergey M․
15846398ca
[utils] Improve parse_duration
8 years ago
Sergey M․
cb655f34fb
[utils] Add more date formats
8 years ago
Remita Amine
7fe1592073
[common] fix dash codec information for mixed videos and fragment url construction( #11490 )
8 years ago
Sergey M․
b0c65c677f
[utils] Improve urljoin
8 years ago
Sergey M․
e34c33614d
[utils] Add convenience urljoin
8 years ago
Yen Chi Hsuan
582be35847
Update coding style after pycodestyle 2.1.0
...
In pycodestyle 2.1.0, E305 was introduced, which requires two blank
lines after top level declarations, too.
See https://github.com/PyCQA/pycodestyle/issues/400
See also #10689 ; thanks @stepshal for first mentioning this issue and
initial patches
8 years ago
Sergey M․
02dc0a36b7
[utils] Introduce base_url
8 years ago
Sergey M․
c6eed6b8c0
[utils] Lower priority for rare date formats and add tests
8 years ago
Sergey M․
3e4185c396
[utils] Use native french month names
8 years ago
Sergey M․
f6717dec8a
[utils] Improve month_by_name and add tests
8 years ago
Sergey M․
6562d34a8c
[utils] Improve mimetype2ext
8 years ago
Yen Chi Hsuan
70852b47ca
[utils] Recognize units with full names in parse_filename
...
Reference: https://en.wikipedia.org/wiki/Template:Quantities_of_bytes
8 years ago
Yen Chi Hsuan
e4659b4547
[utils] Correct octal/hexadecimal number detection in js_to_json
8 years ago
Sergey M․
13585d7682
[utils] Recognize lowercase units in parse_filesize
8 years ago
Remita Amine
5f2c2b7936
[test_utils] add test for option with not str value
8 years ago
Sergey M․
a8795327ca
[utils] Add support TV Parental Guidelines ratings in parse_age_limit
8 years ago
Yen Chi Hsuan
7dc2a74e0a
[utils] Fix unified_timestamp for formats parsed by parsedate_tz()
8 years ago
Yen Chi Hsuan
0b68de3cc1
Merge pull request #8876 from remitamine/html5_media
...
[extractor/common] add helper method to extract html5 media entries
9 years ago
Yen Chi Hsuan
84c237fb8a
[utils] Add get_element_by_class
...
For #9950
9 years ago
Remita Amine
dfaa86b75e
[test_utils] add test for smuggling a smuggled url
9 years ago
remitamine
4f3c5e0627
[utils] add helper function for parsing codecs
9 years ago
Yen Chi Hsuan
1143535d76
[utils] Add urshift()
...
Used in IqiyiIE and LeIE
9 years ago
Sergey M․
46f59e89ea
[utils] Add unified_timestamp
9 years ago
Yen Chi Hsuan
47212f7bcb
[utils] Don't transform numbers not starting with a zero
...
Fix test_Viidea and maybe others
9 years ago
Yen Chi Hsuan
55b2f099c0
[utils] Decode HTML5 entities
...
Used in test_Vporn_1. Also related to #9270
9 years ago
bzc6p
b96f007eeb
Added sanitization support for Hungarian letters Ő and Ű
9 years ago
Sergey M․
46bc9b7d7c
[utils] Allow None in remove_{start,end}
9 years ago
Sergey M․
364cf465dd
[test_utils] PEP 8
9 years ago
Sergey M․
89ac4a19e6
[utils] Process non-base 10 integers in js_to_json
9 years ago
felix
bd1e484448
[utils] js_to_json: various improvements
...
now JS object literals like { /* " */ 0: ",]\xaa<\/p>", } will be correctly converted to JSON.
9 years ago
Yen Chi Hsuan
778a1ccca7
[utils] Add Œ and œ found in French to ACCENT_CHARS
...
Fixes #9463
9 years ago
Yen Chi Hsuan
dab0daeeb0
[utils,compat] Move struct_pack and struct_unpack to compat.py
9 years ago
Adam Thalhammer
31c4448f6e
Instead of replacing accented characters with an underscore when sanitizing file names in restricted mode, replace them with their non-accented equivalents fixes #9347
9 years ago
Adam Thalhammer
79a2e94e79
Instead of replacing accented characters with an underscore when sanitizing file names in restricted mode, replace them with their non-accented equivalents fixes #9347
9 years ago
Sergey M
b6c0d4f431
Merge pull request #9110 from remitamine/parse_duration
...
[utils] imporove parse_duration to handle more formats
9 years ago
remitamine
acaff49575
[utils] imporove parse_duration to handle more formats
9 years ago
Jaime Marquínez Ferrándiz
eb9c3edd5e
[test/utils] Add test for date_from_str
9 years ago
Yen Chi Hsuan
81f36eba88
[test/test_utils] Update for escape_url change (again)
9 years ago
Yen Chi Hsuan
2d60465e44
[test/test_utils] Update for escape_url change
9 years ago
Jaime Marquínez Ferrándiz
782b1b5bd1
[utils] lookup_unit_table: Match word boundary instead of end of string
9 years ago
Sergey M․
c5229f3926
[utils] PEP 8
9 years ago
remitamine
83548824c2
Merge pull request #8092 from bpfoley/twitter-thumbnail
...
[utils] Add extract_attributes for extracting html tag attributes
9 years ago
Sergey M․
fb47597b09
[bbc] Generalize unit table lookup and add parse_count
9 years ago
remitamine
3201a67f61
[test/test_utils] add more tests for update_url_query
9 years ago
remitamine
fb640d0a3d
[test/test_utils] add tests for update_url_query
9 years ago
Brian Foley
8bb56eeeea
[utils] Add extract_attributes for extracting html tag attributes
...
This is much more robust than just using regexps, and handles all
the common scenarios, such as empty/no values, repeated attributes,
entity decoding, mixed case names, and the different possible value
quoting schemes.
9 years ago
Yen Chi Hsuan
5eb6bdced4
[utils] Multiple changes to base_n()
...
1. Renamed to encode_base_n()
2. Allow tables longer than 62 characters
3. Raise ValueError instead of AssertionError for invalid input data
4. Return the first character in the table instead of '0' for number 0
5. Add tests
9 years ago
Sergey M․
f160785c5c
[utils] Remove AM/PM from unified_strdate patterns
9 years ago
Yen Chi Hsuan
5bc880b988
[utils] Add OHDave's RSA encryption function
9 years ago
Sergey M․
8411229bd5
[utils] Allow dot in strip_jsonp
9 years ago
Sergey M․
86296ad2cd
[utils] Add ability to control skipping false values in dict_get
9 years ago
Sergey M․
cbecc9b903
[utils] Add dict_get convenience method
9 years ago
Sergey M․
6b77d52b1f
[test_utils] Add tests for encode_compat_str
9 years ago
Yen Chi Hsuan
db2fe38b55
[utils] Support alternative timestamp format in TTML
...
Fixes #7608
9 years ago
Yen Chi Hsuan
d631d5f9f2
[utils] Fix TTML conversion
...
Tolerate invalid timestamps (closes #7909 )
9 years ago
Sergey M․
31b2051e21
[utils] Add remove_quotes
9 years ago
Sergey M․
9cb9a5df77
[utils] Check ext with trailing slash against the list of known extensions
9 years ago
Sergey M․
5035536e3f
[test_utils] Add tests for determine_ext
9 years ago
Sergey M․
7aefc49c40
[utils] Skip invalid/non HTML entities ( Closes #7518 )
9 years ago
Jaime Marquínez Ferrándiz
6a75040278
[utils] unified_strdate: Return None if the date format can't be recognized ( fixes #7340 )
...
This issue was introduced with ae12bc3ebb
, it returned 'None'.
9 years ago
Sergey M
30eecc6a04
Merge pull request #7296 from jaimeMF/xml_attrib_unicode
...
Use a wrapper around xml.etree.ElementTree.fromstring in python 2.x (…
9 years ago
Sergey M․
578c074575
[utils] Support list of xpath in xpath_element
9 years ago
Sergey M․
52c3a6e49d
[utils] Improve parse_iso8601
9 years ago
Jaime Marquínez Ferrándiz
36e6f62cd0
Use a wrapper around xml.etree.ElementTree.fromstring in python 2.x ( #7178 )
...
Attributes aren't unicode objects, so they couldn't be directly used in info_dict fields (for example '--write-description' doesn't work with bytes).
9 years ago
Sergey M․
d01949dc89
[utils:js_to_json] Fix bad escape in double quoted strings
9 years ago
Sergey M․
f71264490c
[test_utils] Add tests for cli option converters
9 years ago
Sergey M․
87f70ab39d
[test_utils] Add more tests for xpath
9 years ago
Sergey M․
ee114368ad
[utils] Make value optional for find_xpath_attr
...
This allows selecting particular attributes by name but without specifying the value and similar to xpath syntax `[@attrib]`
9 years ago
Yen Chi Hsuan
9c29bc69f7
[utils] Improve parse_duration
...
Now dots are parsed. For example '87 Min.'
9 years ago
Yen Chi Hsuan
1b0427e6c4
[utils] Support TTML without default namespace
...
In a strict sense such TTML is invalid, but Yahoo uses it.
10 years ago
Yen Chi Hsuan
7dff03636a
[utils] Support 'dur' field in TTML
10 years ago
Yen Chi Hsuan
d39e0f05db
[utils] Remove sanitize_url_path_consecutive_slashes()
...
This function is used only in SohuIE, which is updated to use a new
extraction logic.
10 years ago
Yen Chi Hsuan
0fe2ff78e6
[NBC] Enhance embedURL extraction ( closes #2549 )
10 years ago
Sergey M․
b3ed15b760
[utils] Add replace_extension
10 years ago
Sergey M․
a4bcaad773
[test_utils] Add tests for prepend_extension
10 years ago
Yen Chi Hsuan
bf6427d2fb
[ffmpeg] Add dfxp (TTML) subtitles support ( #3432 , #5146 )
10 years ago
Yen Chi Hsuan
0a1603634b
[utils] Remove url_infer_protocol
10 years ago
Yen Chi Hsuan
418c5cc3fc
[udn] Add new extractor
10 years ago
Sergey M․
8cf70de428
[test_utils] Add test for unified_strdate
10 years ago
Sergey M․
ba9e68f402
[utils] Drop trailing comma before closing brace
10 years ago
Naglis Jonaitis
91757b0f37
[utils] Escape all HTML entities written in hexadecimal form
10 years ago
Jaime Marquínez Ferrándiz
5379a2d40d
[test/utils] Test xpath_text
10 years ago
Sergey M․
92a4793b3c
[utils] Place sanitize url function near other sanitizing functions
10 years ago
Sergey M․
dc03a42537
Merge branch 'sohu_fix' of https://github.com/yan12125/youtube-dl into yan12125-sohu_fix
10 years ago
Sergey M․
2ebfeacabc
[utils] Keep dot and dotdot unmodified ( Closes #5171 )
10 years ago
Sergey M․
f18ef2d144
[utils] Disallow trailing dot in sanitize_path for a path part
10 years ago
Sergey M․
a2aaf4dbc6
[utils] Add sanitize_path
10 years ago
Yen Chi Hsuan
55969016e9
[utils] Add a function to sanitize consecutive slashes in URLs
10 years ago
Philipp Hagemeister
a7440261c5
[utils] Streap leading dots
...
Fixes #2865 , closes #5087
10 years ago
Philipp Hagemeister
3e675fabe0
[airmozilla] Be more tolerant when nonessential items are missing ( #5030 )
10 years ago
Philipp Hagemeister
5a42414b9c
[utils] Prevent hyphen at beginning of filename ( Fixes #5035 )
10 years ago
Philipp Hagemeister
d305dd73a3
[utils] Fix js_to_json
...
Previously, the runtime could be atrocious for longer inputs.
10 years ago
Philipp Hagemeister
347de4931c
[YoutubeDL] Add generic video filtering ( Fixes #4916 )
...
This functionality is intended to eventually encompass the current format filtering.
10 years ago
Philipp Hagemeister
9bb8e0a3f9
[wsj] Add new extractor ( Fixes #4854 )
10 years ago
Philipp Hagemeister
8f4b58d70e
[ntvde] Add new extractor ( Fixes #4850 )
10 years ago
Philipp Hagemeister
cfb56d1af3
Add --list-thumbnails
10 years ago
Philipp Hagemeister
61ca9a80b3
[generic] Add support for BOMs ( Fixes #4753 )
10 years ago
Naglis Jonaitis
a69801e2c6
[utils] Add additional format to unified_strdate
10 years ago
Sergey M․
a5fb718c50
[test_utils] Add more tests for parse_duration
10 years ago
Philipp Hagemeister
2aeb06d6dc
[utils] Improve colon handling ( Fixes #4683 )
10 years ago
Philipp Hagemeister
0590062925
Respect age_limit when listing extractors ( Fixes #4653 )
10 years ago
Philipp Hagemeister
cae97f6521
Improve and test ffmpeg version detection
10 years ago
Philipp Hagemeister
42bdd9d051
[cinchcast] Add new extractor ( Fixes #4428 )
10 years ago
Philipp Hagemeister
47d7c64274
[test_utils] Make test more realistically ( #4377 )
10 years ago
Philipp Hagemeister
5f9b83944d
[ffmpeg] Improve version check and call it from hls ( Fixes #4377 )
10 years ago
Philipp Hagemeister
e8df5cee12
[minhateca] Fix duration parsing
10 years ago
Philipp Hagemeister
4349c07dd7
[minhateca] Add extractor ( Fixes #4094 )
10 years ago
Philipp Hagemeister
e075a44afb
[tests] Remove useless u prefixes
10 years ago
Philipp Hagemeister
be64b5b098
[xminus] Simplify and extend ( #4302 )
10 years ago
Jouke Waleson
8bcc875676
PEP8: more applied
10 years ago
Jouke Waleson
5f6a1245ff
PEP8 applied
10 years ago
Philipp Hagemeister
7d4111ed14
Provide guidance when called with a YouTube ID starting with a dash.
...
Reported at https://news.ycombinator.com/item?id=8648121
10 years ago
Philipp Hagemeister
6a68bb574a
[eporner] Fix duration ( Closes #4188 )
10 years ago
Philipp Hagemeister
609a61e3e6
[npo] Improve npo.nl ( Fixes #4173 )
10 years ago
Philipp Hagemeister
4c0924bb24
[utils] Fix intlist_to_bytes in Python 2 ( #4181 )
10 years ago
Philipp Hagemeister
e4bdb37ec6
[spiegel] Add support for embeds
10 years ago
Philipp Hagemeister
12ea2f30cf
[utils] Remove unused get_meta_content function
10 years ago
Philipp Hagemeister
8c25f81bee
[util] Move compatibility functions out of util
...
utils is large enough without these compatibility functions.
Everything that is present in newer versions of Python (i.e. with dev Python it's just an import) goes into compat.py .
Everything else (i.e. youtube-dl-specific helpers) goes into utils.py .
10 years ago
Philipp Hagemeister
7d11297f3f
Merge branch 'master' of github.com:rg3/youtube-dl
10 years ago
Philipp Hagemeister
6ad4013d40
[drtv] Allow fractional timestamps ( Fixes #4059 )
10 years ago
Jaime Marquínez Ferrándiz
8abec2c8bb
[test_utils] Fix compat_getenv and compat_expanduser tests on python 3.x
10 years ago
Sergey M․
f889cea109
Merge branch 'compat-getenv-and-expanduser' of https://github.com/dstftw/youtube-dl into dstftw-compat-getenv-and-expanduser
...
Conflicts:
test/test_utils.py
youtube_dl/__init__.py
10 years ago
Philipp Hagemeister
e7b6d12254
[utils] Improve and test js_to_json
10 years ago
Philipp Hagemeister
410f3e73ab
[utils] Fix js_to_json
10 years ago
Sergey M․
4644ac5527
[core] Decode environment variables with filesystem encoding ( Fixes #3854 , Fixes #3217 , Fixes #2918 )
...
Introduces compat versions of os.getenv and os.path.expanduser
10 years ago
Philipp Hagemeister
99b67fecc5
[arte] Fix upload date extraction
10 years ago
Philipp Hagemeister
9c44d2429b
[vimeo:likes] Support large like lists ( Fixes #3847 )
10 years ago
Philipp Hagemeister
0003a5c416
Merge remote-tracking branch 'dstftw/escape-non-ascii-in-urls'
...
Conflicts:
test/test_utils.py
10 years ago
Philipp Hagemeister
a020a0dc20
[facebook] Fix support for untitled videos ( Fixes #3757 )
10 years ago
Sergey M․
d05cfe0600
[YoutubeDL/utils] Clarify rationale for URL escaping in comment, move escape routines to utils and add some tests
10 years ago
Sergey M․
ca7b3246b6
[utils] Improve parse_duration
10 years ago
Philipp Hagemeister
4e408e479b
[utils] Modernize tests
10 years ago
Philipp Hagemeister
7adcbe7594
[rtlnl] Extract duration
10 years ago
Jaime Marquínez Ferrándiz
173a7026d5
[test/test_utils] Fix typo in method name
10 years ago
Philipp Hagemeister
676eb3f2dd
Fix unicode_escape ( Fixes #2695 )
11 years ago
Philipp Hagemeister
fac55558ad
[washingtonpost] Add extractor ( Fixes #2622 )
11 years ago
Philipp Hagemeister
912b38b428
[instagram] Fix info_dict key name
11 years ago
Jaime Marquínez Ferrándiz
b74fa8cd2c
[facebook] Fix login process
...
It was broken and didn't work in python 3.
And use `_download_webpage` instead of `compat_urllib_request.urlopen`.
11 years ago
Philipp Hagemeister
62e609ab77
Ignore BOM in batch files ( Fixes #2450 )
11 years ago
Sergey M.
2db806b4aa
Improve parse_duration
11 years ago
Philipp Hagemeister
b53466e168
Fix f4m downloading on Python 2.6
11 years ago
Philipp Hagemeister
026fcc0495
Fix #2355 (date parsing with dashes)
11 years ago
Philipp Hagemeister
65697b3bf3
Merge branch 'paged-lists'
...
Conflicts:
test/test_utils.py
youtube_dl/extractor/youtube.py
11 years ago
Philipp Hagemeister
5aafe895fc
Correct XML ampersand fixup
11 years ago
Philipp Hagemeister
b7ab059084
Add infrastructure for paged lists
...
This commit allows to download pages in playlists as needed instead of all at once.
Before this commit,
youtube-dl http://www.youtube.com/user/ANNnewsCH/videos --playlist-end 2 --skip-download
took quite some time - now it's almost instantaneous.
As an example, the youtube:user extractor has been converted.
Fixes #2175
11 years ago
Philipp Hagemeister
608d11f515
[cnn] Add multiple formats, duration, and upload_date
11 years ago
Philipp Hagemeister
d6c7a367e8
[utils] Fix url_basename
11 years ago