Commit Graph

673 Commits (master)

Author SHA1 Message Date
Sergey M․ 042b664933
Revert "[utils] Add support for cookies with spaces used instead of tabs"
According to [1] TABs must be used as separators between fields.
Files produces by some tools with spaces as separators are considered
malformed.

1. https://curl.haxx.se/docs/http-cookies.html

This reverts commit cff99c91d1.
4 years ago
Sergey M․ cff99c91d1
[utils] Add support for cookies with spaces used instead of tabs 4 years ago
Sergey M․ fca6dba8b8
[YoutubeDL] Force redirect URL to unicode on python 2 4 years ago
Sergey M․ 42db58ec73
[utils] Improve str_to_int 4 years ago
Remita Amine 348c6bf1c1 [utils] handle int values passed to str_to_int 5 years ago
Sergey M․ 1ced222120
[utils] Add generic caesar cipher and rot47 5 years ago
InfernalUnderling 9d30c2132a [utils] Handle rd-suffixed day parts in unified_strdate (#23199) 5 years ago
Sergey M․ 53896ca5be
[utils] Actualize major IPv4 address blocks per country 5 years ago
Sergey M․ 824fa51165
[utils] Improve subtitles_filename (closes #22753) 5 years ago
Sergey M․ f7a147e3b6
[utils] Introduce random_user_agent and use as default User-Agent (closes #21546) 5 years ago
Sergey M․ 28cc2241e4
[utils] Restrict parse_codecs and add theora as known vcodec (#21381) 5 years ago
Sergey M․ 53cd37bac5
[utils] Improve strip_or_none 5 years ago
Sergey M․ 3089bc748c
Fix W504 and disable W503 (closes #20863) 5 years ago
Jakub Wilk fd35d8cdfd [utils] Transliterate "þ" as "th" (#20897)
Despite visual similarity "þ" is unrelated to "p".
It is normally transliterated as "th":

    $ echo þ-Þ | iconv -t ASCII//TRANSLIT
    th-TH
5 years ago
Sergey M․ 5e1271c56d
[utils] Improve int_or_none and float_or_none (#20403) 5 years ago
Sergey M․ 0dc41787af
[utils] Introduce parse_bitrate 5 years ago
Sergey M․ 067aa17edf
Start moving to ytdl-org 5 years ago
remitamine e7e62441cd [utils] strip #HttpOnly_ prefix from cookies files (#20219) 5 years ago
Ales Jirasek 22f5f5c6fc
[malltv] Add extractor (closes #18058) 5 years ago
Sergey M․ fad4ceb534
[utils] Fix urljoin for paths with non-http(s) schemes 5 years ago
Sergey M․ e9a50fba86
[utils] Fix typo 5 years ago
Sergey M․ b7acc83550
[utils] Add language codes replaced in 1989 revision of ISO 639 to ISO639Utils (closes #18765) 5 years ago
Sergey M․ 1bab343704
[YoutubeDL] Introduce YoutubeDLCookieJar and clarify the rationale behind session cookies (closes #12929) 6 years ago
Alexander Seiler aa374bc78e [utils] Fix random_birthday to generate existing dates only 6 years ago
Sergey M․ 25d110be30
[utils] Properly recognize AV1 codec (closes #17506) 6 years ago
Sergey M․ 9e21e6d96b
[utils] Improve remote address skipping and add support for python 2.6 (closes #17362) 6 years ago
Andrew Udvare 8959018a5f
[utils] Skip remote IP addresses non matching to source address' IP version (closes #13422) 6 years ago
Sergey M․ 60c0856223
[utils] Use pure browser header for User-Agent (closes #17236) 6 years ago
Huyuumi 38e87f6c2a [utils] Remove return from __init__ 6 years ago
Sergey M․ af03000ad5
[utils] Introduce url_or_none 6 years ago
Sergey M․ e9c671d5e8
[utils] Allow JSONP with empty func name (closes #17028) 6 years ago
Sergey M․ 0685d9727b
[utils] Share JSON-LD regex 6 years ago
Enes 85750f8972 [openload] Improve ext extraction 6 years ago
Remita Amine 261f47306c [utils] fix style id extraction for namespaced id attribute(closes #16551) 6 years ago
Remita Amine 5a16c9d9d3 [utils] keep the original TV_PARENTAL_GUIDELINES dict 6 years ago
Remita Amine b836118724 [utils] Relax TV Parental Guidelines matching 6 years ago
Sergey M․ 5f95927a62
Improve geo bypass mechanism
* Introduce geo bypass context
* Add ability to bypass based on IP blocks in CIDR notation
* Introduce --geo-bypass-ip-block
6 years ago
Sergey M․ 6cc622327f
[utils] Introduce merge_dicts 6 years ago
Sergey M․ 1cc47c6674
[utils] Fix match_str for boolean meta fields 6 years ago
Philipp Hagemeister f226880c6d [tennistv] Add support for tennistv.com 6 years ago
Sergey M․ b871d7e954
[utils] Add parse_resolution 6 years ago
Sergey M․ befa4708fd
[utils] Fixup some common URL's typos in sanitize_url (closes #15649) 6 years ago
Remita Amine b12cf31bb1 [cbc] add new extractor for olympics.cbc.ca(closes #15535) 6 years ago
Sergey M․ 65220c3bd6
Add support for IronPython 6 years ago
Mike Fährmann c384d537f8 [util] Improve scientific notation handling in js_to_json (closes #14789) 6 years ago
Philipp Hagemeister d7cd9a9e84
[utils] Fix youtube-dl under PyPy3 on Windows 6 years ago
Sergey M․ b555ae9bf1
[utils] Add another date format pattern (#14999) 7 years ago
Remita Amine d4f05d4731 [utils] add sami mimetype to mimetype2ext 7 years ago
Remita Amine ffe6979ef9 [utils] add hvc1 codec code to parse_codecs 7 years ago
Sergey M․ 056653bbb1
[utils] Add support for zero years and months in parse_duration 7 years ago
Sergey M․ 6be08ce602
[utils] Use in OnDemandPagedList by default
Not using cache results in redundant network I/O due to downloading the same pages while using --playlist-items n-m
7 years ago
Sergey M․ 4c54b89e03
Hide experimental phantomjs wrapper 7 years ago
Yen Chi Hsuan 4ed2d7b7d1 Fix flake8 issues after #14225 7 years ago
Yen Chi Hsuan 3869028ffb [utils] Use bytes-like objects in dfxp2srt
This fixes handling of non-UTF8 TTML subtitles

Closes #14191
7 years ago
Yen Chi Hsuan a88d461dff Merge pull request #14225 from Tithen-Firion/openload-phantomjs-method
Openload phantomjs method
7 years ago
Sergey M․ c7e327c4d4
[utils] Introduce bool_or_none 7 years ago
Yen Chi Hsuan 95f3f7c20a
[utils] Fix unescapeHTML for misformed string like "&a"" (#13935) 7 years ago
Sergey M․ 5b232f46dc
[utils] Skip missing params in cli_bool_option (closes #13865) 7 years ago
Tithen-Firion feee8d32e4 [phantomjs] add exe version to debug info 7 years ago
Yen Chi Hsuan 609ff8ca19 [utils] Support attributes with no values in get_elements_by_attribute() 7 years ago
Sergey M․ aefce8e6dc
[utils] Use compat_shlex_quote in shell_quote 7 years ago
Sergey M․ b4a3d461e4
[utils] Handle HTMLParseError in extract_attributes (closes #13349) 7 years ago
Sergey M․ 2ae2ffda5e
[utils] Improve unified_timestamp 7 years ago
Yen Chi Hsuan b407d8533d
[utils] Drop an compatibility wrapper for Python < 2.6
addinfourl.getcode is added since Python 2.6a1. As youtube-dl now
requires 2.6+, this is no longer necessary.

See 9b0d46db11
7 years ago
Yen Chi Hsuan 5552c9eb0f
[utils] Recognize more patterns in strip_jsonp()
Used in Youku Show pages
7 years ago
Sergey M․ 60f5c9fb19
[utils] Recognize more audio codecs (#13081) 7 years ago
Yen Chi Hsuan 10c87c151b
[utils] Rename try_multipart_encode to _multipart_encode_impl
To state that this is an internal function and people should be careful
when using it outside youtube-dl.
7 years ago
Yen Chi Hsuan b2ad479d17
[utils] Fix multipart_encode for Python < 3.5 7 years ago
Tithen-Firion c89267d31a Merge branch 'master' into openload-phantomjs-method 7 years ago
Yen Chi Hsuan 0c26548601
[cda] Implement birthday verification (closes #12789) 7 years ago
Sergey M․ deef31955b
[utils] Improve unified_timestamp
Seen at http://zaq1.pl/video/xev0e
7 years ago
Remita Amine 98ce1a3fd3 [utils] add video/mp2t to mimetype2ext 7 years ago
Tithen-Firion 98f9d87381 [phantomjs] Add required version checking 7 years ago
Tithen-Firion edd9221cd2 [utils] Fix inconsistent output of clean_html
`\s` in Python 2.x doesn't match unicode whitespace characters by
default
7 years ago
Tithen-Firion 40e41780f1 [phantomjs] add cookie support 7 years ago
Tithen-Firion da57ebaf84 [openload] separate PhantomJS code from extractor 7 years ago
Remita Amine 5b995f713b [utils] add support for ttml styles 7 years ago
Sergey M․ a32a9a7ef5
[extractor/common] Add support multiple getters in try_get 7 years ago
Sergey M․ 51098426b8
[utils] Introduce expand_path 7 years ago
Sergey M․ 4b5de77bdb
[utils] Process bytestrings in urljoin (closes #12369) 7 years ago
Sergey M․ 158af5242e
[utils] Carry long doc string 7 years ago
Yen Chi Hsuan 0a5445ddbe [utils] Add bytes_to_long() and long_to_bytes()
Used in daisuki.net (#4738)

Both are adapted from public domain PyCrypto:
https://github.com/dlitz/pycrypto/blob/master/lib/Crypto/Util/number.py
7 years ago
Yen Chi Hsuan f48409c7ac [utils] Add pkcs1pad
Used in daisuki.net (#4738)
7 years ago
Sergey M․ 4248dad92b Improve geo bypass mechanism
* Rename options to preffixly match with --geo-verification-proxy
* Introduce _GEO_COUNTRIES for extractors
* Implement faking IP right away for sites with known geo restriction
7 years ago
Sergey M․ 18a0defab0 [utils] Make random_ipv4 return unicode string 7 years ago
Sergey M․ 773f291dcb Add experimental geo restriction bypass mechanism
Based on faking X-Forwarded-For HTTP header
7 years ago
Pierre Mdawar bf5b9d859a [utils] Introduce YoutubeDLError base class for all youtube-dl exceptions 7 years ago
Sergey M․ db13c16ef8
[utils] Add support for quoted string literals in --match-filter (closes #8050, closes #12142, closes #12144) 7 years ago
Ricardo Constantino 2f49bcd690
utils.py: Workaround TypeError with Python 2.7.13 in Windows
Fixes #11540

Tested with Windows Python 2.7.12 and 2.7.13.
7 years ago
Thomas Christlieb 2af12ad9d2 Introduce get_elements_by_class and get_elements_by_attribute utility functions 7 years ago
Sergey M․ 4195096ea8
[utils] Improve comments processing in js_to_json (closes #11947) 7 years ago
Michal Čihař b3ee552e4b
[utils] Handle single-line comments in js_to_json 7 years ago
Sergey M․ 15846398ca
[utils] Improve parse_duration 7 years ago
Yen Chi Hsuan 0c1c6f4b9f
[utils] Add another date format seen in NextTV 7 years ago
Sergey M․ cb655f34fb
[utils] Add more date formats 7 years ago
Yen Chi Hsuan ec85ded83c
Fix "invalid escape sequences" error on Python 3.6 7 years ago
Sergey M․ b0c65c677f
[utils] Improve urljoin 8 years ago
Sergey M․ e34c33614d
[utils] Add convenience urljoin 8 years ago
Sergey M․ fb37eb25d9
[utils] Add common user agents map 8 years ago
Sergey M․ 02dc0a36b7
[utils] Introduce base_url 8 years ago