Commit Graph

514 Commits (b271e3352603c523bf2c6762973937fd36925aae)

Author SHA1 Message Date
Philipp Hagemeister d3e5bbf437 Correct --max-downloads with --ignore-errors 11 years ago
Philipp Hagemeister 65697b3bf3 Merge branch 'paged-lists'
Conflicts:
	test/test_utils.py
	youtube_dl/extractor/youtube.py
11 years ago
Philipp Hagemeister 50317b111d Merge branch 'youtube-dash-manifest'
Conflicts:
	youtube_dl/extractor/youtube.py
11 years ago
Philipp Hagemeister de3ef3ed58 Default to -f best-audio when only audio is requested 11 years ago
Philipp Hagemeister ba7678f9cc Add -f bestaudio (Fixes #2163) 11 years ago
Philipp Hagemeister 04b4d394d9 Add new --default-search option (#2193) 11 years ago
Philipp Hagemeister 7b0817e8e1 [servingsys] Add support
This also adds support for brightcove advertisements.
Fixes #2181
11 years ago
Philipp Hagemeister b7ab059084 Add infrastructure for paged lists
This commit allows to download pages in playlists as needed instead of all at once.
Before this commit,
    youtube-dl http://www.youtube.com/user/ANNnewsCH/videos --playlist-end 2 --skip-download
took quite some time - now it's almost instantaneous.
As an example, the youtube:user extractor has been converted.
Fixes #2175
11 years ago
Philipp Hagemeister dd27fd1739 [youtube] Download DASH manifest
If given, download and parse the DASH manifest file, in order to get ultra-HQ formats.
Fixes #2166
11 years ago
Jaime Marquínez Ferrándiz 58c3c7ae38 Don’t try to merge the formats if ffmpeg or avconv are not installed 11 years ago
Jaime Marquínez Ferrándiz 76b1bd672d Add ‘--prefer-avconv’ and ‘--prefer-ffmpeg’ options (#2115)
Affects the ffmpeg post processors, if ‘--prefer-ffmpeg’ is given and both avconv and ffmpeg are installed, it will use ffmpeg. Otherwise it will follow the old behaviour.
11 years ago
Philipp Hagemeister 282962bd36 --list-formats: Only add "@" if vbr is given 11 years ago
Philipp Hagemeister 56327689a2 Move postprocessor into its own package 11 years ago
Philipp Hagemeister a9c5e5ca6e Set required properties for format merging 11 years ago
Philipp Hagemeister a015dce0e2 Merge remote-tracking branch 'jaimeMF/merge-formats' 11 years ago
Philipp Hagemeister 6febd1c1df Prepare widespread unicode literal use 11 years ago
Jaime Marquínez Ferrándiz 6350728be2 Allow merging formats (closes #1612)
Multiple formats can be requested using `-f 137+139`, each one is downloaded and then the two are merged with ffmpeg.
11 years ago
Philipp Hagemeister eadaf08c16 Merge remote-tracking branch 'origin/master' 11 years ago
Jaime Marquínez Ferrándiz 241bce7aaf Merge pull request #2061 from rzhxeo/var
Correct variable name in YoutubeDL.list_formats
11 years ago
Philipp Hagemeister 33ec2ae8d9 Merge remote-tracking branch 'origin/master' 11 years ago
Jaime Marquínez Ferrándiz e9f9a10fba Fix initialization of YoutubeDL with params set to None
Set it to an empty dictionary because it’s directly accessed when setting some properties
11 years ago
rzhxeo 1cdfc31e1f Correct variable name in YoutubeDL 11 years ago
Philipp Hagemeister a0ddb8a2fa Add new --print-traffic option 11 years ago
Philipp Hagemeister 7217e148fb [yahoo] Use centralized sorting, and add tbr field 11 years ago
Philipp Hagemeister c7deaa4c74 [zdf] Use centralized sorting 11 years ago
Philipp Hagemeister 4bcc7bd1f2 Add temporary _sort_formats helper function 11 years ago
Philipp Hagemeister f49d89ee04 Add a resolution field and improve general --list-formats output 11 years ago
Philipp Hagemeister 1538eff6d8 [bliptv] Remove support for direct downloads
This is now handled by the generic IE
11 years ago
Jaime Marquínez Ferrándiz 933605d7e8 YoutubeDL: rename `_fd_progress_hooks` back to `_progress_hooks`
In the future it may report more things.
11 years ago
Jaime Marquínez Ferrándiz b3d9ef88ec YoutubeDL: only set the ‘formats’ field of the info_dict if it was already set before
It caused a circular reference error, when trying to dump it to json (for example with the test video for myvideo.de or any other video without formats)
11 years ago
Philipp Hagemeister 9fc3bef87a Merge remote-tracking branch 'jaimeMF/split-downloaders' 11 years ago
Philipp Hagemeister f82b18efc1 Merge remote-tracking branch 'rzhxeo/youtube' 11 years ago
Philipp Hagemeister 196938835a Remove debugging code
Introduced by accident in 5d681e960d
11 years ago
Philipp Hagemeister 5d681e960d Use bidiv instead of fribidi if available (Fixes #1912) 11 years ago
rzhxeo 62d68c43ed Make prefer_free_formats sorting more robust 11 years ago
rzhxeo bfaae0a768 Filter and sort videos before calling list_formats 11 years ago
rzhxeo 4ea3be0a5c [YoutubeIE] Externalize format selection 11 years ago
Philipp Hagemeister 29eb517403 Add webpage_url_basename info_dict field (Fixes #1938) 11 years ago
alimirjamali 0a9ce268ba Incorrect variable is used to check whether thumbnail exists
Dear @phihag

I believe in line 848, the correct variable to check is 'thumb_filename' rather than 'infofn'

Kindly advise

Mit freundlichen Gruessen
Ali
11 years ago
Philipp Hagemeister a19fd00cc4 Simplify --playlist-start / --playlist-end interface 11 years ago
Philipp Hagemeister 7b6fefc9d4 Apply --no-overwrites for --write-* files as well (Fixes #1980) 11 years ago
Philipp Hagemeister 525ef9227f Add --get-duration (Fixes #859) 11 years ago
Philipp Hagemeister 5fe18bdbde Add --min-views / --max-views (Fixes #1979) 11 years ago
Jaime Marquínez Ferrándiz 3bc2ddccc8 Move FileDownloader to its own module and create a new class for each download process
A suitable downloader can be found using the 'get_suitable_downloader' function.

Each subclass implements 'real_download', for downloading an info dict you call the 'download' method, which first checks if the video has already been downloaded
11 years ago
Jaime Marquínez Ferrándiz 8ab470f1b2 Now a new FileDownloader is created when downloading a video
The progress hooks can be added using the method "add_downloader_progress_hook"
11 years ago
Philipp Hagemeister 357ddadbf5 Fix thumbnail filename determination (Fixes #1945) 11 years ago
Philipp Hagemeister 45598aab08 [YoutubeDL] Simplify filename preparation 11 years ago
Jaime Marquínez Ferrándiz 26e6393134 Set 'NA' as the default value for missing fields in the output template (fixes #1931)
Remove the `except KeyError` clause, it won't get raised anymore
11 years ago
Philipp Hagemeister 1c088fa89d Improve --bidi-workaround support 11 years ago
Philipp Hagemeister 395293a889 [--load-info] Always read file as UTF-8
This allows editing the file (and not escaping non-ASCII characters) and reloading it in.
11 years ago
Philipp Hagemeister db4da14027 Merge remote-tracking branch 'jaimeMF/load-info' 11 years ago
Philipp Hagemeister 2101830c0d Remove unused imports 11 years ago
Philipp Hagemeister a0d96c9843 Add filename to --dump-json output (Fixes #1908) 11 years ago
Philipp Hagemeister 0783b09b92 Add a workaround for terminals without bidi support (Fixes #1912) 11 years ago
Philipp Hagemeister ef4fd84857 [wistia] Add extractor 11 years ago
Philipp Hagemeister 7fc3fa0545 [9gag] Add extractor 11 years ago
Jaime Marquínez Ferrándiz d494389821 Option '--load-info': if the download fails, try extracting the info with the 'webpage_url' field of the info dict
The video url may have expired.
11 years ago
Jaime Marquínez Ferrándiz 1dcc4c0cad Add --load-info option (#972)
It just calls the 'YoutubeDL.process_ie_result' with the dictionary from the json file
11 years ago
Philipp Hagemeister 6ad14cab59 Add --socket-timeout option 11 years ago
Philipp Hagemeister e344693b65 Make socket timeout configurable, and bump default to 10 minutes (#1862) 11 years ago
Philipp Hagemeister a3fb4675fb Do not mutate default arguments
In this case, it looks rather harmless (since the conditions for --restrict-filenames should not change while a process is running), but just to be sure.
This also simplifies the interface for callers, who can just pass in the idiomatic None for "I don't care, whatever is the default".
11 years ago
Jaime Marquínez Ferrándiz 4a98cdbf3b YoutubeDL: set the 'params' property before any message/warning/error is sent (fixes #1840)
If it sets the 'restrictfilenames' param, it will first report a warning. It will try to get the logger from the 'params' property, which would be set at that moment to None, raising the error 'AttributeError: 'NoneType' object has no attribute 'get''
11 years ago
Jaime Marquínez Ferrándiz d31209a144 Use the 'extractor_key' field for the download archive file
It has the same value as the ie_key.
11 years ago
Jaime Marquínez Ferrándiz 529a2e2cc3 Fix typo in the documentation of the 'download_archive' param 11 years ago
Philipp Hagemeister fb04e40396 [soundcloud] Support for listing of audio-only files 11 years ago
Philipp Hagemeister b0b9eaa196 Merge pull request #1829 from jaimeMF/ydl-empty-params
Allow to initialize a YoutubeDL object without parameters
11 years ago
Philipp Hagemeister 0c75c3fa7a Do not warn about fixed output template if --max-downloads is 1
Fixes #1828
11 years ago
Jaime Marquínez Ferrándiz a3927cf7ee Allow to initialize a YoutubeDL object without parameters
Having to pass the 'outtmpl' parameter feels really strange when you just want to extract the info of a video.
11 years ago
Philipp Hagemeister d46cc192d7 Reduce socket timeout 11 years ago
Philipp Hagemeister 5db07df634 Fix --download-archive (Fixes #1826) 11 years ago
Philipp Hagemeister 79d09f47c2 Merge branch 'opener-to-ydl' 11 years ago
Philipp Hagemeister c059bdd432 Remove quality_name field and improve zdf extractor 11 years ago
Philipp Hagemeister 02dbf93f0e [zdf/common] Use API in ZDF extractor.
This also comes with a lot of extra format fields
Fixes #1518
11 years ago
Philipp Hagemeister e03db0a077 Merge branch 'master' into opener-to-ydl 11 years ago
Philipp Hagemeister a1ee09e815 Document proxy 11 years ago
Philipp Hagemeister 8bf9319e9c Simplify logger code(#1811) 11 years ago
Itay Brandes 43afe28588 Log to an external logger (fixes #1810)
Sadly applications using youtube-dl's python sources can't directly
access it's log stream. It's pretty much limited to stdout and stderr
only.

It should log to logging.Logger instance passed to YoutubeDL's params
dictionary.
11 years ago
Philipp Hagemeister 7012b23c94 Match --download-archive during playlist processing (Fixes #1745) 11 years ago
Philipp Hagemeister 00ea0f11eb Print full title in --get-title output (#1806) 11 years ago
Philipp Hagemeister dca0872056 Move the opener to the YoutubeDL object.
This is the first step towards being able to just import youtube_dl and start using it.
Apart from removing global state, this would fix problems like #1805.
11 years ago
Philipp Hagemeister 86bd5f2ca9 Merge remote-tracking branch 'dz0ny/patch-1' 11 years ago
Philipp Hagemeister 8694c60000 import json for --dump-json 11 years ago
Mohamedh Fazal 9d1538182f Add an option to dump json information 11 years ago
Jaime Marquínez Ferrándiz efd6c574a2 Correctly write and restore the console title on the stack (fixes #1782) 11 years ago
Philipp Hagemeister 749febf4d1 Allow --console-title when --quiet is given (Fixes #1783) 11 years ago
Philipp Hagemeister bdde425cbe Save and restore console title (Fixes #1782) 11 years ago
Philipp Hagemeister ce02ed60f2 Remove * imports 11 years ago
Philipp Hagemeister 1e5b9a95fd Move console_title to YoutubeDL 11 years ago
Philipp Hagemeister b5349e8721 Fix indentation of (best) and (worst) in --list-formats 11 years ago
Philipp Hagemeister 7150858d49 [spiegel] Implement format selection 11 years ago
Philipp Hagemeister 91c7271aab Add automatic generation of format note based on bitrate and codecs 11 years ago
Jaime Marquínez Ferrándiz feee2ecfa9 Pass the 'download' argument to 'process_video_result' (fixes #1769) 11 years ago
Janez Troha 9771cceb2c Fix filename extension leaking to json filename
Makes writeinfojson behaving exactly as writethumbnail in case where filename contains mediafile extension.

Case:

video.mp4 converted to music.mp3 would yield music.mp4.info.json instead music.mp3.info.json or music.info.json
11 years ago
Jaime Marquínez Ferrándiz ca715127a2 Don't assume the 'subtitlesformat' is set in the params dict (fixes #1750) 11 years ago
Jaime Marquínez Ferrándiz 897d6cc43a Improve format listing for long format ids
Now arte.tv videos have quite long ids.
11 years ago
Jaime Marquínez Ferrándiz be97abc247 Set the 'extractor_key' field in the info_dict
It's the string returned by the class method 'ie_key', which allows to retrieve the extractor with 'get_info_extractor'
11 years ago
Jaime Marquínez Ferrándiz 9103bbc5cd Add the 'webpage_url' field to info_dict
The url for the video page, it must allow to reproduce the result.
It's automatically set by YoutubeDL if it's missing.
11 years ago
Jaime Marquínez Ferrándiz b6c45014ae Set the extra_info inside YoutubeDL.process_ie_result and set only if the keys are missing 11 years ago
Philipp Hagemeister 7193498811 Use index in formt string (Fixes vevo test on Python 2.6) 11 years ago
Philipp Hagemeister 94badb2599 Fix output indenting for --list-formats 11 years ago
Philipp Hagemeister 57dd9a8f2f Nicer --list-formats output 11 years ago
Jaime Marquínez Ferrándiz 702665c085 tests: build the filename from the info_dict if the 'file' key is missing
It will need to have the 'id' and 'ext' keys to work.
11 years ago
rzhxeo 216d71d001 Check if description and thumbnail are None to prevent crash 11 years ago
Philipp Hagemeister 78a3a9f89e Make "requested format not available" expected (#1655) 11 years ago
Philipp Hagemeister a7685f3bf4 mixcloud does not do any format selection 11 years ago
Philipp Hagemeister 8abeeb9449 Nicer --list-formats output 11 years ago
Philipp Hagemeister c1002e96e9 Let extractors omit ext in formats 11 years ago
rzhxeo 7df286540f [YouPornIE] Extract all encrypted links and remove doubles at the end 11 years ago
Philipp Hagemeister 7853cc5ae1 Merge remote-tracking branch 'origin/master'
Conflicts:
	youtube_dl/YoutubeDL.py
11 years ago
Philipp Hagemeister 586a91b67f Expand tilde in template (Fixes #1639) 11 years ago
Jaime Marquínez Ferrándiz fe7e0c9825 Style fixes in YoutubeDL.py
Fixed some of the problems reported by pep8
11 years ago
Jaime Marquínez Ferrándiz 12893efe01 Respect the download parameter in YoutubeDL.process_video_result if the extractor handle the format selection 11 years ago
Jaime Marquínez Ferrándiz 8c51aa6506 The 'format' field now defaults to '{format_id} - {width}x{height}{format_note}'
Following the YoutubeIE format. The 'format_note' gives additional info about the format, for example '3D' or 'DASH video'.
11 years ago
Jaime Marquínez Ferrándiz 3fd39e37f2 YoutubeDL: remove method that came from FileDownloader 11 years ago
Jaime Marquínez Ferrándiz 49e86983e7 Allow to use the extension for the format selection
The best format with the extension is downloaded.
11 years ago
Jaime Marquínez Ferrándiz a9c58ad945 Accept requested formats to be in the format 35/best (closes #1552)
The format selection code is now an independent function.
11 years ago
Philipp Hagemeister 416a5efce7 fix typos 11 years ago
Philipp Hagemeister f4d96df0f1 Extend #980 with --max-quality support 11 years ago
Philipp Hagemeister 1c1218fefc Merge remote-tracking branch 'jaimeMF/format_selection' 11 years ago
Jai Grimshaw 1fb07d10a3 [youtube] Adds #1312 Download annotations
Adds #1321 Download annotations from youtube
Annotations are downloaded and written to a .annotations.xml file using the https://www.youtube.com/annotations_invideo?features=1&legacy=1&video_id=$VIDEOID API.
Added unit test for annotations.
11 years ago
Jaime Marquínez Ferrándiz 8016c92297 Fix the default values of format_id and format 11 years ago
Jaime Marquínez Ferrándiz e028d0d1e3 Implement the prefer_free_formats in YoutubeDL 11 years ago
Jaime Marquínez Ferrándiz 79819f58f2 Default 'format' field to {width}x{height}
If width is None, use {height}p and if height is None, '???'
11 years ago
Jaime Marquínez Ferrándiz 6ff000b888 Do not handle format selection for IEs that already handle it 11 years ago
Jaime Marquínez Ferrándiz 99e206d508 Implement the max quality option in YoutubeDL 11 years ago
Jaime Marquínez Ferrándiz dd82ffea0c Implement format selection in YoutubeDL
Now the IEs can set a formats field in the info_dict, with the formats ordered from worst to best quality. It's a list of dicts with the following fields:
* Mandatory: url and ext
* Optional: format and format_id

The format_id is used for choosing which formats have to be downloaded.

Now a video result is processed by the method process_video_result.
11 years ago
Tom 1d368c7589 Tiny tpo 11 years ago
Philipp Hagemeister b24f347190 Merge branch 'download-archive'
Conflicts:
	youtube_dl/YoutubeDL.py
	youtube_dl/__init__.py
11 years ago
Philipp Hagemeister ee6c9f95e1 Remove superfluous parenthesis 11 years ago
Philipp Hagemeister cfadd183c4 Call extracted property age_limit everywhere 11 years ago
Philipp Hagemeister 8dbe9899a9 Allow users to specify an age limit (fixes #1545)
With these changes, users can now restrict what videos are downloaded by the intented audience, by specifying their age with --age-limit YEARS .
Add rudimentary support in youtube, pornotube, and youporn.
11 years ago
Philipp Hagemeister c1c9a79c49 Add basic --download-archive option
Often, users want to be able to download only videos they haven't seen before, despite the video files having been deleted or moved in the mean time.
When --download-archive FILE is given, the extractor and ID of every download is recorded in the specified file. If it is already present, the video in question is skipped.
11 years ago
Filippo Valsorda 47192f92d8 implement --no-playlist to only download current video - closes #755 11 years ago
Jaime Marquínez Ferrándiz 0a60edcfa9 Don't fail if the video thumbnail couldn't be downloaded (fixes #1516)
Just report a warning
11 years ago
Philipp Hagemeister c3c88a2664 Allow opts.cachedir == None to disable cache 11 years ago
Philipp Hagemeister c35f9e72ce Move cachedir doc 11 years ago
Philipp Hagemeister 34308b30d6 Warn if no locale is set (#1474) 11 years ago
Jaime Marquínez Ferrándiz c40c6aaaaa Catch socket.error before IOError
Since python 2.6 it's a child class.
11 years ago
Philipp Hagemeister 7459e3a290 Always correct encoding when writing to sys.stderr (Fixes #1435) 11 years ago
Jaime Marquínez Ferrándiz 0b7f31184d Now --all-sub is a modifier to --write-sub and --write-auto-sub (closes #1412)
For keeping backwards compatibility --all-sub sets --write-sub if --write-auto-sub is not given
11 years ago
Jaime Marquínez Ferrándiz 535f59bbcf Merge pull request #1350 from Jaiz909/description-keyerror-fix
Fixed issue #1277 KeyError when no description.
11 years ago
Jai Grimshaw b3f0e53048 Fixed issue #1277 KeyError when no description.
Allows a continue with a warning when an extractor cannot retrieve a description.
11 years ago
Philipp Hagemeister 6d69d03bac Merge remote-tracking branch 'origin/reuse_ies' 11 years ago
Philipp Hagemeister edde6c56ac Print playpath with --get-url (Fixes #1334) 11 years ago
Jaime Marquínez Ferrándiz aa6a10c44a Allow to specify multiple subtitles languages separated by commas (closes #518) 11 years ago
Jaime Marquínez Ferrándiz d4051a8e05 Add a post processor for embedding subtitles in mp4 videos (closes #1052) 11 years ago
Jaime Marquínez Ferrándiz 9e9c164052 Merge pull request #937 from jaimeMF/subtitles_rework
Subtitles rework
11 years ago
Philipp Hagemeister 67fb0c5495 Merge branch 'master' of github.com:rg3/youtube-dl 11 years ago
Philipp Hagemeister 4efba05c56 Clarify template error message (#1209) 11 years ago
user 7edcb8f39c More informative error 11 years ago
Jaime Marquínez Ferrándiz bbcbf4d459 Switch some calls to to_stderr to report_error and report_warning 11 years ago
Jaime Marquínez Ferrándiz 6804038d06 Don't try to write the subtitles if it's None 11 years ago
Jaime Marquínez Ferrándiz 5d51a883c2 Use a dictionary for storing the subtitles
The errors while getting the subtitles are reported as warnings, if no subtitles are found return and empty dict.
11 years ago
Jaime Marquínez Ferrándiz c4a91be726 Save subtitles using the same code for all the options 11 years ago
Jaime Marquínez Ferrándiz d8269e1dfb Don't try to save the thumbnail if it's None
It means the extractor couldn't find it
12 years ago
Jaime Marquínez Ferrándiz cbdbb76665 Use determine_ext when saving the thumbnail
Urls that contain a query produced filenames with wrong extensions
12 years ago
Jaime Marquínez Ferrándiz a431154706 Set the playlist_index and playlist fields for already resolved video results. 12 years ago
Jaime Marquínez Ferrándiz 56c7366547 YoutubeIE: reuse instances of InfoExtractors (closes #998)
When a IE is added to the list, it's also added to a dictionary. When a IE is requested it first looks in the dictionary and if there's no instance it will create a new one.

That way _real_initialize is only called once for each IE, saving time if it needs to login for example.
12 years ago
Jaime Marquínez Ferrándiz 023fa8c440 Add function add_default_info_extractors to YoutubeDL
It adds to the list the ies returned by ge_extractors
12 years ago
Ismael Mejia b98a6b2f72 Fixed typo in subtitle format option (from: sbt => sbv) 12 years ago
Ismael Mejia 65cceef8f4 Added support for additional vtt subtitle format (WebVTT) in youtube-dl. 12 years ago
Jaime Marquínez Ferrándiz b004821fa9 Add the option "--write-auto-sub" to download automatic subtitles from Youtube
Now automatic subtitles are only downloaded if the option is given.
(closes #903)
12 years ago
Jaime Marquínez Ferrándiz c6c1974672 Add "--video-password" option (related #889)
Used only for accessing a private video

Restore the error when the account is missing
12 years ago
Jaime Marquínez Ferrándiz 8222d8de88 Split FileDownloader in two classes: FileDownloader and YoutubeDL
YoutubeDL is the class that coordinates everything
FileDownloader gets a filename and an info dict and downloads the video.
12 years ago