Commit Graph

68 Commits (9da4c426798657a3c266dcd778c3e3c327f7fbdb)

Author SHA1 Message Date
Javier M. Mellid a88d6d9a53 Add s3_url requirement in doc when rgw support is enabled in s3.py
Signed-off-by: Javier M. Mellid <jmunhoz@igalia.com>
9 years ago
Javier M. Mellid a52ff221f7 Add proper version_added for rgw option in s3.py
Signed-off-by: Javier M. Mellid <jmunhoz@igalia.com>
9 years ago
Javier M. Mellid b1724344d5 Add Ceph RGW S3 compatibility
Ceph Object Gateway (Ceph RGW) is an object storage interface built on top of
librados to provide applications with a RESTful gateway to Ceph Storage
Clusters:

http://docs.ceph.com/docs/master/radosgw/

This patch adds the required bits to use the RGW S3 RESTful API properly.

Signed-off-by: Javier M. Mellid <jmunhoz@igalia.com>
9 years ago
codemeup @ Work 7314cc3867 Fixing error exception handling for python3. Does not need to be compa… (#3840)
* Fixing error exception handling for python. Does not need to be compatible with Python2.4 b/c boto is Python 2.6 and above.

* Fixing error exception handling for python. Does not need to be compatible with Python2.4 b/c boto is Python 2.6 and above.

* Fixing compile time errors IRT error exception handling for Python 3.5.
This does not need to be compatible with Python2.4 b/c Boto is Python 2.6 and above.
9 years ago
Doug Luce ec0f2113e0 Fix S3 unavailable region error
This is to address this error:

  fatal: [site]: FAILED! => {"changed": false, "failed": true, "msg": "Failed to connect to S3: Region  does not seem to be available for awsmodule boto.s3. If the region definitely exists, you may need to upgrade boto or extend with endpoints_path"}

Commit 0dd58e9 changed the logic so an exception is thrown (by
`connect_to_aws`) before the `s3 is None` check is performed. This
changes the `None` check to a catch so the old logic can compensate.
9 years ago
Brian Coca 3381e8f76e clarified docs on overwrite 9 years ago
Ted Timmons a87171e280 fix typo in documentation
let's -> lets
9 years ago
Brian Coca 6a48f2207a corrected typo on aws connection arguments var 9 years ago
Toshio Kuratomi 1c4da434c7 Merge pull request #2421 from willthames/ec2_fix_vpc_security_token
Use connect_to_aws where possible
9 years ago
Matt Martz c7845456a6 Various simple linting type cleanups on modules 9 years ago
Brian Coca fbaf4fb628 Merge pull request #2746 from ferrants/patch-2
updated s3 module documentation
9 years ago
Brian Coca 1a298f7948 updated s3 docs to reflect mode choices
fixes #2318
9 years ago
Matt Ferrante 2c75e72e83 updated s3 module documentation 9 years ago
Will Thames 0dd58e9326 Use connect_to_aws where possible
`connect_to_aws` fixes a bug with security tokens in AWS.
Modules should use that rather than calling
`boto.x.connect_to_region`
9 years ago
Greg DeKoenigsberg 344cf5fc0e Remove @ralph-tice from maintainership per his request 9 years ago
Val Komarov 986d4064e9 Remove duplicate logic 9 years ago
Frank van Tol a7667fcaf2 Update s3.py
We are copying from S3,  the bucket is the source, not the target.
9 years ago
Taneli Lepp 15f4b59619 Added option to set multiple ACLs for S3 objects. Also verifies the
selected permissions against Boto's canned ACL strings list.
9 years ago
Benno Joy b5db274362 Merge pull request #1940 from jcowley/s3-permissions
Add support for S3 canned permissions. resolves #1939
9 years ago
Till Backhaus 39e7e05a8d Delete dead and broken code 9 years ago
Jason Cowley 8215aad3b3 Add support for S3 canned permissions. resolves #1939 9 years ago
James Cammarata 79f21c6ec9 Fix missing params to download_s3file in s3 9 years ago
James Cammarata 4c4e978360 Fixing s3 failures when bucket names contain dots 9 years ago
Brian Coca 92f32a0253 added version added to headers in s3 9 years ago
Brian Coca 4b78b5597f Merge pull request #160 from acaire/s3-headers
S3 headers
9 years ago
Feanil Patel 22461e335a If overwrite is set, download eagerly.
If the we are going to overwrite the file anyway,
there is no need to do any checksums locally or
grab the remote etag.
9 years ago
khassen 5f2b365faa Use the common/shared MD5 function. 9 years ago
ayush 3f7313cc43 Updated doc strings so each character isn't considered a line 9 years ago
Apoorva Kulkarni c0e4c50eeb s3 module: Add missing version tag to "encrypt" parameter 9 years ago
Ash Caire 3ccf2e8a86 add headers param to s3 uploads 10 years ago
Toshio Kuratomi 406402bb26 Merge pull request #332 from proby/add-s3-list
Add support for listing keys in a specific S3 bucket
10 years ago
Patrick Roby e90d02c35c iterate through all keys in a more pythonic manner 10 years ago
Patrick Roby 8ba96aaf4b update documentation, adding new params 10 years ago
Toshio Kuratomi d824a7a9f5 Merge pull request #255 from SDFE/s3_connect_params
Use aws connect calls that allow boto profile use
10 years ago
Patrick Roby 00322c43fc Add support for listing keys in a specific S3 bucket
Includes support for specifying a prefix, marker, and/or max_keys. Returns a list of key names (as strings).
10 years ago
Greg DeKoenigsberg 28a869a030 Updating cloud modules with proper github author information 10 years ago
Tyler Cross de617bb3ae Get specific object version with S3 module.
- allow specifying version for mode=get and mode=getstr
- when version specified doesn't exist give an error message that indicates so
10 years ago
Brian Coca acfa1b4c39 Merge pull request #1464 from qrtt1/devel
Prevent s3 module getting the memory-error from uploading or downloading a large file
10 years ago
Bruce Pennypacker 51bc602f42 Added delobj command to delete an object within a bucket 10 years ago
Ching Yi, Chan 1517ae8ab2 Refactoring for easier to read 10 years ago
Ching Yi, Chan ed5ea7e921 Prevent memory-error from a large file 10 years ago
tedder b138411671 feature pull request: catch and retry recoverable errors
boto can throw SSLError when timeouts occur (among other SSL errors). Catch these so proper JSON can be returned, and also add the ability to retry the operation.

There's an open issue in boto for this: https://github.com/boto/boto/issues/2409

Here's a sample stacktrace that inspired me to work on this. I'm on 1.7, but there's no meaningful differences in the 1.8 release that would affect this. I've added line breaks to the trace for readability.

    failed to parse: Traceback (most recent call last):
      File "/home/ubuntu/.ansible/tmp/ansible-tmp-1419895753.17-160808281985012/s3", line 2031, in <module> main()
      File "/home/ubuntu/.ansible/tmp/ansible-tmp-1419895753.17-160808281985012/s3", line 353, in main download_s3file(module, s3, bucket, obj, dest)
      File "/home/ubuntu/.ansible/tmp/ansible-tmp-1419895753.17-160808281985012/s3", line 234, in download_s3file key.get_contents_to_filename(dest)
      File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 1665, in get_contents_to_filename response_headers=response_headers)
      File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 1603, in get_contents_to_file response_headers=response_headers)
      File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 1435, in get_file query_args=None)
      File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 1488, in _get_file_internal for bytes in self:
      File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 368, in next data = self.resp.read(self.BufferSize)
      File "/usr/local/lib/python2.7/dist-packages/boto/connection.py", line 416, in read return httplib.HTTPResponse.read(self, amt)
      File "/usr/lib/python2.7/httplib.py", line 567, in read s = self.fp.read(amt)
      File "/usr/lib/python2.7/socket.py", line 380, in read data = self._sock.recv(left)
      File "/usr/lib/python2.7/ssl.py", line 341, in recv return self.read(buflen)
      File "/usr/lib/python2.7/ssl.py", line 260, in read return self._sslobj.read(len) ssl.SSLError: The read operation timed out
10 years ago
Benno Joy 4e206e3790 Fixes issue 1197 for s3 module, where the file gets downloaded even if checksums match 10 years ago
billwanjohi 40eef6c3ec s3 put: add support for server-side encryption
- added 'encrypt' boolean option
- reordered module options alphanumerically
10 years ago
Toshio Kuratomi 5f9592248a Minor fixups found during review of #582 10 years ago
Toshio Kuratomi dcc04f9505 Merge branch 's3_retry' of https://github.com/stansonhealth/ansible-modules-core into stansonhealth-s3_retry
Conflicts:
	cloud/amazon/s3.py
10 years ago
Benno Joy ed62de6d3c Fixes issue 1197 for s3 module, where the file gets downloaded even if checksums match 10 years ago
Brian Coca 0e23dd6c55 avoid exception when boto is not available 10 years ago
Dan Abel c5f5dfd8a0 use aws connect calls that allow boto profile use 10 years ago
steenzout d960dc1584 issue #994: use HAS_BOTO to determine if import was successful:
- removed import of sys module.
- HAS_BOTO constant to check if import was successful.
- trigger a failure when import fails.
- removed unnecessary imports.
10 years ago