....

• When setting contents, the file pointer now starts reading content where you positioned it and returns the file pointer up to where you finished reading. This is useful for multipart uploads of large files where it is desirable to split them into different parts.
• Computing md5 for a file pointer now returns the file pointer to the initial position instead of at the start of the file. It also takes a size of bytes to read from the file pointer which enables it to work well with multipart uploads.
• If the provider supports chunking, delay the md5 calculation to when the file is being sent on the wire and send the Content-MD5 in the trailer. [THIS IS UNTESTED as I don't have a google storage account and AWS doesn't support chunking for uploads.]
• Improve the callback so that it doesn't call too many times and also doesn't call twice or more for the final chunk.
• Fix callback for get_file such that the Key size is available when figuring out cb_count.
• get_file() now calculates the md5 as the stream is being read. It is useful to have key.md5 available after reading contents in case you want to verify what you read without re-calculating the md5 of a paricularly large piece of data. key.md5 is set.
• after send_file() key.size will be accurate for the data that you sent.
• Added unit tests for most of the work.

I've been searching all night for a solution to this and nothing has worked so far. I have Python 2.7.8 installed, and boto is also installed on OS X 10.10.3. The problem is that unless I'm running as my root user, when I try to run the AWS ec2.py script while deploying with Ansible, I get- ERROR: Inventory script (ec2.py) had an execution error: Traceback (most recent call last): File "/Users/peterjohnjoseph/Sites/massvapors.com/ansible/ec2.py", line 121, in <module> import boto ImportError: No module named boto

Any idea what I might want to try? I'll give whatever information you need.

'ExpiredToken' errors are occasionally thrown when IAM role's temporary credentials are used.

--- kvs.py

--- Note that 'connection' and 'bucket' objects are created once and reused for put requests.

connection = connect_s3() bucket = connection.get_bucket(bucket_name, False)

def put_raw(bucket, key, value, headers=None): k = Key(bucket) k.key = key

File "/home/myapp/kvs.py", line 160, in put_raw retval = k.set_contents_from_string(value, headers=headers) File "/usr/local/lib/python2.7/dist-packages/boto-2.5.2-py2.7.egg/boto/s3/key.py", line 1129, in set_contents_from_string encrypt_key=encrypt_key) File "/usr/local/lib/python2.7/dist-packages/boto-2.5.2-py2.7.egg/boto/s3/key.py", line 993, in set_contents_from_file size=size) File "/usr/local/lib/python2.7/dist-packages/boto-2.5.2-py2.7.egg/boto/s3/key.py", line 724, in send_file query_args=query_args) File "/usr/local/lib/python2.7/dist-packages/boto-2.5.2-py2.7.egg/boto/s3/connection.py", line 470, in make_request override_num_retries=override_num_retries) File "/usr/local/lib/python2.7/dist-packages/boto-2.5.2-py2.7.egg/boto/connection.py", line 913, in make_request return self._mexe(http_request, sender, override_num_retries) File "/usr/local/lib/python2.7/dist-packages/boto-2.5.2-py2.7.egg/boto/connection.py", line 790, in _mexe request.body, request.headers) File "/usr/local/lib/python2.7/dist-packages/boto-2.5.2-py2.7.egg/boto/s3/key.py", line 681, in sender response.status, response.reason, body) boto.exception.S3ResponseError: S3ResponseError: 400 Bad Request

ExpiredTokenThe provided token has expired.A0E4730BE76CC9F0VdBvtwjLvBhr5QN1VZ2v/LJeBTxso52+q59kmfspLUdbWWnrz714qhXAMMgnvZYV ...

As mentioned on the boto email list, I'd like to see slick53's more intuitive methods available in the boto package. Aside from minor changes to make the code more idiomatic, I've basically left the code alone, though I may revisit it pending review of this request.

There are some basic unit tests.

If accepted, I'll submit some changes to the docs.

When using assume_role_with_web_identity, the doc says : "AssumeRoleWithWebIdentity is an API call that does not require the use of AWS security credentials"

However, calls to STSConnection() fail with

2013-06-14 09:30:08,699 boto [ERROR]:Unable to read instance data, giving up
Traceback (most recent call last):
  File "test.py", line 77, in <module>
    credentials = doGetAccessCredentials(token['id_token'], profile)
  File "test.py", line 17, in doGetAccessCredentials
    conn = boto.sts.STSConnection()
  File "/Library/Python/2.7/site-packages/boto/sts/connection.py", line 83, in __init__
    validate_certs=validate_certs)
  File "/Library/Python/2.7/site-packages/boto/connection.py", line 965, in __init__
    validate_certs=validate_certs)
  File "/Library/Python/2.7/site-packages/boto/connection.py", line 552, in __init__
    host, config, self.provider, self._required_auth_capability())
  File "/Library/Python/2.7/site-packages/boto/auth.py", line 683, in get_auth_handler
    'Check your credentials' % (len(names), str(names)))

This is because STSConnection uses 'sign-v2' auth handler. How can we specify STSConnection to use anon handler instead ?

Hi, using HEAD, and having checked Issue 526: VPC 2.0: The parameter groupName cannot be used with the parameter subnet.

I still get the same error:

conn = boto.connect_vpc(aws_access_key_id= AWS_ACCESS_KEY_ID,aws_secret_access_key=AWS_SECRET_ACCESS_KEY)

res = conn.run_instances(AMI_ID, ... instance_type = 'm1.xlarge', ... key_name = 'AWSEC2key', ... min_count = 1, max_count = 1, ... security_groups=['lm'], ... user_data = None, ... subnet_id = 'subnet_f03c1f99', ... private_ip_address='10.0.0.189', ... security_group_ids=['sg-2c342940'], ... )

Traceback (most recent call last): File "", line 9, in File "/Library/Python/2.6/site-packages/boto-2.0-py2.6.egg/boto/ec2/connection.py", line 625, in run_instances return self.get_object('RunInstances', params, Reservation, verb='POST') File "/Library/Python/2.6/site-packages/boto-2.0-py2.6.egg/boto/connection.py", line 871, in get_object raise self.ResponseError(response.status, response.reason, body) boto.exception.EC2ResponseError: EC2ResponseError: 400 Bad Request

InvalidParameterCombinationThe parameter groupName cannot be used with the parameter subnet0a9bac66-e484-4d87-b4bf-62e845016c7e

Used mk-ca-bundle.pl locally to generate a recent cacerts.txt file that fixes https certificate validation for OSX 10.8x. These certs were built with:

$ perl mk-ca-bundle.pl -w 64 boto/cacerts/cacerts.txt

fixes boto/boto#1535 closes jtriley/StarCluster#267

This adds support from network acls. I was not able to find a method AssociateNetworkAcl on this page: http://docs.amazonwebservices.com/AWSEC2/latest/APIReference/query-apis.html

Hi,

Yesterday (Jan-29), Amazon has added support to add or delete global indexes at any time, which is by far the feature I was most looking forward using for over a year.

I've changed dynamodb2.table.update method taking into account the backward campatibility as well. Documentation amendments and a unit test case has been provided too accordingly to the contributing guidelines.

Looking forward to see this in the new version of boto.

http://www.appneta.com/blog/s3-list-get-bucket-default/

When get_bucket is called, by default it also does a full LIST because validate defaults to True. Thus, anyone using this library is accidentally paying way more for S3 than they should. This default should be False.

A bunch of changes to make boto.ec2 tests pass on Python 3.2 and Python 3.3 while all boto tests continue to pass on Python 2. There has been some manual testing as well to verify that basic functionality works (e.g.: printing all available images) when connecting to Eucalyptus Community Cloud (see here) and EC2 (see here).

Passing Travis CI build: https://travis-ci.org/msabramo/boto/builds/3465072

This is the work that I alluded to in #1127.

{"mobile":{"S":"+9198765432100"},"taxRegistrationNumber": {"S":"TRN2255"},"phone":{"S":"+919876543211"},"companyRegistrationNumber":{"S":"REG0000152"},"email":{"S":"[email protected]"}} i have to insert this data into table by map datatype ..im using putitem ..but its not working

Please see the AWS official guide for allocate-address; there is allowed to assign a tag for the IP address (--tag-specifications).

  allocate-address
[--domain <value>]
[--address <value>]
[--public-ipv4-pool <value>]
[--network-border-group <value>]
[--customer-owned-ipv4-pool <value>]
[--dry-run | --no-dry-run]
[--tag-specifications <value>]
[--cli-input-json <value>]
[--generate-cli-skeleton <value>]

https://github.com/boto/boto/blob/91ba037e54ef521c379263b0ac769c66182527d7/boto/ec2/connection.py#L1830-L1834

However, I hit the following errors when I passed the TagSpecification to the client.

We observed in our production clusters that if we are doing sync between two different accounts/buckets String literals transforming into Unicode, after successful migration.

As a result we are seeing failures when we are trying to read the value after sync.

A complete list of supported websites endpoints are listed in https://docs.aws.amazon.com/general/latest/gr/s3.html#s3_website_region_endpoints these changes add all the missing endpoints.

Note that as of today Feb 2021 cn-north-1 region isn't listed, but looks like is still in use.

This command breaks with my typical ~/.aws/config

aws --profile default --region us-east-1 ec2 describe-snapshots

'str' object has no attribute 'get'

With values in my ~/.aws/config like it fails.

[default]
region = us-east-1
cli_pager =
s3 =
signature_version = s3v4

Note: These blank entries are supported and recommended in other places (or injected by other tools). Not supporting it for some CLI calls feels like a bug.

With simpler values in my ~/.aws/config like it will work without issue (probably any set that doesn't have empty values).

[default]
region = us-east-1

Environment:

aws-cli/2.4.17 Python/3.9.10 Darwin/20.5.0 source/x86_64 prompt/off

I haven't done an exhaustive check for this, but I've ran about 50 other commands since updating my aws-cli version and this is the first one that choked on it.

Thank you for looking at this issue.