Questions tagged [boto]

boto is an open-source Python Interface to Amazon Web Services

boto
Filter by
Sorted by
Tagged with
71 votes
5 answers
89k views

How to peek at messages in the queue

I don't want the message to count as "read" but I'd like to know what's in the queue. The documentation: http://boto.s3.amazonaws.com/ref/sqs.html#module-boto.sqs Isn't very straight forward about ...
0 votes
0 answers
103 views

ERROR - Runtime.ImportModuleError: Unable to import module 'main': cannot import name 'DEPRECATED_SERVICE_NAMES' from 'botocore.docs'

I'm supporting a lambda project (python 3.8) and when I run the command "sam local invoke LambdaFunction --event events/get_users.json", it gives the following error `Invoking main....
0 votes
0 answers
44 views

retrieveAndGenerate Syntax Error: Unknown parameter generationConfiguration or retrievalConfiguration (Claude-v3, Amazon Bedrock)

I am trying to retrieve and generate response from knowledge base use claude-v3 model. To do so I followed the boto3 documentation and Blog Post on Amazon and wrote this method: def ...
-1 votes
0 answers
41 views

Botocore generates incorrect S3 URLs, leading to a 403 Access Denied

I'm using Boto3(v1.34) client to list objects from an S3 bucket, but I've encountered some odd behavior. Note- 403 is not a real issue but due to different root cause which i have explained below. ...
56 votes
14 answers
161k views

Move files between two AWS S3 buckets using boto3

I have to move files between one bucket to another with Python Boto API. (I need it to "Cut" the file from the first Bucket and "Paste" it in the second one). What is the best way to do that? ** Note:...
-1 votes
0 answers
36 views

AWS Bedrock Stream responses model

I am trying to call the invokeModelWithResponseStream API endpoint ( not using any boto3 library or sdk). As per the docs https://docs.aws.amazon.com/bedrock/latest/APIReference/...
2 votes
2 answers
2k views

How to upload the folder to digital ocean spaces?

From the link, https://www.digitalocean.com/community/questions/how-to-upload-an-object-to-digital-ocean-spaces-using-python-boto3-library. It only states to upload files to the spaces. I want to ...
30 votes
8 answers
54k views

Boto3 error: The AWS Access Key Id you provided does not exist in our records

I am currently trying to get access to Amazon S3 inside a virtual machine and download files like so: s3 = boto3.resource('s3', aws_access_key_id="xxxxxxxxxxx", ...
0 votes
1 answer
10k views

How to check give directory or folder exist in given s3 bucket and if exist how to delete the folder from s3?

I want to check whether folder or directory exist in give s3 bucket, if exist i want delete folder from s3 bucket using python code. example for : s3:/bucket124/test Here "bucket124" is bucket and "...
147 votes
9 answers
134k views

Amazon S3 boto - how to delete folder?

I created a folder in s3 named "test" and I pushed "test_1.jpg", "test_2.jpg" into "test". How can I use boto to delete folder "test"?
3 votes
2 answers
15k views

How to get last modified date of latest file from S3 with Boto Python?

This is structure of my s3 bucket Bucket 1 Company A File A-02/01/20 File A-01/01/20 File B-02/01/20 File B-01/01/20 Company B File A-02/01/20 File ...
1 vote
1 answer
1k views

get aws cli 'multipart_chunksize' value using boto

I am developing an application in python and I need to calculate the number of S3 Puts, for this I require the total size of file to be divided with multipart_chunksize to find total number of chunks. ...
29 votes
8 answers
87k views

Boto [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed while connecting to S3

I am trying to connect to S3 using boto, but it seems to fail. I've tried some workarounds, but they don't seem to work. Can anyone please help me with this. Below is the code. import boto if not ...
9 votes
2 answers
11k views

How to validate Amazon access key and secret key is correct?

I wrote a function to validate the AWS keys by just creating the ec2 connection object import boto.ec2 try: ec2Conn = boto.ec2.connect_to_region(region, aws_access_key_id=access_key, ...
1 vote
0 answers
32 views

botocore >= 1.28.0 slower in multithread application

The official Boto3 docs recommends creating a new resource per thread: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/resources.html#multithreading-or-multiprocessing-with-resources ...
0 votes
1 answer
663 views

Python : XML file downloaded from S3 full of string escaping characters

I have a number of XML files that I have added to S3 (localstack sever). I can view these files through Cyberduck and they are valid xml files. However, when I download the objects, the XML data is ...
0 votes
1 answer
65 views

How to set timeout and retry strategy when I enrypt/decrypt data using python aws_encryption_sdk?

I'm using aws_encryption_sdk to encrypt and decrypt data. Once we hit an issue and our process stuck for ~24 hours. One of the hypotheses was that there was a problem with string value decryption, ...
0 votes
0 answers
67 views

How to save logs, to S3Boto3Storage, with Django Logging

I need to save my logs to a file on dedicated storage (using S3BotoStorage). I haven't found any working solution yet. Does anyone know how to make it work? I need to incorporate something similar to ...
53 votes
13 answers
130k views

Getting S3 objects' last modified datetimes with boto

I'm writing a Python script that uploads files to S3 using boto librairy. I only want to upload changed files (which I can check by their "last modified" datetimes), but I can't find the ...
1 vote
1 answer
50 views

Asyncio with boto and for range?

I'm trying first time the asyncio botocore implementation. However, I'm quite sure I'm not getting the expected asynchronicity, likely due to my own lack of experience with it. :) The goal of the ...
64 votes
11 answers
168k views

AWS: Boto3: AssumeRole example which includes role usage

I'm trying to use the AssumeRole in such a way that i'm traversing multiple accounts and retrieving assets for those accounts. I've made it to this point: import boto3 stsclient = boto3.client('sts') ...
12 votes
11 answers
15k views

Amazon EC2 AutoScaling CPUUtilization Alarm- INSUFFICIENT DATA

So I've been using Boto in Python to try and configure autoscaling based on CPUUtilization, more or less exactly as specified in this example: http://boto.readthedocs.org/en/latest/autoscale_tut.html ...
16 votes
8 answers
17k views

Python/Boto - Writing to AWS CloudWatch Logs without sequence token

I am trying to send logs to AWS CloudWatch Logs using Python and Boto framework. I am doing this: res=logs.put_log_events("FOO", "BAR", [{'timestamp':int(round(time.time() * 1000)), '...
0 votes
0 answers
17 views

AWS::SNS::Subscription not created by moto by CloudFormation.create_stack

I have following CloufFormation template AWSTemplateFormatVersion: '2010-09-09' Parameters: ServiceName: Default: test Type: String Resources: DynamoDBTable: Properties: ...
1 vote
2 answers
282 views

get-log-events returns only a few results of many

I am logging to CloudWatch from a python lambda. In the AWS console I can see several hundred log entries, but if I try to use get-log-events from either a Python Boto call, or through the AWS CLI I ...
79 votes
8 answers
78k views

Can you upload to S3 using a stream rather than a local file?

I need to create a CSV and upload it to an S3 bucket. Since I'm creating the file on the fly, it would be better if I could write it directly to S3 bucket as it is being created rather than writing ...
35 votes
6 answers
49k views

How to use the AWS Python SDK while connecting via SSO credentials

I am attempting to create a python script to connect to and interact with my AWS account. I was reading up on it here https://boto3.amazonaws.com/v1/documentation/api/latest/guide/quickstart.html and ...
232 votes
19 answers
465k views

Boto3 Error: botocore.exceptions.NoCredentialsError: Unable to locate credentials

When I simply run the following code, I always gets this error. s3 = boto3.resource('s3') bucket_name = "python-sdk-sample-%s" % uuid.uuid4() print("Creating new bucket with name:",...
0 votes
2 answers
40 views

Lookup Key from S3 Bucket using Boto

I've taken a script written by Paul Davies about reingesting Splunk Logs from the AWS Cloud. The When my logs have failed to process in Kinesis Firehose they get placed in a backup S3 bucket. The ...
1 vote
2 answers
10k views

ImportError: No module named zlib

I am getting this error while running a python script to send file on S3 File "docs/setup/s3transfer.py", line 121, in <module> main(*args, **kwargs) File "docs/setup/s3transfer.py", line ...
100 votes
12 answers
116k views

The role defined for the function cannot be assumed by Lambda

I'm getting the error "The role defined for the function cannot be assumed by Lambda" when I'm trying to create a lambda function with create-function command. aws lambda create-function --region ...
0 votes
0 answers
32 views

Sending data from LAMBDA FUNCTION to AWS IOT TWIN MAKER

So I wanted to send data from Lambda to aws iot twin maker however I have getting some errors. Here, is my code: import json import boto3 iot_twin_maker_client = boto3.client('iottwinmaker') def ...
14 votes
4 answers
16k views

SES error missing final '@domain'

I'm using python lib boto for sending emails via SES. And when I tried to send to more than 30 (but less than 50, so limit is not exceeded) recipients, I got an error: <Error> <Type>...
27 votes
9 answers
50k views

[Django][AWS S3] botocore.exceptions.clienterror an error occurred (accessdenied) when calling the PutObject operation

I am trying to connect Django project to AWS S3. settings.py contains below: AWS_ACCESS_KEY_ID = #ID AWS_SECRET_ACCESS_KEY = #Key AWS_STORAGE_BUCKET_NAME = #Bucket AWS_S3_CUSTOM_DOMAIN = '%s.s3....
7 votes
3 answers
9k views

Why might boto be denied access to S3 with proper IAM keys?

I am trying to access a bucket on S3 with boto. I have been given read access to the bucket and my keys are working when I explore it in S3 Browser. The following code is returning 403 Forbidden ...
0 votes
2 answers
401 views

Django Storages S3 gets all images in every query

I Setup my project to use amazon S3 using django-storages with the S3Boto backend. Somo of my models have ImageFields that are uploaded to S3 and it's working fine. The problem comes when I try to do ...
0 votes
1 answer
1k views

ANSIBLE: boto - "TypeError: a bytes-like object is required, not 'str'"

Using ansible=2.10.0 and boto=2.49.0, When trying to execute ec2_vol playbook for manage disk to AWS instance with Proxy environment. Getting this error - connection.py",line 796, in proxy_ssl, ...
0 votes
1 answer
691 views

Error on s3 file upload with python

Trying to upload an image to a bucket on s3 amazon. The image is uploaded via form and the image is stored in 'file': file = request.files['file'] I also try to rename it as 'filename' which holds ...
6 votes
5 answers
30k views

s3 - how to get fast line count of file? wc -l is too slow

Does anyone have a quick way of getting the line count of a file hosted in S3? Preferably using the CLI, s3api but I am open to python/boto as well. Note: solution must run non-interactively, ie in an ...
73 votes
7 answers
105k views

How do I get the file / key size in boto S3?

There must be an easy way to get the file size (key size) without pulling over a whole file. I can see it in the Properties of the AWS S3 browser. And I think I can get it off the "Content-length" ...
1 vote
2 answers
197 views

AWS DynamoDB Delete Item

I'm working on deleting DynamoDB item through Lambda. I have a simple table setup and Partition key is Date (Number) and I don't have sort key. I tested below command in AWS CLI (via CMD) and it is ...
3 votes
2 answers
7k views

How to calculate Dynamodb item size? Getting validationerror 400KB Boto

ValidationException: ValidationException: 400 Bad Request {u'message': u'Item size has exceeded the maximum allowed size', u'__type': u'com.amazon.coral.validate#ValidationException'} The item ...
0 votes
0 answers
256 views

S3FS fails when given aiobotocore session to initialize the File System

I am trying to use a combination of aiobotocore, botocore and s3fs to build an S3 client which refreshes its credentials automatically. This is my code so far: Code to create a refreshable session. It ...
39 votes
6 answers
88k views

Retrieve all items from DynamoDB using query?

I am trying to retrieve all items in a dynamodb table using a query. Below is my code: import boto.dynamodb2 from boto.dynamodb2.table import Table from time import sleep c = boto.dynamodb2....
15 votes
3 answers
19k views

boto3 how to create object with metadata?

In the example below I want to set a timestamp metadata attribute when created an S3 object. How do I do that? The documentation is not clear. import uuuid import json import boto3 import botocore ...
5 votes
3 answers
6k views

Python Boto EC2 find instance given its IP address

Using boto in Python, how can I find the boto.ec2 instance object given an IP address?
67 votes
12 answers
170k views

DynamoDB : The provided key element does not match the schema

Is there a way to get an item depending on a field that is not the hashkey? Example My Table Users: id (HashKey), name, email And I want to retrieve the user having email as '[email protected]' How ...
16 votes
4 answers
26k views

Can you get the AWS account name from boto?

I have an AWS key and secret key and would like to call boto to get the account name. I can get account ID, but the AWS account name is a mystery.
1 vote
1 answer
125 views

API for creating behaviour in cloudfront

I have created a CloudFront Distribution through the console , Am trying to create a origin and behaviour in that distribution using boto3 I cant find any api to create origin and behaviour in ...
0 votes
2 answers
83 views

boto equivalent of aws client command

I have this command that works as expected. But I will like to use boto instead. aws cloudwatch get-metric-statistics \ --region ap-south-1 \ --namespace AWS/RDS \ --metric-name DBLoad \ ...

1
2 3 4 5
48