Questions tagged [boto3]
Boto 3 - The Amazon Web Services (AWS) SDK for Python
6,644
questions
0
votes
0
answers
36
views
How to iterate a dict/list/dict Python
I'm trying to obtain a particular SNS Topic ARN using the Boto3 SDK:
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sns.html#SNS.Client.list_topics
What I want to be able ...
0
votes
0
answers
4
views
How to iterate from nested dictionary for RDS with boto3
I am working with boto3 and lambda.
My aim is to take RDS snapshot backup and email will send.
working scenario,
snapshot created and email is sent.
I want to sent snapshot id in email as well.
below ...
0
votes
1
answer
23
views
How to mock __init__ method
I want to mock below code, can you suggest any way to achieve this?
def __init__(self):
self.client = boto3.resource(DYNAMODB)
self.table = self.client.Table(TABLE_NAME)
Below is ...
0
votes
0
answers
17
views
boto3 searialisation outputs strange format
I am writing a Lambda Python function on AWS. It retrieves a DynamoDB item and I want to return that back to the caller in a JSON format. If I don't serialise the item, there are errors with Python's ...
0
votes
1
answer
17
views
Fetch all the records from Dynamo DB table using Query (python boto3)
I have a table in Dynamo Db with 2 million records. The table has date (yyyy-mm-dd) as partition key and account I'd as sort key. Currently, we are using boto3.client.dynamodb.query to fetch the data ...
0
votes
1
answer
24
views
Do I need to have AWS CLI to use Boto3?
I have a Django application that needs to use Boto3 to create and manage EC2 instances. When I host the Django application, do I need to install AWS CLI in the server to use Boto3 in the Django ...
-1
votes
1
answer
29
views
Boto3 does not use specified region
I have the following script to list trails from CloudTrail:
import boto3
import os
os.environ['AWS_DEFAULT_REGION'] = 'us-east-2'
current_session = boto3.session.Session(profile_name='production')
...
3
votes
0
answers
36
views
Objects in S3 bucket are not replicated in target bucket
i have enabled replication for an s3 bucket using boto3 ,needed to enable KMS as well but when i try to test this functionality by uploading files manually in the source bucket, they don't get ...
0
votes
0
answers
18
views
Dynamically build the updateexpression and ExpressionAttribute Value for Dynamodb update_item
I have a table storing user details/settings for an application and have an Lambda called by API Gateway to manage users.
When I want to update a user's settings I am sending a PUT request from API to ...
-1
votes
1
answer
20
views
Modifying Termination protection of Cloud-formation stack based on tags using boto3
I am quite new at working with Boto3. My requirements are to create a boto3 script that will modify the termination protection of Cloud Formation Stack based on tags: For Example, if the tag Value is &...
0
votes
1
answer
13
views
Deleting Snapshots managed by Backup using boto3
I wanted to delete the old snapshots from my aws account i used the following code but i am getting an error can you please suggest any code changes
this is the code
import json
import boto3
import ...
0
votes
2
answers
15
views
AWS S3 - Get folder count inside a particular folder using python's boto3
I am using Python's boto3 library.
In AWS S3, I have a folder. I want to know the number of subfolders present under that folder.
The structure can be like this:
a/x
a/y
a/z
So, I want to know the ...
0
votes
0
answers
8
views
Python - Boto3 STS Token refreshing too early using RefreshableCredentials
I have an S3 upload tool that uses a combination of a bearer token provided through OIDC authentication for our service, as well as an STS token generated using that bearer token to perform uploads to ...
-2
votes
0
answers
23
views
Identifying Accessible File/Folder in Domains using Python
I need some help with creating a script that pulls our domains from AWS route53 and PING/ curls the domain in form of domain.com/file/foldername.
.gitignore
.git
.index.php
.env
and if the file exis ...
0
votes
2
answers
42
views
AttributeError: partially initialized module 'boto3' has no attribute 'session' (most likely due to a circular import)
I tried the below code
import boto3
aws_mg_con=boto3.session.Session(profile_name='demo_user')
iam_con=aws_mg_con.resource('iam')
But i receive this error
'AttributeError: partially initialized ...
0
votes
1
answer
21
views
Cloudwatch Billing Alarm Boto3
I'm trying to create a CloudWatch billing alarm example pic:
Example of end-goal
it all seems fine, it's really strange I even get a HTTPStatusCode 200 when I run it but it does nothing, I've even ...
0
votes
0
answers
10
views
An error occurred (InternalFailure) when calling the CreateDataSet operation in quicksight api with python boto3
When I run the below code in AWS Lambda func to create quicksight data set, I am getting this error: An error occurred (InternalFailure) when calling the CreateDataSet operation
I have this datasource ...
0
votes
0
answers
28
views
Correct way to handle missing data in boto3 response with python
I am writing some python code to do some stuff with AWS autoscaling groups and such. I have several functions that will use boto3 to call to aws and get back a dict from which I want to extract one ...
0
votes
1
answer
7
views
Unable to create Authorizer for REST API using passed Cognito User Pool from separate stack. What is the issue?
I have 2 stacks in total my WebFrontStack and my CRUD stack. I need to create a lambda authorizer for my rest API using my cognito user pool. My cognito user pool was created in the WebFrontStack and ...
0
votes
1
answer
15
views
How to get statemachineARN with name of statemachine
I have a state machine name(e.g. custom-state-machine), and I want the statemachineARN of this, how could I achieve that.
there is function, list_state_machines(), but it gives all the state machines ...
0
votes
0
answers
10
views
AWS - Prisma - Python Boto3 Enhanced auto-remediation
I use that enhanced auto-remediation (https://github.com/PaloAltoNetworks/Prisma-Enhanced-Remediation#getting-started) trying to auto remediate alerts detected in Prisma.
For some reasons some alerts ...
0
votes
0
answers
12
views
mp4 Upload To S3 from external S3 location or presigned url | DJANGO
Currently my client is sending me presigned url and public S3 location of a mp4 video.
below is the format
{
"URL": "https://brytecam-bucket-1.amazonaws.com/video-url",
"...
1
vote
1
answer
28
views
Test if an S3 object was created or updated when using boto3
Is it possible to determine whether or not an S3 object was created or updated when using boto3?
When using boto3.client('s3').put_object() to create or update S3 objects, the underlying PutObject API ...
0
votes
0
answers
39
views
botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the GetObject operation
Even though passing the the correct Access key, Id and token, I am getting an error while running a below code. Anything missing in this code?
import boto3
session = boto3.Session(
region_name='...
0
votes
0
answers
5
views
S3 AWS - generating presigned url in R
I would like to do this in R using the package 'paws':
import boto3
url = boto3.client('s3').generate_presigned_url(
ClientMethod='get_object',
Params={'Bucket': 'BUCKET_NAME', 'Key': '...
-1
votes
1
answer
37
views
How do I find the Prices for different RDS Instance Classes using the Pricing API in Boto3 AWS?
So I started by trying to find the service code for RDS instances to use with the pricing API's .get_products() method. But when I used .describe_servies() and went through there list of services, I ...
0
votes
1
answer
28
views
Error while updating ECR permission using boto3 client
We are trying to update the permission of an ECR repository using boto3 sdk.
import json
import boto3
access_key = "*******"
secret_access = "*******"
ecr_repo_name = 'repo-name'
...
0
votes
2
answers
21
views
How do I get my file to be saved as JSON after writing a JSON file into AWS S3 bucket
I am trying to write a JSON file into my AWS S3 bucket. However, I do not get a JSON file after it has been uploaded.
I get my data from a website using a request.get() and format it into a JSON file ...
-2
votes
0
answers
44
views
Using DynamoDB how to update multiple items with transact_write_items
I'm pretty new to dynamoDB, and I'm facing an issue of updating multiple items using transact_write_items.
So, basically, I have two records that I want to update in one table. Both of them have the ...
0
votes
0
answers
14
views
Boto3 CloudWatch Log returns only 100 logGroups
The AWS account I'm dealing with has 300+ CloudWatch LogGroups.
But when I try to retrieve them with nextToken and pagination techniques, it returns only 100 records.
I used the below codes.
nextToken ...
0
votes
1
answer
22
views
How to create spot instance with boto3 with required instance attributes instead of instance type
I am able to create spot instance with below code
import boto3
import datetime, random, string, json
client = boto3.client('ec2')
response = client.request_spot_instances(
DryRun=False,
...
0
votes
1
answer
18
views
How to get ip address of spot instance created with boto3
I am able to create spot instance with boto3 with the following code
import boto3
import datetime, random, string
client = boto3.client('ec2')
response = client.request_spot_instances(
DryRun=...
-1
votes
1
answer
25
views
expected str instance, set found when using Boto3
I'm using boto3 and putting in my credentials like-so:
dynamodb_client = boto3.client('dynamodb', region_name='us-west-2', aws_access_key_id={access_key}, aws_secret_access_key={secret_key})
I get ...
0
votes
1
answer
36
views
How to load JSON data (call from API) without key directly to S3 bucket using Python?
I am relatively new to AWS s3 I am calling an API to load the JSON data directly to s3 bucket. From s3 bucket data will be read by Snowflake. After researching I found that using Boto3 we can load ...
1
vote
1
answer
43
views
python boto3 DynamoDB validation error on getItem yet schema looks correct
Here is the scheme of my dynamoDB table
Here is the way I am requesting my get_item
if event['hcpcs_codes'] != None:
# codes must be in numerical, alphabetical order
# multi codes ...
2
votes
0
answers
19
views
How can I download a file from an S3 bucket after I SSH to a remote machine/edge node to a particular path in the same remote machine?
I am relatively new to Airflow and am trying to solve a particular problem.
I wanted to know how I could download my file residing in an S3 bucket after SSH to a remote machine/client through the ...
0
votes
1
answer
31
views
botocore.exceptions.ClientError: An error occurred (UnauthorizedOperation) when calling the DescribeInstances operation
I am trying to filter out instances in AWS by matching particular tag added in EC2 instance. For this I have created new IAM role with following limited permissions :-
{
"Version": "...
0
votes
0
answers
32
views
Multiprocessing - PicklingError: Can't pickle <class 'botocore.client.S3'>: attribute lookup S3 on botocore.client failed
The following code gets the error of
_pickle.PicklingError: Can't pickle <class 'botocore.client.S3'>: attribute lookup S3 on botocore.client failed
import boto3
import concurrent.futures as ...
1
vote
0
answers
18
views
Convert boto3.resource(...) to boto3.client?
I found the following code in a class.
self.s3client = boto3.client("s3")
self.s3client.put_object(...)
And later the following code in the same class,
self.s3object = boto3.resource("...
0
votes
0
answers
16
views
Copying AWS Backup snaphots to RDS snapshots automatically
I have a backup of a type "snapshot"(Aurora) created by AWS Backups service.
I want to automate copying it to RDS snapshots. There is such an option in the AWS Console in the AWS Backup tab.
...
0
votes
0
answers
14
views
Possible to use boto3/SDK service resource cross account? [duplicate]
Since these are instantiated with ur current user and not some provided client or credentials, im assuming it is not possible to use them cross account right?
What im hoping to do is to run a Lambda ...
-2
votes
0
answers
20
views
AWS SES intermittently timing out in Lambda [closed]
I'm running a fairly simple python script in a lambda function (from a docker image) and it works fine about half of the time, and the other half it times out. It's getting stuck on making an api call ...
-1
votes
0
answers
31
views
Download all files to tmp directory of lambda from a folder in s3 bucket
I have the below structure in my s3 bucket where multiple text files are stored.I need to download all these files to AWS Lambda and store them into the /tmp dir.
myfolder/
Reports/
test1....
-1
votes
0
answers
56
views
Python | How to Upload Multiple Files in Parallel to an S3 Bucket?
I have recently hit a bottleneck in uploading files to S3 Bucket. I have an API which after scanning the PDF extracts each page of the PDF as an image in a local directory, after which each image is ...
0
votes
0
answers
23
views
Speed up S3 files copy between buckets-Using python
All, I am trying to copy large number of files from one bucket to other, following is my code but looks like copy is not happening! Not sure does bot3.transfer has copy method.
what I am missing here
...
0
votes
1
answer
71
views
How to delete items from dynamodb list
listofvalues to remove = [1,2,3]
myList from dynamodb = [1,2,3,4,5,6,7]
Do i need to loop every single item find the index and remove
Is there any better approach
Gone through link Add or remove an ...
0
votes
0
answers
23
views
Trying to read csv with special characters from AWS S3
Im trying to read a csv from my bucket on S3 using AWS Lambda, but the csv have special characters from brazillian portuguese alphabet.
def rentabilidade_sintetica():
s3_client = boto3.client('s3')
...
0
votes
0
answers
43
views
Will boto3 s3.put_object() run in parallel when wrapped as async function?
I found the following question for converting sync function to async function.
How can I wrap a synchronous function in an async coroutine?.
from functools import wraps, partial
def wrap(func):
@...
-1
votes
1
answer
36
views
Get item as soon as it inserted in dynamodb in python
Looking for example
We insert single record in dynamodb
We need to retrieve that item as soon as it inserted in dynamodb database using python
Just like get record seeking continuously for latest item ...
0
votes
2
answers
29
views
IndexError: list index out of range with moto
I am mocking an internal function which is returning dynamodb query. the query had begins_with which was throwing error IndexError: list index out of range.
I changed the query and removed begins_with ...