Questions tagged [boto3]

Boto 3 - The Amazon Web Services (AWS) SDK for Python

Filter by
Sorted by
Tagged with
0 votes
0 answers
8 views

Unable to run start_thing_registration_task method. Getting the error message: Status Code: 400; Error Code: ResourceRegistrationFailureException

I am getting the following error message. My python code runs till completion but no "Things" gets created. I extracted the error message from the list_thing_registration_task_reports method....
Osama Khalid's user avatar
0 votes
1 answer
20 views

Failed to create folder in S3 bucket using boto3

I am trying to create a folder/directory in S3 bucket using boto3 library. Here is my sample code. import boto3 access_key = "REDACTED" secret_key = "REDACTED" endpoint = "...
Abhishek Kulkarni's user avatar
0 votes
1 answer
10 views

How to pass containerOverrides to an ECS task when triggering as a target from eventbridge rule?

I want to write a python code that uses boto3 to assign targets to an eventbridge rule. The target should be an ECS task. Also when the eventbridge rule runs/triggers the ecs task it should override ...
Akshay Alok's user avatar
0 votes
1 answer
12 views

boto3: ssm client: describe_instance_information: How to filter based on computername

I have a list of managed nodes in the fleet manager. I want to filter the managed nodes using ComputerName field, but in the documentation the filters dont have that ssm_client = boto3.client('...
Santhosh's user avatar
  • 10.1k
0 votes
1 answer
19 views

Boto 3 Redshift client execute_statement lagging

I'm trying to copy some data over from MySQL to RS. The table is 650,000-ish rows, so i'm doing it in batches of 10,000 or so and also using python's multiprocessing module. The first few iterations ...
Raksha's user avatar
  • 1,574
0 votes
0 answers
9 views

How to setup google identity provider with fastapi using cognito, I have done required setup with aws and google developer console

It's really confusing to use google identity data to create user at the AWS cognito end. The process of using cognito boto3 client for google identity is confusing in a sense. There is no clear ...
DEBASISH MOHANTY's user avatar
0 votes
0 answers
20 views

configparser Not Recognizing Sections in config.ini

I'm trying to use the configparser module in Python to read from a config.ini file. Despite having a [example_section] in my config.ini file, when I run my script, it keeps printing that the section ...
coryrobbins's user avatar
0 votes
1 answer
41 views

Python patch object, how to assert on calls

I'm trying to create some pytest unit tests on a custom ansible module. I'm trying to mock the boto3 client and just test the logic of the class. I moved the initialization of boto into a class method ...
robobandit's user avatar
0 votes
2 answers
31 views

Unable to retrieve names of all folders in s3 bucket using boto3

I am using list_objects_v2 to get list of folders in s3 bucket The below function is being used to get folder names def get_folders(source_bucket): pages = paginator.paginate(Bucket=source_bucket) ...
user3435047's user avatar
0 votes
1 answer
23 views

List bucket files in S3 files permission denied. But upload works

I'm making a simple script in Python to upload and list the files in a S3 bucket, the problem is that I can upload the files, but I receive a Permission Denied error when I try to list the files. This ...
Klian's user avatar
  • 1,530
0 votes
0 answers
11 views

boto3 - Get "Full load throughput rows target" of a DMS Database Migration task

I am building a python script that monitors all DMS Database Migration tasks in an AWS account and checks if there is any task whose Full load throughput rows target value has been zero for certain ...
ambb's user avatar
  • 41
0 votes
0 answers
27 views

Getting error in serverless deploy - AWS Lambda Python [duplicate]

I'm getting below error while running the command "serverless deploy" SyntaxError: Unexpected token '.' node_modules\@aws-sdk\client-acm\dist-cjs\ACMClient.js:28 const ...
Asfar Irshad's user avatar
0 votes
1 answer
18 views

DynamoDb Query and Append item to nested list inside a list, Python Boto3

I have this kind of entity in a dynamodb table: { "fieldAA":152, "Big_List":[ { "fieldA":"value", "Nested_List":[ ...
Daniel A.'s user avatar
1 vote
1 answer
25 views

how to set file type in upload_fileObj() function in boto3 for blackblaze

def getB2Resource(): b2_config = Config( region_name = 'region_name...', signature_version = 'v4', retries = { 'max_attempts': ..., 'mode': '......' ...
Sounak Das's user avatar
1 vote
2 answers
44 views

Query DynamoDB table API response

I am querying/scanning the DynamoDB table and getting the expected response. The response contains data type as well in dict Key :{'S':'String'} , how can I get a response just in Key : Value format i....
K P's user avatar
  • 15
-2 votes
1 answer
47 views

How to start a server on a remote machine? [duplicate]

I'm trying to set up a cluster of EC2 instances that I can use for "big data" modeling. I'm trying to stay close to the metal and would prefer to not use existing frameworks (e.g. dask, ...
Steven Scott's user avatar
0 votes
0 answers
29 views

boto3 Request Entity Too Large when uploading file with size 6MB, even if multipart_thershold is much bigger than 6MB

I'm trying to save file to s3-compatible storage (to be precise, ceph) Here is the code: import boto3 import requests from boto3.s3.transfer import TransferConfig s3 = boto3.client('s3', endpoint_url=...
Egor's user avatar
  • 1
-1 votes
0 answers
34 views

S3 file download through S3 Transfer manager

In python there is a class called S3Transfer in boto3.s3 where configurations can be given which supports multipart upload......configurations like the one below can be mentioned boto3.s3.transfer....
arsenal_random_profile's user avatar
0 votes
1 answer
34 views

Connection timeout to S3 from a python application running in ECS EC2

I have a python ECS task running via EC2 (not Fargate). In this task, I am setting up a boto3 client cls._instance.client = boto3.client( "s3", region_name="eu-west-1", ) ...
Simeon Borisov's user avatar
0 votes
0 answers
21 views

Reuse create_client in aiobotocore for better performences

I am implementing an async Python code that uses aiobotocore to put objects in S3. The code is something like that - class Foo: def __init__(self): self._aiobotocore_session = get_session()...
omer bar lev's user avatar
-1 votes
1 answer
24 views

boto3 failing to upload file in subdirectory of existing bucket

<EDIT> Closing this, as uninstalling and reinstalling moto from scratch fixed the issue. </EDIT> I am using boto3==1.28.45 with moto==1.3.14 running locally on port 5353. This has worked ...
gmoss's user avatar
  • 1,035
0 votes
1 answer
25 views

How can I query List of Dict using filterExpressions for DynamoDB

I am currently working on a project that involves using Amazon DynamoDB as the database for storing and retrieving data. In one of my DynamoDB tables, I have a field that is a List of Dict (Dictionary)...
NAVEEN KUMAR's user avatar
0 votes
1 answer
13 views

How to fetch DynamoDB records with FilterExpression=Attr.exists via boto3?

I have the following code: import boto3 from boto3.dynamodb.conditions import Attr resource = boto3.resource("dynamodb") table = resource.Table(MY_TABLE_NAME) records = [] pagination = {} ...
MartinSolie's user avatar
0 votes
0 answers
23 views

Django storages adding “;amp” to staticfiles query string parameter only for Django-jet files resulting in unauthorized access

Some staticfiles links have wrong params, what i noticed that they are only related to "django-jet" package. Normal Django staticfiles URL: https://daal.nyc3.digitaloceanspaces.com/static/...
Ji---'s user avatar
  • 115
0 votes
0 answers
44 views

Why am I getting an error from this boto3 syntax?

I am attempting to write a tool in python to add or remove users from an EC2 security group on AWS. The tool is meant to enable/disable access to a server-side private system for specific people via ...
amariani's user avatar
0 votes
1 answer
14 views

Cognito not running custom auth challenge

I have created in Cognito the following custom challenge triggers in Python. They are identical to the AWS ones but written in Python. Define auth challenge: def lambda_handler(event, context): if ...
João Pedro Schmitt's user avatar
0 votes
1 answer
21 views

Setup boto3 to connect to custom host, similar to s3cmd

With s3cmd I am able to connect to a bucket with the following config file: [default] access_key = <VARIABLE> secret_key = <VARIABLE> host_base = <VARIABLE> host_bucket = <...
user3289867's user avatar
0 votes
0 answers
12 views

How to filter using start_query in CloudWatch?

I'm writing a python script to query cloudwatch: This query works fine: query = ''' fields @timestamp, @message ''' But this is not working: query = ''' fields @timestamp, @message | ...
DenCowboy's user avatar
  • 13.9k
0 votes
0 answers
25 views

Sagemaker AutoML not finding .gzip file

I have a file saved in an s3 bucket with the extension .gzip. I am then launching an AutoML job using boto3, but it does not find the file (I am certain it is in there). I get this error in the log of ...
Aaron England's user avatar
0 votes
0 answers
42 views

ERROR: ALTER TABLE APPEND cannot run inside a multiple commands statement

I'm trying to iteratively pull data from MySQL and build a table in RedShift. But getting this error on APPEND db = <some_db> table_name = 'test' table_name_full = f'{db}.{table_name}' cols = ['...
Raksha's user avatar
  • 1,574
0 votes
1 answer
33 views

DynamoDB append list to nested list with boto3

I have a DynamoDB object which has a list with lists: options: [ ['fish', 'cat', 'dog', 'cow'] ] I want to append another list to it so it looks like this: options: [ ['fish', 'cat', 'dog', 'cow']...
Denny Crane's user avatar
1 vote
0 answers
37 views

How to properly mock functions inside FastAPI endpoint handler

I'm having trouble mocking a function that resides inside a FastAPI endpoint handler. No matter what I've tried it doesn't seem to work. A brief explanation of what I'm trying to test. I have a ...
BlindPrimate's user avatar
0 votes
1 answer
14 views

Trying to connect boto3 to linode object storage

I cannot seem to connect to Linode Object Storage with boto3. It seems like this is something that is supported, but I cannot find the right syntax. I have tried adding http/https to endpoint_url and ...
Caullyn's user avatar
  • 146
-2 votes
2 answers
29 views

Bucket name validation fails in Boto3 in a Junyper notebook

Been running into an error with my S3 bucket and folder using Boto3 to download a file into a Junyper notebook. This is the code that generates the error, which I'm assuming I'm using the wrong ...
Emmanuel Henri's user avatar
-2 votes
0 answers
18 views

Unable to start EC2 in sequential order using lambda,python

Unable to start EC2 instances in sequential order [x, y, z] also have to make sure each instance is in running state before starting the other instance also skip if the instance is already running. ...
Rajeshwar rao Gone's user avatar
0 votes
1 answer
26 views

Amazon SES creds working with smtplib but not with boto3

I'm trying to send emails using Amazon SES. Created the roles and API keys and policies and tested it out using Python smtplib. Pasting the snippet below: import smtplib fromaddr = "some@email....
Krishh's user avatar
  • 612
-1 votes
0 answers
13 views

How to create AWS s3 bucket with specific capacity (size limit) using python boto3 client

I'm using boto3 sdk for python and I have created s3 client session = boto3.session.Session() s3_client = session.client('s3', endpoint_url=self....
Madeva Tuppad's user avatar
1 vote
0 answers
36 views

AWS Transfer How To See In-Flight File Transfers From SFTP Connector?

I have an AWS Lambda function configured to call start_file_transfer on an AWS Transfer Family SFTP Connector at X minute intervals. The problem I'm having is that sometimes the Lambda runs again ...
gbeaven's user avatar
  • 1,572
0 votes
0 answers
17 views

Passing metadata of trigger to ECS task -fargate- from S3 via Eventbridge

i am receiving bucket name and key as an empty string which causes script to fail. I am trying to retrieve filename which caused the trigger by in eventbridge from S3 but unable to get it. aws events ...
Vandalism's user avatar
0 votes
0 answers
18 views

Error while running a DAG task from Airflow CLI

ERROR - Unable to retrieve connection from secrets backend (MetastoreBackend). Checking subsequent secrets backend. Traceback (most recent call last): File "/usr/local/lib/python3.8/site-...
Adheeban's user avatar
0 votes
1 answer
16 views

How to put_parameter where value is type file using boto3?

Using the aws-cli, I can do this: aws ssm put-parameter --name my-param --value file://my-file.txt --type String How can I do the same in boto3 where the value is of type file?
alltej's user avatar
  • 6,817
0 votes
0 answers
58 views

error as - "cannot import name 'DEFAULT_CIPHERS' from 'urllib3.util.ssl_'" on AWS Lambda using a layer

for python 3.9,imported libraries "boto3_version": "1.26.90", "botocore_version": "1.29.90", "urllib3": "1.26.11". Fetch data from public ...
ANKITA KUMARI's user avatar
0 votes
1 answer
47 views

DynamoDB Query with partition key and sort key giving me Query condition missed key schema element error

I'm using boto3 in Python to make queries to a DynamoDB. I'm using a Local Secondary Index to query my results sorting by an attribute (start_date). However, when connecting two Key( ) elements with &...
Luigi Trevisan's user avatar
-1 votes
0 answers
63 views

AccessDenied with boto3 operations in Lambda and s3

I'm developing in terraform a AWS Lambda that is getting triggered by an object in s3 and then Im posting parsed data to api gateway. Im having a problem with: { "errorMessage": "An ...
woodbine's user avatar
0 votes
1 answer
70 views

Cognito: Using admin_user_global_sign_out in pre_authentication trigger sometimes fail due to apparent async behavior

I'm trying to sing-out all old sessions from Cognito for a specific user credentials (revoke the Refresh tokens) when a new session is started using the same user credentials (in this case e-mail and ...
João Pedro Schmitt's user avatar
-1 votes
0 answers
21 views

An error occurred: An error occurred when calling the SendCommand operation: Instances [[<instance-ID>]] not in a valid state for account

An error occurred: An error occurred (InvalidInstanceId) when calling the SendCommand operation: Instances [[]] not in a valid state for account 309178984042. def pass_command(imageId, user_command, ...
ravi bansal's user avatar
-1 votes
0 answers
21 views

Best practices to use IAM user credentials in an ECS task

I have an app (written in Python) that runs a Fargate instance on the AWS Elastic Container Service (ECS). Credentials are injected on the container through an .env file located in S3 and the code ...
André Lourenço's user avatar
-1 votes
0 answers
11 views

ERROR KeyError : Records Error while moving files from one s3 bucket folder to another

I am new to lambda functions. I am trying to create lambda function which basically moves datafiles from one s3 bucket to Archive folder in the same bucket, which get triggered by cloud watch schedule ...
Anusha's user avatar
  • 61
0 votes
1 answer
58 views

Automation to find EC2 instance with a public IP address

Goal: automation to find an ec2 instance information (aws ec2 describe-instances) in any project/region of an organization by public IP address (should be at most one because public IPs are unique). ...
Johnny W's user avatar
3 votes
0 answers
33 views

AWS S3 Public (with restricted regional access) bucket access with Python?

I have a public bucket in S3 with a policy that only allows regional access from us-west-1 and us-west-2. I want to access this bucket anonymously in Python using boto3/s3fs and list all files in it, ...
Minh Phan's user avatar

1
2 3 4 5
153