Questions tagged [aiohttp]
Use this tag for questions about AIOHTTP – a client/server framework for asyncio Python.
1,465
questions
0
votes
0
answers
28
views
Request a lot of URLs with asyncio
I have a list of 30,000 URLs and for each of the URLs of this list, I want obtain a list which contains the text of the page for every URL.
For this I use asyncio, it's already faster than with ...
1
vote
0
answers
22
views
What is the correct way to reuse a aiohttp ClientSession() object?
I would like to verify whether the code I have created is an appropriate and effective way to reuse a aiohttp.ClientSession() object.
from asyncio import tasks
import aiohttp
import asyncio
import ...
0
votes
0
answers
33
views
Streaming upload
Is it possible on an aiohttp server to send an uploaded file to another server without loading it into memory? How can I do that?
async def forward_huge_upload_to_upstream(request):
reader = await ...
0
votes
1
answer
32
views
how to connect nginx proxy_pass to aiohttp webserver in python (Error)
Im trying to connect my nginx proxy_pass to my aiohttp web server
But im keep getting errors
Here is my Nginx config:
server {
server_name www.example.com;
location /nextpay {
...
3
votes
1
answer
69
views
How to gracefully close aioredis pool in aiohttp server?
I create redis pool the following way:
async def create_redis_connection_pool(app) -> aioredis.Redis:
redis = aioredis.from_url(
"redis://localhost", encoding="utf-8"...
-2
votes
0
answers
38
views
Python Gremlin - Aiohttp throws "Connection was already closed."
I'm getting an error while working with cosmos db and gremlin database via python (gremlin_python) module.
The error is intermittent and raised via the aiohttp wrapper class where it says "...
4
votes
0
answers
65
views
+50
How to handle graceful shutdown inside of aiohttp coroutines?
I want to save state into the database of a background aiohttp coroutine before server is shut down. I was thinking of creating a global array of coroutine jobs that need to be finished and do an ...
1
vote
0
answers
54
views
Restore state of long lived coroutine in aiohttp service
I have a service that can start an "operation" when you call an API endpoint with some parameters. This starts a background coroutine using get_scheduler(request).spawn(coro_fn). This ...
0
votes
0
answers
12
views
How to Detect which Element of Task Crashed
I'm rotating proxies using aiohttp and aiohttp_socks5. The problem is sometimes some of my proxies stop working and code crashes.
aiohttp_socks5.socks.SOCKSAuthError: SOCKS server rejected username ...
0
votes
1
answer
27
views
After running aiohttp server and make a get request crashes error
There is a program, but, when i running the program and go to http://127.0.0.1:8080/scan/192.168.0.1/1/10 server appears "500 Internal Server Error
Server got itself in trouble" and throw ...
0
votes
0
answers
10
views
Can connection.AioHttpApp handle multiple requests?
I have an aiohttp application that I generated from a JSON openapi document.
It works well if I send the requests one at a time, for example:
curl -X GET http://localhost:8000/v1/foo
The problem is ...
1
vote
1
answer
18
views
Setting up CORS with Faust webview
I'm attempting to directly post information to a Kaka topic via a Faust web view as per their example. However, I'm getting the familiar CORS issue in the response.
Looking into the docs only leads me ...
0
votes
1
answer
28
views
Url loads when accessed from the browser but not when loading from aiohttp
I am trying to load a URL from https://coinfomania.com/feed
It works perfectly when viewed from the browser
When I try to load data from the same URL with aiohttp with the following code, it gives me ...
1
vote
1
answer
42
views
How to share (initialize and close) aiohttp.ClientSession between Django async views to use connection pooling
Django supports async views since version 3.1, so it's great for non-blocking calls to e.g. external HTTP APIs (using, for example, aiohttp).
I often see the following code sample, which I think is ...
1
vote
1
answer
35
views
Asyncio Sleep Does not seem to Satisfy Rate Limits for certain status codes
I am working with an API that limits to 4 requests/s. I am using asyncio and aiohttp to make asynchronous http requests. I am using Windows.
When working with the API I receive 3 status codes most ...
0
votes
0
answers
22
views
How to make handler of this local host on python?
I have made a design for web-app with forms to input. Now i need to connect this forms with Python script. HTML design here:
<!DOCTYPE html>
<html lang="en">
<head>
<...
0
votes
0
answers
27
views
return value from aiohttp post method
I am currently working with the aiohttp post method. With await post.read() I get the return value when the method is executed. But now I have the problem that I want to get the return value as soon ...
-1
votes
0
answers
37
views
Error Parsing HTML of BeautifulSoup Using Async/Aiohttp Libraries
I am getting the following error back after asyncio finishes grabbing the html of the following pages.
object of type 'UnboundLocalError' has no len()
I tried to catch this error using the try/...
0
votes
0
answers
14
views
bad api request Aiohttp Pastebin
So my code looks like this:
self.bot.session = aiohttp.ClientSession()
""" Another Class """
data = {
'api_option': 'paste',
'api_dev_key': "...
0
votes
0
answers
14
views
set_wakeup_fd only works in main thread on linux
more info about error - https://github.com/aio-libs/aiohttp/issues/6783
I have python3.8.13
0
votes
1
answer
52
views
aiohttp request get block async/await- Python
I'm a newbie at asyncio and aiohttp. Recently, I try to practice for understanding how does the eventloop actually working.
when I practice for sending urls simultaneously, I encounter some problems. ...
0
votes
0
answers
37
views
How do you POST multipart formdata with an octet stream using aiohttp?
I am looking to send a request to an API that takes multipart formdata as an octet stream using aiohttp in Python. I took a look at the docs and was still confused about how to send a request properly....
2
votes
2
answers
71
views
Python aiohttp returns a different reponse than python requests. I need help understanding why
after the whole evening yesterday and morning today i really an help for understanding why, aiohttp request returns differently than requests request.
import requests
reqUrl = "https://api-...
0
votes
0
answers
44
views
two data streams into one ffmpeg stream
I have two data streams, each representing a rabbitmq subscriber. Inside audio and video streams, data is in the form of bytes. Something like:
vp = VideoProcess()
ap = AudioProcess()
async def ...
0
votes
1
answer
65
views
How to resolve RuntimeError: await wasn't used with future using aiohttp and asyncio in python?
I made some pretty simple script which pulls data from clicky.com api but for some reason it does not work as expected from time to time.
Sometimes it gets results but another time I am getting the ...
1
vote
0
answers
36
views
Python asynchronous requests with timer
I'm trying to run a list of http get request to an url, following a trace of interarrival times between the requests (occasionally with concurrent requests at the same time).
I'm currently using ...
0
votes
0
answers
19
views
Why are 404 codes returned from Twitter when using aiohttp and Python 3.8?
I've written a Python script to validate URLs in a website. The script has been used for many years and uses aiohttp to check multiple links in parallel.
Recently (like the last few days), checks ...
0
votes
0
answers
52
views
How to fix "ConnectionResetError" and "aiohttp.client_exceptions.ClientConnectorError" while web scraping using asyncio and aiohttp?
I am learning web scraping with asyncio and aiohttp. The scraper seems to work for few times and then shows some errors for multiple tries and then works again. The same URL opens normally in browser ...
1
vote
0
answers
65
views
Is there jwt authentication for aiohttp?
I have django backend with JWT authentication and want to work on client side using aiohttp. I didn't found any ready-made implementations. All I found is aiohttp-jwt, which used on server side. For ...
0
votes
2
answers
195
views
How to use asyncio, aiohttp web scraper with fastapi?
I am learning web scraping using asyncio and aiohttp with beautifulsoup. I want to create a RESTful API to get user input, scrape the data and then show the response in json format. This is how my ...
2
votes
1
answer
210
views
TypeError: 'coroutine' object is not iterable
I build a chatapplication using Aiohttp server and using python-socket-io. When i tried to host this application in nginx i found this error in supervisor error from error log of supervisor(error log ...
1
vote
1
answer
67
views
google drive file upload with aiohttp in Python
I am trying to make a function that upload files to my google drive.
The below code uploads a single file with requests.
import requests
import json
with open('storage.json', 'r') as file:
token =...
0
votes
0
answers
24
views
Random ConnectTimeout using aiohttp.ClientSession / httpx.AsyncClient via nginx into aiohttp.web.Application
I am trying to debug random ConnectTimeout happening in our infrastructure.
The symptom: Every now and then we receive a ConnectTimeout.
What I have tested:
Change host file: We changed the host file ...
0
votes
1
answer
61
views
Asyncio concurrency rate limit
I am new to python and have been trying to set a rate limit for a program that works with asyncio for concurrent requests but after trying almost everything I found in stackoverflow and other ...
0
votes
0
answers
36
views
aiohttp requests with persistent session take longer vs instant session
I want to use persistent session across all requests vs creating & closing new sessions each time I initiate request. It's recommended way in aiohttp docs and persistent session other advantages ...
0
votes
1
answer
38
views
Python annotations for decorated async functions
I have difficulties with annotations for my coroutines which are decorated to prevent aiohttp errors.
There are my two functions:
from typing import Callable, Awaitable, Optional
from os import sep
...
1
vote
1
answer
38
views
python coroutine asyncio/ await / aiohttp
new in asyncio world.
going straight to the point...
I want to do/make a request(aiohttp) to a site.
if the wait for an answer pass than N seconds I want to stop the process of waiting.
Do the process ...
1
vote
1
answer
25
views
aiohttp check request headers after receiving response
here's the problem:
I have a list of authorization headers to check.
For that, I use aiohhtp
def make_tasks(session, proxies, unchecked_headers):
tasks = list()
for unchecked_header in ...
0
votes
0
answers
72
views
ConnectionResetError: Cannot write to closing transport
I am trying to build a discord bot for my server and at present, I am building a web dashboard for my bot, unfortunately, I am facing the below problem whenever I click the back button please help me ...
1
vote
1
answer
44
views
How can I catch an error in aiohttp get request?
Let's say I have some list of URLs to get some data from them. I try to do it in async way, but one of the URLs is incorrect. How can I catch this error and is it possible to change the URL address ...
0
votes
0
answers
23
views
How can i check if aiohttp post data exists
i'm trying to check if an post parameter in aiohttp exists but the check fails idk why, the code:
async def agentsHandler(request):
global currentId, agentsList
try:
data = await ...
0
votes
0
answers
43
views
Make .netrc authentication work with aiohttp
I'd like to use a .netrc file with credentials to authenticate to an API using aiohttp. As far as I can tell this should be possible, as long as the file is in the home directory (or the relevant env ...
0
votes
0
answers
23
views
Why aiohttp "unpacks" file?
app.router.add_static(
'/' + config['prefix'] + '/',
config['path'],
show_index=True
)
File on server:
/path/name.ext.gz 53kB
Downloaded with browser and with aiohttp client:
name.ext.gz ...
1
vote
1
answer
71
views
How can I speed up parsing with aiohttp workers queue?
I need to collect 256 urls from the main page, 653 more of them, 11000 more of these 653. I collected 653, guy helped me here.Help me add an async to the worker_iso(q) function, which collects 11000 ...
1
vote
0
answers
38
views
python - aiohttp.ClientSession().post
So, I had this code and it worked perfectly fine:
def upload_to_imgur(url):
j1 = requests.post(
"https://api.imgur.com/3/upload.json",
headers=headers,
data={
...
2
votes
1
answer
104
views
How can I speed up the aiohttp parser bs4?
The task is to get data from the site. I have 800 URLs to request. But it takes a long time .I use aiohttp. At this stage, I have received links, by clicking on each of them, I also get a number of ...
0
votes
0
answers
113
views
AttributeError: __aenter__
I'm trying to make a bulk update in my database. Using aiohtpp + sqlachemy. I'm getting a error
async with sess.begin():
AttributeError: __aenter__
with this code:
async def ...
0
votes
0
answers
24
views
What to do when asyncio http takes a long time in python?
import aiohttp
import asyncio
import time
start_time = time.time()
async def get_pokemon(session, url):
async with session.get(url) as resp:
pokemon = await resp.json()
return ...
1
vote
1
answer
92
views
Python aiohttp how to handle client session token timeout
I am making several 100's of http request using aiohttp. I am relatively new to the async world but have managed to get the basic code working.
First I am generating a token. Then, making aiohttp ...
0
votes
1
answer
72
views
How to get json response from aiohttp?
how to get json response from aiohttp?
if i use .json() i getting the error:
File "main.py", line 371, in _checker
user_name = f'{res_json["username"]}#{res_json["...