0

I want to use persistent session across all requests vs creating & closing new sessions each time I initiate request. It's recommended way in aiohttp docs and persistent session other advantages per my use case.

The problem is that, persistent session takes much more time than instantly created sessions. I expect that using persistent session should take around same time.

Below is the sample script which demonstrates the problem:

import asyncio
import time
from contextlib import contextmanager
from typing import Optional

import aiohttp

LINK: str = 'http://google.com'
SESSION: Optional[aiohttp.ClientSession] = None


async def request(link: str, session: Optional[aiohttp.ClientSession] = None):
    if session is not None:
        async with session.get(link) as response:
            return await response.text()
    else:
        async with aiohttp.ClientSession() as session:
            async with session.get(link) as response:
                return await response.text()


async def make_session():
    global SESSION

    SESSION = aiohttp.ClientSession()


@contextmanager
def timer(msg: str):
    start_time = time.time()

    try:
        yield None
    finally:
        print(f'{msg} elapsed: {time.time() - start_time}')


if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    loop.run_until_complete(make_session())

    persistent_session_futures = [request(LINK, SESSION) for _ in range(10)]
    instant_session_futures = [request(LINK) for _ in range(10)]

    with timer('instant session'):
        loop.run_until_complete(asyncio.gather(*instant_session_futures))

    with timer('persistent session'):
        loop.run_until_complete(asyncio.gather(*persistent_session_futures))

    loop.run_until_complete(SESSION.close())

"""
sample output:
instant session elapsed: 1.0750820636749268
persistent session elapsed: 8.006160020828247
"""

Can you please help me in understanding the behavior / possible bug?

These are the versions used in the script:
Python: 3.8.10
aiohttp: 3.8.1
OS version: Ubuntu 20.04.4 LTS

UPDATE 26.05.2022:
Dockerized versions output expected results:

FROM python:3.9-alpine

RUN pip install --upgrade pip
RUN pip install aiohttp

COPY test.py test.py

CMD ["python", "test.py"]
# python3.8-alpine / aiohttp
instant session elapsed: 0.4182624816894531
persistent session elapsed: 0.4050147533416748

# python3.9-alpine / aiohttp
instant session elapsed: 0.42077016830444336
persistent session elapsed: 0.41016197204589844

But running locally, in either versions (using venv) results in instant session are faster - very weird!

3
  • Are you sure this is the code you're running? I tested it on my computer and got instant session elapsed: 0.2039353847503662 persistent session elapsed: 0.13978099822998047. Python 3.9.7, aiohttp 3.8.1. Looking at your code I see no problem with it. May 25, 2022 at 21:45
  • Thanks @AndrejKesely for the feedback - really appreciate it! Yes - I'm sure - that's the code :) I've put code in the docker images & now outputs are as expected, i.e. persistent session is quicker (can't even believe myself :D). On the other hand, I've tested on 2 different Ubuntu (20.04.4 LTS) machines & results are different - persistent session takes much longer. Please see the update May 26, 2022 at 6:10
  • The first test was done in Docker. I even ran the test in my host machine (Ubuntu 20.04 LTS, Python 3.8.10, latest aiohttp and I see the exact results (instant session elapsed: 0.5182437896728516 persistent session elapsed: 0.36999082565307617). Something on your configuration seems bad (do you use proxy? inspect environment variables on your host) May 26, 2022 at 6:55

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.