I am fairly new to computer networking and want to use the python requests
library for downloading large files from an external FTP server. I have a conceptual question as to when the content of a large file is received and how the client tells the server when to send over the content.
My code looks somewhat like
import requests
...
response = requests.get(url_to_very_large_file, stream=True)
...
with open(save_path, "wb") as file:
for chunk in response.iter_chunks(chunk_size):
file.write(chunk)
Now response arrives back from the server very quickly (less than a second), but the content of the file (say 2 GB heavy for the sake of argument) surely cannot arrive that fast. I'm also confused that response already has a content
attribute. What happens under the hood?
More precisely:
- What is in
response.content
? - Does the server now bombard my client with the 2 GB content right away, or is another request sent to the server when I ask for
response.iter_chunks
orresponse.content.read()
? At which point does the server start sending over the 2GB of content? - Does the server know in which chunk_size I am reading /expecting the files?
- Where are the chunks stored in the meantime, if they are received by the client but not read into memory?