How does Cloudflare even know that this request came from a script even if I provided all the data, cookies and parameters when making a normal request? What does it check for? Am I doing something wrong? For example (I have redacted some of the values):
import requests
cookies = {
'__Host-next-auth.csrf-token': '...',
'cf_clearance': '...',
'oai-asdf-ugss': '...',
'oai-asdf-gsspc': '...',
'intercom-id-dgkjq2bp': '...',
'intercom-session-dgkjq2bp': '',
'intercom-device-id-dgkjq2bp': '...',
'_cfuvid': '...',
'__Secure-next-auth.callback-url': '...',
'cf_clearance': '...',
'__cf_bm': '...',
'__Secure-next-auth.session-token': '...',
}
headers = {
'authority': 'chat.openai.com',
'accept': 'text/event-stream',
'accept-language': 'en-IN,en-US;q=0.9,en;q=0.8',
'authorization': 'Bearer ...',
'content-type': 'application/json',
'cookie': '__Host-next-auth.csrf-token=...',
'origin': 'https://chat.openai.com',
'referer': 'https://chat.openai.com/chat',
'sec-ch-ua': '"Brave";v="111", "Not(A:Brand";v="8", "Chromium";v="111"',
'sec-ch-ua-mobile': '?0',
'sec-ch-ua-platform': '"Linux"',
'sec-fetch-dest': 'empty',
'sec-fetch-mode': 'cors',
'sec-fetch-site': 'same-origin',
'sec-gpc': '1',
'user-agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36',
}
json_data = {
...
}
response = requests.post('https://chat.openai.com/backend-api/conversation', cookies=cookies, headers=headers, json=json_data)
I have tried different useragents to no avail, but I can't seem to figure out whats causing the problem in the first place.
The response comes back with error code 403
and HTML something like:
<html>
...
...
<h1>Access denied</h1>
<p>You do not have access to chat.openai.com.</p><p>The site owner may have set restrictions that prevent you from accessing the site.</p>
<ul class="cferror_details">
<li>Ray ID: ...</li>
<li>Timestamp: ...</li>
<li>Your IP address: ...</li>
<li class="XXX_no_wrap_overflow_hidden">Requested URL: chat.openai.com/backend-api/conversation </li>
<li>Error reference number: ...</li>
<li>Server ID: ...</li>
<li>User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36</li>
</ul>
...
...
</html>
chat.openai.com
require some type of authentication, such as a login or an API token? So how are you authenticating to theopenai.com
service prior to sending a post request?chat.openai.com
?