11

I am currently using Python Requests, and need a CSRF token for logging in to a site. from my understanding requests.Session() gets the cookie, but obviously I need the token. And Also I would like to know where to place it in my code. import requests

user_name = input('Username:')
payload = {
'username': 'user_name',
'password': 'randompass123'
}


with requests.Session() as s:
p = s.post('https://examplenotarealpage.com', data=payload)
1
  • This website talks you through the process fairly simply: kazuar.github.io/scraping-tutorial Basically, look in the page source for an input tag called something like "csrftoken" or similar.
    – wp78de
    Jul 15, 2018 at 19:52

3 Answers 3

16

See the following code example. You can use it directly to login into a website that only uses cookies to store login information.

import requests

LOGIN_URL = 'https://examplenotarealpage.com'
headers = {
    'accept': 'text/html,application/xhtml+xml,application/xml',
    'user-agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36'
}

response = requests.get(LOGIN_URL, headers=headers, verify=False)

headers['cookie'] = '; '.join([x.name + '=' + x.value for x in response.cookies])
headers['content-type'] = 'application/x-www-form-urlencoded'
payload = {
    'username': 'user_name',
    'password': 'randompass123'
}

response = requests.post(LOGIN_URL, data=payload, headers=headers, verify=False)
headers['cookie'] = '; '.join([x.name + '=' + x.value for x in response.cookies])

There are a few possible locations of the CSRF token. Different websites use different ways to pass it to browser. Here are some of them:

  • It can come with response headers, in that case getting it is easy.
  • Sometimes page meta holds the CSRF token. You have to parse the html content of the page to get it. Find the proper CSS selector for it. See an example:

    from bs4 import BeautifulSoup
    soup = BeautifulSoup(response.text, 'lxml')
    csrf_token = soup.select_one('meta[name="csrf-token"]')['content']
    
  • It can be inside of a script tag with JavaScript code. Getting it will be tricky. But, you can always use regex to isolate it.

1
  • 1
    Thank you for setting me on the right path, I'm trying to send requests to a website and for the life of my couldn't figure out how the hell Chrome was sending X-CSRF tokens without getting them somewhere first
    – Amon
    Sep 9, 2020 at 5:47
4
import requests
from bs4 import BeautifulSoup
headers = {'user-agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 
           (KHTML, like Gecko) Chromium/80.0.3987.160 Chrome/80.0.3987.163 
           Safari/537.36'
 }
login_data = {
             'name' : 'USERNAME',
             'pass' : 'PASSWORD',
             'form_id':'new_login_form',
             'op':'login'
  }

with requests.Session() as s:
    url = 'https://www.codechef.com/'
    r = s.get(url,headers=headers,verify=False)
    #print(r.content) # to find name of csrftoken and form_build_id
    soup = BeautifulSoup(r.text, 'lxml')

    csrfToken = soup.find('input',attrs = {'name':'csrfToken'})['value']
    form_build_id = soup.find('input',attrs = {'name':'form_build_id'}) 
    ['value']

    login_data['csrfToken'] = csrfToken
    login_data['form_build_id'] = form_build_id

    r = s.post(url,data=login_data,headers = headers)
    print(r.content)

You can directly use this but their are few things to change:
1.check your user-agent in your browser network option
2.check your name attribute for csrf-token and form_build_id by print(r.content) and find csrftoken and form-build-id and check their name attribute.

final step :

search logout in your r.content if it is their then you are login.

1
  • Works like a charm after installing "bs4" and "lxml" using pip.
    – Cagy79
    Jul 19 at 11:23
0

I put it out here because it took me a lot of time and analysis of the network interaction to find this answer...

I had to login to a swagger/openAPI with python/requests. I could login to the site with a browser, but to login with requests I would need the x_csrf_token/sails.sid combo...

After trying and failing all answers here and otherwhere, checked the browser communication. It turns out the only way was to first get the 'sails.sid', and then do a GET to the undocumented(?) /csrfToken...

base_host = '...'
base_path= '/api/v2'
base_url = base_host + base_path
data = {
  "email": "...",
  "password": "..."
}
resp = requests.post(f"{base_url}/login", data=data)
session_cookie = resp.cookies
session_dict=session_cookie.get_dict()

sails_sid = session_dict.get('sails.sid','could not get valid [sails.sid]')
print(f'sails.sid:{[sails_sid]}')

Then:

cookies = {
    'sails.sid': sails_sid,
}
headers = {
    'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:100.0) Gecko/20100101 Firefox/100.0',
}
r2 = requests.get(f"{base_url}/csrfToken",  cookies=cookies, headers=headers)
print(r2.json())

Notice that in my case it was emai/password... I found all this from analysing the browser via Firefox inspect, so that might be your last option too...

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.