Questions tagged [urllib2]

urllib2 is a builtin python 2 module that defines functions and classes to help with URL actions. It is notably unsatisfactory and has been replaced in python 3 and by third-party libraries.

Filter by
Sorted by
Tagged with
0 votes
0 answers
14 views

Receiving an error when attempting to open a url link with urlopen [duplicate]

I am attempting to run this portion of my code that used to work before, but now I get the following error, urllib2.URLError: <urlopen error [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol ...
Kyle knowles's user avatar
0 votes
0 answers
105 views

I'm getting error "urllib.error.HTTPError: HTTP Error 400: Bad Request" for programmatic access to NCBI Entrez

I'm trying to get sequence information by parsing and reading files by biopython, I have to use efetch and I'm making a feather file format for speed and to save disk space. I know the usage ...
Mayur Kedare's user avatar
1 vote
1 answer
54 views

Python urllib2 -> urllib3 request.urlopen

I need to switch from urllib2 to urllib3. There is a problem with a request. python2 code: reqdata='{"PM1OBJ1":{"FREQ":"","U_AC":"","I_AC":&...
Michael's user avatar
  • 43
0 votes
0 answers
126 views

how to automate authentication SAML request using urllib or selenium with python?

unable to reach required website as per the below code , authentication request page is popped up and unable move next I tried using selenium python but unable to automate SAML authentication page ...
Rajashekar Birudu's user avatar
0 votes
1 answer
131 views

urllib.error.HTTPError: HTTP Error 404: Not Found Issue when Trying to read url from Yahoo Finance given Stock Symbol

I am trying to write a program that will help me with accounting work. I am trying to pull information by using stock prices for companies over time. I am utilizing urllib.request to read and open ...
davo1176's user avatar
0 votes
0 answers
34 views

Read binary data response as uint16array using urllib2 in python 3.9

I am trying to get url response in a uint16array buffer where I am using urllib for http communication in python3.9. I tried np.frombuffer(url.read(),dtype=np.uint16) but its not giving correct array. ...
Kuldeep Shivran's user avatar
0 votes
1 answer
165 views

How do I pass a user-agent to panda's pd.read_html()?

some websites automatically decline requests due to lack of user-agent, and it's a hassle using bs4 to scrape many different types of tables. This issue was resolved before through this code: url = '...
Dadash Vespek's user avatar
0 votes
0 answers
597 views

Get Network Error (dns_unresolved_hostname) using Requests, Urllib and Pandas

Trying to call a URL and get an HTML table from an internal server on network. When entering a URL into a browser, the page and table load. When calling URL in Python using Requests, URLLIB, and ...
Bobby Python's user avatar
0 votes
1 answer
2k views

Why does I get SSLV3_ALERT_HANDSHAKE_FAILURE error when requesting a website?

My code is below: import urllib.request import urllib.parse from lxml import etree HEADERS = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko)...
marlon's user avatar
  • 6,089
-1 votes
1 answer
33 views

get status_code for http.urlopen

How can I print the status code for this response? (eg 200/401 etc) resp = http.urlopen('POST', 'https://api.bitbucket.org/2.0/repositories/6789oh', headers=headers, body=json.dumps(data)) print(str(...
x89's user avatar
  • 2,870
0 votes
1 answer
2k views

add authentication info for urllib.request

I had this query with with the requests library: import requests headers = { 'Content-type': 'application/json', } data = """ {"target": { "ref_type": "...
user avatar
0 votes
1 answer
778 views

Can't get header location from HTTP request python [duplicate]

I want to get the value of header location in python. When I use requests, the GET request basically headers without Location. headers = {"content-type": "application/json", "...
Omar Abdelrazik's user avatar
0 votes
1 answer
48 views

Forge app problem downloading with Urllib2 Authentication failed because the remote party has closed the transport stream

I've successfully downloaded the file from BIM360 using Postman for testing. Now I'm trying to implement this request in Ironpython urllib2 (cause urllib2 is my only choice for this task in the ...
Marcus's user avatar
  • 21
0 votes
0 answers
37 views

How do I open all the links and save the images into a folder in a specific directory on my pc?

Code: import urllib.request from bs4 import BeautifulSoup from requests import get import requests import dload import pandas as pd pd.set_option('display.max_colwidth', None) week_11_picURL = "...
chickenbutt's user avatar
0 votes
1 answer
515 views

python3(lambda function) failing to send response back to cloud formation template

I have created a cloud formation template for cross s3-bucket object replication using lambda which is written in python, Below is the lambda code. import json import logging import signal import ...
Just curious's user avatar
0 votes
2 answers
6k views

urllib.error.HTTPError: HTTP Error 403: Forbidden with urllib.requests

I am trying to read an image URL from the internet and be able to get the image onto my machine via python, I used example used in this blog post https://www.geeksforgeeks.org/how-to-open-an-image-...
patricebailey1998's user avatar
0 votes
0 answers
283 views

<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed for python2 but not in python3

I have a server (RHEL 7.5) with both python2 (2.7.5) and python3 (3.7.9) When I run the below code it's passed successfully for python3 but failed for python2 try: import urllib2 #python2 ...
Shurik's user avatar
  • 562
0 votes
1 answer
957 views

Using python to login to a website that use cookies

Im trying to log on into this website (https://phishtank.org/login.php) to use it with python, but the site use cookies. I tried this: import urllib cookies = { 'PHPSESSID': '...
Richard's user avatar
  • 53
0 votes
0 answers
49 views

Any ideas on how to make it so that I dont get Error 429: Too Many Request

Any ideas on how to make it so that I dont get Error 429: Too Many Request Just a beginner here, I am trying to get ipData from my database which consists of Office365 Audit data.. Am i using the ...
Tech Marc's user avatar
0 votes
0 answers
260 views

Python - not sure if I'm connecting via FTP or SFTP to a server

I'm using urllib to make a request to a server. When the URL has the format ftp://ftp.piplatam.com//..., it runs OK. When the URL has the format ftp://sftp.piplatam.com//..., it runs OK. When the URL ...
Juan Gabriel Griggio's user avatar
3 votes
5 answers
3k views

ModuleNotFoundError: No module named 'urllib.request'; 'urllib' is not a package

I have the following code and i tried all i could find on stackoverflow but nothing worked. Here is the code: import urllib.request, urllib.parse, urllib.error fhand = urllib.request.urlopen('data....
Mihai's user avatar
  • 61
0 votes
2 answers
6k views

ModuleNotFoundError: No module named 'cookielib'

ModuleNotFoundError: No module named 'cookielib' and 'urllib2' import urllib, urllib2, cookielib username = 'yourusername' password = 'yourpassword' ck = cookielib.CookieJar() opener = urllib2....
Nghĩa Haru's user avatar
2 votes
1 answer
1k views

while reading json data from url using python gives error "urllib.error.HTTPError: HTTP Error 403: Forbidden"

with this code I am reading a URL and using the data for filtration but urllib could not work url = "myurl" response = urllib.request.urlopen(url) data = json.loads(response.read()) ...
divine's user avatar
  • 77
0 votes
1 answer
544 views

If condition on error message after response.read

response = urllib2.urlopen("http://example.com/Hi") html = response.read() Is it possible that when the above code is run while the Hi directory has been removed (or the server is not ...
User122's user avatar
  • 25
0 votes
1 answer
726 views

Web Scrape - How do I click options based on select name using contains value?

I'm trying to scrape the below select dropdown menu in order to get the text content. I cannot use the name as the "_P889O1" changes for each product I will be trying to extract the data ...
Andrew Glass's user avatar
1 vote
1 answer
374 views

How can I recognize a "parked free" website?

I have a long list with urls and some of them are "parked-free" by godaddy Is there any technical way to recognize such pages without opening them on browser? The page is technically live ...
IoaTzimas's user avatar
  • 10.5k
2 votes
2 answers
695 views

Python - How to make sure resource is closed

I want to close f before returning from the method. I added finally blocked but it needs initialization. What to initialize it with? def test_close_resource(url): try: f = urllib2.urlopen(...
user1298426's user avatar
  • 3,487
1 vote
0 answers
368 views

CERTIFICATE_VERIFY_FAILED in urllib2 but not requests, on Python 2.7.18

My longstanding Python 2.7.17 environment got corrupted, and so I installed Python 2.7.18 using the macOS 64-bit installer downloadable from python.org. Now I get errors when loading HTTPS sites ...
Iron Pillow's user avatar
  • 2,152
0 votes
1 answer
257 views

How to reconnect after WIFI connection is lost and restored later in Python urllib

I'm trying to make a python program that checks the title of a website every 2 seconds. My program works fine when the connection is good. But when I lose the wifi connection, the python program ...
Abhishek Mishra's user avatar
0 votes
1 answer
521 views

How to change source port and ip in urllib2 request in python 2.x

while sending the urlib2 request os is choosing the source ip and port by default, The requirement is we need to set a different port for each request using urllib2. please refer to the attached ...
Nandakishor Jahagirdar's user avatar
1 vote
0 answers
2k views

Using Python requests module to fetch URL behind VPN

I have few URLs behind my company VPN whose IP I can get. I want to create a connection to those URLs using request module of Python without switching on VPN. Using Proxy was one of the popular ...
PRATEEK's user avatar
  • 57
2 votes
0 answers
220 views

Output from HTTPConnection debug level when using python's http.client

I'm trying to get debug information from python's built in httpclient. In a console I tried from http.client import HTTPSConnection HTTPSConnection.debuglevel = 1 a_url = 'https://feeds.simplecast....
Bhaskar's user avatar
  • 594
0 votes
2 answers
341 views

How to pull the table data that is generated with javascript from a website using Python?

My goal is to get only the data from a specific table in a website. If possible, opened website could be run on the background preventing to pop-up. I have tried Get page generated with Javascript in ...
alper's user avatar
  • 2,918
1 vote
1 answer
114 views

Changing the Header of urllib2 to get different search results on google

Can I change the header on a google query to get the results I would get for mobile? I'm trying to get different results on google according to different headers in the library urllib2 or requests. I ...
grumpyp's user avatar
  • 432
0 votes
1 answer
458 views

Using urllib.request to write an image

I am trying to use this code to download an image from the given URL import urllib.request resource = urllib.request.urlretrieve("http://farm2.static.flickr.com/1184/1013364004_bcf87ed140.jpg&...
Onur-Andros Ozbek's user avatar
2 votes
1 answer
6k views

requests.exceptions.MissingSchema: Invalid URL Python API Get request

I am trying to pull the data using get request through the below URL and I'm getting below error. I'm not understanding what is the issue with the URL. Any help would be appreciated import requests ...
Rahul's user avatar
  • 467
1 vote
1 answer
1k views

Using urllib/urllib2 get a session cookie and use it to login to a final page

I need to use urllib/urllib2 libraries to login to a first website to retrieve session cookie that will allow me to log in to the proper, final website. Using requests library is pretty straight ...
hod's user avatar
  • 750
0 votes
0 answers
124 views

web scraping in Python using urllib: urllib.error.HTTPError: HTTP Error 404: Not Found

I had obtained this code for url extractor from github.I am quiet new to python and I tried to use requests instead of urllib as suggested in some answers but could not quiet figure it out. import re ...
maya's user avatar
  • 71
1 vote
2 answers
5k views

Python Urllib ContextualVersionConflict

Till Oct 20, it was fine. Oct 21, it failed with the below messages. Can any one help here command: /usr/local/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/...
Akbar B's user avatar
  • 43
0 votes
1 answer
39 views

How to exclude 404 pages from python downloading

This code works fine, but how do I stop 404 pages from downloading? Urllib requests always return 403 because the host does not allow python. Is there another way to detect if the file exists? import ...
LBDCOM12's user avatar
-2 votes
1 answer
178 views

Is there any modification to execute curl command in python 3.x?

I have a runnable code in python 2.x version for following shell command : curl -XPOST localhost:5055/parse -d '{"q":"tell me about more info on covid", "projects": "...
user34's user avatar
  • 103
3 votes
2 answers
15k views

Python certificate verify failed: unable to get local issuer certificate

I am a novice programmer so pardon my mistakes. I have written the below code to verify a list of Websites are still active and all my work is based off this problem statement. The script is able to ...
zulfi123786's user avatar
0 votes
1 answer
132 views

I'm trying to find a specific link in a page with a python script

I'm trying to figure out how to extract from a given site only the links with a specific text inside This is the program i use: from BeautifulSoup import BeautifulSoup import urllib2 import re ...
CyberJugger's user avatar
3 votes
0 answers
2k views

SSL: CERTIFICATE_VERIFY_FAILED certificate verify failed: Hostname mismatch, certificate is not valid for 'url''

I am trying to access Solr using urllib2 as instaructed here: https://lucene.apache.org/solr/guide/7_3/using-python.html using urllib but running into error raise URLError(err) urllib.error.URLError: &...
Tannu Priya's user avatar
0 votes
2 answers
859 views

geturl() in python urllib library not working

I'm using the urllib to build a simple web scraper and need to know if a url redirects after being opened with the urlopen method. I'm calling the geturl() method on the response object but getting ...
Zalman Kelber's user avatar
0 votes
1 answer
160 views

json.load(json_ob) results in error "JSONDecodeError: Expecting value: line 1 column 1 (char 0)"

I've created a json object by calling an API: import urllib.request as urllib2 import json api_key = '...api_key_here...' request_url = 'http://api....url_here....xml?attribute=XXXX&api_key=blah'...
code_to_joy's user avatar
0 votes
1 answer
103 views

Replacing Requests with urllib2

I have the following code using requests to generate CSRFToken import requests: import requests class TokenGenerator: def GenerateToken(self, uid, pwd): TokenGenerator.session = requests....
user13852938's user avatar
0 votes
0 answers
297 views

python3 - urllib - ssl - UnicodeDecodeError

I want to run this code. But error why is it error? import requests import ssl from urllib import request urlMain = "https://www.example.com/" urlLogin = "https://www.example.com/join/" istekBaba = ...
emre bey's user avatar
0 votes
1 answer
59 views

Python: CERTIFICATE_VERIFY_FAILED, Windows

I have a script that is throwing a CERTIFICATE_VERIFY_FAILED on one of the PCs available to me, but not the other. The certificate was updated last week. How do I check if there is a cached version of ...
brihp's user avatar
  • 1
1 vote
1 answer
764 views

Requests error only with one specific website : OS Error 10054 Connection aborted

I get this error only on this particular website. I have googled and read everything but none of them solved my problem. Very simple piece of code, can anybody figure out what's going on? Other ...
SCool's user avatar
  • 3,104

1
2 3 4 5
60