Questions tagged [urllib2]
urllib2 is a builtin python 2 module that defines functions and classes to help with URL actions. It is notably unsatisfactory and has been replaced in python 3 and by third-party libraries.
2,960
questions
0
votes
0
answers
14
views
Receiving an error when attempting to open a url link with urlopen [duplicate]
I am attempting to run this portion of my code that used to work before, but now I get the following error, urllib2.URLError: <urlopen error [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol ...
0
votes
0
answers
105
views
I'm getting error "urllib.error.HTTPError: HTTP Error 400: Bad Request" for programmatic access to NCBI Entrez
I'm trying to get sequence information by parsing and reading files by biopython, I have to use efetch and I'm making a feather file format for speed and to save disk space. I know the usage ...
1
vote
1
answer
54
views
Python urllib2 -> urllib3 request.urlopen
I need to switch from urllib2 to urllib3. There is a problem with a request.
python2 code:
reqdata='{"PM1OBJ1":{"FREQ":"","U_AC":"","I_AC":&...
0
votes
0
answers
126
views
how to automate authentication SAML request using urllib or selenium with python?
unable to reach required website as per the below code , authentication request page is popped up and unable move next
I tried using selenium python but unable to automate SAML authentication page ...
0
votes
1
answer
131
views
urllib.error.HTTPError: HTTP Error 404: Not Found Issue when Trying to read url from Yahoo Finance given Stock Symbol
I am trying to write a program that will help me with accounting work. I am trying to pull information by using stock prices for companies over time. I am utilizing urllib.request to read and open ...
0
votes
0
answers
34
views
Read binary data response as uint16array using urllib2 in python 3.9
I am trying to get url response in a uint16array buffer where I am using urllib for http communication in python3.9.
I tried np.frombuffer(url.read(),dtype=np.uint16) but its not giving correct array.
...
0
votes
1
answer
165
views
How do I pass a user-agent to panda's pd.read_html()?
some websites automatically decline requests due to lack of user-agent, and it's a hassle using bs4 to scrape many different types of tables.
This issue was resolved before through this code:
url = '...
0
votes
0
answers
597
views
Get Network Error (dns_unresolved_hostname) using Requests, Urllib and Pandas
Trying to call a URL and get an HTML table from an internal server on network.
When entering a URL into a browser, the page and table load.
When calling URL in Python using Requests, URLLIB, and ...
0
votes
1
answer
2k
views
Why does I get SSLV3_ALERT_HANDSHAKE_FAILURE error when requesting a website?
My code is below:
import urllib.request
import urllib.parse
from lxml import etree
HEADERS = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko)...
-1
votes
1
answer
33
views
get status_code for http.urlopen
How can I print the status code for this response? (eg 200/401 etc)
resp = http.urlopen('POST', 'https://api.bitbucket.org/2.0/repositories/6789oh', headers=headers, body=json.dumps(data))
print(str(...
0
votes
1
answer
2k
views
add authentication info for urllib.request
I had this query with with the requests library:
import requests
headers = {
'Content-type': 'application/json',
}
data = """
{"target": {
"ref_type": "...
0
votes
1
answer
778
views
Can't get header location from HTTP request python [duplicate]
I want to get the value of header location in python.
When I use requests, the GET request basically headers without Location.
headers = {"content-type": "application/json", "...
0
votes
1
answer
48
views
Forge app problem downloading with Urllib2 Authentication failed because the remote party has closed the transport stream
I've successfully downloaded the file from BIM360 using Postman for testing. Now I'm trying to implement this request in Ironpython urllib2 (cause urllib2 is my only choice for this task in the ...
0
votes
0
answers
37
views
How do I open all the links and save the images into a folder in a specific directory on my pc?
Code:
import urllib.request
from bs4 import BeautifulSoup
from requests import get
import requests
import dload
import pandas as pd
pd.set_option('display.max_colwidth', None)
week_11_picURL = "...
0
votes
1
answer
515
views
python3(lambda function) failing to send response back to cloud formation template
I have created a cloud formation template for cross s3-bucket object replication using lambda which is written in python, Below is the lambda code.
import json
import logging
import signal
import ...
0
votes
2
answers
6k
views
urllib.error.HTTPError: HTTP Error 403: Forbidden with urllib.requests
I am trying to read an image URL from the internet and be able to get the image onto my machine via python, I used example used in this blog post https://www.geeksforgeeks.org/how-to-open-an-image-...
0
votes
0
answers
283
views
<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed for python2 but not in python3
I have a server (RHEL 7.5) with both python2 (2.7.5) and python3 (3.7.9)
When I run the below code it's passed successfully for python3 but failed for python2
try:
import urllib2 #python2
...
0
votes
1
answer
957
views
Using python to login to a website that use cookies
Im trying to log on into this website (https://phishtank.org/login.php) to use it with python, but the site use cookies. I tried this:
import urllib
cookies = {
'PHPSESSID': '...
0
votes
0
answers
49
views
Any ideas on how to make it so that I dont get Error 429: Too Many Request
Any ideas on how to make it so that I dont get Error 429: Too Many Request
Just a beginner here, I am trying to get ipData from my database which consists of Office365 Audit data..
Am i using the ...
0
votes
0
answers
260
views
Python - not sure if I'm connecting via FTP or SFTP to a server
I'm using urllib to make a request to a server.
When the URL has the format ftp://ftp.piplatam.com//..., it runs OK.
When the URL has the format ftp://sftp.piplatam.com//..., it runs OK.
When the URL ...
3
votes
5
answers
3k
views
ModuleNotFoundError: No module named 'urllib.request'; 'urllib' is not a package
I have the following code and i tried all i could find on stackoverflow but nothing worked. Here is the code:
import urllib.request, urllib.parse, urllib.error
fhand = urllib.request.urlopen('data....
0
votes
2
answers
6k
views
ModuleNotFoundError: No module named 'cookielib'
ModuleNotFoundError: No module named 'cookielib' and 'urllib2'
import urllib, urllib2, cookielib
username = 'yourusername'
password = 'yourpassword'
ck = cookielib.CookieJar()
opener = urllib2....
2
votes
1
answer
1k
views
while reading json data from url using python gives error "urllib.error.HTTPError: HTTP Error 403: Forbidden"
with this code I am reading a URL and using the data for filtration but urllib could not work
url = "myurl"
response = urllib.request.urlopen(url)
data = json.loads(response.read())
...
0
votes
1
answer
544
views
If condition on error message after response.read
response = urllib2.urlopen("http://example.com/Hi")
html = response.read()
Is it possible that when the above code is run while the Hi directory has been removed (or the server is not ...
0
votes
1
answer
726
views
Web Scrape - How do I click options based on select name using contains value?
I'm trying to scrape the below select dropdown menu in order to get the text content.
I cannot use the name as the "_P889O1" changes for each product I will be trying to extract the data ...
1
vote
1
answer
374
views
How can I recognize a "parked free" website?
I have a long list with urls and some of them are "parked-free" by godaddy
Is there any technical way to recognize such pages without opening them on browser?
The page is technically live ...
2
votes
2
answers
695
views
Python - How to make sure resource is closed
I want to close f before returning from the method. I added finally blocked but it needs initialization. What to initialize it with?
def test_close_resource(url):
try:
f = urllib2.urlopen(...
1
vote
0
answers
368
views
CERTIFICATE_VERIFY_FAILED in urllib2 but not requests, on Python 2.7.18
My longstanding Python 2.7.17 environment got corrupted, and so I installed Python 2.7.18 using the macOS 64-bit installer downloadable from python.org. Now I get errors when loading HTTPS sites ...
0
votes
1
answer
257
views
How to reconnect after WIFI connection is lost and restored later in Python urllib
I'm trying to make a python program that checks the title of a website every 2 seconds. My program works
fine when the connection is good. But when I lose the wifi connection, the python program ...
0
votes
1
answer
521
views
How to change source port and ip in urllib2 request in python 2.x
while sending the urlib2 request os is choosing the source ip and port by default,
The requirement is we need to set a different port for each request using urllib2.
please refer to the attached ...
1
vote
0
answers
2k
views
Using Python requests module to fetch URL behind VPN
I have few URLs behind my company VPN whose IP I can get. I want to create a connection to those URLs using request module of Python without switching on VPN.
Using Proxy was one of the popular ...
2
votes
0
answers
220
views
Output from HTTPConnection debug level when using python's http.client
I'm trying to get debug information from python's built in httpclient. In a console I tried
from http.client import HTTPSConnection
HTTPSConnection.debuglevel = 1
a_url = 'https://feeds.simplecast....
0
votes
2
answers
341
views
How to pull the table data that is generated with javascript from a website using Python?
My goal is to get only the data from a specific table in a website. If possible, opened website could be run on the background preventing to pop-up.
I have tried Get page generated with Javascript in ...
1
vote
1
answer
114
views
Changing the Header of urllib2 to get different search results on google
Can I change the header on a google query to get the results I would get for mobile?
I'm trying to get different results on google according to different headers in the library urllib2 or requests. I ...
0
votes
1
answer
458
views
Using urllib.request to write an image
I am trying to use this code to download an image from the given URL
import urllib.request
resource = urllib.request.urlretrieve("http://farm2.static.flickr.com/1184/1013364004_bcf87ed140.jpg&...
2
votes
1
answer
6k
views
requests.exceptions.MissingSchema: Invalid URL Python API Get request
I am trying to pull the data using get request through the below URL and I'm getting below error.
I'm not understanding what is the issue with the URL.
Any help would be appreciated
import requests
...
1
vote
1
answer
1k
views
Using urllib/urllib2 get a session cookie and use it to login to a final page
I need to use urllib/urllib2 libraries to login to a first website to retrieve session cookie that will allow me to log in to the proper, final website. Using requests library is pretty straight ...
0
votes
0
answers
124
views
web scraping in Python using urllib: urllib.error.HTTPError: HTTP Error 404: Not Found
I had obtained this code for url extractor from github.I am quiet new to python and I tried to use requests instead of urllib as suggested in some answers but could not quiet figure it out.
import re
...
1
vote
2
answers
5k
views
Python Urllib ContextualVersionConflict
Till Oct 20, it was fine. Oct 21, it failed with the below messages. Can any one help here
command: /usr/local/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/...
0
votes
1
answer
39
views
How to exclude 404 pages from python downloading
This code works fine, but how do I stop 404 pages from downloading? Urllib requests always return 403 because the host does not allow python. Is there another way to detect if the file exists?
import ...
-2
votes
1
answer
178
views
Is there any modification to execute curl command in python 3.x?
I have a runnable code in python 2.x version for following shell command :
curl -XPOST localhost:5055/parse -d '{"q":"tell me about more info on covid", "projects": "...
3
votes
2
answers
15k
views
Python certificate verify failed: unable to get local issuer certificate
I am a novice programmer so pardon my mistakes. I have written the below code to verify a list of Websites are still active and all my work is based off this problem statement.
The script is able to ...
0
votes
1
answer
132
views
I'm trying to find a specific link in a page with a python script
I'm trying to figure out how to extract from a given site only the links with a specific text inside
This is the program i use:
from BeautifulSoup import BeautifulSoup
import urllib2
import re
...
3
votes
0
answers
2k
views
SSL: CERTIFICATE_VERIFY_FAILED certificate verify failed: Hostname mismatch, certificate is not valid for 'url''
I am trying to access Solr using urllib2 as instaructed here: https://lucene.apache.org/solr/guide/7_3/using-python.html using urllib
but running into error raise URLError(err) urllib.error.URLError: &...
0
votes
2
answers
859
views
geturl() in python urllib library not working
I'm using the urllib to build a simple web scraper and need to know if a url redirects after being opened with the urlopen method. I'm calling the geturl() method on the response object but getting ...
0
votes
1
answer
160
views
json.load(json_ob) results in error "JSONDecodeError: Expecting value: line 1 column 1 (char 0)"
I've created a json object by calling an API:
import urllib.request as urllib2
import json
api_key = '...api_key_here...'
request_url = 'http://api....url_here....xml?attribute=XXXX&api_key=blah'...
0
votes
1
answer
103
views
Replacing Requests with urllib2
I have the following code using requests to generate CSRFToken import requests:
import requests
class TokenGenerator:
def GenerateToken(self, uid, pwd):
TokenGenerator.session = requests....
0
votes
0
answers
297
views
python3 - urllib - ssl - UnicodeDecodeError
I want to run this code. But error why is it error?
import requests
import ssl
from urllib import request
urlMain = "https://www.example.com/"
urlLogin = "https://www.example.com/join/"
istekBaba = ...
0
votes
1
answer
59
views
Python: CERTIFICATE_VERIFY_FAILED, Windows
I have a script that is throwing a CERTIFICATE_VERIFY_FAILED on one of the PCs available to me, but not the other. The certificate was updated last week. How do I check if there is a cached version of ...
1
vote
1
answer
764
views
Requests error only with one specific website : OS Error 10054 Connection aborted
I get this error only on this particular website.
I have googled and read everything but none of them solved my problem.
Very simple piece of code, can anybody figure out what's going on? Other ...