Questions tagged [jsoup]
Jsoup is a Java HTML parser for extracting and manipulating HTML data, using the best of DOM, CSS, and jQuery-like methods.
6,795
questions
0
votes
2
answers
59
views
Jsoup parsing weird behavior
I'm trying do parsing with jsoup in loop, but after several iterations the loop starts all over again like in parallel thread, why does this happen?
Code of my method:
public void parser(String type, ...
-1
votes
2
answers
95
views
Jsoup doesn't detect closing </div> tags in the list
I have an HTML string with <div style="position: relative;"> </div> that contains list of ~200 elements, like this:
<div style="position: relative;">
<div ...
0
votes
1
answer
40
views
JSoup - Unable to retrieve tag of type video from Instagram
Firstly, let me say I am a backend developer and haven't tried to parse a HTML document in probably 15 years so grant me patience. Plus, I don't really know how Instagram works, which is why I'm ...
0
votes
2
answers
55
views
Java Jsoup extracting <td> tags
I need to collect all languages into my collection from this site.
Maybe someone knows how can I extract all of them without the column "ISO-639 code"?
I am using Jsoup for extracting HTML ...
-1
votes
2
answers
49
views
Select element value from html via XPath
I've got a html element that looks like this:
<p>
<strong>Popular resonses: </strong>
bat, butterfly, moth
</p>
Html contains about all elements with <p> tag.
I need to ...
0
votes
1
answer
76
views
Jsoup does not escape special characters (<, >) inside quoted Element attribute values
I got a surprise result when I tried to programmatically build an Element with attribute with value that contains < and >.
Am I missing something? https://www.w3.org/TR/xml/#syntax
The test:
...
0
votes
1
answer
41
views
prevent Jsoup from converting utf-8 to utf-16
I would like the following to return the encoded bullet point the same way it was supplied:
Jsoup.parse("<div>•</div>").text()
but instead I get a utf-16 string out ...
0
votes
0
answers
33
views
Using Jsoup to print the content of HTML td, but the print stops at new york
I want to view the entire values of the webpage and then print each value, currently, my code stops at New York. How can I print out the entire data?
This is my webpage:
"https://studyinthestates....
0
votes
0
answers
37
views
Xss prevention in JSP [duplicate]
XSS is well known vulnerability in JSP. There are lot of information on web (and on stack overflow) about preventing XSS in JSP.
I have found printing user input with <c:out> is most commonly ...
0
votes
1
answer
23
views
jsoup.clean replaces single quotes with double quotes. How can I prevent that?
For our application, we allow the user to enter HTML, that later is displayed for other users.
For security reasons, we sanitize this HTML and do not allow the user to save HTML, that might not be ...
0
votes
0
answers
39
views
Replacing redundant HTML string using Jsoup, Pattern class and StringBuilder
I have a big html string in form of byte[] which size is about 59kb and I need to decrease it as much as I can. I manage to remove some redundant string and hit size of 36kb after parsing. However, I ...
1
vote
0
answers
204
views
Jsoup.connect is throwing 403 for valid login credential / cookie value
Below working code has suddenly started failing with error
HTTP error fetching URL. Status=403, URL=[https://www.valueresearchonline.com/funds/26123/motilal-oswal-flexi-cap-fund-regular-plan]
I have ...
0
votes
0
answers
17
views
Simple jsoup parsing and webview output
i am new to android studio and now i tried to make a simple app, that just get html code from page https://www.oishare/DCIM/100OLYMP/, replace http://oishare/DCIM/100OLYMP/FILE.html?/DCIM/100OLYMP/ ...
0
votes
0
answers
35
views
General sibling selector (tilda, ~) within :has() pseudoclass in JSoup doesn't work as expected
I have a problem with results returned by method select from library JSoup when used on a selector that contains general sibling selector placed within :has() pseudoclass.
Let's say we have the ...
-3
votes
1
answer
72
views
Jsoup parsing just output null
i making an app to get raw files from my olympus camera.
i can connect to it via wifi and go to 192.168.0.10 or oishare/DCIM/100OLYMP to get list of pictures, but when i tap on some of it, i get ...
0
votes
0
answers
30
views
How can I scrape, crawl, collect results from Google search with JSoup
It already worked a few weeks before, but now I am getting a page with the choice to accept or reject cookies.
My code:
String useragent = "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, ...
0
votes
0
answers
28
views
Closing an HTML <p> tag without the added line break [duplicate]
I'm working on a java function that generates some html code for an email using a thymeleaf template. I then want to unit test the resulting html string using JSoup. The problem I'm running into is ...
0
votes
1
answer
20
views
Disable JSoup whitespace removal
I'm working on a project where I need to test some html output. To do this, I am using JSoup to extract the elements I want to test against and running assertions on the output. The problem I'm ...
2
votes
2
answers
76
views
How search in a website using Jsoup in kotlin
I want to use https://www.devicespecifications.com/ website in my application. There is a search frame in this website which user enters a phone model, then presses the enter key and navigates to ...
0
votes
0
answers
28
views
Parsing site with csrf
Could you please advise on how to deal with parsing the dns-shop website (https://www.dns-shop.ru/)? I need to retrieve some data from the page using Jsoup. It's said that dns-shop has good protection ...
0
votes
0
answers
19
views
Can I make JSOUP to click a button in android studio?
I've been searching to automatically click the web button after the webview loads the link
BUTTON IMAGE
Here if you see the photo above you can see two buttons, If the webview loads the link it will ...
2
votes
1
answer
100
views
HTTP error fetching URL. Status=500 while getting data from a website
I'm trying to get data from a website using Jsoup, the website is taking payload in Json ({"SEARCH_VALUE":"ab","STARTS_WITH_YN":false,"ACTIVE_ONLY_YN":false,&...
-3
votes
1
answer
50
views
How to extract aria-label value using java?
This is example :
String document = "<div class=\"tertiary\" if=\"tertiaryHyperlinks.length\"> \n" +
" <a rel=\"nofollow\" data-...
1
vote
1
answer
47
views
JSOUP find identical tag elements that are not separated by other tags
Code example:
<body>
<p>123124</p>
<p>214565</p>
<p>123333</p>
<img src = "aboba"/>
<p>faaaa</p>
<p>...
0
votes
0
answers
35
views
jsoup call is changing url parameter from "%26" to "%2526" during execution
I've below code
Document doc = null;
doc = Jsoup.connect(url).
ignoreContentType(true).
timeout(maxTimeout + (tryCount * ...
0
votes
2
answers
51
views
Getting an element's subtending text nodes (including indirect)
We have a need to retrieve all the subtending text nodes of an element, whether direct or indirect. The textNodes() method on the Element class is returning only the direct child text nodes, and not ...
0
votes
0
answers
34
views
read URL in JTextField (swing) and pass URL to JSoup
the user gives a URL in a JTextField (user interface made with Swing). I then read the URL and pass it to JSoup.
This is my code:
private class FileButtonHandler implements ActionListener {
...
0
votes
0
answers
14
views
Jsoup adds userAgent and still reports an error,how should I end it?
String url = "https://www.ozon.ru";
Connection connect = Jsoup.connect(url).timeout(10000).userAgent("Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20100101 Firefox/5.0"...
0
votes
0
answers
71
views
403 – forbidden exception while using the Jsoup
I need to fetch the HTTP status code of this website "https://in.bookmyshow.com/". So I tried using Jsoup to fetch the status of this particular website but the site denies the request hence ...
1
vote
1
answer
98
views
Unable to get required data from paginated data while scraping a website for analysis
I'm scraping a website for analysis purpose which uses request verification token in its payload. I'm using JsoupBrowser() (library: "net.ruippeixotog" %% "scala-scraper" % "3....
0
votes
1
answer
57
views
Jsoup can't find existing element by class name
I'm trying to parse pastebin page with all params(likes, views_count, etc). And all fine except raw text.
I know that there is a \<paste_id\>/raw, but I don't want to use it since I will have to ...
0
votes
1
answer
17
views
Receiving issues with Android.Volley.Servererror when logcatting a volley attempt
I am trying to follow a tutorial that will build a very basic app that pulls a JSON from a server, I have been waylaid due to problems; I'm having no end of problems trying to pull from the server
I ...
-1
votes
1
answer
114
views
Java JSoup: Could not parse query ... unexpected token at /
I'm learning jsoup using a news article about football (https://www.bbc.com/sport/football/66241967).
My goal is to extract the text body of the news article.
Since the text body of the article seems ...
0
votes
1
answer
27
views
Java jsoup - all links belonging to a specific classes - no output, no error message
I'm learning jsoup in Java.
I'd like to print all links belonging to a specific classes.
I'm not getting any output when my expected output is my beutiful link : /sport/golf/66227692\.
I've looked at ...
0
votes
1
answer
953
views
Getting "Request unsuccessful. Incapsula incident ID: 1220001110238789559-263723949970693063" while scraping website [closed]
I'm trying to scrape website using scala scraper.
Code:
val document = JsoupBrowser().get("https://esos.nv.gov/NotarySearchOnline/NotarySearchExternal")
println(document)
But it is ...
-1
votes
1
answer
42
views
how to find out AJAX network call url populating particular field on a webpage?
On the page https://www.moneycontrol.com/india/stockpricequote/banks-private-sector/karurvysyabank/KVB
TTM PE data gets fetched a bit slower and till that time data is shown as "--" instead.
...
0
votes
0
answers
34
views
Whatsapp web in webview by using Jsoup
Unable to load Whatsapp web URL using android Jsoup library and here I am using this code.
webView = findViewById(R.id.webview);
WebSettings webSettings = webView.getSettings();
...
0
votes
0
answers
64
views
JSoup Request with "X-Requested-With" = "XMLHttpRequest" Header - Not Working
I am using jsoup to scrape a web site that requires a login. I have had no issue scraping several different types of pages this way, passing in the cookies I obtain on login.
On one page, an ajax ...
0
votes
0
answers
44
views
JSONObject(doc.text()) not working in a particular case
I'm trying to parse HTML output by converting it into JSON object as below
JSONObject json = new JSONObject(doc.text());
But it's working in first case and other case throwing error
as Method threw '...
1
vote
1
answer
90
views
Vaadin 14.10.1 error in flow-server 2.9.2 with java.lang.NoSuchMethodError: org.jsoup.nodes.Element.traverse(org.jsoup.select.NodeVisitor)
I have recently updated the vaadin version from 14.4.4 to 14.10.1 due to https://github.com/vaadin/flow/issues/16905 and now i am facing a new issue in flow-server 2.9.2 in class at line number 244 ...
0
votes
0
answers
19
views
Stop Jsoup document toString method from outputting nodes was not in the input string passed to the html parser
We use jsoup html parser to parse a malformed document, such as
<div><span>span 1</span><span>span 2</span>
Note that the html document is malformed, missing the trailing ...
1
vote
0
answers
43
views
Jsoup gives the same document from two different urls
I am making a java aplication for kakuro and I wanted to scrape the boards from a site. Now the dimensions "3x3", and "4x4" don't work but in the meantime "5x5", "...
2
votes
1
answer
82
views
What does `:first-of-type` really mean?
Given the following HTML:
<p>paragraph text 1</p>
<p>paragraph text 1</p>
<div class="heading-h3">Category Title 1</div>
<p>1. <a href="#...
1
vote
0
answers
49
views
Jsoup: different Document structure while loading HTML from URL and from file
I'm using Jsoup to parse data from website.
I need to parse data both from a URL and from a local HTML file, but the contents of the Document for the same web page, obtained from the link and from its ...
0
votes
0
answers
22
views
Why is the same http post request failing with jsoup but working with AsyncHttpClient?
want to get the data from reddit profile pages. Some of these pages are tagged as nsfw, so users have to click accept in gui. (e.g. https://old.reddit.com/user/3DigitIQ) After clicking the page send a ...
-1
votes
1
answer
36
views
Jsoup 1.16.1 is throwing error with Jetty 11.0.15
Jsoup 1.16.1 is throwing error with Jetty 11.0.15.
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.11.0:testCompile (default-testCompile) on project jsoup: Compilation ...
0
votes
1
answer
112
views
How to scrap data from website that uses cookies using scala scraper?
I'm trying to scrap data from website (https://www.notary.org/find-notary-in-pa) but getting 403 (forbidden).
I'm using scala version 2.13.1, sbt version 1.2.8 and libraries ("net.ruippeixotog&...
0
votes
0
answers
27
views
How to switch between pages in JSOUP, if pagination jquery
There is a service that sends messages from the database. I want to import all clients, but the service doesn't provide such functionality. I thought that it is possible to do parsing with the help of ...
0
votes
0
answers
123
views
Not able to scrap data from a website that contains cookies
I'm trying to scrap data from site (https://www.notary.org/find-notary-in-pa), but when I scrap it, it don't provide all data. It is just showing some tags related to Cookies and giving 403 Forbidden ...
0
votes
2
answers
457
views
How to sanitize window objects to prevent reflected XSS attacks in java
I'm writing servlet-based application in which I need to handle the XSS vernability. I've implemented following logic to sanitize input using ESAPI and JSOUP library.
For each request I am getting ...