Questions tagged [jsoup]

Jsoup is a Java HTML parser for extracting and manipulating HTML data, using the best of DOM, CSS, and jQuery-like methods.

Filter by
Sorted by
Tagged with
0 votes
2 answers
59 views

Jsoup parsing weird behavior

I'm trying do parsing with jsoup in loop, but after several iterations the loop starts all over again like in parallel thread, why does this happen? Code of my method: public void parser(String type, ...
Teck's user avatar
  • 1
-1 votes
2 answers
95 views

Jsoup doesn't detect closing </div> tags in the list

I have an HTML string with <div style="position: relative;"> </div> that contains list of ~200 elements, like this: <div style="position: relative;"> <div ...
NikMAX's user avatar
  • 17
0 votes
1 answer
40 views

JSoup - Unable to retrieve tag of type video from Instagram

Firstly, let me say I am a backend developer and haven't tried to parse a HTML document in probably 15 years so grant me patience. Plus, I don't really know how Instagram works, which is why I'm ...
ecronin's user avatar
0 votes
2 answers
55 views

Java Jsoup extracting <td> tags

I need to collect all languages into my collection from this site. Maybe someone knows how can I extract all of them without the column "ISO-639 code"? I am using Jsoup for extracting HTML ...
Egor1705's user avatar
-1 votes
2 answers
49 views

Select element value from html via XPath

I've got a html element that looks like this: <p> <strong>Popular resonses: </strong> bat, butterfly, moth </p> Html contains about all elements with <p> tag. I need to ...
DeMmAge's user avatar
  • 123
0 votes
1 answer
76 views

Jsoup does not escape special characters (<, >) inside quoted Element attribute values

I got a surprise result when I tried to programmatically build an Element with attribute with value that contains < and >. Am I missing something? https://www.w3.org/TR/xml/#syntax The test: ...
ondruska's user avatar
  • 135
0 votes
1 answer
41 views

prevent Jsoup from converting utf-8 to utf-16

I would like the following to return the encoded bullet point the same way it was supplied: Jsoup.parse("<div>&#8226;</div>").text() but instead I get a utf-16 string out ...
DavesPlanet's user avatar
0 votes
0 answers
33 views

Using Jsoup to print the content of HTML td, but the print stops at new york

I want to view the entire values of the webpage and then print each value, currently, my code stops at New York. How can I print out the entire data? This is my webpage: "https://studyinthestates....
sam tada's user avatar
0 votes
0 answers
37 views

Xss prevention in JSP [duplicate]

XSS is well known vulnerability in JSP. There are lot of information on web (and on stack overflow) about preventing XSS in JSP. I have found printing user input with <c:out> is most commonly ...
LuMa's user avatar
  • 13
0 votes
1 answer
23 views

jsoup.clean replaces single quotes with double quotes. How can I prevent that?

For our application, we allow the user to enter HTML, that later is displayed for other users. For security reasons, we sanitize this HTML and do not allow the user to save HTML, that might not be ...
Matthias's user avatar
0 votes
0 answers
39 views

Replacing redundant HTML string using Jsoup, Pattern class and StringBuilder

I have a big html string in form of byte[] which size is about 59kb and I need to decrease it as much as I can. I manage to remove some redundant string and hit size of 36kb after parsing. However, I ...
Wrapper's user avatar
  • 814
1 vote
0 answers
204 views

Jsoup.connect is throwing 403 for valid login credential / cookie value

Below working code has suddenly started failing with error HTTP error fetching URL. Status=403, URL=[https://www.valueresearchonline.com/funds/26123/motilal-oswal-flexi-cap-fund-regular-plan] I have ...
vikramvi's user avatar
  • 3,398
0 votes
0 answers
17 views

Simple jsoup parsing and webview output

i am new to android studio and now i tried to make a simple app, that just get html code from page https://www.oishare/DCIM/100OLYMP/, replace http://oishare/DCIM/100OLYMP/FILE.html?/DCIM/100OLYMP/ ...
laguro's user avatar
  • 1
0 votes
0 answers
35 views

General sibling selector (tilda, ~) within :has() pseudoclass in JSoup doesn't work as expected

I have a problem with results returned by method select from library JSoup when used on a selector that contains general sibling selector placed within :has() pseudoclass. Let's say we have the ...
Snusmumriken's user avatar
-3 votes
1 answer
72 views

Jsoup parsing just output null

i making an app to get raw files from my olympus camera. i can connect to it via wifi and go to 192.168.0.10 or oishare/DCIM/100OLYMP to get list of pictures, but when i tap on some of it, i get ...
laguro's user avatar
  • 1
0 votes
0 answers
30 views

How can I scrape, crawl, collect results from Google search with JSoup

It already worked a few weeks before, but now I am getting a page with the choice to accept or reject cookies. My code: String useragent = "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, ...
Peter Graf's user avatar
0 votes
0 answers
28 views

Closing an HTML <p> tag without the added line break [duplicate]

I'm working on a java function that generates some html code for an email using a thymeleaf template. I then want to unit test the resulting html string using JSoup. The problem I'm running into is ...
pbuchheit's user avatar
  • 1,431
0 votes
1 answer
20 views

Disable JSoup whitespace removal

I'm working on a project where I need to test some html output. To do this, I am using JSoup to extract the elements I want to test against and running assertions on the output. The problem I'm ...
pbuchheit's user avatar
  • 1,431
2 votes
2 answers
76 views

How search in a website using Jsoup in kotlin

I want to use https://www.devicespecifications.com/ website in my application. There is a search frame in this website which user enters a phone model, then presses the enter key and navigates to ...
Sara-Omidvar's user avatar
0 votes
0 answers
28 views

Parsing site with csrf

Could you please advise on how to deal with parsing the dns-shop website (https://www.dns-shop.ru/)? I need to retrieve some data from the page using Jsoup. It's said that dns-shop has good protection ...
Михаил А's user avatar
0 votes
0 answers
19 views

Can I make JSOUP to click a button in android studio?

I've been searching to automatically click the web button after the webview loads the link BUTTON IMAGE Here if you see the photo above you can see two buttons, If the webview loads the link it will ...
Gines Frank Ellis's user avatar
2 votes
1 answer
100 views

HTTP error fetching URL. Status=500 while getting data from a website

I'm trying to get data from a website using Jsoup, the website is taking payload in Json ({"SEARCH_VALUE":"ab","STARTS_WITH_YN":false,"ACTIVE_ONLY_YN":false,&...
user avatar
-3 votes
1 answer
50 views

How to extract aria-label value using java?

This is example : String document = "<div class=\"tertiary\" if=\"tertiaryHyperlinks.length\"> \n" + " <a rel=\"nofollow\" data-...
ire_A's user avatar
  • 1
1 vote
1 answer
47 views

JSOUP find identical tag elements that are not separated by other tags

Code example: <body> <p>123124</p> <p>214565</p> <p>123333</p> <img src = "aboba"/> <p>faaaa</p> <p>...
Jake Morgan's user avatar
0 votes
0 answers
35 views

jsoup call is changing url parameter from "%26" to "%2526" during execution

I've below code Document doc = null; doc = Jsoup.connect(url). ignoreContentType(true). timeout(maxTimeout + (tryCount * ...
vikramvi's user avatar
  • 3,398
0 votes
2 answers
51 views

Getting an element's subtending text nodes (including indirect)

We have a need to retrieve all the subtending text nodes of an element, whether direct or indirect. The textNodes() method on the Element class is returning only the direct child text nodes, and not ...
Sandeep's user avatar
  • 1,253
0 votes
0 answers
34 views

read URL in JTextField (swing) and pass URL to JSoup

the user gives a URL in a JTextField (user interface made with Swing). I then read the URL and pass it to JSoup. This is my code: private class FileButtonHandler implements ActionListener { ...
user22322280's user avatar
0 votes
0 answers
14 views

Jsoup adds userAgent and still reports an error,how should I end it?

String url = "https://www.ozon.ru"; Connection connect = Jsoup.connect(url).timeout(10000).userAgent("Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20100101 Firefox/5.0"...
Qi Zhang's user avatar
0 votes
0 answers
71 views

403 – forbidden exception while using the Jsoup

I need to fetch the HTTP status code of this website "https://in.bookmyshow.com/". So I tried using Jsoup to fetch the status of this particular website but the site denies the request hence ...
Hari's user avatar
  • 1
1 vote
1 answer
98 views

Unable to get required data from paginated data while scraping a website for analysis

I'm scraping a website for analysis purpose which uses request verification token in its payload. I'm using JsoupBrowser() (library: "net.ruippeixotog" %% "scala-scraper" % "3....
Zaryab Ali's user avatar
0 votes
1 answer
57 views

Jsoup can't find existing element by class name

I'm trying to parse pastebin page with all params(likes, views_count, etc). And all fine except raw text. I know that there is a \<paste_id\>/raw, but I don't want to use it since I will have to ...
KostromDan's user avatar
0 votes
1 answer
17 views

Receiving issues with Android.Volley.Servererror when logcatting a volley attempt

I am trying to follow a tutorial that will build a very basic app that pulls a JSON from a server, I have been waylaid due to problems; I'm having no end of problems trying to pull from the server I ...
John B Gilbert's user avatar
-1 votes
1 answer
114 views

Java JSoup: Could not parse query ... unexpected token at /

I'm learning jsoup using a news article about football (https://www.bbc.com/sport/football/66241967). My goal is to extract the text body of the news article. Since the text body of the article seems ...
Robbie's user avatar
  • 45
0 votes
1 answer
27 views

Java jsoup - all links belonging to a specific classes - no output, no error message

I'm learning jsoup in Java. I'd like to print all links belonging to a specific classes. I'm not getting any output when my expected output is my beutiful link : /sport/golf/66227692\. I've looked at ...
Robbie's user avatar
  • 45
0 votes
1 answer
953 views

Getting "Request unsuccessful. Incapsula incident ID: 1220001110238789559-263723949970693063" while scraping website [closed]

I'm trying to scrape website using scala scraper. Code: val document = JsoupBrowser().get("https://esos.nv.gov/NotarySearchOnline/NotarySearchExternal") println(document) But it is ...
Zaryab Ali's user avatar
-1 votes
1 answer
42 views

how to find out AJAX network call url populating particular field on a webpage?

On the page https://www.moneycontrol.com/india/stockpricequote/banks-private-sector/karurvysyabank/KVB TTM PE data gets fetched a bit slower and till that time data is shown as "--" instead. ...
vikramvi's user avatar
  • 3,398
0 votes
0 answers
34 views

Whatsapp web in webview by using Jsoup

Unable to load Whatsapp web URL using android Jsoup library and here I am using this code. webView = findViewById(R.id.webview); WebSettings webSettings = webView.getSettings(); ...
Malik Farooq's user avatar
0 votes
0 answers
64 views

JSoup Request with "X-Requested-With" = "XMLHttpRequest" Header - Not Working

I am using jsoup to scrape a web site that requires a login. I have had no issue scraping several different types of pages this way, passing in the cookies I obtain on login. On one page, an ajax ...
user1821431's user avatar
0 votes
0 answers
44 views

JSONObject(doc.text()) not working in a particular case

I'm trying to parse HTML output by converting it into JSON object as below JSONObject json = new JSONObject(doc.text()); But it's working in first case and other case throwing error as Method threw '...
vikramvi's user avatar
  • 3,398
1 vote
1 answer
90 views

Vaadin 14.10.1 error in flow-server 2.9.2 with java.lang.NoSuchMethodError: org.jsoup.nodes.Element.traverse(org.jsoup.select.NodeVisitor)

I have recently updated the vaadin version from 14.4.4 to 14.10.1 due to https://github.com/vaadin/flow/issues/16905 and now i am facing a new issue in flow-server 2.9.2 in class at line number 244 ...
Abhishek-M's user avatar
0 votes
0 answers
19 views

Stop Jsoup document toString method from outputting nodes was not in the input string passed to the html parser

We use jsoup html parser to parse a malformed document, such as <div><span>span 1</span><span>span 2</span> Note that the html document is malformed, missing the trailing ...
Cmak Mak's user avatar
1 vote
0 answers
43 views

Jsoup gives the same document from two different urls

I am making a java aplication for kakuro and I wanted to scrape the boards from a site. Now the dimensions "3x3", and "4x4" don't work but in the meantime "5x5", "...
Clipz's user avatar
  • 11
2 votes
1 answer
82 views

What does `:first-of-type` really mean?

Given the following HTML: <p>paragraph text 1</p> <p>paragraph text 1</p> <div class="heading-h3">Category Title 1</div> <p>1. <a href="#...
Introspective's user avatar
1 vote
0 answers
49 views

Jsoup: different Document structure while loading HTML from URL and from file

I'm using Jsoup to parse data from website. I need to parse data both from a URL and from a local HTML file, but the contents of the Document for the same web page, obtained from the link and from its ...
NikMAX's user avatar
  • 17
0 votes
0 answers
22 views

Why is the same http post request failing with jsoup but working with AsyncHttpClient?

want to get the data from reddit profile pages. Some of these pages are tagged as nsfw, so users have to click accept in gui. (e.g. https://old.reddit.com/user/3DigitIQ) After clicking the page send a ...
CampingCow's user avatar
-1 votes
1 answer
36 views

Jsoup 1.16.1 is throwing error with Jetty 11.0.15

Jsoup 1.16.1 is throwing error with Jetty 11.0.15. [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.11.0:testCompile (default-testCompile) on project jsoup: Compilation ...
ASI's user avatar
  • 11
0 votes
1 answer
112 views

How to scrap data from website that uses cookies using scala scraper?

I'm trying to scrap data from website (https://www.notary.org/find-notary-in-pa) but getting 403 (forbidden). I'm using scala version 2.13.1, sbt version 1.2.8 and libraries ("net.ruippeixotog&...
Zaryab Ali's user avatar
0 votes
0 answers
27 views

How to switch between pages in JSOUP, if pagination jquery

There is a service that sends messages from the database. I want to import all clients, but the service doesn't provide such functionality. I thought that it is possible to do parsing with the help of ...
Aleksey T.'s user avatar
0 votes
0 answers
123 views

Not able to scrap data from a website that contains cookies

I'm trying to scrap data from site (https://www.notary.org/find-notary-in-pa), but when I scrap it, it don't provide all data. It is just showing some tags related to Cookies and giving 403 Forbidden ...
Zaryab Ali's user avatar
0 votes
2 answers
457 views

How to sanitize window objects to prevent reflected XSS attacks in java

I'm writing servlet-based application in which I need to handle the XSS vernability. I've implemented following logic to sanitize input using ESAPI and JSOUP library. For each request I am getting ...
Coder's user avatar
  • 410

1
2 3 4 5
136