0

I am scrapping Walmart page at URL http://www.walmart.com/search/search-ng.do?tab_value=all&search_query=camera&search_constraint=0&Find=Find&ss=false&ic=16_32 using JSOUP DOM parser in JAVA.

I am building URL based on user parameters and building a DOM object using

Document doc = Jsoup.parse(contentVar); 

For the next step i want to print all the products/price. I used the following code:

String price = doc.getElementsByClass("camelPrice").text();
String title = doc.getElementsByClass("ListItemLink").text();       
System.out.println("Product: " + title);
System.out.println("Price: "+ price);

Here i am using the tags for the price and product description. However my results are :

Title/Product Name: C1, C2, ... C16 (c is camera title)
Price: $279.95 $279.95 $479.00 $479.00 $60.00 $60.00 $99.00 $99.00 $429.00 $429.00 $129.00 $129.00 $109.00 $109.00 $89.00 $89.00 $384.00 $384.00 $69.00 $69.00 $279.00 $279.00 $129.00 $129.00 $55.20 - $69.00 $55.20 - $69.00 $74.00 $74.00 $119.00 $119.00

here the prices are duplicated because of a possible quickview tag. Is there any way to remove the duplicacy in prices using any JSOUP method

2 Answers 2

2

Well seeing the html dom I noticed that there are duplicates in the sense that there is a price

<div class="ItemShelfAvail">     <----------- SEE HERE
<div class="OnlinePriceAvail">
<div class="PriceHeader OnlineHead">Online</div>
<div class="PriceContent">
<div class="PriceDisplay" id="price_display_23204350_2">
<div class="PriceCompare">
<div class="camelPrice"><span class="prefixPriceText2"></span><span class="bigPriceText2">$279.</span><span class="smallPriceText2">00</span><span></span></div>

and a price

<div class="OnlinePriceAvail">
<div class="PriceHeader OnlineHead">Online</div>
<div class="PriceContent">
<div class="PriceDisplay" id="price_display_23204350_2">
<div class="PriceCompare">
<div class="camelPrice"><span class="prefixPriceText2"></span><span class="bigPriceText2">$279.</span><span class="smallPriceText2">00</span><span></span></div>

You must see what list you want from the two and then put a proper selector. If you want both of them just take the Elements list returned by the getElementsByClass and manipulate each price.

getElementsByClass returns Elements which is a list where every node is of type Element. You can do

Elements elPrice = doc.getElementsByClass("camelPrice");
1

I know that this may now be useless to the creator of the thread but I found this when looking on how to find the price of a product on amazon uk.

String pricing = doc.getElementsByClass("priceLarge").text();
System.out.println("price : " + pricing);

Here is the code to do it :)

1
  • Yes it is not very useful now. Appreciate your response :)
    – CodeMonkey
    Jun 4, 2013 at 14:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.