1

I am trying to scrape data from two websites. I want to build a generic web parser to scrape those website. I need a generic parser because i might need to parse additional websites in the future. I want to parse the websites using java and store specific data in MySQL. What are the steps I should take? Can anyone guide me in this?

1
  • 1
    This is much too broad, and you haven't indicated you've done any work on the problem yet. SO is intended to answer specific questions, not provide general tutorial/education on broad topics such as this. Dec 6, 2013 at 21:16

1 Answer 1

2

I've done this same thing for scraping the web using java and building a parser is an absolute nightmare, there are so many more things to consider that you realize at the start.

Use jSoup. It is sooo easy, and leaves you time to enjoy the finer things in life.

3
  • 1
    Yes, that's what I plan to use. I think using Jsoup and regular expressions should make it a bit easy. Thanks!
    – Nemin
    Dec 6, 2013 at 21:37
  • 1
    No worries! I really like too how jsoup has a "sandbox" so to speak where you can test stuff out - try.jsoup.org, and the main page for learning how to use it is their "cookbook" (took me a while to find it, but it's a gem)
    – Josh T
    Dec 6, 2013 at 21:39
  • Hey if this works for you, would you mark it correct?
    – Josh T
    Dec 6, 2013 at 21:54

Not the answer you're looking for? Browse other questions tagged or ask your own question.