0

I am trying to scrape lottery results data but have come accross two problems.

I am using:

$a = file_get_contents('website.com/directory')

storing the data into an array, Then using var_dump to output the results. My main problem is that no website I find has lottery results from multiple sources arranged in any comprehensive way; and writing individual functions per lottery source seems extremely inefficient. Plus all of the websites have different ways of their lottery results being listed

I know I will probably receive a lot of votes down but I am really unsure of how to proceed and this is my last resort and I can foresee a lot of problems arising from scraping many sources into one database. How can I ensure consistency in the data?

I am sure I am not the only one to have this kind of problem scraping from multiple sources

Am I going about the scraping in the wrong way? Any guidance would help me a lot

1
  • 1
    There are plenty of sites/services that have to scrape/combine data from multiple sources, and then clean up the data to make it consistent. For you, you will probably have to write some sub-scrapers that are specialized to scraping from different sites, and can handle the act of cleaning up the data from each source into a unified format for your purposes.
    – wkl
    Dec 30, 2015 at 15:51

2 Answers 2

1

I did a quick google search and came up with a few I see what you mean about them not being very good. I did find one site though which looks fairly easy to scrape. https://www.hityah.com/lotto/

Easy lottery has it's own div with class latto-box than in the h3 of each is the lottery name, so you can scrape that and they keep making it easy by putting lottery result numbers in easy classes as well.

If you go further in to say https://www.hityah.com/lotto/mega-sena/ you can see a large list of results, they even have archived data...

If you need help with the code don't hesitate to ask!

1
  • Great reference! If only other sites would take this kind of care! I will deal with the individual scraping requirements for the other resources but use this as my primary source for now! (We are still building the backend). As for the code I think it will be fine just trial and error and organizing it into something that makes sense Dec 30, 2015 at 16:02
1

As stated from @birryree, you will often face the problem that you need to collect information from different sources, that all have different aproaches and ways to share their data.

We also have a project that faced the same problem, and we did the following:

We have a huge database that will be filled from different data sources. Those data then will be imported and merged from that import database into another database, that will contain all the information in one pattern for all data sources.

Our website then gets it data from that one database, where all the information from other sources are combined.

3
  • Thanks for your input. Does this happen live? Obviously the time delay in the lottery results is a major factor here. You said "Those data then will be imported and merged..." does that imply that you did it manually? Dec 30, 2015 at 16:05
  • 1
    @DannyMahoney No, this runs completly automatic. Merged because there possibly could be multiple data sources for the same entry. It completly runs over multiple server scripts and cronjobs whatsoever, it's pretty complex in its architecture. But that's how it's done.
    – KhorneHoly
    Dec 30, 2015 at 16:11
  • that's impressive! Definitely something I will be working towards (in small steps). Thank you for your help Dec 30, 2015 at 16:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.