6

I was wondering about the Google Analytics database design, how they handle the huge values in hourly basis, even in minutes.

Let's say, they have 100 million users and almost every user has 300 counters at every minute. For one user, 300 counters have 18000 rows in one hour. For one day it is 432K rows and almost 3 million rows.

I have thought that they are not using a relational database, but not sure about it...

Is there any suggestion about it?

Regards,

2
  • 2
    I'm pretty sure Google use a custom developed database called BigTable (en.wikipedia.org/wiki/BigTable)
    – Kane
    Sep 15, 2009 at 10:20
  • google says : # BigTable is a distributed hash mechanism built on top of GFS(Google File System). It is not a relational database. It doesn't support joins or SQL type queries. # It provides lookup mechanism to access structured data by key. GFS stores opaque data and many applications needs has data with structure. From this comment KEY word is important. Lookup mechanishm is handled just for one key. Seems it has an very big HashTable inside...
    – Melih
    Sep 15, 2009 at 11:46

2 Answers 2

12

BigTable

And you're right, they are not using a relational database.

1

High Scalability has a summary of Google's architecture here. It doesn't discuss Analytics directly but it shows how BigTable fits into the entire infratructure. I'm not sure the details of Google's schema is available - as the article says "Infrastructure can be a competitive advantage" - but I would guess it's a lot more tightly bound to the hardware implementation than a regular data model would be.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.