1

I have built and app which does random stuff and I want to collect various statistics which I want to display in graphs. My problem is I'm not sure how to store the data in a database except writing each log into new row which seems very inefficient.

Example Data (5 minute averages):

  • Number of Users
  • Number of Online Users
  • Number of Actions

What would be the best way to store this information? Do I need a separate table for thing that I'm logging or could they all go into one table?

1 Answer 1

2

Often you don't need the full resolution data kept for all time and you can re-process it periodically into lower resolution data to save space. For example you could store one day of full resolution (5 minute averages) but periodically re-average that data into 1 hour bins/1 day bins/ 1 month bins/ etc while also culling the data.

This allows you to have the data you need to display nice graphs of the activity over different time ranges (hour, day, week, month, etc) while limiting the number of rows to just what your application requires.

There are also excellent applications to store and display time-series data. MRTG and RRDTool come to mind. See this tutorial for good food for thought: rrdtutorial

4
  • Thanks for the info. The links don't do a very good job of explaining what they do so ill just use mysql and index the relevant columns.
    – redslazer
    Feb 6, 2012 at 8:51
  • Yes index necessary columns and don't worry about the links unless you want a separate tool with built in data storage and graphs. Sounds like you want to implement it yourself so have fun and good luck! Feb 6, 2012 at 9:03
  • Should i have a separate table for each variable or can i chuck them all in 1 table and have an indexed type column?
    – redslazer
    Feb 6, 2012 at 9:07
  • Seems simplest to have one row for each time period (per 5 minutes in your example) and then a column for each variable along with the time and whatever else you may need. Feb 6, 2012 at 9:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.