0

Background:

I've just started learning php to create a website to display user data from a betting program held in a mysql database.

The program in question (An IRC bet bot client written in python) originally used .csv text file arrays to hold user data. However I realized that this method of data storage really wasn't scale-able and held a lot of data in memory (thus required lots of file writes and flat text file searches).

I re-implemented the program to use a mySQL database instead. I'm currently re-writing the website.

Is it best that for every user I query the mySQL database once for each variable stored (thus hundreds of queries need to construct a user list page)? Or should I query the database once but the data into an array and cherry pick the variables out of that array that I need to display? Thus only querying once but holding more data in memory and doing much more server based processing.

The original website (That still works using text.csv files) can currently be found here: http://xcoins.co.uk (However this will soon be updated with the mysql based version (hopefully))

Question:

Is it better to query a mySQL database once per user get lots of data and sort it using php to display a list of users sorted by a value (some of the returned data from the query may be discarded or maybe cached). OR should I query the database lots of times for each variable of user data I need on a page but thus it will only grab what data is needed (but would require around 100 queries to be sent an received to display a single user list page)?

How many queries are generally acceptable to display a single webpage with lots of data on it?

5
  • Caching re-useable data is always a great idea ofcourse. Could you explain why you need 100+ queries for one page? If you need userdata for example it would take 1 query to get all the users' info so I don't really know why you would need that many
    – Crecket
    Sep 22, 2015 at 9:33
  • Try to fetch all data you need with a minimum of queries, but avoid fetching anything you do not need (as with SELECT *). So user list page would only need one single query that retrieves the data you want to display. Data which is used multiple times should be stored, for example in associative arrays.
    – syck
    Sep 22, 2015 at 9:41
  • @Crecket Depending on how I implement the list page for user coin balance, I guess I should do it with one query, for some reason I was thinking, I'd use a get username balance query for each user and then sort them in php, but obviously this wouldn't be a good idea. Sep 22, 2015 at 11:35
  • SELECT * FROM users for example will get you everything in the users table. Than you can very easily manipulate it to form a list with all users. Or if you are looking at a specific profile it would be possible to SELECT * FROM users WHERE username = ? to reduce traffic.
    – Crecket
    Sep 22, 2015 at 11:51
  • This post can help you: [what-is-faster-one-big-query-or-many-small-queries?][1] [1]: dba.stackexchange.com/questions/76973/… Sep 24, 2015 at 11:10

1 Answer 1

0

The common way is to retrieve as much information as you need for a given page (but no more), and to use this result set to populate the page.

So, to create the "customer profile" page, you might run a query like:

select p.name, 
       p.address, 
       sum(t.amount) as balance
from   profile p, 
       transactions t
where  p.id = ?
and    p.id = t.id

(Apologies for archaic join syntax - old habits die hard).

This would give you a PHP result set, which you'd print to the page in the appropriate place.

Of course, often you cannot retrieve all the data you need in a single query, either because there isn't a natural join (e.g. showing the number of active users on the system), or for performance/complexity reasons. In this case, you'd write individual queries for those data elements.

You may want to look at PHP frameworks which abstract away a lot of this stuff for you.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.