1

I'm working on a hobby project that is a online game. That game stores player data in one big flat file. The data itself contains all the information of the player from Name to even items on the player itself. It's a rather large amount of columns by itself and having dozens of items only increases the flat file size to boot.

To give you a visual. My current player file is 192 columns (not accounting for items).

Player Data

There is 51 columns in my flat files for player data after I reduced the fluff. This does not include the items or the abilities for the players. I've already decided those can be separated into separate tables and linked with a FK.

The 51 columns of data are unique to the player and should not be duplicated. They are not what I've been told as good candidates for normalization.

Table

  • id
  • name
  • password
  • race
  • sex
  • class
  • level
  • gold
  • silver
  • experience
  • quest
  • armor
  • strength
  • wisdom
  • dexterity
  • etc

Activity

However, the activity of when some of these columns are selected and updated is vastly different from one another. Some are updated when the player moves, others are rarely utilized outside of when the player logs into the game and loaded into memory. Records are never dropped or rebuilt. Every column has a value. frequency of activity is anywhere from every second to once a month.

Question

That leads me to a question. Instead of traditional way of normalizing data, can I split these columns up based on activity and increase performance if they were in the same table? Or should I leave them the same table all together and just rely on proper indexing? Most of the columns look good to go, but like I said, some are used more than others. But, there is a vast difference in when some are used more than others. This sort of scares me.

7
  • You can split them up by creating a table called Activities that stores activity information. Beyond that, it is really hard to say anything, because your question does not have many details. Aug 25, 2013 at 18:10
  • Not too sure I understand what you mean. I updated the activity description to include more details on activity. But, the question is whether or not you can split based on some columns being more active (selected or updated) more than other columns in the same table?
    – Fastidious
    Aug 25, 2013 at 18:18
  • . . Your data does not sound normalized to begin with. You seem to be storing different entities on a single row (user and activity to name two from the question). I am unclear on the intent of the question. Aug 25, 2013 at 18:19
  • 1
    Tables should reflect entities and their relation, like Player, Item, player_has_item, ... If you normalize your tables, I don't believe they'd have ~200 columns X_X
    – Daniel W.
    Aug 25, 2013 at 18:20
  • 1
    If you can't normalize stuff and put into proper entities/relations, try NoSQL (MongoDB, ...) which can hold documents and doesn't care how many "columns" a document holds.
    – Daniel W.
    Aug 25, 2013 at 18:23

1 Answer 1

1

What you're mentioning is called denormalization and is actually a quite known and frequent matter.

There are no general rules and indications as to when to denormalize. This depends on so many things specific to each project (like the hardware, the type of DB, and the "activity" you mention to name a few) that it comes down to profiling each application to get to a conclusion.

Also, sometimes denormalization means splitting a table into two tables with a one-to-one relationship (like in your case). Sometimes it means getting rid of FKs and putting everything in a BIG table with many columns to avoid the joins when selecting.

Most importantly, keep in mind that your question is as much about performance than it is about scalability. Separating into different tables/databases mean you could eventually store the data in different machines, each having a specific hardware architecture with a database that fits the use case.


Example of denormalization in the gaming industry

One example of denormalization I can think of when it comes to MMORPGs is to store all the unfrequently changed user data in a BLOB. Not only is this denormalizing, but the whole row is stored as a series of bytes. Dr. E.F. Codd wouldn't be happy at all.

One company that does this is Playfish.

This means that you have faster selects at the cost of slower updates and, most importantly, changing the schema for the user becomes a real hassle (but the reasoning here is it will always be Username, Password, E-mail until the end of time). This also means that your user data can now be stored in a simpler key/value store instead of an RDBMS with more overhead. Of course, the login server fetching user information won't need to be as performant as the one handling the gameplay.


So try reading about use cases for denormalization (this is a very active topic) and see where you can apply your findings in your case. Also, keep in mind that pre-optimization can be sometimes counter-productive, maybe you should focus now on developing your game. When you have scaling/performance problems, you will most probably have the funding that comes with the high number of users to address the problem. Good luck!

6
  • This is for a MUD, which is the first generation MMORPG's. I don't plan to shard anything as it won't scale to millions of non-unique account and unique player records such as a Playfish game. But, the schema, data model, whatever would likely still be the same as any other MMORPG. Just without the big data. But yes, thanks for sharing. I guess I was looking at more about pre-optimization before optimizing on real data and scale.
    – Fastidious
    Aug 25, 2013 at 19:20
  • 1
    Ahh, I see what you mean now by this: During normalization, the database designer stores different but related types of data in separate logical tables called relations. When a query combines data from multiple tables into a single result table, it is called a join. Multiple joins in the same query can have a negative impact on performance. Introducing denormalization and adding back a small number of redundancies can be a useful for cutting down on the number of joins.
    – Fastidious
    Aug 25, 2013 at 19:25
  • If I split everything up into separate tables. I will have to do multiple joins of that data in order to piece together the player file. This would need to happen pretty frequently too as the player is playing the game. That's ideally why I asked the question. So, by denormalizing, I can as the above states, reduce the number of joins. Check!
    – Fastidious
    Aug 25, 2013 at 19:27
  • Thinking about how to redo my data model with this in mind has made my task of creating the new model harder. Thanks :D
    – Fastidious
    Aug 25, 2013 at 19:28
  • 1
    Oh, don't worry about that. I have the game up and running. MUD is open source (not for profit). I've been doing this for 15 years. It actually lead me to work in the game industry where I helped launch multiple AAA MMORPG's. I'm just not a DBA or a programmer. I worked in marketing. I do this for fun. ;)
    – Fastidious
    Aug 25, 2013 at 19:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.