31

When would a database design be described as overnormalized? Is this characterization an absolute one? Or is it dependent on the way it is used in the application? Thanks.

11 Answers 11

35

In the general sense, I think that overnormalized is when you are doing so many JOINs to retrieve data that it is causing notable performance penalties and deadlocks on your database, even after you've tuned the heck out of your indexes. Obviously, for huge applications and sites like MySpace or eBay, de-normalization is a scaling requirement.

As a developer for several small businesses, I tell you that in my experience it's always been easier to go from normalized -> denormalized than the other way around, and in fact going the other way around (to avoid duplication of data now that the business requirements have changed a year or so later) is much more difficult.

When I read general statements such as "you should put the address in your customers table instead of a separate address table so you can avoid the join", I shudder, because you just know that a year from now somebody's going to ask you to do something with addresses that you totally didn't foresee, like maintaining an audit trail, or storing multiple per customer. If your database allows you to create an indexed view, you can sidestep that issue until you get to the point where your dataset is so large that it can't possibly exist or be served by a single server or set of servers in a 1-write, many-read environment. For most of us, I don't think that scenario happens very often.

When in doubt, I aim for third normal form with some exceptions (for example, having a field contain a CSV-list of separated strings because I know I'll never ever look at the data from the other angle). When I need to consolidate, I'll look at my views or indexes first. Hope this helps.

5
  • You should aim for at least BCNF (basically a version of 3NF that eliminates an edge case that the official 3NF does not), and very often you'll find that the data is in fact in 5NF at that point anyway. Nov 16, 2008 at 4:58
  • It's worth noting that as of SQL Server 2005 you can use Inline-Table-Valued-Functions (ITVF). You can join to these like a table and pass in a few parameters. It may sound overdone to suggest that you could even query from a View and serve it up in an ITVF, but if you find yourself using the same parameters and joins over and over again across multiple sprocs, then it may make sense to encapsulate it in a call to an ITVF.
    – MikeTeeVee
    Feb 15, 2012 at 17:45
  • 2
    @JonathanLeffler Every DB is different, so a rule like "always aim for BCNF" is contraindicated. Normalization has benefits, but it can also have disadvantages. Did you know that in an insert heavy environment inserting into indexed columns can have a significant performance penalty (don't want to join without an index) depending on index type? Also, joining isn't a free operation, so if you're joining 1 table to get a subset of another and so on down the chain 8 deep, join performance can add up some nasty overhead for large tables (>100M records). Sometimes denormalization has benefits.
    – hsanders
    Dec 4, 2012 at 15:22
  • @Nicholas Piasecki I know that this post is almost 11 years old, but I was just wondering, could you shed some light on how normalization may affect maintaining an audit trail? Thanks. Oct 3, 2019 at 22:22
  • 1
    @scrnjakovic Well 11 years later, I guess my thought at the time was that a common way (not the only way) to implement an audit trail in a database is with a "shadow" table, where you have dbo.Customers and dbo.AuditCustomers, where a new row is inserted into AuditCustomers whenever there are changes to the original data. If your data is normalized that means the data is edited in one place and auditing is easy. If it isn't normalized then you may have to update it in multiple places. Oct 7, 2019 at 13:01
17

It's always a question of the application domain. It's usually a question of correctness, but occasionally a question of performance.

There's one case where I can think of a prima facie case of overnormalization: say you have an order + orderitem, and the orderitem references productID, and leaves pricing to the product.price. Since that introduces temporal coupling, you've incorrectly normalized because the overnormalization affects already shipped orders, unless prices absolutely never change. You can certainly argue that this is simply a modeling error (as in the comments), but I see under-normalization as a modeling error in most cases, too.

The other category is performance related. In principle, I think there are generally better solutions to performance than denormalizing data, such as materialized views, but if your application suffers from the performance consequences of many joins, it may be worth assessing whether denormalizing can help you. I think these cases are often over-emphasized, because people sometimes reach for denormalization before they properly profile their application.

People also often forget about alternatives, like keeping a canonical form of the database and using warehousing or other strategies for frequently-read, but infrequently changed data.

7
  • 2
    Temporal coupling is a good point and is something that is easy to overlook until 30 days after your implementation goes live. Not that I've been there. Nov 15, 2008 at 17:49
  • 2
    I like your emphasis on alternatives. Note that your first case isn't related to normalization at all. It's a failure of the domain designer to distinguish between a product price and a sale price.
    – HTTP 410
    Nov 15, 2008 at 19:24
  • @RoadWarrior - yes, or more precisely, between 'current product price' and 'sale price'. Nov 16, 2008 at 4:59
  • i would think that the first example is not "over-normalization", as the product still logically may have a current price, but under-modeling, since the order-item is (as you point out) temporally bound and should thus snapshot the price at the time of the sale. Nov 16, 2008 at 5:23
  • All these are fair points, though probably a matter of definition. For me, over-normalization includes cases where correctness is compromised (as a result of poor modeling). Non-normalized schemas compromise correctness unless precautions are taken.
    – JasonTrue
    Nov 16, 2008 at 5:32
11

Normalization is absolute. A database follows Normal Forms or it does not. There are a half-dozen normal forms. Mostly, they have names like First through Fifth. Plus there's a Boyce-Codd Normal Form.

Normalization exists for precisely one purpose -- to prevent "update anomalies".

Normalization isn't subjective. It isn't a judgement. Each table and relationship among tables either does or does not follow a normal form.

Consequently, you can't be "over-normalized" or "under-normalized".

Having said that, normalization has a performance cost. Some people elect to denormalize in various ways to improve performance. The most common sensible denormalization is to break 3NF and include derived data.

A common mistake is to break 2NF and have duplicate copies of a functional dependency between a key and non-key value. This requires extra updates or -- worse -- triggers to keep the copies in parallel.

Denormalization of a transactional database should be a case-by-case situation.

A data warehouse, also, rarely follows any of the transactional normalization rules because it's (essentially) never updated.

"Over-normalization" could mean that a database is too slow because of a large number of joins. This may also mean that the database has outgrown the hardware. Or that the applications haven't been designed to scale.

The most common issue here is that folks try to use a transactional database for reporting while transactions are going on. The locking for transactions interferes with reporting.

"Under-normalization," however, means that there are NF violations and needless processing is being done to handle the replicated data and correct update anomalies.

2
  • you can't be "over-normalized" or "under-normalized" but "Over-normalization" could mean... and "Under-normalization," however, means... Though both are helpful, I'm not sure which @SLott to believe. ;^)
    – ruffin
    Apr 19, 2016 at 13:03
  • Turns out update anomalies first stop at ETNF (Fagin & Date 2012) between 4NF & 5NF (& there were already then anomaly-free NFs between it & 5NF). But 5NF eliminates further cases of redundancy where a table can be advantageously replaced by 3 or more that join back to it.
    – philipxy
    Mar 12, 2018 at 22:10
5

When the performance cost exceeds the benefit towards the application's intended purpose.

3
  • 17
    I always liked the phrase "Normalize 'til it hurts, denormalize 'til it works." :)
    – vfilby
    Nov 15, 2008 at 18:32
  • Exactly - the perfect balance. Nov 16, 2008 at 1:35
  • A very nice statement vfilby. It summarizes my comment below in one clear and simple sentence. :)
    – Eigir
    Nov 16, 2008 at 16:35
4

Normalize your OLTP databases, and denormalize your OLAP databases. Each has a mission that dictates its schema. Like normalized transaction databases, data warehouses exist for a reason. A complete system needs both.

3

A lot of people are talking about performance. I think a key issue is flexibility. In general, the more normalized your database, the more flexible it is.

We currently use an "over-normalized" database because, in our operating environment, client requirements change on a monthly basis. By "over-normalizing" we can adopt our software accordingly, without changing the database structure.

1
  • 1
    I completely agree. I've used databases with millions of records, and performance was never an issue. The structure of the data needs to flexible enough to allow for multiple different uses and changing requirements without having to change the data structure. Normalization is the answer to this. Sep 22, 2009 at 18:23
2

My take on this:

Always normalize as much as you are able to do. I usually go crazy on normalization, and try to design something that could handle every thinkable future extensions. What I end up with is a database design that is extremely flexible... and impossible to implement.

Then the real job starts: De-normalization. Here you solve what you know would be problematic to implement and/or would slow the queries down because of too many joins.

This way you know what you scarify for make the design usable.

Edit: Documentations! I forgot to mention that documenting the de-normalization is very important. It is extremely helpful when you take over a project to know the reason behind the choices.

2
  • The 'every thinkable future extension' is overkill; at most you need to handle probable extensions (not ones that are just possible). This is part of the agile technique - not worrying too much about the future. With a DBMS, some concern about the future is good, but not too much. Nov 16, 2008 at 5:01
  • 1
    I understand what you mean, but I believe that the design of the DBMS is the most fundamental part of a project. Mistakes done on that level are the most difficult ones to correct later, since a redesign of the database has a high chance of breaking large parts of the code.
    – Eigir
    Nov 16, 2008 at 16:29
2

Third Normal Form (3NF) is considered the optimal level of normalization for many a rational database application. This is a state in which, as Bill Kent once summarized, every "non-key field [in every table within a particular a relational database management system, or RDBMS] must provide a fact about the key, the whole key, and nothing but the key." 3NF is a term that was introduced by E.F. Codd, inventor of the relational model for database management. Generally, the data that a software application is dependent on, especially an application used for an Online Transaction Processing System (OLTP), will fare well in 3NF. This normal form by definition reduces database size by calling for a minimum repetition of row/column data, and maximizes query efficiency and ease of application maintenance. 3NF achieves that by requiring that a database's tables (i.e., its schema) be broken down into separate tables related by primary/foreign keys--basically until Kent's rule holds true (well, I've stated it this way for ease of reading but the actual definition of 3NF is much more detailed than that). In contrast, overnormalization implies increasing the number of joins required in a query between related tables. This comes as a result of breaking down the database schema into a much more granular level than 3NF. However, though normalization past the 3rd degree can often be considered overnormalization, the negative connotation of the term "overnormalization" can sometimes be unwarranted. Overnormalization may be desirable in some applications which by design require 4NF (and beyond) due to the complexity and versatility of the application software. An example of that is a highly customizable and extensible commercial database program for some industry in which it is sold to end users requiring an open API. But then the reverse can be desirable as well--that is, denormalization--most notably, when designing an Online Analytical Processing (OLAP) database used strictly to summarize data from an OLTP database just for querying/reporting--such as a data warehouse. In this case the data must by necessity reside in a highly denormalized format (i.e, 1NF, or 2NF). It's often under these constraints--when there are high demands for efficient querying and reporting--that we find database and application programmers calling a database, "overnormalized". But as Redgate's Tony Davis once said--taking into account today's much more advanced and efficient database software and storage systems--"the performance hit from multiple joins in a query is negligible. If your database is slow, it isn’t because it is ‘over-normalized’!" So in conclusion, this characterization--overnormalization--isn't an absolute one, and it is dependent on the way it is used in the application. In Kent's words, "The normalization rules are designed to prevent update anomalies and data inconsistencies. . . [but] there is no obligation to fully normalize all records when actual performance requirements are taken into account. . . The normalized design enhances the integrity of the data, by minimizing redundancy and inconsistency, but at some possible performance cost for certain retrieval applications. . . [Thus,] the desirability of normalization has to be assessed, in terms of its performance impact on retrieval applications."

1

..or hitting limits on the number of joins your RDBMS will do.

2
  • 2
    That's a defective or toy DBMS - time to replace it with a real one. Nov 16, 2008 at 4:57
  • 1
    Pfft .. everyone knows that a "real" RDMS should be doing trillions of joins. Limits are for wimps. Anything that can't handle a trillion joins ... must be a "toy" !!
    – Michael M
    Jul 7, 2016 at 20:04
1

If performance is affected by too many joins, creating de-normalized tables for reporting purposes can speed things up. By copying the data into new tables, it may be possible to run reports with no joins at all.

0

In my experience, I've never seen a normalized database that contains postal addresses, as it's usually acceptable to store the address as a string. Ideally, there would be tables for countries, counties / states, cities, districts and streets. I've not come across anyone who needs to report on street level, so it hasn't been necessary. The addresses have only be used for postal contact, so are treated as a single entity.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.