1

Using ASP.NET and Windows Stack.

Purpose: Ive got a website that takes in over 1GB of data about every 6 months. So as you can tell my database can become huge.

Problem: Most hosting providers only offer Databases in 1GB increments. This means that every time I go over another 1GB, I will need to create another Database. I have absolutely no experience in this type of setup and Im looking for some advice on what to do?

Wondering:

  1. Do I move the membership stuff over to a separate database? This still won't solve much because of the size of the other data I have.
  2. Do I archive data into another database? If I do, how to I allow users to access it?
  3. If I split the data between two databases, do I name the tables the same?
  4. I query all my data with LINQ. So establishing a few different connections wouldn't be a horrible thing.
  5. Is there a hosting provider that anyone knows of that can scale their databases?

I just want to know what to do? How can I solve this dilemma? I don't have the advertising dollars coming in to spend more than $50 a month so far...

While http://www.ultimahosts.net/windows/vps/ seems to offer the best solution for the best price, they still split the databases up. So where do I go from here?

Again, I am a total amateur to multiple databases. Ive only used one at a time..

2
  • Depending on your database model/design, it could be normalized which would decrease the inflation over time.
    – OMG Ponies
    Nov 3, 2009 at 18:25
  • In the PHP world, this would be easy. However, you might try contacting a "custom" provider like www.mityman.com and tell them exactly what you need in terms of storage and bandwidth. One of their "standard" packages is 5GB storage and 25Gb of bandwith for $49.99, but they are often willing to work with you on storage issues(hard drives are cheap.)
    – Tom
    Nov 3, 2009 at 18:37

6 Answers 6

1

I'd be genuinely surprised if they actually impose a hard 1GB per DB limit and create a new one for each additional GB, but on the assumption that that actually is the case -

  1. Designate a particular database as your master database. This is the only one your app will directly connect to.
  2. Create a clone of all the tables you'll need in your second (and third, fourth etc) databases.
  3. Within your master database, create a view that does a UNION on the tables as a cross-DB query - SELECT * FROM Master..TableName UNION SELECT * FROM DB2..TableName UNION SELECT * FROM DB3..TableName

For writing, you'll need to use sprocs to locate the relevant records and update them, but you shouldn't have a major problem there. In principle you could extend the view above to return which DB the record was in if you wanted.

0

Answering this question is very hard for it requires knowing at least some basic facts about the data model, the way the data is queried, etc. Also as suggested by rexem, a better understanding of the use model may allow using normalization to limit the growth (and I had may also allow introducing compression, if applicable)

I'm more puzzled at the general approach and business model (and I do understand the need to keep cost down with a startup application based on ad revenues). Wouldn't you be able to contract an amount that will fit your need for the next 6 months, then, when you start outgrowing this space, purchase additional storage (for an extra 6 month/year, by then you may be "rich"); such may not even require anything on your end (depends on the way hosting service manages racks etc.), or at worse, may require you to copy the old database to the new (bigger) storage?

In this fashion, you wouldn't need to split the database in any artificial fashion, and hence focus on customer-oriented features, rather than optimizing queries that need to compile info from multiple servers.

2
  • Can one Database grow into 1 Terabyte and still be alright? Do you know any hosting provider that can offer that or is that when I should buy my own server? Or rent one? Nov 5, 2009 at 15:36
  • There is something missing from this equation. You ask if it can grow into 1 terabyte, but you say you only increase your data by 1 gigabyte per six months (2 gigabytes per year). Unless you're planning for storage for the next 500 years, where did the 1 terabyte number come from?
    – BarrettJ
    Nov 5, 2009 at 15:59
0

I believe solution is much more simpler than that: also if your provider manage database in 1 GB space it does not means that you have N databases of 1 GB each, it means that once you reach 1 GB the database could be increased to move to 2 GB, 3 GB and so on...

Regards
Massimo

0

You would have multiple questions to answer:

  1. It seems the current hosting provider can not be very reliable if it is the way you say: they create a new database every time the initial one gets more then 1GB - this sounds strange... at least they should increase the storage for the current db and announce you that you'll be charged more... Find other hosting solutions with better options...
  2. Is there any information into your current DB that could be archived? That's a very important question since you may carry over "useless" data that could be archived into separate databases and queried only when special requests. As other colleagues told you already, that would be difficult for us to evaluate since we do not know the data model.
  3. Can you split the data model into two total different storages and only replicate between them the common information? You could use SQL Server Replication (http://technet.microsoft.com/en-us/library/ms151198.aspx) to maintain the same membership information between the databases. If the data model can not be splited then I do not see any practical choice to have multiple databases - just find a bigger storage solution.
0

You may want to look for a better hosting provider.

Even SQL Express supports a 4GB database, and it's free. Some hosts don't like using SQL Express in a shared environment, but disk space is so cheap these days that finding a plan that starts at or grows in chunks of more than 1GB should be pretty easy.

0

You should go for a Windows VPS solution. Most of the Windows VPS providers will offer SQL 2008 Web Edition that can support upto 10 GB of database space ...

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.