9

I am just a beginner in MySQL, I need to know how much data can be stored in MySQL. I am developing a web crawler, can I store all the data in MySQL, or do I need to use another Database? Which is more faster? What I mean is, which has the highest Writing/Reading Rate? Do I need to reconfigure to add more data?

6
  • 4
    You mean the entire data of the web? :) Commented Aug 5, 2010 at 16:54
  • You can store as much data in MySQL as you have disk space and your OS can cope with. There's really not a lot of advice anyone can provide, given so vague information.
    – nos
    Commented Aug 5, 2010 at 16:55
  • @Daniel :yep @MarkJ:why? Commented Aug 5, 2010 at 16:58
  • @Tom Gullen sure it is(given the OS can account for all the disk space), you might not be able to store it all in one thable though.
    – nos
    Commented Aug 5, 2010 at 16:58
  • @Nos, you edited your comment and it is now correct. What you said previously however was not! :-)
    – Tom Gullen
    Commented Aug 5, 2010 at 17:05

3 Answers 3

12

Depends on the operating system.

Operating System             File-size Limit
Win32 w/ FAT/FAT32           2GB/4GB
Win32 w/ NTFS                2TB (possibly larger)
Linux 2.2-Intel 32-bit       2GB (LFS: 4GB)
Linux 2.4+                   4TB
Solaris 9/10                 16TB
MacOS X w/ HFS+              2TB
NetWare w/NSS file system    8TB

http://dev.mysql.com/doc/refman/5.0/en/full-table.html

Your write/read rate is of little concern to you; your bottleneck will be your internet connection.

10
  • i am using a win32/NTFS system,what will i do if the data goes higher than 4tb??its already 3TB Commented Aug 5, 2010 at 16:58
  • Find a different DBS, SQL server on a 64bit machine can have 16TB. msdn.microsoft.com/en-us/library/ms143432.aspx Just shop around and see what tickles your onion jar. Oracle can handle 8589541376 G on a 'BFT' addressing scheme. I don't think MYSql was designed with your needs in mind.
    – Tom Gullen
    Commented Aug 5, 2010 at 17:02
  • @Tom Gullen: So you are suggesting SQL Server,is there any Open Source DB for my needs?what DB did google using? Commented Aug 5, 2010 at 17:04
  • @Alex: google is using homegrown BigTable (stackoverflow.com/questions/362956/…).
    – Wrikken
    Commented Aug 5, 2010 at 17:08
  • @Wrikken : Thks for the info,in my case what all DB can use?what are the solutions? what about Hadoop?is it a DB,can i use that for a webcrawler? Commented Aug 5, 2010 at 17:11
0

https://forums.mysql.com/read.php?22,379547,381106

InnoDB Size Limits

  • Max # of tables: 4 G
  • Max size of a table: 32TB
  • Columns per table: 1000
  • Max row size: n*4 GB
  • 8 kB if stored on the same page
  • n*4 GB with n BLOBs
  • Max key length: 3500
  • Maximum tablespace size: 64TB
  • Max # of concurrent trxs: 1023

Nanda Kishore Toomula

Sr DBA,Nokia India

CMDBA 5.0

1
  • what is the max total size please Commented Feb 10, 2019 at 16:18
0

This comes from their docs. MyISAM permits data and index files to grow up to 256TB by default, but this limit can be changed up to the maximum permissible size of 65,536TB (2567 − 1 bytes).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.