I can find cheap VPS hosts with 128MB RAM, and I wonder if that is enough to run a crate node for a tiny database, initially for testing. (I'm not looking for recommended memory, but the minimum one, for not running into out-of-memory exceptions. Crate is supposed to be the only service in the node.)
-
what kind of queries do you expect to be able to run - and how many records do you expect? clearly, the less memory, the less data your response can handle.– ChristianMar 30, 2016 at 6:27
-
~10000 records of ~10Kb each; at peak it must handle 10 writes/minute and 100 reads/minute. Simple queries (no joins) on columns' values, array elements and one fulltext column.– Sony SantosMar 30, 2016 at 14:22
-
Each read must bring at most 100 records of 10Kb each (total max 1MB/read). Each write saves only one record.– Sony SantosMar 30, 2016 at 15:10
2 Answers
It is possible to run Crate in such an environment. I wouldn't recommend it, though. In any case you need to take a few precautions:
- Select a lean Linux distribution that actually boots and runs with such a small memory footprint. Alpine might be one choice.
- Install Java. You need at least openjdk7 (update 55 and up).
- Install and start Crate from the tarball as explained on the Crate website.
On a virtual machine with 128 MB RAM on top of Alpine 3.3, I installed openjdk8-jre
(you have to enable community repositories in /etc/apk/repositories
) on disk. I downloaded the Crate 0.54.7 tarball and just extracted it. I set CRATE_HEAP_SIZE=64m
as this is the recommeded half of the available memory.
I created a table "demo"
DROP TABLE IF EXISTS demo;
CREATE TABLE demo (
data string
);
and filled it up with 10,000 records of 10 KB random strings each with a slow bash script:
head -c7380 /dev/urandom | uuencode - | grep ^M | tr -d '\n\047'
This took a few minutes (about 20 records/s), but with bulk inserts it should be way faster and just take seconds.
The net amount of data was about 100 MB and took 287 MB gross on disk as reported by the admin UI.
Operating system, the installed software, and the data altogether claimed 820 MB on the disk.
I configured twice the amount of memory as swapspace and got the following footprint (the Crate process itself without data takes up about 40 MB):
# free
total used free shared buffers cached
Mem: 120472 117572 2900 0 652 6676
-/+ buffers/cache: 110244 10228
Swap: 240636 131496 109140
A fulltext search over all 10,000 records (SELECT count(*) FROM demo WHERE data LIKE '%ABC%'
) took about 1.9 seconds.
Summary: Yes, it's possible, but you lose a lot of features if you actually do so. Your results will heavily depend on the type of queries you actually run.
I just played around a bit how much you could reduce the HEAP size to, and it looks like that 64MB
heap (128MB
memory) could work out for your use case.
Make sure you set the HEAP size correctly using the CRATE_HEAP_SIZE
(docs) environment variable and also set bootstrap.mlockall: true
(docs) to the JVM does not swap memory.
However, I would recommend at least 256MB
HEAP (512MB
memory).