In BigTable/GFS and Cassandra terminology, what is the definition of a SSTable?
5 Answers
Sorted Strings Table (borrowed from google) is a file of key/value string pairs, sorted by keys
-
5Thanks for yet another excellent SO Cassandra answer! BTW, have you seen this question: stackoverflow.com/questions/2573106/…– knorvCommented Apr 5, 2010 at 19:15
-
-
1yes, sstables are immutable by design -- which is an awesome feature Commented Jul 3, 2015 at 17:19
-
3
-
3@xjcl The SSTable segment is immutable. The sorted behavior is designed on the in-memory level by leveraging a sorted data structure such as AVL tree. When you flush the in-memory on disk, you write it in sorted format. Once a segment is formed on disk, there is no way you can edit it hence it is immutable. Commented Sep 7, 2021 at 16:59
"An SSTable provides a persistent,ordered immutable map from keys to values, where both keys and values are arbitrary byte strings. Operations are provided to look up the value associated with a specified key, and to iterate over all key/value pairs in a specified key range. Internally, each SSTable contains a sequence of blocks (typically each block is 64KB in size, but this is configurable). A block index (stored at the end of the SSTable) is used to locate blocks; the index is loaded into memory when the SSTable is opened. A lookup can be performed with a single disk seek: we first find the appropriate block by performing a binary search in the in-memory index, and then reading the appropriate block from disk. Optionally, an SSTable can be completely mapped into memory, which allows us to perform lookups and scans without touching disk."
-
9"without touching disk" -> "without being aware that the disk is being touched". Memory mapped IO is a very handy technique because it delegates the actual IO to the OS, assuming it can do a good job at caching (especially when several processes share the same file). But it has the disadvantage that you don't have control of it. If the page is not resident in memory, the thread will block and cannot perform other operations; contrast it with "async IO", where you can register a callback and do other stuff in the same thread, while the IO is pending.– mkmCommented Jul 27, 2011 at 14:51
-
4@ithkuil: You can absolutely have control of memory mapped IO at least to the point of being able to assure that certain pages are in memory or have been committed to disk (there is still wiggle room for pages that aren't guaranteed to be in memory but very well could be). That's what wondrous things like mlock(), msync(), and MAP_LOCKED are all about. You can also get an understanding of what currently is and isn't paged in through mincore(). Commented Apr 18, 2012 at 4:58
-
3@ChristopherSmith: yes you are right, there are ways to control it. However, usually it's used for critical performance sections (realtime) or security related issues (like avoiding that a in-memory password gets swapped on disk). Memory mapped files are very useful exactly because of the fact that you don't have to decide how much of them to keep in memory; otherwise you could just read the whole file in memory without mmap and achieve the same effect. In fact, I just grepped through the cassandra code; the only call is
mlockall(MCL_CURRENT);
done at startup. See also: goo.gl/AEgPM– mkmCommented Apr 18, 2012 at 17:03 -
6
SSTable (engl. Sorted Strings Table) is a file of key/value string pairs, sorted by keys.
An SSTable provides a persistent,ordered immutable map from keys to values, where both keys and values are arbitrary byte strings.
Internally, each SSTable contains a sequence of blocks (typically
each block is 64KB in size, but this is configurable).
A tablet is stored in the form of SSTables.
SSTable (directly mapped to GFS) is key-value based immutable storage. It stores chunks of data, each is of 64KB.
Definitions:
- Index of the keys: key and starting location
- Chunk is a storage unit in GFS, replica management are by chunk
SSTable means "sorted string table" based on key-value pair.In Cassandra, SSTables are immutable and sorted by keys.