I just started using R, and came across data.table. I found it brilliant.
A very naive question: Can I ignore data.frame to use data.table to avoid syntax confusion between two packages?
I just started using R, and came across data.table. I found it brilliant.
A very naive question: Can I ignore data.frame to use data.table to avoid syntax confusion between two packages?
From the data.table FAQ
As FAQ 1.1 highlights,
j
in[.data.table
is fundamentally different fromj
in[.data.frame
. Even something as simple asDF[,1]
would break existing code in many packages and user code. This is by design, and we want it to work this way for more complicated syntax to work. There are other differences, too (see FAQ 2.17).Furthermore,
data.table
inherits fromdata.frame
. It is adata.frame
, too. Adata.table
can be passed to any package that only acceptsdata.frame
and that package can use[.data.frame
syntax on thedata.table
.We have proposed enhancements to R wherever possible, too. One of these was accepted as a new feature in R 2.12.0 :
unique()
andmatch()
are now faster on character vectors where all elements are in the globalCHARSXP
cache and have unmarked encoding (ASCII). Thanks to Matthew Dowle for suggesting improvements to the way the hash code is generated inunique.
c.A second proposal was to use
memcpy
induplicate.c
, which is much faster than a for loop in C. This would improve the way that R copies data internally (on some measures by 13 times). The thread on r-devel is here : http://tolstoy.newcastle.edu.au/R/e10/devel/10/04/0148.html.
DT[3]
refers to the 3rd row, butDF[3]
refers to the 3rd columnDT[3,] == DT[3],
butDF[,3] == DF[3]
(somewhat confusingly)- For this reason we say the comma is optional in DT, but not optional in DF
DT[[3]] == DF[3] == DF[[3]]
DT[i,]
where i is a single integer returns a single row, just likeDF[i,]
, but unlike a matrix single row subset which returns a vector.DT[,j,with=FALSE]
where j is a single integer returns a one column data.table, unlikeDF[,j]
which returns a vector by defaultDT[,"colA",with=FALSE][[1]] == DF[,"colA"]
.DT[,colA] == DF[,"colA"]
DT[,list(colA)] == DF[,"colA",drop=FALSE]
DT[NA]
returns 1 row of NA, butDF[NA]
returns a copy of DF containing NA throughout.- The symbol
NA
is type logical in R, and is therefore recycled by[.data.frame
. Intention wasprobablyDF[NA_integer_]
.[.data.table
does this automatically for convenience.DT[c(TRUE,NA,FALSE)]
treats the NA as FALSE, butDF[c(TRUE,NA,FALSE)]
returns NA rows
for eachNA
DT[ColA==ColB]
is simpler thanDF[!is.na(ColA) & !is.na(ColB) & ColA==ColB,]
data.frame(list(1:2,"k",1:4))
creates 3 columns,data.table
creates one list column.check.names
is by defaultTRUE
indata.frame
butFALSE
indata.table
, for convenience.stringsAsFactors
is by default TRUE indata.frame
but FALSE indata.table
, for efficiency.- Since a global string cache was added to R, characters items are a pointer to the single cached string and there is no longer a performance benefit of coverting to factor.
- Atomic vectors in list columns are collapsed when printed using ", " in data.frame, but "," in data.table with a trailing comma after the 6th item to avoid accidental printing of large embedded objects.
- In
[.data.frame
we very often setdrop=FALSE
. When we forget, bugs can arise in edge cases where single columns are selected and all of a sudden a vector is returned rather than a single column data.frame. In[.data.table
we took the opportunity to make it consistent and drop drop.- When a data.table is passed to a data.table-unaware package, that package it not concerned with any of these differences; it just works
There will possibly be cases where some packages use code that falls down when given a data.frame, however, given that data.table
is constantly being maintained to avoid such problems, any problems that may arise will be fixed promptly.
For example
From the NEWS for v 1.8.2
- base::unname(DT) now works again, as needed by plyr::melt(). Thanks to Christoph Jaeckel for reporting. Test added.
- An as.data.frame method has been added for ITime, so that ITime can be passed to ggplot2 without error, #1713. Thanks to Farrel Buchinsky for reporting. Tests added. ITime axis labels are still displayed as integer seconds from midnight; we don't know why ggplot2 doesn't invoke ITime's as.character method. Convert ITime to POSIXct for ggplot2, is one approach.
data.frame
must be used yet?? – MichaelChirico May 3 '15 at 15:20