19

I thought that the exclamation mark ! is the symbol for the logical NOT operator. Now, learning indexing in the DataFrames package, I came across this: data[!,:Treatment]. Which seems to be the same as using the known colon symbol :

data[:,:Treatment]==data[!,:Treatment] is true.

Why this redundancy then?

2 Answers 2

16

! in indexing is specific to DataFrames, and signals that you want a reference to the underlying vector storing the data, rather than a copy of it. You can read all about indexing DataFrames here. In your example the are both == because all values are identical, but they are not === since df[:, :Treatment] gives you a copy of the underlying data.

Example:

julia> using DataFrames

julia> df = DataFrame(y = [1, 2, 3]);

julia> df[:, :y] == df[!, :y] # true because all values are equal
true

julia> df[:, :y] === df[!, :y] # false because they are not the same vector
false
Sign up to request clarification or add additional context in comments.

3 Comments

So is this the same as the dereference operator * in C-like languages?
You can think of a dataframe as (more or less) a struct containing vectors -- one per column. df[!, :y] is the same vector as df.y, meaning that both vectors point to the same chunk of memory.
a small comment is that df[!, :y] is allowed as LHS of broadcasting assignment while df.y is not allowed if :y is not present in df (due to restrictions of Base). Also df[!, cols] can be used to create a new data frame containing multiple columns from df without copying them (as opposed to df[:, cols] which will freshly allocate all columns).
11

Quoting the documentation of DataFrames.jl:

Columns can be directly (i.e. without copying) accessed via df.col or df[!, :col]. [...] Since df[!, :col] does not make a copy, changing the elements of the column vector returned by this syntax will affect the values stored in the original df. To get a copy of the column use df[:, :col]: changing the vector returned by this syntax does not change df.

An example might make this clearer:

julia> using DataFrames

julia> df = DataFrame(x = rand(5), y=rand(5))
5×2 DataFrame
│ Row │ x        │ y         │
│     │ Float64  │ Float64   │
├─────┼──────────┼───────────┤
│ 1   │ 0.937892 │ 0.42232   │
│ 2   │ 0.54413  │ 0.932265  │
│ 3   │ 0.961372 │ 0.680818  │
│ 4   │ 0.958788 │ 0.923667  │
│ 5   │ 0.942518 │ 0.0428454 │
julia> # `a` is a copy of `df.x`: modifying it will not affect `df`

julia> a = df[:, :x]
5-element Array{Float64,1}:
 0.9378915597741728
 0.544130347207969
 0.9613717853719412
 0.958788066884128
 0.9425183324742632

julia> a[2] = 1;

julia> df
5×2 DataFrame
│ Row │ x        │ y         │
│     │ Float64  │ Float64   │
├─────┼──────────┼───────────┤
│ 1   │ 0.937892 │ 0.42232   │
│ 2   │ 0.54413  │ 0.932265  │
│ 3   │ 0.961372 │ 0.680818  │
│ 4   │ 0.958788 │ 0.923667  │
│ 5   │ 0.942518 │ 0.0428454 │
julia> # `b` is a view of `df.x`: any change made to it will be reflected in df

julia> b = df[!, :x]
5-element Array{Float64,1}:
 0.9378915597741728
 0.544130347207969
 0.9613717853719412
 0.958788066884128
 0.9425183324742632

julia> b[2] = 1;

julia> df
5×2 DataFrame
│ Row │ x        │ y         │
│     │ Float64  │ Float64   │
├─────┼──────────┼───────────┤
│ 1   │ 0.937892 │ 0.42232   │
│ 2   │ 1.0      │ 0.932265  │
│ 3   │ 0.961372 │ 0.680818  │
│ 4   │ 0.958788 │ 0.923667  │
│ 5   │ 0.942518 │ 0.0428454 │

Note that, since the indexing with ! does not involve any data copy, it will generally be more efficient.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.