1

When trying to clean up the code using @code_warntype I get the type instability of ::AbstractArray{T,1} when it is not expected.

A data frame is the argument of the function FUNC1, and a particular COLUMN in it is used within the function. I've defined the type for this COLUMN within the function as Array{Float64,1}. But, when I run @code_warntype on the function, ::AbstractArray{T,1} appears in the output.

function FUNC1(df::DataFrame)
    df_COL=df[:COLUMN]::Array{Float64,1}

.......
end

Expected result is that there should be no type instability because the type has been specified for that column.

Actual results:

Body::Tuple{Float64,Float64}
│           159 1 ── %1   = invoke Base.getindex(_2::DataFrame, :COLUMN::Symbol)::AbstractArray{T,1} where T
│               │           (Core.typeassert)(%1, Array{Float64,1})
│               │    %3   = π (%1, Array{Float64,1})

1 Answer 1

2

That's printing out exactly as I'd expect. There are three things that are happening here:

  • First the indexing: %1 = invoke Base.getindex — this is doing the indexing. It can return a vector of any type. This is indeed type-unstable.
  • Then the typeassert: (Core.typeassert)(%1, Array{Float64,1}) — this ensures that what getindex returned (in %1) is a Vector{Float64}. If it's not, Julia will throw an error.
  • And now the payoff: %3 = π (%1, Array{Float64,1}) — now that vector can be considered a Vector{Float64} since every other type would result in an error. From here on out, computations with the vector should be type-stable.

Adding type assertions like that don't "fix" the instability at its root, they simply patch it up so everything afterwards is fast.

Sign up to request clarification or add additional context in comments.

6 Comments

How do you suggest to make the indexing part type-stable?
Instead of passing the whole DataFrame, I have opted to pass only the array of floats. Let me know if it is possible to make it type stable passing the whole DataFrame
Pass Tables.columntable(df) to the function instead of df. The only limitation of this approach is that you will not be able to change columns in the data frame.
If it is efficient depends on what kind of workflow you use. DataFrame is type unstable on purpose as there are situations that it is preferable. If you want type stability use Tables.columntable function and pass the result to the function (at the cost of more rigidity).
Don't worry about it, I'd say. Not all instabilities are your enemy. In particular, this "instability" probably costs you few nanoseconds, and this is likely an O(1) cost in terms of the rest of your algorithm.
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.