2

I ran a benchmark test on the below mentioned flux code which is taken from model-zoo. I have noticed few performance issues:

  1. Flux is slower than python equivalent.
  2. Flux doesn't utilize all the threads for execution (usually the CPU usage is about 50%).

Code:

#model
using Flux
vgg19() = Chain(            
    Conv((3, 3), 3 => 64, relu, pad=(1, 1), stride=(1, 1)),
    Conv((3, 3), 64 => 64, relu, pad=(1, 1), stride=(1, 1)),
    MaxPool((2,2)),
    Conv((3, 3), 64 => 128, relu, pad=(1, 1), stride=(1, 1)),
    Conv((3, 3), 128 => 128, relu, pad=(1, 1), stride=(1, 1)),
    MaxPool((2,2)),
    Conv((3, 3), 128 => 256, relu, pad=(1, 1), stride=(1, 1)),
    Conv((3, 3), 256 => 256, relu, pad=(1, 1), stride=(1, 1)),
    Conv((3, 3), 256 => 256, relu, pad=(1, 1), stride=(1, 1)),
    MaxPool((2,2)),
    Conv((3, 3), 256 => 512, relu, pad=(1, 1), stride=(1, 1)),
    Conv((3, 3), 512 => 512, relu, pad=(1, 1), stride=(1, 1)),
    Conv((3, 3), 512 => 512, relu, pad=(1, 1), stride=(1, 1)),
    MaxPool((2,2)),
    Conv((3, 3), 512 => 512, relu, pad=(1, 1), stride=(1, 1)),
    Conv((3, 3), 512 => 512, relu, pad=(1, 1), stride=(1, 1)),
    Conv((3, 3), 512 => 512, relu, pad=(1, 1), stride=(1, 1)),
    BatchNorm(512),
    MaxPool((2,2)),
    flatten,
    Dense(512, 4096, relu),
    Dropout(0.5),
    Dense(4096, 4096, relu),
    Dropout(0.5),
    Dense(4096, 10),
    softmax
)

#data

using MLDatasets: CIFAR10
using Flux: onehotbatch
# Data comes pre-normalized in Julia
trainX, trainY = CIFAR10.traindata(Float32)
testX, testY = CIFAR10.testdata(Float32)
# One hot encode labels
trainY = onehotbatch(trainY, 0:9)
testY = onehotbatch(testY, 0:9)

#training

using Flux: crossentropy, @epochs
using Flux.Data: DataLoader
model = vgg19()
opt = Momentum(.001, .9)
loss(x, y) = crossentropy(model(x), y)
data = DataLoader(trainX, trainY, batchsize=64)
@epochs 100 Flux.train!(loss, params(model), data, opt)

I have tried running this code with sysimage including pre-compilation file, however the results were still not in favor of flux.

Please comment on my mistake in this code which is making it slower than python. As i was wondering the julia is supposed to be faster than python.

I have also posted this question on julia-discourse.

Thanks in advance!

6
  • 1
    x-ref: discourse.julialang.org/t/why-is-flux-model-slower-than-python/… Commented Jan 29, 2021 at 7:07
  • 1
    It is worth mentioning that most of the high performance libraries in python are written in c language and use python only as a kind of interface. So you should be careful on your expectation like "julia is supposed to be faster than this because this is python". Sometimes you will be comparing a c implementation against a julia one. Commented Jan 30, 2021 at 0:16
  • 1
    You need to start Julia with the parameter -t nThreads. See docs.julialang.org/en/v1/manual/multi-threading. Using Juno this should have been done automatically. Commented Jan 31, 2021 at 2:47
  • @crstnbr thanks for listing the reference, you may click on julia-discourse hyperlink in my article to reach there too. Commented Feb 2, 2021 at 7:02
  • @aramirezreyes thanks for the feedback. I see, you are right as listed by community on the discourse, tensorflow is written in C++ and using python as front end while flux is in julia. This means i am technically comparing julia to C++. Commented Feb 2, 2021 at 7:06

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.