This answer of @Dunes states, that due to pipeline-ing there is (almost) no difference between floating-point multiplication and division. However, from my expience with other languages I would expect the division to be slower.
My small test looks as follows:
A=np.random.rand(size)
command(A)
For different commands and size=1e8
I get the following times on my machine:
Command: Time[in sec]:
A/=0.5 2.88435101509
A/=0.51 5.22591209412
A*=2.0 1.1831600666
A*2.0 3.44263911247 //not in-place, more cache misses?
A+=A 1.2827270031
The most interesting part: dividing by 0.5
is almost twice as fast as dividing by 0.51
. One could assume, it is due to some smart optimization, e.g. replacing division by A+A
. However the timings of A*2
and A+A
are too far off to support this claim.
In general, the division by floats with values (1/2)^n
is faster:
Size: 1e8
Command: Time[in sec]:
A/=0.5 2.85750007629
A/=0.25 2.91607499123
A/=0.125 2.89376401901
A/=2.0 2.84901714325
A/=4.0 2.84493684769
A/=3.0 5.00480890274
A/=0.75 5.0354950428
A/=0.51 5.05687212944
It gets even more interesting, if we look at size=1e4
:
Command: 1e4*Time[in sec]:
A/=0.5 3.37723994255
A/=0.51 3.42854404449
A*=2.0 1.1587908268
A*2.0 1.19793796539
A+=A 1.11329007149
Now, there is no difference between division by .5
and by .51
!
I tried it out for different numpy versions and different machines. On some machines (e.g. Intel Xeon E5-2620) one can see this effect, but not on some other machines - and this does not depend on the numpy version.
With the script of @Ralph Versteegen (see his great answer!) I get the following results:
- timings with i5-2620 (Haswell, 2x6 cores, but a very old numpy version which does not use SIMD):
- timings with i7-5500U (Broadwell, 2 cores, numpy 1.11.2):
The question is: What is the reason for higher cost of the division by 0.51
compared to division by 0.5
for some processors, if the array sizes are large (>10^6).
The @nneonneo's answer states, that for some intel processors there is an optimization when divided by powers of two, but this does not explain, why we can see the benefit of it only for large arrays.
The original question was "How can these different behaviors (division by 0.5
vs. division by 0.51
) be explained?"
Here also, my original testing script, which produced the timings:
import numpy as np
import timeit
def timeit_command( command, rep):
print "\t"+command+"\t\t", min(timeit.repeat("for i in xrange(%d):"
%rep+command, "from __main__ import A", number=7))
sizes=[1e8, 1e4]
reps=[1, 1e4]
commands=["A/=0.5", "A/=0.51", "A*=2.2", "A*=2.0", "A*2.2", "A*2.0",
"A+=A", "A+A"]
for size, rep in zip(sizes, reps):
A=np.random.rand(size)
print "Size:",size
for command in commands:
timeit_command(command, rep)
%timeit
magic in IPython both seem to be take the same amount of time, whatever the array size.0.5
and0.51
:(