Does an algorithm exist that finds the maximum of an unsorted array in O(log n) time?
This question is asked a lot (is this a popular CS homework question or something?) and the answer is always the same: no.
Think about it mathematically. Unless the array is sorted, there is nothing to "cut in half" to give you the log(n)
behavior.
Read the question comments for a more in-depth discussion (which is probably way out of the question's scope anyhow).
-
What about Bitonic array? how do we find Bitonic number( max number )in an array with log(n) complexity? – Prasanna May 1 '20 at 19:28
-
Obviously the question refers to parallel computation in which context it is certainly possible. – Gregory Morse Feb 15 at 1:44
Consider this: without visiting every element, how do you know that some element you haven't visited isn't larger than the largest you have found so far?
It's not possible to do this in O(log(N))
. It is O(N)
in the best/worst/average case because one would need to visit every item in the array to determine if it is the larges one or not. Array is unsorted, which means you cannot cut corners.
Even in the case of parallelisation, this cannot be done in O(N)
, because Big-O notation doesn't care about how many CPU one has or what is the frequency of each CPU. It is abstracted from this specifically to give crude estamate of the problem.
Parallelisation can be neglected because time spent dividing a job can be considered equal to the time of sequential execution. This is due to the reason of constants being disregarded. The following are all the same:
O(N) = O(Const * N) = O(N / Const) = O(N + Const) = O(N - Const)
From the other hand, in practise, divide-and-conquer parallel algorithms can give you some performance benefits, so it may run a little bit faster. Fortunately, Big-O doesn't deal with this fine-grained algorithmic complexity analysis.
-
Big O analysis certainly deals with divide and conquer complexity. And if your parallelism is n/2 comparisons simultaneously then it reduces to O(log n) complexity. And the fine grained points are totally irrelevant if n is large enough and you actually had such an absurdly large number of processors. You would get practical time savings. Dividing the job is potentially a one time task, while the computation could be repeated many times. It's not particularly realistic to have huge numbers of dedicated processors waiting to solely do this task, but its theoretically and practically posible – Gregory Morse Feb 15 at 1:51
-
@GregoryMorse assuming it is possible to have an N-Core supercomputer. One can assume that it is possible to divide the task between every core. So that each core executes some computation task. Which would make this is an
O(1)
task, only if we didn't have to then conquer - compare the computation between each other to figure out the largest of the computed values. Comparison isO(1)
operation, but it needs to be repeatedN
times, as there is no way to not-compare something and still get a max value. ThusO(log n)
is impossible for this task, irrespective of a number of cores. – oleksii Feb 15 at 15:54 -
This is incorrect the conquer part is O(log n) steps. The tournament winner style algorithm means each step half as many comparisons are needed. So first compare n/2 in O(1) then n/4 in O(1) ... until 1 remains, the max. It's the archetypal log n algo – Gregory Morse Feb 16 at 16:14
-
The problem is, the array is not sorted. If it was a sorted array then one could try using something like this, but then the first (last) element would provide the result in
O(1)
. How would this idea work on4, 1, 0, 22, 7, 3, 5
, what if I move22
to the first/last place? This logic cannot be applied to unsorted data. – oleksii Feb 16 at 17:07 -
Actually the tournament algorithm applies EXACTLY to unsorted arrays. [4, 1, 0, 22, 7, 3, 5] -> [4, 22, 7, 5] -> [22, 7], -> [22]. It doesnt matter where it is. [22, 4, 1, 0, 7, 3, 5] -> [22, 1, 7, 5] -> [22, 7] -> [22]. [4, 1, 0, 7, 3, 5, 22] -> [4, 7, 5, 22] -> [7, 22] -> [22]. I suggest taking a formal algorithms course. It is where I learned about this. The odd number at any stage is a "by" to the next round of a tournament - just a ceiling function ultimately in the recurrence relation. Remember we are talking about parallelism here!!! Of course this is right for the non-parallel – Gregory Morse Feb 22 at 18:14
No. It's O(n). In the worst case all members of the array have to be visited and compared.
Of course NOT . suppose that there's still an element which you haven't still compared it with any other element . so there is no guarantee that the element you haven't compared is not the maximum element
and suppose that your comparing graph (vertices for elements and edges for comparing ) has more than one component . in this case you must put an edge (in the best way between maximum of two components).we can see that at n-1 operation MUST be done
This is very old, but I don't agree with the answers given. YES, it can be done, with parallel hardware, in logarithmic time.
Time complexity would be:
O(log(n) * log(m))
n
is the quantity of numbers to compare; m
is the size of each number.
However, hardware size would be:
O(n * m)
The algorithm would be:
Compare numbers in pairs. Time of this is
O(log(m))
, and size isO(n * m)
, using carry look-ahead comparators.Use the result in 1 to multiplex both inputs of 1. Time of this is
O(1)
, and size isO(n * m)
.Now you have an array half the initial size; go to step 1. This loop is repeated
log(n)
times, so total time isO(log(n) * log(m))
, and total size isO(n * m)
.
Adding some more MUXes you can also keep track of the index of the largest number, if you need it, without increasing the complexity of the algorithm.
-
2It is also possible to do it in O(1), but that is not practical; hardware size would grow very fast with n and m. – alx Apr 1 '18 at 0:16
If you use N
processors, it can be done in O(log N)
time.
But the Work Complexity is still O(N)
.
If using N^2
processors, you can reduce time complexity to O(1)
by applying the Usain Bolt algorithm.
k > n
. – Mysticial Sep 15 '12 at 20:36k
as some constant and thus ignore it. (2) Use thek
notation explicitly. I have never seen any book/article analyzing an algorithm assuming the number of coresk = f(n)
for some functionf
(besides constant, of course). If someone did - please reference me to this source and I'll revert my comment. – amit Sep 15 '12 at 20:43O(n)
, so you end up withO(n log n)
. Brent's theorem may help with some algorithm cascading here (the proof is nontrivial), but maybe I've misunderstood the concept. See uni-graz.at/~haasegu/Lectures/GPU_CUDA/Lit/reduction.pdf slide 30. – David Titarenco Sep 15 '12 at 20:45