354

I might have an array that looks like the following:

[1, 4, 2, 2, 6, 24, 15, 2, 60, 15, 6]

Or, really, any sequence of like-typed portions of data. What I want to do is ensure that there is only one of each identical element. For example, the above array would become:

[1, 4, 2, 6, 24, 15, 60]

Notice that the duplicates of 2, 6, and 15 were removed to ensure that there was only one of each identical element. Does Swift provide a way to do this easily, or will I have to do it myself?

8
  • 13
    The easiest way is to convert the array in an NSSet, NSSet is an unordered collection of objects, if need to keep order NSOrderedSet.
    – Andrea
    Sep 9, 2014 at 7:28
  • You could use the intersection function as you can find in this class with functions for arrays: github.com/pNre/ExSwift/blob/master/ExSwift/Array.swift Sep 9, 2014 at 7:31
  • 3
    Probably the most elegant, smartest and fastest answer is provided by mxcl's answer below. Which also helps maintain order
    – mfaani
    May 10, 2018 at 16:56
  • 1
    Why don't you just use Set from Swift ? You'll be able to provide a list of unordered and unique elements.
    – TibiaZ
    Mar 11, 2019 at 10:27
  • 1
    Apple finally realized this is a common problem and added their own solution in swift-algorithms. See MH175's answer.
    – Senseful
    Jun 18, 2022 at 21:11

49 Answers 49

663

You can convert to a Set and back to an Array again quite easily:

let unique = Array(Set(originals))

This is not guaranteed to maintain the original order of the array.

15
  • 55
    Is there a way to use a set while preserving the original order of the array?
    – Crashalot
    Jul 27, 2015 at 19:38
  • 8
    @Crashalot See my answer. Jul 29, 2015 at 18:21
  • 5
    If you need to keep the objects unique by a specific property, than also implement the Hashable and Equatable protocol on that class, instead of just using the Array->Set->Array transformation
    – Fawkes
    Sep 9, 2015 at 5:54
  • 10
    Fails if the elements in originals are not Hashable; only Hashable data types can be added to a Set, yet any data type can be added to an array.
    – Mecki
    Jul 9, 2019 at 15:21
  • 7
    I don't understand why this answer has so many upvotes. Seems like maintaining the order of the array is almost certainly a requirement. Otherwise you might as well just use a Set instead of an Array in the first place. Jun 17, 2020 at 23:49
233

You can roll your own, e.g. like this:

func unique<S : Sequence, T : Hashable>(source: S) -> [T] where S.Iterator.Element == T {
    var buffer = [T]()
    var added = Set<T>()
    for elem in source {
        if !added.contains(elem) {
            buffer.append(elem)
            added.insert(elem)
        }
    }
    return buffer
}

let vals = [1, 4, 2, 2, 6, 24, 15, 2, 60, 15, 6]
let uniqueVals = uniq(vals) // [1, 4, 2, 6, 24, 15, 60]

And as an extension for Array:

extension Array where Element: Hashable {
    func uniqued() -> Array {
        var buffer = Array()
        var added = Set<Element>()
        for elem in self {
            if !added.contains(elem) {
                buffer.append(elem)
                added.insert(elem)
            }
        }
        return buffer
    }
}

Or more elegantly (Swift 4/5):

extension Sequence where Element: Hashable {
    func uniqued() -> [Element] {
        var set = Set<Element>()
        return filter { set.insert($0).inserted }
    }
}

Which would be used:

[1,2,4,2,1].uniqued()  // => [1,2,4]
17
  • 13
    You could also implement the body of that function as var addedDict = [T:Bool](); return filter(source) { addedDict(true, forKey: $0) == nil } Dec 23, 2014 at 17:10
  • 1
    @AirspeedVelocity: Did you mean updateValue(true, forKey: $0)... instead of addedDict(true, forKey: $0)...
    – Jawwad
    Jan 25, 2015 at 1:38
  • 1
    Oops yes sorry I accidentally the method! Should be return filter(source) { addedDict.updateValue(true, forKey: $0) == nil } as you say. Jan 25, 2015 at 1:48
  • 27
    Just a word of caution: Avoid discussing performance for simple functions like this until you provably depend on their performance, at which point the only thing you should do is benchmark. Too often have I seen unmaintainable code or even less performant code due to making assumptions. :) Also, this is probably easier to grasp: let uniques = Array(Set(vals))
    – Blixt
    Feb 4, 2016 at 14:52
  • 20
    @Blixt Agreed. Once again, here the advantage lies in respecting the order of elements of the original array. Feb 4, 2016 at 19:28
118

Swift 4

public extension Array where Element: Hashable {
    func uniqued() -> [Element] {
        var seen = Set<Element>()
        return filter{ seen.insert($0).inserted }
    }
}

every attempt to insert will also return a tuple: (inserted: Bool, memberAfterInsert: Set.Element). See documentation.

Using the returned value means we can avoid doing more than one loop, so this is O(n).

5
  • 12
    After simple profiling, this method is really fast. Its hundreds times faster then using reduce( _: _:), or even reduce(into: _:)
    – Kelvin
    Apr 4, 2018 at 19:36
  • 7
    @Kelvin Because all those other algorithm were O(n^2), and nobody noticed.
    – Alexander
    Dec 16, 2018 at 20:24
  • @Kelvin this answer is identical to Eneko Alonso answer + my comment (Jun 16 '17).
    – Cœur
    Dec 16, 2018 at 23:59
  • 2
    Beware that duplicity is determined by Hash values of two objects/structs and not the == equality in Equatable protocol. If you want to determine duplicity lets say in struct hased only on some properties (not all properties), you need to use custom Hashable implementation with func hash(into hasher: inout Hasher) Nov 22, 2022 at 14:14
  • We will use Set<Element>(minimumCapacity: count) initialiser in this answer. Mar 23 at 9:17
93

Use a Set or NSOrderedSet to remove duplicates, then convert back to an Array:

let uniqueUnordered = Array(Set(array))
let uniqueOrdered = Array(NSOrderedSet(array: array))
5
  • 2
    let uniqueOrderedNames = Array(NSOrderedSet(array: userNames)) as! [String] if u have array of String, not of Any Jun 18, 2018 at 13:07
  • Fails if the elements in array are not Hashable; only Hashable data types can be added to a Set, yet any data type can be added to an array.
    – Mecki
    Jul 9, 2019 at 15:18
  • Tested in Swift 5.1b5, given that the elements are Hashable and a desire to retain ordering, the NSOrderedSet(array: array).array is marginally faster than the pure swift func uniqued() using a set with filter. I tested with 5100 strings that resulted in 13 unique values.
    – dlemex
    Aug 1, 2019 at 22:41
  • 5
    Array(NSOrderedSet(array: array)) is not working in Swift 5. Use NSOrderedSet(array: array).array as! [String] instead.
    – Lindemann
    Dec 5, 2020 at 3:40
  • 1
    The second one only works for "primitive" types Mar 27, 2021 at 9:15
80

Many answers available here, but I missed this simple extension, suitable for Swift 2 and up:

extension Array where Element:Equatable {
    func removeDuplicates() -> [Element] {
        var result = [Element]()

        for value in self {
            if result.contains(value) == false {
                result.append(value)
            }
        }

        return result
    }
}

Makes it super simple. Can be called like this:

let arrayOfInts = [2, 2, 4, 4]
print(arrayOfInts.removeDuplicates()) // Prints: [2, 4]

Filtering based on properties

To filter an array based on properties, you can use this method:

extension Array {

    func filterDuplicates(@noescape includeElement: (lhs:Element, rhs:Element) -> Bool) -> [Element]{
        var results = [Element]()

        forEach { (element) in
            let existingElements = results.filter {
                return includeElement(lhs: element, rhs: $0)
            }
            if existingElements.count == 0 {
                results.append(element)
            }
        }

        return results
    }
}

Which you can call as followed:

let filteredElements = myElements.filterDuplicates { $0.PropertyOne == $1.PropertyOne && $0.PropertyTwo == $1.PropertyTwo }
5
  • @Antoine Thank you for the Filtering based on properties extension. It's really useful. But can you please explain how it works. It's too hard to understand for me. Thank you
    – Mostafa
    Oct 4, 2016 at 0:08
  • Updates for swift 3: func filterDuplicates(_ includeElement: (_ lhs:Element, _ rhs:Element) -> Bool) -> [Element]{
    – cbartel
    May 9, 2017 at 4:02
  • The first part of this answer (extension Array where Element: Equatable) is being superseded by stackoverflow.com/a/36048862/1033581 which offers a more powerful solution (extension Sequence where Iterator.Element: Equatable).
    – Cœur
    Jun 15, 2017 at 9:27
  • 13
    This will have O(n²) time performance, which is really bad for large arrays.
    – Duncan C
    Sep 3, 2018 at 16:01
  • 2
    You should use a set to keep track of elements seen so far, to bring this terrible O(n²) complexity back down to O(n)
    – Alexander
    Dec 16, 2018 at 20:45
65

If you put both extensions in your code, the faster Hashable version will be used when possible, and the Equatable version will be used as a fallback.

public extension Sequence where Element: Hashable {
  /// The elements of the sequence, with duplicates removed.
  /// - Note: Has equivalent elements to `Set(self)`.
  var firstUniqueElements: [Element] {
    let getSelf: (Element) -> Element = \.self
    return firstUniqueElements(getSelf)
  }
}

public extension Sequence where Element: Equatable {
  /// The elements of the sequence, with duplicates removed.
  /// - Note: Has equivalent elements to `Set(self)`.
  var firstUniqueElements: [Element] {
    let getSelf: (Element) -> Element = \.self
    return firstUniqueElements(getSelf)
  }
}

public extension Sequence {
  /// The elements of the sequences, with "duplicates" removed
  /// based on a closure.
  func firstUniqueElements<Hashable: Swift.Hashable>(
    _ getHashable: (Element) -> Hashable
  ) -> [Element] {
    var set: Set<Hashable> = []
    return filter { set.insert(getHashable($0)).inserted }
  }

  /// The elements of the sequence, with "duplicates" removed,
  /// based on a closure.
  func firstUniqueElements<Equatable: Swift.Equatable>(
    _ getEquatable: (Element) -> Equatable
  ) -> [Element] {
    reduce(into: []) { uniqueElements, element in
      if zip(
        uniqueElements.lazy.map(getEquatable),
        AnyIterator { [equatable = getEquatable(element)] in equatable }
      ).allSatisfy(!=) {
        uniqueElements.append(element)
      }
    }
  }
}

If order isn't important, then you can always just use this Set initializer.

9
  • 1
    @DavidSeek like this, uniqueArray = nonUniqueArray.uniqueElements
    – Mert Celik
    Aug 16, 2018 at 21:29
  • 1
    yeah dont worry. got it working right after. been almost 2 years now :P
    – David Seek
    Aug 16, 2018 at 22:11
  • 2
    This will have O(n²) time performance, which is really bad for large arrays.
    – Duncan C
    Sep 3, 2018 at 15:59
  • 1
    The hahsable version will have better performance, but won't preserve the order of elements in the original array. Leo's answer will give both O(n) performance AND preserve object ordering.
    – Duncan C
    Sep 3, 2018 at 16:03
  • 1
    @Jessy There are already multiple O(1) answers, but they have way less votes than most of the naive O(n^2) solutions. This one is particularly good for its simplicity: stackoverflow.com/a/46354989/3141234
    – Alexander
    Dec 17, 2018 at 0:41
49

edit/update Swift 4 or later

We can also extend RangeReplaceableCollection protocol to allow it to be used with StringProtocol types as well:

extension RangeReplaceableCollection where Element: Hashable {
    var orderedSet: Self {
        var set = Set<Element>()
        return filter { set.insert($0).inserted }
    }
    mutating func removeDuplicates() {
        var set = Set<Element>()
        removeAll { !set.insert($0).inserted }
    }
}

let integers = [1, 4, 2, 2, 6, 24, 15, 2, 60, 15, 6]
let integersOrderedSet = integers.orderedSet // [1, 4, 2, 6, 24, 15, 60]

"abcdefabcghi".orderedSet  // "abcdefghi"
"abcdefabcghi".dropFirst(3).orderedSet // "defabcghi"

Mutating method:

var string = "abcdefabcghi"
string.removeDuplicates() 
string  //  "abcdefghi"

var substring = "abcdefabcdefghi".dropFirst(3)  // "defabcdefghi"
substring.removeDuplicates()
substring   // "defabcghi"

For Swift 3 click here

15
  • 1
    I like this, it works with an array of dictionarys too!
    – DeyaEldeen
    Sep 28, 2016 at 12:01
  • 2
    @Alexander Leo Dabus has replaced the reduce implementation, so now the complexity is different.
    – Cœur
    Jun 16, 2017 at 3:08
  • 1
    The results are interesting. For both 1 million unique items and 8 million, the filter version is faster. However, the filter-based version takes 8.38x longer for 8 million unique items (a hair over O(n) time), where the flatmap-based version takes 7.47x longer for 8 million unique entries than 1 million, suggesting that the flatmap based version scales better. Somehow the flatmap based version does slightly better than O(n) time!
    – Duncan C
    Sep 12, 2017 at 16:00
  • 1
    In fact, when I run the test with 64x more items in the array, the flatmap based version is faster.
    – Duncan C
    Sep 12, 2017 at 16:09
  • 1
    @Jessy yes it keeps the original order. It is not called sortedSet. Btw it uses the same name Apple uses in NSOrderedSet just dropping the NS
    – Leo Dabus
    Sep 3, 2018 at 23:26
32

Swift 4

Guaranteed to keep ordering.

extension Array where Element: Equatable {
    func removingDuplicates() -> Array {
        return reduce(into: []) { result, element in
            if !result.contains(element) {
                result.append(element)
            }
        }
    }
}
8
  • I use this now, only changed the method name to removeDuplicates :)
    – J. Doe
    Mar 28, 2018 at 19:22
  • I guess this solution is compact, but I believe that deanWombourne solution posted a year earlier may be slightly more efficient than a reduce: overall, it's just one more line in your whole project to write your function as: var unique: [Iterator.Element] = []; for element in self where !unique.contains(element) { unique.append(element) }; return unique. I admit I haven't tested the relative performances yet.
    – Cœur
    Aug 21, 2018 at 11:25
  • 12
    This will have O(n²) time performance, which is really bad for large arrays.
    – Duncan C
    Sep 3, 2018 at 15:58
  • @NickGaens No it's not, it's O(n²). There's nothing swift about this.
    – Alexander
    Dec 16, 2018 at 20:32
  • @Cœur reduce or reduce(into:) wouldn't make a critical difference. Rewriting this to not repeatedly call contains would make a MUCH bigger difference.
    – Alexander
    Dec 16, 2018 at 20:33
29

Inspired by https://www.swiftbysundell.com/posts/the-power-of-key-paths-in-swift, we can declare a more powerful tool that is able to filter for unicity on any keyPath. Thanks to Alexander comments on various answers regarding complexity, the below solutions should be near optimal.

Non-mutating solution

We extend with a function that is able to filter for unicity on any keyPath:

extension RangeReplaceableCollection {
    /// Returns a collection containing, in order, the first instances of
    /// elements of the sequence that compare equally for the keyPath.
    func unique<T: Hashable>(for keyPath: KeyPath<Element, T>) -> Self {
        var unique = Set<T>()
        return filter { unique.insert($0[keyPath: keyPath]).inserted }
    }
}

Note: in the case where your object doesn't conform to RangeReplaceableCollection, but does conform to Sequence, you can have this additional extension, but the return type will always be an Array:

extension Sequence {
    /// Returns an array containing, in order, the first instances of
    /// elements of the sequence that compare equally for the keyPath.
    func unique<T: Hashable>(for keyPath: KeyPath<Element, T>) -> [Element] {
        var unique = Set<T>()
        return filter { unique.insert($0[keyPath: keyPath]).inserted }
    }
}

Usage

If we want unicity for elements themselves, as in the question, we use the keyPath \.self:

let a = [1, 4, 2, 2, 6, 24, 15, 2, 60, 15, 6]
let b = a.unique(for: \.self)
/* b is [1, 4, 2, 6, 24, 15, 60] */

If we want unicity for something else (like for the id of a collection of objects) then we use the keyPath of our choice:

let a = [CGPoint(x: 1, y: 1), CGPoint(x: 2, y: 1), CGPoint(x: 1, y: 2)]
let b = a.unique(for: \.y)
/* b is [{x 1 y 1}, {x 1 y 2}] */

Mutating solution

We extend with a mutating function that is able to filter for unicity on any keyPath:

extension RangeReplaceableCollection {
    /// Keeps only, in order, the first instances of
    /// elements of the collection that compare equally for the keyPath.
    mutating func uniqueInPlace<T: Hashable>(for keyPath: KeyPath<Element, T>) {
        var unique = Set<T>()
        removeAll { !unique.insert($0[keyPath: keyPath]).inserted }
    }
}

Usage

If we want unicity for elements themselves, as in the question, we use the keyPath \.self:

var a = [1, 4, 2, 2, 6, 24, 15, 2, 60, 15, 6]
a.uniqueInPlace(for: \.self)
/* a is [1, 4, 2, 6, 24, 15, 60] */

If we want unicity for something else (like for the id of a collection of objects) then we use the keyPath of our choice:

var a = [CGPoint(x: 1, y: 1), CGPoint(x: 2, y: 1), CGPoint(x: 1, y: 2)]
a.uniqueInPlace(for: \.y)
/* a is [{x 1 y 1}, {x 1 y 2}] */
7
  • 1
    Now that's a good implementation! I only with that key paths were convertible to closures, so that you can use a closure arg to support both arbitrary code (in closures) and mere property look ups (via key paths). Only change I would make is to make keyPath default to \.self, because that's probably the majority of use cases.
    – Alexander
    Apr 15, 2019 at 15:42
  • 1
    @Alexander I tried to default to Self, but then I would need to make Element always Hashable. An alternative to a default value is adding a simple overload without parameters: extension Sequence where Element: Hashable { func unique() { ... } }
    – Cœur
    Apr 15, 2019 at 18:33
  • Ah yes, makes sense!
    – Alexander
    Apr 15, 2019 at 20:49
  • 1
    Brilliant ... simple, and best of all 'flexible'. Thx. Oct 1, 2019 at 22:08
  • @Alexander-ReinstateMonica: This looks very similar to your own solution from March 2018: gist.github.com/amomchilov/fbba1e58c91fbd4b5b767bcf8586112b 👏👏👏
    – FizzBuzz
    Jul 2, 2020 at 22:51
28

Here's a category on SequenceType which preserves the original order of the array, but uses a Set to do the contains lookups to avoid the O(n) cost on Array's contains(_:) method.

public extension Sequence where Element: Hashable {

    /// Return the sequence with all duplicates removed.
    ///
    /// i.e. `[ 1, 2, 3, 1, 2 ].uniqued() == [ 1, 2, 3 ]`
    ///
    /// - note: Taken from stackoverflow.com/a/46354989/3141234, as 
    ///         per @Alexander's comment.
    func uniqued() -> [Element] {
        var seen = Set<Element>()
        return self.filter { seen.insert($0).inserted }
    }
}

If you aren't Hashable or Equatable, you can pass in a predicate to do the equality check:

extension Sequence {

    /// Return the sequence with all duplicates removed.
    ///
    /// Duplicate, in this case, is defined as returning `true` from `comparator`.
    ///
    /// - note: Taken from stackoverflow.com/a/46354989/3141234
    func uniqued(comparator: @escaping (Element, Element) throws -> Bool) rethrows -> [Element] {
        var buffer: [Element] = []

        for element in self {
            // If element is already in buffer, skip to the next element
            if try buffer.contains(where: { try comparator(element, $0) }) {
                continue
            }

            buffer.append(element)
        }

        return buffer
    }
}

Now, if you don't have Hashable, but are Equatable, you can use this method:

extension Sequence where Element: Equatable {

    /// Return the sequence with all duplicates removed.
    ///
    /// i.e. `[ 1, 2, 3, 1, 2 ].uniqued() == [ 1, 2, 3 ]`
    ///
    /// - note: Taken from stackoverflow.com/a/46354989/3141234
    func uniqued() -> [Element] {
        return self.uniqued(comparator: ==)
    }
}

Finally, you can add a key path version of uniqued like this:

extension Sequence {

    /// Returns the sequence with duplicate elements removed, performing the comparison using the property at
    /// the supplied keypath.
    ///
    /// i.e.
    ///
    /// ```
    /// [
    ///   MyStruct(value: "Hello"),
    ///   MyStruct(value: "Hello"),
    ///   MyStruct(value: "World")
    ///  ].uniqued(\.value)
    /// ```
    /// would result in
    ///
    /// ```
    /// [
    ///   MyStruct(value: "Hello"),
    ///   MyStruct(value: "World")
    /// ]
    /// ```
    ///
    /// - note: Taken from stackoverflow.com/a/46354989/3141234
    ///
    func uniqued<T: Equatable>(_ keyPath: KeyPath<Element, T>) -> [Element] {
        self.uniqued { $0[keyPath: keyPath] == $1[keyPath: keyPath] }
    }
}

You can stick both of these into your app, Swift will choose the right one depending on your sequence's Iterator.Element type.


For El Capitan, you can extend this method to include multiple keypaths like this:

    /// Returns the sequence with duplicate elements removed, performing the comparison using the property at
    /// the supplied keypaths.
    ///
    /// i.e.
    ///
    /// ```
    /// [
    ///   MyStruct(value1: "Hello", value2: "Paula"),
    ///   MyStruct(value1: "Hello", value2: "Paula"),
    ///   MyStruct(value1: "Hello", value2: "Bean"),
    ///   MyStruct(value1: "World", value2: "Sigh")
    ///  ].uniqued(\.value1, \.value2)
    /// ```
    /// would result in
    ///
    /// ```
    /// [
    ///   MyStruct(value1: "Hello", value2: "Paula"),
    ///   MyStruct(value1: "Hello", value2: "Bean"),
    ///   MyStruct(value1: "World", value2: "Sigh")
    /// ]
    /// ```
    ///
    /// - note: Taken from stackoverflow.com/a/46354989/3141234
    ///
    func uniqued<T: Equatable, U: Equatable>(_ keyPath1: KeyPath<Element, T>, _ keyPath2: KeyPath<Element, U>) -> [Element] {
        self.uniqued {
            $0[keyPath: keyPath1] == $1[keyPath: keyPath1] && $0[keyPath: keyPath2] == $1[keyPath: keyPath2]
        }
    }

but (imho) you're probably better off just passing in your own block to self.uniqued.

5
  • Heyyy finally someone with an O(n) solution. You can combine the "check" and "insert" set operations into one, by the way. See stackoverflow.com/a/46354989/3141234
    – Alexander
    Dec 16, 2018 at 20:35
  • Oh, that's clever :) Dec 18, 2018 at 9:59
  • @deanWombourne How to distinct elements by multiple keypaths? Aug 15, 2021 at 6:39
  • @EICaptainv2.0 You can just extend the uniqued method to take two generic parameters and check them both for equality - check out the edit I just made. The items are only duplicates if both of the values specified by the keypaths are the same. Aug 16, 2021 at 11:56
  • Cool. Thanks @deanWombourne Aug 19, 2021 at 4:05
17

No need to write any extensions.

Apple has finally introduced uniqued() method in its Algorithms package.

Points to consider:

  1. Not just Arrays. It can be used on any type conforming to Sequence protocol.
  2. The Element type in Sequence must conform to Hashable protocol

Example:

import Algorithms

let numbers = [1, 2, 3, 3, 2, 3, 3, 2, 2, 2, 1]
print(numbers.uniqued()) // prints [1, 2, 3]

More info https://github.com/apple/swift-algorithms/blob/main/Guides/Unique.md

3
  • Algorithms not from Apple i guess.
    – Umit Kaya
    Mar 4, 2022 at 9:09
  • @UmitKaya It is absolutely from Apple, just published as a separate package to avoid littering Foundation. Mar 12, 2022 at 18:51
  • 1
    Note that this still requires that the elements conform to Hashable
    – damd
    Apr 21, 2022 at 12:16
16

Think like a functional programmer :)

To filter the list based on whether the element has already occurred, you need the index. You can use enumerated to get the index and map to return to the list of values.

let unique = myArray
    .enumerated()
    .filter{ myArray.firstIndex(of: $0.1) == $0.0 }
    .map{ $0.1 }

This guarantees the order. If you don't mind about the order then the existing answer of Array(Set(myArray)) is simpler and probably more efficient.


UPDATE: Some notes on efficiency and correctness

A few people have commented on the efficiency. I'm definitely in the school of writing correct and simple code first and then figuring out bottlenecks later, though I appreciate it's debatable whether this is clearer than Array(Set(array)).

This method is a lot slower than Array(Set(array)). As noted in comments, it does preserve order and works on elements that aren't Hashable.

However, @Alain T's method also preserves order and is also a lot faster. So unless your element type is not hashable, or you just need a quick one liner, then I'd suggest going with their solution.

Here are a few tests on a MacBook Pro (2014) on Xcode 11.3.1 (Swift 5.1) in Release mode.

The profiler function and two methods to compare:

func printTimeElapsed(title:String, operation:()->()) {
    var totalTime = 0.0
    for _ in (0..<1000) {
        let startTime = CFAbsoluteTimeGetCurrent()
        operation()
        let timeElapsed = CFAbsoluteTimeGetCurrent() - startTime
        totalTime += timeElapsed
    }
    let meanTime = totalTime / 1000
    print("Mean time for \(title): \(meanTime) s")
}

func method1<T: Hashable>(_ array: Array<T>) -> Array<T> {
    return Array(Set(array))
}

func method2<T: Equatable>(_ array: Array<T>) -> Array<T>{
    return array
    .enumerated()
    .filter{ array.firstIndex(of: $0.1) == $0.0 }
    .map{ $0.1 }
}

// Alain T.'s answer (adapted)
func method3<T: Hashable>(_ array: Array<T>) -> Array<T> {
    var uniqueKeys = Set<T>()
    return array.filter{uniqueKeys.insert($0).inserted}
}

And a small variety of test inputs:

func randomString(_ length: Int) -> String {
  let letters = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"
  return String((0..<length).map{ _ in letters.randomElement()! })
}

let shortIntList = (0..<100).map{_ in Int.random(in: 0..<100) }
let longIntList = (0..<10000).map{_ in Int.random(in: 0..<10000) }
let longIntListManyRepetitions = (0..<10000).map{_ in Int.random(in: 0..<100) }
let longStringList = (0..<10000).map{_ in randomString(1000)}
let longMegaStringList = (0..<10000).map{_ in randomString(10000)}

Gives as output:

Mean time for method1 on shortIntList: 2.7358531951904296e-06 s
Mean time for method2 on shortIntList: 4.910230636596679e-06 s
Mean time for method3 on shortIntList: 6.417632102966309e-06 s
Mean time for method1 on longIntList: 0.0002518167495727539 s
Mean time for method2 on longIntList: 0.021718120217323302 s
Mean time for method3 on longIntList: 0.0005312927961349487 s
Mean time for method1 on longIntListManyRepetitions: 0.00014377200603485108 s
Mean time for method2 on longIntListManyRepetitions: 0.0007293639183044434 s
Mean time for method3 on longIntListManyRepetitions: 0.0001843773126602173 s
Mean time for method1 on longStringList: 0.007168249964714051 s
Mean time for method2 on longStringList: 0.9114790915250778 s
Mean time for method3 on longStringList: 0.015888616919517515 s
Mean time for method1 on longMegaStringList: 0.0525397013425827 s
Mean time for method2 on longMegaStringList: 1.111266262292862 s
Mean time for method3 on longMegaStringList: 0.11214958941936493 s
12
  • 1
    unlike Array(Set(myArray)), this works for things that aren't Hashable Oct 21, 2019 at 20:11
  • 2
    ... and unlike Array(Set(myArray)) the order of your array is maintained. Feb 10, 2020 at 6:50
  • It looks like the best answer to me, at least at the present when Swift 5 is already the current version.
    – oradyvan
    Feb 13, 2020 at 10:22
  • 1
    @TimMB Oh I misread your post. I saw someone's adaptation that usedthe lastIndex(of:). I totally disagree over the clarity vs optimization point in this case. I don't think this implementation is particularly clear, especially compared to a simple set-based solution. In any case, such code should be extracted out to an extension function. This algorithm becomes basically unusable at even a low input size, like in the thousands to tens of thousands. It's not hard to find such data sets, people can have thousands of songs, files, contacts, etc.
    – Alexander
    Feb 16, 2020 at 16:21
  • 1
    Check out my bench mark results: drive.google.com/a/ryerson.ca/file/d/… full code: gist.github.com/amomchilov/299d012dccba375bf15880355684ebed
    – Alexander
    Feb 18, 2020 at 15:16
13

One more Swift 3.0 solution to remove duplicates from an array. This solution improves on many other solutions already proposed by:

  • Preserving the order of the elements in the input array
  • Linear complexity O(n): single pass filter O(n) + set insertion O(1)

Given the integer array:

let numberArray = [10, 1, 2, 3, 2, 1, 15, 4, 5, 6, 7, 3, 2, 12, 2, 5, 5, 6, 10, 7, 8, 3, 3, 45, 5, 15, 6, 7, 8, 7]

Functional code:

func orderedSet<T: Hashable>(array: Array<T>) -> Array<T> {
    var unique = Set<T>()
    return array.filter { element in
        return unique.insert(element).inserted
    }
}

orderedSet(array: numberArray)  // [10, 1, 2, 3, 15, 4, 5, 6, 7, 12, 8, 45]

Array extension code:

extension Array where Element:Hashable {
    var orderedSet: Array {
        var unique = Set<Element>()
        return filter { element in
            return unique.insert(element).inserted
        }
    }
}

numberArray.orderedSet // [10, 1, 2, 3, 15, 4, 5, 6, 7, 12, 8, 45]

This code takes advantage of the result returned by the insert operation on Set, which executes on O(1), and returns a tuple indicating if the item was inserted or if it already existed in the set.

If the item was in the set, filter will exclude it from the final result.

4
  • 1
    Not to be picky but you will be performing the insert and the membership test as many times as there are elements so you should count their cost as O(n) as well. This doesn't mean 3xO(n) however because these O's and don't have equal cost with the filter so the addition of O(n)'s is apples to oranges. If we consider set operations to be a O(1) part of the filter cost, the complexity is merely O(n), albeit with a larger "O". Pushing this to the limit, you could also avoid the insertions when the element is already in the set.
    – Alain T.
    Mar 16, 2017 at 17:10
  • You are right, using defer the code would do the set test operation twice, one with contains and one with insert. Further reading Swift documentation, I found that insert returns a tuple indicating if the element was inserted or not, so I've simplified the code removing the contains check. Mar 16, 2017 at 17:41
  • 3
    Nice. Your extension could be optimal by doing it on extension Sequence where Iterator.Element: Hashable { ... }
    – Cœur
    Jun 16, 2017 at 3:00
  • 2
    @AlainT. Nope. Both insert and contains have O(1) complexity. O(1) + O(1) = O(1). These two operations are then done n times (once per call of the closure passed to filter, which is called once per element) I.e. if an operation takes a constant amount of time regardless of input size, then doing it twice still makes it take a constant time that is regardless of input size. The total complexity of this is O(n).
    – Alexander
    Dec 16, 2018 at 20:44
13

An alternate (if not optimal) solution from here using immutable types rather than variables:

func deleteDuplicates<S: ExtensibleCollectionType where S.Generator.Element: Equatable>(seq:S)-> S {
    let s = reduce(seq, S()){
        ac, x in contains(ac,x) ? ac : ac + [x]
    }
    return s
}

Included to contrast Jean-Pillippe's imperative approach with a functional approach.

As a bonus this function works with strings as well as arrays!

Edit: This answer was written in 2014 for Swift 1.0 (before Set was available in Swift). It doesn't require Hashable conformance & runs in quadratic time.

2
  • 8
    Beware, there are not one, but two ways this runs in quadratic time – both contains and array append run in O(n). Though it does have the benefit of only requiring equatable, not hashable. Dec 23, 2014 at 17:03
  • this is a really complicated way of writing filter. It's O(n^2) (which is required if you don't want to require Hashable conformance), but you should at least call that out explicitly
    – Alexander
    Dec 16, 2018 at 21:13
13

Swift 5

extension Sequence where Element: Hashable {
    func unique() -> [Element] {
        NSOrderedSet(array: self as! [Any]).array as! [Element]
    }
}
4
  • I did some variation so I could select a key to compare. extension Sequence { // Returns distinct elements based on a key value. func distinct<key: Hashable>(by: ((_ el: Iterator.Element) -> key)) -> [Iterator.Element] { var existing = Set<key>() return self.filter { existing.insert(by($0)).inserted } } } Aug 7, 2018 at 14:10
  • There no need to use a Bool, when the only value you use is true. You're reaching for a "unit type" (a type with only one possible value). Swift's unit type is Void, whose only value is () (a.k.a. the empty tuple). So you can just use [T: Void]. Though you shouldn't do that, because you've basically just invented Set. Use Set instead. See stackoverflow.com/a/55684308/3141234 Please delete this answer.
    – Alexander
    Feb 15, 2020 at 3:46
  • if your element is Hasable, you can directly use Array(Set(yourElements) Mar 27, 2021 at 8:58
  • This changes the order of the array. Nov 29, 2021 at 23:21
11

swift 2

with uniq function answer:

func uniq<S: SequenceType, E: Hashable where E==S.Generator.Element>(source: S) -> [E] {
    var seen: [E:Bool] = [:]
    return source.filter({ (v) -> Bool in
        return seen.updateValue(true, forKey: v) == nil
    })
}

use:

var test = [1,2,3,4,5,6,7,8,9,9,9,9,9,9]
print(uniq(test)) //1,2,3,4,5,6,7,8,9
1
  • The Bool value obviously is redundant, as your code never reads it. Use a Set instead of a Dictionary and you get my upvote. Feb 8, 2016 at 10:04
10

Swift 4.x:

extension Sequence where Iterator.Element: Hashable {
  func unique() -> [Iterator.Element] {
    return Array(Set<Iterator.Element>(self))
  }

  func uniqueOrdered() -> [Iterator.Element] {
    return reduce([Iterator.Element]()) { $0.contains($1) ? $0 : $0 + [$1] }
  }
}

usage:

["Ljubljana", "London", "Los Angeles", "Ljubljana"].unique()

or

["Ljubljana", "London", "Los Angeles", "Ljubljana"].uniqueOrdered()
1
  • This is O(n^2). Don't do this.
    – Alexander
    Dec 16, 2018 at 20:23
10

In Swift 5

 var array: [String] =  ["Aman", "Sumit", "Aman", "Sumit", "Mohan", "Mohan", "Amit"]

 let uniq = Array(Set(array))
 print(uniq)

Output Will be

 ["Sumit", "Mohan", "Amit", "Aman"]
1
  • 2
    This is a repeat of many of the answers already here, and it doesn't preserve ordering.
    – Alexander
    Feb 15, 2020 at 3:37
8

For arrays where the elements are neither Hashable nor Comparable (e.g. complex objects, dictionaries or structs), this extension provides a generalized way to remove duplicates:

extension Array
{
   func filterDuplicate<T:Hashable>(_ keyValue:(Element)->T) -> [Element]
   {
      var uniqueKeys = Set<T>()
      return filter{uniqueKeys.insert(keyValue($0)).inserted}
   }

   func filterDuplicate<T>(_ keyValue:(Element)->T) -> [Element]
   { 
      return filterDuplicate{"\(keyValue($0))"}
   }
}

// example usage: (for a unique combination of attributes):

peopleArray = peopleArray.filterDuplicate{ ($0.name, $0.age, $0.sex) }

or...

peopleArray = peopleArray.filterDuplicate{ "\(($0.name, $0.age, $0.sex))" }

You don't have to bother with making values Hashable and it allows you to use different combinations of fields for uniqueness.

Note: for a more robust approach, please see the solution proposed by Coeur in the comments below.

stackoverflow.com/a/55684308/1033581

[EDIT] Swift 4 alternative

With Swift 4.2 you can use the Hasher class to build a hash much easier. The above extension could be changed to leverage this :

extension Array
{
    func filterDuplicate(_ keyValue:((AnyHashable...)->AnyHashable,Element)->AnyHashable) -> [Element]
    {
        func makeHash(_ params:AnyHashable ...) -> AnyHashable
        { 
           var hash = Hasher()
           params.forEach{ hash.combine($0) }
           return hash.finalize()
        }  
        var uniqueKeys = Set<AnyHashable>()
        return filter{uniqueKeys.insert(keyValue(makeHash,$0)).inserted}     
    }
}

The calling syntax is a little different because the closure receives an additional parameter containing a function to hash a variable number of values (which must be Hashable individually)

peopleArray = peopleArray.filterDuplicate{ $0($1.name, $1.age, $1.sex) } 

It will also work with a single uniqueness value (using $1 and ignoring $0).

peopleArray = peopleArray.filterDuplicate{ $1.name } 
5
  • This could give random results depending on the behavior of "\()", as it may not gives you unique values like conforming to Hashable should. Example, if your elements conform to Printable by all returning the same description, then your filtering fails.
    – Cœur
    Jun 16, 2017 at 1:41
  • Agreed. Selection of the fields (or formula) that will produce the desired uniqueness pattern will have to take this into account. For many use cases, this provides a simple ad-hoc solution that requires no alteration of the element's class or struct.
    – Alain T.
    Jun 16, 2017 at 12:27
  • 2
    @AlainT. Don't do this, really. String's purpose is not to be some ghetto ad-hoc key generation mechanism. Just constrain T to being Hashable.
    – Alexander
    Dec 16, 2018 at 20:37
  • @Alexander I've applied this idea in a new answer: stackoverflow.com/a/55684308/1033581
    – Cœur
    Apr 15, 2019 at 7:26
  • Perfect answer as i want. Thank you so much. Jun 7, 2019 at 7:24
7

As was noted at WWDC 2021, Swift has community-developed Algorithms, Collections, and Numerics Packages. The Algorithms package features a uniqued() algorithm.

These are not yet part of the Swift Standard library. You can currently download them from Apple's Github page and/or install them via Swift Package Manager.

WWDC Video:

https://developer.apple.com/videos/play/wwdc2021/10256/

Github page:

https://github.com/apple/swift-algorithms

uniqued() and uniqued(on:) documentation:

https://github.com/apple/swift-algorithms/blob/main/Guides/Unique.md

2
  • 1
    This should be the top answer
    – maxpower
    Dec 1, 2021 at 17:51
  • How do I do this in iOS?
    – daniel
    Aug 9, 2022 at 4:44
5
  1. First add all the elements of an array to NSOrderedSet.
  2. This will remove all the duplicates in your array.
  3. Again convert this orderedset to an array.

Done....

Example

let array = [1,1,1,1,2,2,2,2,4,6,8]

let orderedSet : NSOrderedSet = NSOrderedSet(array: array)

let arrayWithoutDuplicates : NSArray = orderedSet.array as NSArray

output of arrayWithoutDuplicates - [1,2,4,6,8]

4

You can use directly a set collection to remove duplicate, then cast it back to an array

var myArray = [1, 4, 2, 2, 6, 24, 15, 2, 60, 15, 6]
var mySet = Set<Int>(myArray)

myArray = Array(mySet) // [2, 4, 60, 6, 15, 24, 1]

Then you can order your array as you want

myArray.sort{$0 < $1} // [1, 2, 4, 6, 15, 24, 60]
1
  • 1
    "Then you can order your array as you want" What if I want the same ordering as from the original array? It's not that easy.
    – Alexander
    Dec 16, 2018 at 21:11
4

In case you need values sorted, this works (Swift 4)

let sortedValues = Array(Set(array)).sorted()

8
  • 2
    You loosing elements order in this case.
    – Shmidt
    Nov 16, 2018 at 18:13
  • Not at all, that's what the .sorted() at the end is for. Regards. Nov 19, 2018 at 18:44
  • @MauricioChirino And if your original array was [2, 1, 1]? It would come out [1, 2], that's not ordered :p
    – Alexander
    Dec 16, 2018 at 20:27
  • 2
    @MauricioChirino No, I'm not. If the goal is to remove duplicate values from a sequence, while retaining the order in which elements uniquely appeared, this doesn't do that. The very clear counter example is [2, 1, 1]. The first appearance of unique elements, in order is [2, 1]. That's the correct answer. But using your (incorrect) algorithm, you get [1, 2], which is sorted, but is not in the correct, original, order.
    – Alexander
    Dec 27, 2018 at 18:30
  • 3
    Fails if the elements in array are not Hashable; only Hashable data types can be added to a Set, yet any data type can be added to an array.
    – Mecki
    Jul 9, 2019 at 15:19
4

Here is a solution that

  • Uses no legacy NS types
  • Is reasonably fast with O(n)
  • Is concise
  • Preserves element order
extension Array where Element: Hashable {

    var uniqueValues: [Element] {
        var allowed = Set(self)
        return compactMap { allowed.remove($0) }
    }
}
1
  • This is nice but would only works for Hashable elements Mar 27, 2021 at 9:19
3

Slightly more succinct syntax version of Daniel Krom's Swift 2 answer, using a trailing closure and shorthand argument name, which appears to be based on Airspeed Velocity's original answer:

func uniq<S: SequenceType, E: Hashable where E == S.Generator.Element>(source: S) -> [E] {
  var seen = [E: Bool]()
  return source.filter { seen.updateValue(true, forKey: $0) == nil }
}

Example of implementing a custom type that can be used with uniq(_:) (which must conform to Hashable, and thus Equatable, because Hashable extends Equatable):

func ==(lhs: SomeCustomType, rhs: SomeCustomType) -> Bool {
  return lhs.id == rhs.id // && lhs.someOtherEquatableProperty == rhs.someOtherEquatableProperty
}

struct SomeCustomType {

  let id: Int

  // ...

}

extension SomeCustomType: Hashable {

  var hashValue: Int {
    return id
  }

}

In the above code...

id, as used in the overload of ==, could be any Equatable type (or method that returns an Equatable type, e.g., someMethodThatReturnsAnEquatableType()). The commented-out code demonstrates extending the check for equality, where someOtherEquatableProperty is another property of an Equatable type (but could also be a method that returns an Equatable type).

id, as used in the hashValue computed property (required to conform to Hashable), could be any Hashable (and thus Equatable) property (or method that returns a Hashable type).

Example of using uniq(_:):

var someCustomTypes = [SomeCustomType(id: 1), SomeCustomType(id: 2), SomeCustomType(id: 3), SomeCustomType(id: 1)]

print(someCustomTypes.count) // 4

someCustomTypes = uniq(someCustomTypes)

print(someCustomTypes.count) // 3
1
  • There no need to use a Bool, when the only value you use is true. You're reaching for a "unit type" (a type with only one possible value). Swift's unit type is Void, whose only value is () (a.k.a. the empty tuple). So you can just use [T: Void]. Though you shouldn't do that, because you've basically just invented Set. Use Set instead. See stackoverflow.com/a/55684308/3141234
    – Alexander
    Feb 15, 2020 at 12:47
2

here I've done some O(n) solution for objects. Not few-lines solution, but...

struct DistinctWrapper <T>: Hashable {
    var underlyingObject: T
    var distinctAttribute: String
    var hashValue: Int {
        return distinctAttribute.hashValue
    }
}
func distinct<S : SequenceType, T where S.Generator.Element == T>(source: S,
                                                                distinctAttribute: (T) -> String,
                                                                resolution: (T, T) -> T) -> [T] {
    let wrappers: [DistinctWrapper<T>] = source.map({
        return DistinctWrapper(underlyingObject: $0, distinctAttribute: distinctAttribute($0))
    })
    var added = Set<DistinctWrapper<T>>()
    for wrapper in wrappers {
        if let indexOfExisting = added.indexOf(wrapper) {
            let old = added[indexOfExisting]
            let winner = resolution(old.underlyingObject, wrapper.underlyingObject)
            added.insert(DistinctWrapper(underlyingObject: winner, distinctAttribute: distinctAttribute(winner)))
        } else {
            added.insert(wrapper)
        }
    }
    return Array(added).map( { return $0.underlyingObject } )
}
func == <T>(lhs: DistinctWrapper<T>, rhs: DistinctWrapper<T>) -> Bool {
    return lhs.hashValue == rhs.hashValue
}

// tests
// case : perhaps we want to get distinct addressbook list which may contain duplicated contacts like Irma and Irma Burgess with same phone numbers
// solution : definitely we want to exclude Irma and keep Irma Burgess
class Person {
    var name: String
    var phoneNumber: String
    init(_ name: String, _ phoneNumber: String) {
        self.name = name
        self.phoneNumber = phoneNumber
    }
}

let persons: [Person] = [Person("Irma Burgess", "11-22-33"), Person("Lester Davidson", "44-66-22"), Person("Irma", "11-22-33")]
let distinctPersons = distinct(persons,
    distinctAttribute: { (person: Person) -> String in
        return person.phoneNumber
    },
    resolution:
    { (p1, p2) -> Person in
        return p1.name.characters.count > p2.name.characters.count ? p1 : p2
    }
)
// distinctPersons contains ("Irma Burgess", "11-22-33") and ("Lester Davidson", "44-66-22")
1
2

I used @Jean-Philippe Pellet's answer and made an Array extension that does set-like operations on arrays, while maintaining the order of elements.

/// Extensions for performing set-like operations on lists, maintaining order
extension Array where Element: Hashable {
  func unique() -> [Element] {
    var seen: [Element:Bool] = [:]
    return self.filter({ seen.updateValue(true, forKey: $0) == nil })
  }

  func subtract(takeAway: [Element]) -> [Element] {
    let set = Set(takeAway)
    return self.filter({ !set.contains($0) })
  }

  func intersect(with: [Element]) -> [Element] {
    let set = Set(with)
    return self.filter({ set.contains($0) })
  }
}
1
  • There no need to use a Bool, when the only value you use is true. You're reaching for a "unit type" (a type with only one possible value). Swift's unit type is Void, whose only value is () (a.k.a. the empty tuple). So you can just use [T: Void]. Though you shouldn't do that, because you've basically just invented Set. Use Set instead. See stackoverflow.com/a/55684308/3141234
    – Alexander
    Feb 15, 2020 at 12:47
2

This is just a very simple and convenient implementation. A computed property in an extension of an Array that has equatable elements.

extension Array where Element: Equatable {
    /// Array containing only _unique_ elements.
    var unique: [Element] {
        var result: [Element] = []
        for element in self {
            if !result.contains(element) {
                result.append(element)
            }
        }

        return result
    }
}
1
  • 1
    This is also O(n^2).
    – Alexander
    Dec 16, 2018 at 20:38
2
func removeDublicate (ab: [Int]) -> [Int] {
var answer1:[Int] = []
for i in ab {
    if !answer1.contains(i) {
        answer1.append(i)
    }}
return answer1
}

Usage:

let f = removeDublicate(ab: [1,2,2])
print(f)
3
  • I think this is the simplest
    – Jack Rus
    Jan 2, 2017 at 0:39
  • it keeps the order and gives you an array you want
    – Jack Rus
    Jan 2, 2017 at 0:40
  • This is also O(n²).
    – Alexander
    Dec 16, 2018 at 20:34
2

Swift 3/ Swift 4/ Swift 5

Just one line code to omit Array duplicates without effecting order:

let filteredArr = Array(NSOrderedSet(array: yourArray))
1
  • Here we are typecasting an array into Orderedset. Definition of "Sets" - sets allow only distinct values (It does not allow duplicates). Hence duplicates will be omitted As we are typecasting with NSOrderedSet hence array order will not be disturbed. Mar 14, 2020 at 10:42

Not the answer you're looking for? Browse other questions tagged or ask your own question.