13

In Elixir, what would be an efficient way to filter a Map by its values.

Right now I have the following solution

%{foo: "bar", biz: nil, baz: 4}
|> Enum.reject(fn {_, v} -> is_nil(v) end)
|> Map.new

This solution seems pretty inefficient to me. When called on a Map, Enum.reject/2 returns a Keywords. Since I want a Map, I need to call Map.new/1 to convert that Keywords back to me.

This seems inefficient because Enum.reject/2 has to iterate over the Map once and then presumably, Map.new/1 has to iterate over the Keywords another time.

What would be a more efficient solution?

6 Answers 6

27

You can use :maps.filter/2, which filters a map and does not create any intermediate list:

iex(1)> :maps.filter fn _, v -> v != nil end, %{foo: "bar", biz: nil, baz: 4}
%{baz: 4, foo: "bar"}

A simple benchmark confirms this is faster than Enum.filter + Map.new:

map = for i <- 1..100000, into: %{}, do: {i, Enum.random([nil, 1, 2])}

IO.inspect :timer.tc(fn ->
  map
  |> Enum.reject(fn {_, v} -> is_nil(v) end)
  |> Map.new
end)

IO.inspect :timer.tc(fn ->
  :maps.filter fn _, v -> v != nil end, map
end)
{44728,
 %{48585 => 1, 60829 => 2, 12995 => 1, 462 => 2, 704 => 2, 28954 => 2,
   29635 => 2, 78798 => 1, 92572 => 1, 89750 => 2, 39389 => 2, 62855 => 2,
   79313 => 1, 92062 => 2, 61871 => 1, 92856 => 2, 75920 => 1, 59922 => 1,
   37912 => 2, 30420 => 2, 51211 => 2, 7994 => 2, 78269 => 2, 9765 => 2,
   38352 => 2, 6653 => 1, 82555 => 2, 54031 => 2, 45138 => 1, 41351 => 1,
   40746 => 1, 5961 => 1, 66704 => 2, 33823 => 1, 47603 => 1, 86873 => 1,
   81009 => 2, 96255 => 1, 36219 => 1, 1328 => 2, 33314 => 1, 54477 => 2,
   40189 => 2, 27028 => 1, 31676 => 1, 94037 => 1, 32388 => 1, 4351 => 1,
   46309 => 1, ...}}
{28638,
 %{48585 => 1, 60829 => 2, 12995 => 1, 462 => 2, 704 => 2, 28954 => 2,
   29635 => 2, 78798 => 1, 92572 => 1, 89750 => 2, 39389 => 2, 62855 => 2,
   79313 => 1, 92062 => 2, 61871 => 1, 92856 => 2, 75920 => 1, 59922 => 1,
   37912 => 2, 30420 => 2, 51211 => 2, 7994 => 2, 78269 => 2, 9765 => 2,
   38352 => 2, 6653 => 1, 82555 => 2, 54031 => 2, 45138 => 1, 41351 => 1,
   40746 => 1, 5961 => 1, 66704 => 2, 33823 => 1, 47603 => 1, 86873 => 1,
   81009 => 2, 96255 => 1, 36219 => 1, 1328 => 2, 33314 => 1, 54477 => 2,
   40189 => 2, 27028 => 1, 31676 => 1, 94037 => 1, 32388 => 1, 4351 => 1,
   46309 => 1, ...}}
1
3

In this case comprehension would be a good idea, because it also does not create an intermediate list and returns you a map:

map = %{baz: 4, biz: nil, foo: "bar"}
for {key, value} <- map, !is_nil(value), into: %{}, do: {key, value}
2
  • 3
    This should not create an intermediate list, but in practice, I've found for ..., into: %{} to be even slower than creating a list with Enum.filter/2 and passing it to Map.new/1. I tried it with the benchmark I posted in my answer and this runs slower than OP's code by about 20-25% for me.
    – Dogbert
    May 24, 2017 at 7:34
  • Thanks :) I'm not surprised that erlang's solution is the fastest one.
    – PatNowak
    May 24, 2017 at 7:36
2

Since Elixir version 1.13, Map.filter(map, fun) exists natively. See https://hexdocs.pm/elixir/Map.html#filter/2

0

It could be a little expensive but it is more declarative, which IMO adds more value. Also consider how big is going to be your collection and if it makes sense to optimize this filter.

However, I understand your concern, so here is what I did:

%{foo: "bar", biz: nil, baz: 4}
|> Enum.reduce(%{}, filter_nil_values/2)

Where filter_nil_values/2 is defined as

defp filter_nil_values({_k, nil}, accum), do: accum
defp filter_nil_values({k, v}, accum), do: Map.put(accum, k, v)

I tried to do this in a one-line function, but it looks awful.

0

You can also write like this:

m = %{foo: "bar", biz: nil, baz: 4}

Enum.reduce(m, m, fn 
  {key, nil}, acc -> Map.delete(acc, key)
  {_, _}, acc -> acc
end)

The code above is quite efficient if there are few nil values in m.

0
Map.filter(%{foo: "bar", biz: nil, baz: 4}, 
                         fn {_, v} -> v != nil end)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.