8 Things To Know To Count Unique Values in a List Using Python

How to count unique values in a Python list? There are multiple ways and in this tutorial we will go through them as a way for you to practice several Python constructs.

The simplest way to count unique values in a Python list is to convert the list to a set considering that all the elements of a set are unique. You can also count unique values in a list using a dictionary, the collections.Counter class, Numpy.unique() or Pandas.unique().

Most of our examples will work on the same list of numbers:

numbers = [2, 3, 3, 6, 8, 8, 23, 24, 24, 30, 45, 50, 51, 51]

We will implement every approach in its own function so it’s easy to call them from the “main” of our code. Every function takes the numbers list as input and returns the list with unique values.

The main of our program will then count the number of unique values using the list len() method.

Let’s get started!

1. Using a For Loop to Count Unique Values in a Python List

The first approach uses a Python for loop to go through all the elements.

def get_unique_values_with_for_loop(values):
    unique_values = []

    for value in values:
        if value not in unique_values:
            unique_values.append(value)

    return unique_values

The for loop goes through every element of the list and adds that element to a new list if it’s not already present in the new list.

Let’s write a generic code to print the result returned by this function so we can use pretty much the same code to print the result returned by the other functions we will create in the next sections.

numbers = [2, 3, 3, 6, 8, 8, 23, 24, 24, 30, 45, 50, 51, 51]
unique_values = get_unique_values_with_for_loop(numbers)
number_of_unique_values = len(unique_values)
print("The list of unique values is: {}".format(unique_values))
print("The number of unique values is: {}".format(number_of_unique_values))

The output is:

The list of unique values is: [2, 3, 6, 8, 23, 24, 30, 45, 50, 51]
The number of unique values is: 10

The only line we will have to update when calling a different function is the following:

unique_values = get_unique_list_with_for_loop(numbers)

Everything else in the main of our program will stay the same because it depends on the variable unique_values.

2. Count Unique Values in a Python List Using a Set

The set is a data structure that only contains unique elements. We can use this property of the set to get back just unique values.

def get_unique_values_with_set(values):
    return set(values)

...
unique_values = get_unique_values_with_set(numbers)
...

Notice the curly brackets in the output due to the fact that the function returns a set:

The list of unique values is: {2, 3, 6, 8, 45, 50, 51, 23, 24, 30}
The number of unique values is: 10

Do you see that the order of the elements has changed compared to the original list?

That’s because…

The set data type is unordered.

The nice thing of this approach is that the code is a lot more concise (just a single line function).

3. Using a Dictionary to Get the Number of Unique Values in a List

Could we use the fact that dictionary keys are unique to get unique elements from the original list?

We could create a dictionary where the unique elements in the list are the keys of the dictionary. The values of the dictionary are counters that tell us how many time each element appears in the list.

def get_unique_values_with_dictionary(values):
    new_values = {}

    for value in values:
        if value in new_values.keys():
            new_values[value] += 1
        else:
            new_values[value] = 1
    
    print(new_values)
    return new_values.keys()

...
unique_values = get_unique_values_with_dictionary(numbers)
...

At each iteration of the for loop we verify if a specific value already exists as dictionary key. If it doesn’t we add that key to the dictionary and we set the value to 1.

Otherwise we increase the value associated to the existing key in the dictionary.

I have added a print() statement in the function to show you the dictionary created.

{2: 1, 3: 2, 6: 1, 8: 2, 23: 1, 24: 2, 30: 1, 45: 1, 50: 1, 51: 2}
The list of unique values is: dict_keys([2, 3, 6, 8, 23, 24, 30, 45, 50, 51])
The number of unique values is: 10

4. Collections.Counter To Count Unique Values in a List

The same approach explained in the previous section (using a standard dictionary) can also be implemented using the Counter class of the Collections module.

Let’s find out what Counter does exactly:

>>> from collections import Counter
>>> Counter([1,2,3,3])
Counter({3: 2, 1: 1, 2: 1}) 

Interesting…

It generates a dictionary-like data structure where the keys are the elements of the list and the values indicate the count of each element in the original list.

Basically the same thing we have created in the previous section using a standard Python dictionary.

Let’s apply it to our list of numbers:

>>> from collections import Counter
>>> numbers = [2, 3, 3, 6, 8, 8, 23, 24, 24, 30, 45, 50, 51, 51]
>>> new_values = Counter(numbers) 

These are the methods available for the Counter object:

>>> new_values.
new_values.clear(       new_values.get(         new_values.pop(         new_values.update(     
new_values.copy(        new_values.items(       new_values.popitem(     new_values.values(     
new_values.elements(    new_values.keys(        new_values.setdefault( 
new_values.fromkeys(    new_values.most_common( new_values.subtract(    

The .keys() method is the one we need to get the unique elements in the original list:

>>> new_values.keys()
dict_keys([2, 3, 6, 8, 23, 24, 30, 45, 50, 51]) 

So, we can add the following function to our code:

from collections import Counter
...
...

def get_unique_values_with_counter(values):
    return Counter(values).keys()

...
unique_values = get_unique_values_with_counter(numbers)
...

The output is:

The list of unique values is: dict_keys([2, 3, 6, 8, 23, 24, 30, 45, 50, 51])
The number of unique values is: 10

In the same way we have seen when using the set, with this approach we can write a one-line function.

5. Counting Unique Values in a List Using NumPy

The NumPy library also provides a way to count unique values in a list (or array).

NumPy.unique not only allows to find the unique elements of an array but it also provides the number of times each element is present by using the return_counts parameter.

>>> import numpy as np
>>> print(np.unique(numbers))
[ 2  3  6  8 23 24 30 45 50 51] 
>>> print(np.unique(numbers, return_counts=True))
(array([ 2,  3,  6,  8, 23, 24, 30, 45, 50, 51]), array([1, 2, 1, 2, 1, 2, 1, 1, 1, 2])) 

So, let’s create a new function that uses np.unique():

import numpy as np

...

def get_unique_values_with_numpy(values):
    return np.unique(values)

...
unique_values = get_unique_values_with_numpy(numbers)
...

And here are the unique elements we get back:

The list of unique values is: [ 2  3  6  8 23 24 30 45 50 51]
The number of unique values is: 10

6. Using Pandas to Get Unique Values in a List

We can also get unique values in a list using Pandas. To do that we will use pandas.unique.

It takes as input a one-dimensional array, so let’s write our function:

import pandas as pd

...

def get_unique_values_with_pandas(values):
    return pd.unique(pd.Series(values))

...
unique_values = get_unique_values_with_pandas(numbers)
...

The output returned by our program is:

The list of unique values is: [ 2  3  6  8 23 24 30 45 50 51]
The number of unique values is: 10

7. Count Unique Values in a List of Dictionaries

Let’s write code to do something a bit more complex.

I have a list of dictionaries that all contain the same key and I want to get the count of unique values across all the dictionaries in the list.

countries = [{"name": "Italy"}, {"name": "UK"}, {"name": "Germany"}, {"name": "Brazil"}, {"name": "Italy"}, {"name": "UK"}]

The first step would be to get a list that contains all the values in each dictionary. To do that we can use a list comprehension.

>>> names = [country['name'] for country in countries]
>>> print(names)
['Italy', 'UK', 'Germany', 'Brazil', 'Italy', 'UK']         

Now we can use one of the approaches already used before. For example we can pass the list to collections.Counter:

>>> from collections import Counter
>>> Counter(names)
Counter({'Italy': 2, 'UK': 2, 'Germany': 1, 'Brazil': 1})
>>> Counter(names).keys()
dict_keys(['Italy', 'UK', 'Germany', 'Brazil'])
>>> len(Counter(names).keys())
4                           

Makes sense?

8. Which One is the Faster Approach to Count Unique List Values?

We will complete this tutorial by comparing the performance of all the approaches we have implemented to solve our problem.

To do that we will generate a list with random elements and then use the timeit module to measure the execution time for each implementation.

We will generate the list of random numbers using the Python random module:

>>> import random
>>> random_numbers = [random.randrange(10) for i in range(10)]
>>> print(random_numbers)
[7, 6, 0, 7, 9, 8, 1, 6, 7, 4]

This is just an example of list that contains 10 random numbers between 0 and 9. To measure the performance of our function we will use a bigger list with random numbers in a wider range.

random_numbers = [random.randrange(100) for i in range(1000000)]

Let’s take the first function we have created, the one that uses the for loop. Here is how we can pass it to the timeit.timeit function:

import timeit

testfunction = '''
def get_unique_values_with_for_loop(values):
    unique_values = []

    for value in values:
        if value not in unique_values:
            unique_values.append(value)

    return unique_values
'''

random_numbers = [random.randrange(100) for i in range(1000000)]
print(timeit.timeit(testfunction, number=100000000))

The number parameter represents the number of executions.

The output is:

5.022220958

To measure the execution time for every function with timeit simply surround a given function with the following two lines in the same way we have done above:

testfunction = '''
{function_to_test}
'''

Let’s compare the execution time between the six approaches implemented in this tutorial:

get_unique_values_with_for_loop: 5.022220958
get_unique_values_with_set: 5.150171875
get_unique_values_with_dictionary: 5.140621958
get_unique_values_with_counter: 5.145550625
get_unique_values_with_numpy: 5.187875792000001
get_unique_values_with_pandas: 5.070051584

The fastest implementation is the first one that uses a Python for loop and most of them have a comparable performance.

Conclusion

After reading this article you have lots of different ways to count unique values in a Python list.

I hope you have found this tutorial useful to see how you can solve the same problem in many different ways using Python.

Which one of the solutions we have implemented do you prefer? 🙂

Related article: do you want to improve your knowledge about Python list? Have a look at this CodeFatherTech tutorial about list comprehensions!