What’s New ?

The Top 10 favtutor Features You Might Have Overlooked

Read More

Python: 5 Ways to Remove Duplicates from List

  • Apr 30, 2021
  • 5 Minute Read
Python: 5 Ways to Remove Duplicates from List

In this article, we will learn what is a list in python. As a python list is a collection of multiple elements even containing duplicates, sometimes it is necessary to make the list unique. Here, we are going to study the multiple ways to remove duplicates from the list in python. So, let's get started!

What is a List?

The list is the most important data type in python language. In Python language, the list is written as the list of commas separated values inside the square bracket. The most important advantage of the list is that the elements inside the list are not compulsorily of the same data type and negative indexing. Also, all the operation of the string is similarly applied on list data type such as slicing, concatenation, etc. Also, we can create the nested list i.e list containing another list.

For example:

# creating a list of items with different data types 
sample_list = [6,"mark",[A,I]] 
print(sample_list)

 

Output:

 [6, mark, ['A', 'I']]

5 Ways to Remove Duplicates from a List in Python

There are many ways to remove the duplicates from a list in python. Let’s study them below:

Method 1: Naïve Method

In this method, we traverse the list and then append the first occurrence of the element in the new list, and then all other elements are ignored.

Example:

# removing duplicated from the list using naive methods 

# initializing list 
sam_list = [11, 13, 15, 16, 13, 15, 16, 11] 
print ("The list is: " + str(sam_list)) 

# remove duplicated from list 
result = [] 
for i in sam_list: 
    if i not in result: 
        result.append(i) 

# printing list after removal 
print ("The list after removing duplicates : " + str(result)) 

 

Output:

 The list is: [11, 13, 15, 16, 13, 15, 16, 11]

 The list after removing duplicates: [11, 13, 15, 16]

Method 2: Using a list comprehensive

This method is similar to the above method, but this method is shorter than the above one.

Example:

# removing duplicated from the list using list comprehension 

# initializing list 
sam_list = [11, 13, 15, 16, 13, 15, 16, 11] 
print ("The list is: " + str(sam_list)) 

 
# to remove duplicated from list 
result = [] 
[result.append(x) for x in sam_list if x not in result] 

# printing list after removal 
print ("The list after removing duplicates: " + str(result)) 

 

Output:

 The list is: [11, 13, 15, 16, 13, 15, 16, 11]

 The list after removing duplicates: [11, 13, 15, 16]

Method 3: Using set()

This method is the most popular method to remove the duplicate from the python list. This is because the set data structure does not allow duplicates. But the drawback of this method is that the order of the elements is lost.

Example:

# removing duplicated from the list using set() 

# initializing list 
sam_list = [11, 15, 13, 16, 13, 15, 16, 11] 
print ("The list is: " + str(sam_list)) 

# to remove duplicated from list 
sam_list = list(set(sam_list)) 

# printing list after removal 
# ordering distorted
print ("The list after removing duplicates: " + str(sam_list)) 

 

Output:

 The list is: [11, 15, 13, 16, 13, 15, 16, 11]

 The list after removing duplicates: [16, 11, 13, 15]

Method 4: Using list comprehensive + enumerate()

List comprehensive when merged with enumerate function we can remove the duplicate from the python list. Basically in this method, the already occurred elements are skipped, and also the order is maintained.

Example:

# removing duplicated from the list using list comprehension + enumerate() 

# initializing list 
sam_list = [11, 15, 13, 16, 13, 15, 16, 11] 
print ("The list is: " + str(sam_list)) 

# to remove duplicated from list 
result = [i for n, i in enumerate(sam_list) if i not in sam_list[:n]] 

# printing list after removal 
print ("The list after removing duplicates: " + str(result)) 

 

Output:

 The list is: [11, 13, 15, 16, 13, 15, 16, 11]

 The list after removing duplicates: [11, 13, 15, 16]

Method 5: Using collections.OrderedDict.fromkeys()

This is the fastest method to achieve the target of removing duplicates from the python list. This method will first remove the duplicates and return a dictionary that has converted to a list. Also, this method works well in the case of a string.

Example:

# removing duplicated from list using collections.OrderedDict.fromkeys() 
from collections import OrderedDict 

# initializing list 
sam_list = [11, 15, 13, 16, 13, 15, 16, 11] 
print ("The list is: " + str(sam_list)) 

# to remove duplicated from list 
result = list(OrderedDict.fromkeys(sam_list)) 

# printing list after removal 
print ("The list after removing duplicates: " + str(result)) 

 

Output:

 The list is: [11, 15, 13, 16, 13, 15, 16, 11]

 The list after removing duplicates: [11, 15, 13, 16]

These are a few of the methods with which we can remove the duplicate from the python list.

Conclusion

Hence, in this article, we learned about the python list and different methods to remove the duplicate elements from the list in python. Also, we studied the example along with the output for different methods.

FavTutor - 24x7 Live Coding Help from Expert Tutors!

About The Author
Shivali Bhadaniya
I'm Shivali Bhadaniya, a computer engineer student and technical content writer, very enthusiastic to learn and explore new technologies and looking towards great opportunities. It is amazing for me to share my knowledge through my content to help curious minds.