One of the most common tasks in data analysis is to loop through rows of a Pandas DataFrame to perform various operations, like data manipulation and wrangling. In this article, we will discover different ways to iterate over rows of Pandas DataFrames.
What is a DataFrame?
Pandas DataFrames are a two-dimensional array with labeled data structures having different column types. It is a convenient way to work with structured data in Python. DataFrames are a standard and convenient way to store data in a tabular format, with rows to store the information and columns to name the information.
To learn about DataFrames in detail, you can check out how to Convert Pandas Series to DataFrame.
3 Ways to Iterate Over Rows in Pandas DataFrame
Looping through rows in a DataFrame allows us to access and manipulate the values of each row individually. Let us now explore the various methods to iterate over rows of a Pandas DataFrame:
1) Using the iterrows() Function
The pandas library in Python provides itterows() Function, which we can use to iterate over rows of a DataFrame. It returns an iterator that yields the index and a Series object containing the values of each row. By using the iterrows() function, we can get the values of every row using the column names as indices of the DataFrame.
The following Python code shows how to use iterrows() function to iterate over rows in Pandas DataFrame:
import pandas as pd
data = {
"Name": ["John", "Emma", "Michael"],
"Age": [45, 30, 55],
"City": ["New York", "London", "Paris"]
}
df = pd.DataFrame(data)
# Display the original DataFrame
print('Original DataFrame:\n', df)
# Loop through rows using iterrows()
print('Iterated Rows:')
for index, row in df.iterrows():
print(row['Name'], row['Age'], row['City'])Output:
Original DataFrame:
Name Age City
0 John 45 New York
1 Emma 30 London
2 Michael 55 Paris
Iterated Rows:
John 45 New York
Emma 30 London
Michael 55 Paris
2) Using List Comprehension
List comprehension is a brief and efficient way to iterate over rows in a DataFrame while performing operations on the data. We can just simply use a list comprehension to iterate over the values of a particular row. This technique is particularly useful when you need to perform calculations or transformations on the data while iterating.
Let us see how to do it in Python:
import pandas as pd
data = {
"Name": ["John", "Emma", "Michael"],
"Age": [45, 30, 55],
"City": ["New York", "London", "Paris"]
}
df = pd.DataFrame(data)
# Display the original DataFrame
print('Original DataFrame:\n', df)
# Loop through rows using list comprehension
print('Iterated Rows:')
result = [row['Name'] for index, row in df.iterrows()]
print(result)Output:
Original DataFrame:
Name Age City
0 John 45 New York
1 Emma 30 London
2 Michael 55 Paris
Iterated Rows:
['John', 'Emma', 'Michael']
3) Using the apply() Function
One more way we can iterate over rows of Pandas DataFrame is by using the apply() function. The apply() function in Pandas is very productive as it can allow us to apply a function to each row or column of a DataFrame. This function can be defined using a lambda function or a custom-defined function.
Let us try an example:
import pandas as pd
data = {
"Name": ["John", "Emma", "Michael"],
"Age": [45, 30, 55],
"City": ["New York", "London", "Paris"]
}
df = pd.DataFrame(data)
# Display the original DataFrame
print('Original DataFrame:\n', df)
# Use apply() with a lambda function
df['Name'] = df.apply(lambda row: row['Name'].upper(), axis=1)
# Display the new DataFrame.
print('Updated DataFrame:\n',df)Output:
Original DataFrame:
Name Age City
0 John 45 New York
1 Emma 30 London
2 Michael 55 Paris
Updated DataFrame:
Name Age City
0 JOHN 45 New York
1 EMMA 30 London
2 MICHAEL 55 Paris
Conclusion
In this article, we understood the various techniques and methods we can use to iterate over rows of a Pandas data frame. Now that you have a solid understanding of looping through Pandas, you can confidently apply your learnings from this article to your data analysis projects. If you need help with them, our Data Science Tutors are always available!
