Polars DataFrame Tutorial

last modified March 1, 2025

Polars is a fast, efficient DataFrame library for Python. It is designed for high-performance data manipulation and analysis. This tutorial covers how to use Polars DataFrames with practical examples.

Polars provides a DataFrame API similar to Pandas but with better performance. It is optimized for large datasets and supports lazy evaluation for efficient query execution.

Creating a DataFrame

This example shows how to create a Polars DataFrame from a dictionary.

create_df.py

import polars as pl

data = {
    'Name': ['Alice', 'Bob', 'Charlie'],
    'Age': [25, 30, 35],
    'City': ['New York', 'Los Angeles', 'Chicago']
}

df = pl.DataFrame(data)
print(df)

The pl.DataFrame() function creates a DataFrame from a dictionary. This is useful for small datasets or quick prototyping.

Reading a CSV File

This example demonstrates how to read a CSV file into a Polars DataFrame.

read_csv.py

import polars as pl

df = pl.read_csv('data.csv')
print(df)

The pl.read_csv() function reads a CSV file into a DataFrame. This is useful for loading large datasets from files.

Filtering Rows

This example shows how to filter rows in a Polars DataFrame.

filter_rows.py

import polars as pl

data = {
    'Name': ['Alice', 'Bob', 'Charlie'],
    'Age': [25, 30, 35],
    'City': ['New York', 'Los Angeles', 'Chicago']
}

df = pl.DataFrame(data)
filtered_df = df.filter(pl.col('Age') > 30)
print(filtered_df)

The filter() method filters rows based on a condition. This is useful for selecting specific subsets of data.

Selecting Columns

This example demonstrates how to select specific columns in a Polars DataFrame.

select_columns.py

import polars as pl

data = {
    'Name': ['Alice', 'Bob', 'Charlie'],
    'Age': [25, 30, 35],
    'City': ['New York', 'Los Angeles', 'Chicago']
}

df = pl.DataFrame(data)
selected_df = df.select(['Name', 'City'])
print(selected_df)

The select() method selects specific columns from the DataFrame. This is useful for focusing on relevant data.

Adding a New Column

This example shows how to add a new column to a Polars DataFrame.

add_column.py

import polars as pl

data = {
    'Name': ['Alice', 'Bob', 'Charlie'],
    'Age': [25, 30, 35],
    'City': ['New York', 'Los Angeles', 'Chicago']
}

df = pl.DataFrame(data)
df = df.with_column((pl.col('Age') * 2).alias('DoubleAge'))
print(df)

The with_column() method adds a new column to the DataFrame. This is useful for creating derived data.

Grouping and Aggregating

This example demonstrates how to group data and calculate aggregate statistics.

group_agg.py

import polars as pl

data = {
    'City': ['New York', 'Los Angeles', 'New York', 'Chicago'],
    'Sales': [100, 200, 150, 300]
}

df = pl.DataFrame(data)
grouped_df = df.groupby('City').agg([
    pl.col('Sales').sum().alias('TotalSales')
])
print(grouped_df)

The groupby() and agg() methods group data and calculate aggregate statistics. This is useful for summarizing data.

Sorting Data

This example shows how to sort a Polars DataFrame by a column.

sort_data.py

import polars as pl

data = {
    'Name': ['Alice', 'Bob', 'Charlie'],
    'Age': [25, 30, 35],
    'City': ['New York', 'Los Angeles', 'Chicago']
}

df = pl.DataFrame(data)
sorted_df = df.sort('Age', reverse=True)
print(sorted_df)

The sort() method sorts the DataFrame by a column. This is useful for organizing data in a specific order.

Joining DataFrames

This example demonstrates how to join two Polars DataFrames.

join_df.py

import polars as pl

data1 = {
    'ID': [1, 2, 3],
    'Name': ['Alice', 'Bob', 'Charlie']
}

data2 = {
    'ID': [1, 2, 4],
    'City': ['New York', 'Los Angeles', 'Chicago']
}

df1 = pl.DataFrame(data1)
df2 = pl.DataFrame(data2)
joined_df = df1.join(df2, on='ID', how='inner')
print(joined_df)

The join() method joins two DataFrames based on a common column. This is useful for combining related datasets.

Best Practices for Polars DataFrames

Use Lazy Evaluation: Leverage lazy evaluation for efficient query execution.
Optimize Data Types: Use appropriate data types to reduce memory usage.
Batch Operations: Perform batch operations to minimize overhead.
Profile Performance: Profile your code to identify bottlenecks.

Source

Polars Documentation

In this article, we have explored how to use Polars DataFrames for data analysis.

Author

My name is Jan Bodnar, and I am a passionate programmer with extensive programming experience. I have been writing programming articles since 2007. To date, I have authored over 1,400 articles and 8 e-books. I possess more than ten years of experience in teaching programming.

List all Polars tutorials.

Ebooks