Pandas Categorical Data
last modified November 17, 2026 Python 3.15
A categorical dtype is a column whose values are drawn from a fixed
set of categories rather than being arbitrary strings or numbers. A column of
shirt sizes, for instance, may only ever contain small,
medium or large. Pandas stores such a column as a
list of the allowed categories plus a compact array of codes pointing into
that list.
This representation saves memory when values repeat, gives meaningful order to labels that are not numbers, and lets Pandas treat declared-but-unused categories explicitly. All examples below were run with pandas 3.1.1.
Creating a categorical with astype
The quickest way to convert an ordinary column is
astype('category'). Pandas discovers the categories from the
values present and, on this version, lists them in sorted order. The
dtype of the result is category, not
object or str.
import pandas as pd
s = pd.Series(['low', 'high', 'low', 'medium', 'high'])
c = s.astype('category')
print(c)
print()
print(c.dtype)
The printed Series still shows the original labels, but the final line
reports the new dtype and the three categories Pandas found:
high, low and medium. The values
themselves were not rewritten.
$ python cat_astype.py 0 low 1 high 2 low 3 medium 4 high dtype: category Categories (3, str): ['high', 'low', 'medium'] category
Choosing the categories explicitly
To control the set of allowed values, build a Categorical object
and pass a categories list. Values outside that list become
missing values (NaN). The categories attribute shows
the declared set, while codes shows the integer position of each
value inside it; -1 marks a missing value.
import pandas as pd cat = pd.Categorical(['low', 'high', 'low'], categories=['low', 'medium', 'high']) print(cat) print() print(cat.categories) print() print(cat.codes)
Here the categories appear in the order we supplied them, and the codes
0, 2, 0 refer to positions in that list:
low is at index 0 and high at index 2. The declared
category medium is present even though no value uses it.
$ python cat_explicit.py ['low', 'high', 'low'] Categories (3, str): ['low', 'medium', 'high'] Index(['low', 'medium', 'high'], dtype='str') [0 2 0]
Ordered categories and comparisons
By default categories have no order, so comparing two of them is not
meaningful. Setting ordered=True declares that the listed order
is real. Comparisons such as cat > 'medium' then work, and
sorting follows the declared order instead of alphabetical order.
import pandas as pd
cat = pd.Categorical(
['low', 'high', 'medium'],
categories=['low', 'medium', 'high'],
ordered=True
)
print(cat > 'medium')
print()
print(cat.sort_values())
Only the value high is above medium, so the boolean
array is False, True, False. The sorted
result follows low, medium, high; the
dtype line uses < signs to show that the ordering is defined.
$ python cat_ordered.py [False True False] ['low', 'medium', 'high'] Categories (3, str): ['low' < 'medium' < 'high']
Saving memory
The memory_usage method reports how many bytes a column occupies.
Passing deep=True counts the actual string data for text columns;
without it, Pandas only counts the references. Comparing the two forms on a
repeated-value column shows why categoricals are attractive.
import pandas as pd
s = pd.Series(['small', 'medium', 'large'] * 1000)
obj = s.memory_usage(deep=True)
cat = s.astype('category').memory_usage(deep=True)
print('object: ', obj, 'bytes')
print('category:', cat, 'bytes')
On this run the text column needs 40132 bytes while the categorical needs only 3173 bytes: the three distinct strings are stored once and reused via one-byte codes. Exact figures depend on the platform, the values and the Pandas version.
$ python cat_memory.py object: 40132 bytes category: 3173 bytes
Grouping and unused categories
A groupby on a categorical column has a choice about categories
that appear in the dtype but not in the data. With
observed=False every declared category gets a group, including
empty ones; with observed=True only categories that actually
occur are returned.
import pandas as pd
df = pd.DataFrame({
'size': pd.Categorical(['small', 'large'], categories=['small', 'medium', 'large']),
'price': [10, 20]
})
print(df.groupby('size', observed=False)['price'].sum())
print()
print(df.groupby('size', observed=True)['price'].sum())
The first result contains a medium row with a sum of 0 even
though no row has that size. The second result omits it. Use
observed=True when you want only groups backed by real data, and
the default when a stable set of groups matters. See the
grouping data tutorial for more on
grouping.
$ python cat_group.py size small 10 medium 0 large 20 Name: price, dtype: int64 size small 10 large 20 Name: price, dtype: int64
Binning with cut and qcut
Binning means converting continuous numbers into discrete intervals, which
produces an ordered categorical result. cut divides the range
into equal-width bins, while qcut divides the data into bins of
equal size (quantiles). Both return a Series with a
category dtype whose categories are intervals.
import pandas as pd s = pd.Series([1, 7, 12, 19, 25]) print(pd.cut(s, bins=3)) print() print(pd.qcut(s, q=3))
The first block has three equal-width intervals; the second uses quantiles, splitting the values into three groups of roughly equal size. The parenthesised and square brackets describe which endpoints are included. A fuller treatment is in the Pandas binning tutorial.
$ python cat_cut.py
0 (0.976, 9.0]
1 (0.976, 9.0]
2 (9.0, 17.0]
3 (17.0, 25.0]
4 (17.0, 25.0]
dtype: category
Categories (3, interval[float64, right]): [(0.976, 9.0] < (9.0, 17.0] <
(17.0, 25.0]]
0 (0.999, 8.667]
1 (0.999, 8.667]
2 (8.667, 16.667]
3 (16.667, 25.0]
4 (16.667, 25.0]
dtype: category
Categories (3, interval[float64, right]): [(0.999, 8.667] < (8.667, 16.667] <
(16.667, 25.0]]
Source
Pandas Categorical Data Documentation, Pandas Categorical Reference, Pandas cut Documentation, and Pandas qcut Documentation
In this article we have converted columns to the category dtype,
declared explicit and ordered categories, measured the memory saved, grouped
with unused categories, and glimpsed binning. For the memory figures see the
Pandas memory usage tutorial, and for the
interval details see Pandas binning.
Author
List all Pandas tutorials.