ZetCode

Python __reduce_ex__ Method

Last modified April 8, 2025

This comprehensive guide explores Python's __reduce_ex__ method, the special method used for object serialization with pickle. We'll cover basic usage, protocol versions, custom reduction, and practical examples.

Basic Definitions

The __reduce_ex__ method is called by the pickle module to get a tuple representing an object's state. It enables custom serialization behavior for objects.

Key characteristics: it accepts a protocol version number, returns a tuple specifying how to reconstruct the object, and can be used to optimize serialization. It's more advanced than __reduce__.

Basic __reduce_ex__ Implementation

Here's a simple implementation showing how __reduce_ex__ works with the pickle module. It demonstrates the basic return tuple structure.

basic_reduce_ex.py 
import pickle

class SimpleObject:
    def __init__(self, value):
        self.value = value
    
    def __reduce_ex__(self, protocol):
        print(f"Using protocol {protocol}")
        return (self.__class__, (self.value,))

obj = SimpleObject(42)
data = pickle.dumps(obj)
new_obj = pickle.loads(data)
print(new_obj.value)  # 42

This example shows the minimal implementation needed for pickling. The method returns a tuple containing the class and constructor arguments.

The protocol parameter indicates the pickle protocol version being used. Higher protocols support more features but may not be backward compatible.

Custom Reduction with State

For more complex objects, you might need to separately handle the object's state. This example shows how to include instance attributes in the reduction.

custom_reduction.py 
import pickle

class CustomObject:
    def __init__(self, name, data):
        self.name = name
        self.data = data
    
    def __reduce_ex__(self, protocol):
        if protocol >= 2:
            return (self.__class__, (self.name,), {'data': self.data})
        else:
            return (self.__class__, (self.name, self.data))

obj = CustomObject("test", [1, 2, 3])
data = pickle.dumps(obj, protocol=4)
new_obj = pickle.loads(data)
print(new_obj.name, new_obj.data)  # test [1, 2, 3]

This implementation shows protocol-aware reduction. For protocol 2+, it uses a 3-tuple format (class, args, state). For older protocols, it falls back to a simpler format.

The state dictionary allows more control over how instance attributes are restored during unpickling, which can be more efficient for complex objects.

Optimizing for Protocol Versions

Different pickle protocols support different features. __reduce_ex__ can optimize serialization based on the protocol version.

protocol_optimization.py 
import pickle

class OptimizedObject:
    def __init__(self, items):
        self.items = items
    
    def __reduce_ex__(self, protocol):
        if protocol >= 4:
            return (self.__class__, (self.items,), None, None, iter(self.items))
        elif protocol >= 2:
            return (self.__class__, (self.items,), None)
        else:
            return (self.__class__, (self.items,))

obj = OptimizedObject([1, 2, 3])
data = pickle.dumps(obj, protocol=4)
new_obj = pickle.loads(data)
print(new_obj.items)  # [1, 2, 3]

This example shows protocol-specific optimizations. Protocol 4+ supports a 5-tuple return format that can include an iterator for large sequences.

The 5-tuple format is (class, args, state, listitems, dictitems) and allows more efficient pickling of large data structures by streaming them.

Handling External Resources

Some objects manage resources that shouldn't be pickled. __reduce_ex__ can handle these cases by returning alternative reconstruction instructions.

external_resources.py 
import pickle

class DatabaseConnection:
    def __init__(self, connection_string):
        self.connection_string = connection_string
        self.connection = self._connect()
    
    def _connect(self):
        print(f"Connecting to {self.connection_string}")
        return f"Connection to {self.connection_string}"
    
    def __reduce_ex__(self, protocol):
        return (self.__class__, (self.connection_string,))

conn = DatabaseConnection("localhost:5432")
data = pickle.dumps(conn)
new_conn = pickle.loads(data)  # Will re-establish connection
print(new_conn.connection)

This example shows how to handle objects with external resources. The actual connection isn't pickled - just the information needed to recreate it.

When unpickled, the object will be recreated with just the connection string, and the __init__ method will establish a new connection.

Custom Reconstruction Function

For complete control over unpickling, you can provide a custom reconstruction function instead of using the class constructor.

custom_reconstructor.py 
import pickle

class ComplexObject:
    def __init__(self, x, y):
        self.x = x
        self.y = y
        self._calculate()
    
    def _calculate(self):
        self.sum = self.x + self.y
    
    def __reduce_ex__(self, protocol):
        return (reconstruct_complex, (self.x, self.y))

def reconstruct_complex(x, y):
    obj = ComplexObject.__new__(ComplexObject)
    obj.x = x
    obj.y = y
    obj._calculate()
    return obj

obj = ComplexObject(10, 20)
data = pickle.dumps(obj)
new_obj = pickle.loads(data)
print(new_obj.sum)  # 30

This example uses a standalone function for reconstruction instead of the class's __init__ method. This provides complete control over the process.

The reconstruction function creates the object without calling __init__, then manually sets up the state and calls necessary initialization methods.

Best Practices

Source References

Author

My name is Jan Bodnar, and I am a passionate programmer with extensive programming experience. I have been writing programming articles since 2007. To date, I have authored over 1,400 articles and 8 e-books. I possess more than ten years of experience in teaching programming.

List all Python tutorials.