Python os.pwrite Function

Last modified April 11, 2025

This comprehensive guide explores Python's os.pwrite function, which writes data to a file at a specific offset without changing the file pointer. We'll cover its parameters, behavior, and practical use cases.

Basic Definitions

The os.pwrite function writes bytes to a file descriptor at a specified offset. It's similar to os.write but doesn't modify the file pointer position.

Key parameters: fd (file descriptor), data (bytes to write), offset (position in file). Returns number of bytes written. Requires file opened for writing.

Basic File Writing

This example demonstrates basic usage of os.pwrite to write data at a specific position in a file. We first create a file with some content.

basic_write.py

import os

# Create a file with initial content
with open("data.txt", "w") as f:
    f.write("Initial content")

# Open file for writing and get file descriptor
fd = os.open("data.txt", os.O_RDWR)

# Write at offset 8 without changing file pointer
bytes_written = os.pwrite(fd, b"NEW ", 8)
print(f"Wrote {bytes_written} bytes")

# Verify content
os.lseek(fd, 0, os.SEEK_SET)
print(os.read(fd, 100))  # Output: b'Initial NEWtent'

os.close(fd)

This writes "NEW " at position 8 in the file. The original content "content" becomes "NEWtent" because we overwrote part of it. The file pointer remains unchanged after pwrite.

Notice we use os.open to get a file descriptor, as pwrite requires a file descriptor, not a file object.

Writing at Different Offsets

This example shows writing at multiple positions in a file while maintaining the original file pointer position between writes.

multiple_writes.py

import os

# Create a sample file
with open("records.txt", "w") as f:
    f.write("ABCDEFGHIJKLMNOPQRSTUVWXYZ")

fd = os.open("records.txt", os.O_RDWR)

# Get initial position
initial_pos = os.lseek(fd, 0, os.SEEK_CUR)
print(f"Initial position: {initial_pos}")

# Write at various offsets
os.pwrite(fd, b"123", 5)    # Overwrite positions 5-7
os.pwrite(fd, b"456", 10)   # Overwrite positions 10-12
os.pwrite(fd, b"789", 20)   # Overwrite positions 20-22

# Verify position hasn't changed
current_pos = os.lseek(fd, 0, os.SEEK_CUR)
print(f"Position after writes: {current_pos}")

# Read entire file
os.lseek(fd, 0, os.SEEK_SET)
print(os.read(fd, 100))  # Output: b'ABCDE123HIJ456NOPQRST789VWXYZ'

os.close(fd)

Each pwrite operation writes at the specified offset without affecting the others or the file pointer. This allows precise modifications at known file locations.

The output shows the original string with our inserted number sequences at the specified positions.

Appending with pwrite

While pwrite is typically for overwriting, we can use it to append by writing at the end of file. This requires knowing the file size.

append_write.py

import os

# Create initial file
with open("log.txt", "w") as f:
    f.write("Log starts here\n")

fd = os.open("log.txt", os.O_RDWR)

# Get file size for append
file_size = os.lseek(fd, 0, os.SEEK_END)

# Append new entries without moving pointer
os.pwrite(fd, b"Entry 1\n", file_size)
file_size += len("Entry 1\n")

os.pwrite(fd, b"Entry 2\n", file_size)
file_size += len("Entry 2\n")

os.pwrite(fd, b"Entry 3\n", file_size)

# Verify content
os.lseek(fd, 0, os.SEEK_SET)
print(os.read(fd, 1000).decode())

os.close(fd)

This demonstrates how to append data using pwrite by tracking the file size. Each write goes at the end of the current content.

While os.write with O_APPEND is simpler for pure appending, this shows pwrite's flexibility.

Handling Partial Writes

pwrite may write fewer bytes than requested. This example shows how to handle partial writes and ensure complete data is written.

partial_writes.py

import os

fd = os.open("partial.dat", os.O_RDWR | os.O_CREAT)

large_data = b"X" * 1000000  # 1MB of data
offset = 0
remaining = len(large_data)

while remaining > 0:
    written = os.pwrite(fd, large_data[-remaining:], offset)
    if written == 0:  # Disk full?
        raise IOError("Failed to write data")
    offset += written
    remaining -= written
    print(f"Wrote {written} bytes, {remaining} remaining")

os.close(fd)

# Verify
print(f"Final file size: {os.path.getsize('partial.dat')} bytes")

This implements a write loop that continues until all data is written. It updates the offset and remaining data with each partial write.

In practice, disk full situations are rare, but robust code should handle partial writes for large data transfers.

Thread-Safe File Writing

pwrite is atomic when the writes are within a single filesystem block, making it useful for thread-safe operations. This example demonstrates this property.

thread_safe.py

import os
import threading

fd = os.open("counter.txt", os.O_RDWR | os.O_CREAT)
os.write(fd, b"0")  # Initialize counter
os.lseek(fd, 0, os.SEEK_SET)

def increment_counter(thread_id):
    for _ in range(1000):
        # Read current value
        current = int(os.pread(fd, 1, 0))
        # Write incremented value
        os.pwrite(fd, str(current + 1).encode(), 0)

threads = []
for i in range(10):
    t = threading.Thread(target=increment_counter, args=(i,))
    threads.append(t)
    t.start()

for t in threads:
    t.join()

os.lseek(fd, 0, os.SEEK_SET)
print(f"Final counter value: {os.read(fd, 10).decode()}")

os.close(fd)

This implements a simple counter with concurrent increments. While not perfect (race conditions exist between read and write), it demonstrates pwrite's atomic write capability.

For true atomic increments, consider file locking or other synchronization methods in addition to pwrite.

Binary Data Manipulation

pwrite excels at binary file manipulation. This example shows modifying specific bytes in a binary file while leaving others unchanged.

binary_edit.py

import os

# Create a binary file with a pattern
with open("data.bin", "wb") as f:
    f.write(bytes(range(256)))  # 0x00 to 0xFF

fd = os.open("data.bin", os.O_RDWR)

# Modify specific bytes
os.pwrite(fd, b"\xFF\xFF", 0x10)  # Overwrite positions 0x10-0x11
os.pwrite(fd, b"\xAA\xBB", 0x80)  # Overwrite positions 0x80-0x81

# Verify changes
os.lseek(fd, 0, os.SEEK_SET)
content = os.read(fd, 256)

print(f"Byte at 0x10: {hex(content[0x10])}")
print(f"Byte at 0x11: {hex(content[0x11])}")
print(f"Byte at 0x80: {hex(content[0x80])}")
print(f"Byte at 0x81: {hex(content[0x81])}")

os.close(fd)

This creates a binary file with a known pattern, then modifies specific byte ranges using pwrite. The rest of the file remains unchanged.

This technique is useful for binary file formats where you need to modify specific headers or data structures within a larger file.

Performance Comparison

This example compares pwrite performance with regular write when doing random access to a large file.

performance.py

import os
import time
import random

def test_pwrite(fd, positions):
    start = time.time()
    for pos in positions:
        os.pwrite(fd, b"X", pos)
    return time.time() - start

def test_write(fd, positions):
    start = time.time()
    for pos in positions:
        os.lseek(fd, pos, os.SEEK_SET)
        os.write(fd, b"X")
    return time.time() - start

# Create a large test file
fd = os.open("perf_test.dat", os.O_RDWR | os.O_CREAT)
os.ftruncate(fd, 1000000)  # 1MB file

# Generate random positions
positions = [random.randint(0, 999999) for _ in range(1000)]

# Test pwrite
pwrite_time = test_pwrite(fd, positions)

# Test regular write
write_time = test_write(fd, positions)

os.close(fd)

print(f"pwrite time: {pwrite_time:.4f} seconds")
print(f"write time: {write_time:.4f} seconds")
print(f"Ratio: {write_time/pwrite_time:.2f}x")

This benchmarks writing to 1000 random positions in a 1MB file. pwrite typically performs better by avoiding repeated seek operations.

The performance difference grows with more random writes, making pwrite ideal for certain database-like operations.

Security Considerations

File descriptors: Requires proper file descriptor management
Atomic writes: Writes up to PIPE_BUF bytes are atomic
Error handling: Always check return value for partial writes
Concurrency: Safe for multi-threaded use with proper coordination
Platform support: Available on Unix-like systems, not Windows

Best Practices

Use for random access: Ideal for writing at known offsets
Check return values: Verify all bytes were written
Combine with pread: For read-modify-write cycles
Consider file sync: Use fsync for durability when needed
Prefer for large files: More efficient than seek+write

Source References

Author

My name is Jan Bodnar, and I am a passionate programmer with extensive programming experience. I have been writing programming articles since 2007. To date, I have authored over 1,400 articles and 8 e-books. I possess more than ten years of experience in teaching programming.

List all Python tutorials.

Ebooks