Python os.posix_spawn Function

Last modified April 11, 2025

This comprehensive guide explores Python's os.posix_spawn function, a POSIX-compliant process creation mechanism. We'll cover file actions, spawn attributes, and practical examples for efficient process spawning.

Basic Definitions

The os.posix_spawn function creates a new process executing a specified program. It's more efficient than fork-exec on some systems.

Key parameters: path (program to execute), argv (argument list), env (environment), file_actions (file operations), and spawn attributes (process characteristics).

Basic Process Creation

The simplest use of os.posix_spawn launches a program with given arguments. This example runs the 'ls' command to list directory contents.

basic_spawn.py

import os

# Basic process creation
pid = os.posix_spawn(
    "/bin/ls",
    ["ls", "-l", "/tmp"],
    os.environ
)

# Wait for the process to complete
_, status = os.waitpid(pid, 0)
print(f"Process {pid} exited with status {status}")

This creates a new process running 'ls -l /tmp'. The parent waits for the child to complete using os.waitpid. The environment is inherited from the parent.

Note that the first argument in argv should match the program name (convention).

Custom Environment Variables

You can specify custom environment variables when spawning a process. This example sets a special environment for the child process.

custom_env.py

import os

# Create custom environment
custom_env = os.environ.copy()
custom_env["MODE"] = "DEBUG"
custom_env["LOG_LEVEL"] = "VERBOSE"

# Spawn process with custom environment
pid = os.posix_spawn(
    "/usr/bin/env",
    ["env"],
    custom_env
)

# Wait for process
_, status = os.waitpid(pid, 0)
print(f"Process completed with status {status}")

The child process runs 'env' which displays its environment. We see our custom variables MODE and LOG_LEVEL in the output.

Environment variables are passed as a dictionary. Always copy os.environ first to avoid modifying the parent's environment.

File Actions - Redirecting Output

File actions allow modifying file descriptors before program execution. This example redirects stdout to a file.

redirect_output.py

import os

# Prepare file actions
file_actions = os.posix_spawn_file_actions()
file_actions.addopen(1, "output.txt", os.O_WRONLY | os.O_CREAT, 0o644)
file_actions.addclose(2)  # Close stderr

# Spawn process with file actions
pid = os.posix_spawn(
    "/bin/ls",
    ["ls", "-l", "/"],
    os.environ,
    file_actions=file_actions
)

os.waitpid(pid, 0)
print("Output written to output.txt")

This redirects stdout (fd 1) to output.txt and closes stderr (fd 2). The ls output goes to the file instead of the terminal.

File actions must be created before spawning and can include open, close, and dup2 operations.

Spawn Attributes - Process Group

Spawn attributes control process characteristics. This example creates a new process group for the child process.

process_group.py

import os

# Prepare spawn attributes
spawn_attrs = os.posix_spawnattr()
spawn_attrs.setflags(os.POSIX_SPAWN_SETPGROUP)
spawn_attrs.setpgroup(0)  # Create new process group

# Spawn process with attributes
pid = os.posix_spawn(
    "/bin/sleep",
    ["sleep", "10"],
    os.environ,
    spawn_attrs=spawn_attrs
)

print(f"Created process {pid} in new process group")
os.waitpid(pid, 0)

The child process (sleep) runs in its own process group. This is useful for process management and signal handling.

Attributes can control signal handling, scheduling, and other process characteristics.

Combining File Actions and Attributes

This example demonstrates using both file actions and spawn attributes to create a process with customized characteristics.

combined_features.py

import os

# File actions - redirect stdout/stderr
file_actions = os.posix_spawn_file_actions()
file_actions.addopen(1, "out.log", os.O_WRONLY | os.O_CREAT, 0o644)
file_actions.adddup2(1, 2)  # Redirect stderr to stdout

# Spawn attributes - new process group
spawn_attrs = os.posix_spawnattr()
spawn_attrs.setflags(os.POSIX_SPAWN_SETPGROUP)
spawn_attrs.setpgroup(0)

# Spawn process
pid = os.posix_spawn(
    "/bin/bash",
    ["bash", "-c", "echo 'Output'; echo 'Error' >&2"],
    os.environ,
    file_actions=file_actions,
    spawn_attrs=spawn_attrs
)

os.waitpid(pid, 0)
print("Both output and error written to out.log")

The bash command's output and error streams are redirected to out.log. The process runs in its own process group.

This shows how to combine multiple features of posix_spawn for advanced process control.

Error Handling

Proper error handling is crucial when spawning processes. This example shows how to handle various spawn failures.

error_handling.py

import os
import errno

try:
    # Attempt to spawn non-existent program
    pid = os.posix_spawn(
        "/nonexistent/program",
        ["program"],
        os.environ
    )
except OSError as e:
    if e.errno == errno.ENOENT:
        print("Program not found")
    elif e.errno == errno.EACCES:
        print("Permission denied")
    else:
        print(f"Spawn failed: {e}")

try:
    # Invalid file action
    file_actions = os.posix_spawn_file_actions()
    file_actions.addopen(1, "/root/protected", os.O_WRONLY, 0o644)
    pid = os.posix_spawn(
        "/bin/true",
        ["true"],
        os.environ,
        file_actions=file_actions
    )
except PermissionError:
    print("Cannot open protected file")

The first attempt fails because the program doesn't exist. The second fails due to permission issues with file actions.

Always handle potential errors when spawning processes, especially with file operations or privileged paths.

Signal Handling

Spawn attributes can control signal handling in the child process. This example resets signal handlers to defaults.

signal_handling.py

import os
import signal

# Prepare spawn attributes for signal handling
spawn_attrs = os.posix_spawnattr()
spawn_attrs.setsigdefault({signal.SIGINT: signal.SIG_DFL})
spawn_attrs.setflags(os.POSIX_SPAWN_SETSIGDEF)

# Spawn process that ignores Ctrl-C
pid = os.posix_spawn(
    "/bin/bash",
    ["bash", "-c", "echo 'Running...'; sleep 10; echo 'Done'"],
    os.environ,
    spawn_attrs=spawn_attrs
)

print(f"Process {pid} running - Ctrl-C will terminate it")
os.waitpid(pid, 0)

The child process resets SIGINT to default behavior, making it interruptible with Ctrl-C. Without this, it might inherit parent's signal handlers.

Signal handling attributes are particularly important for long-running or interactive child processes.

Security Considerations

Path security: Validate executable paths to prevent command injection
Environment sanitization: Carefully control environment variables
File descriptor handling: Close unnecessary FDs in the child
Privilege separation: Consider dropping privileges in child
Error handling: Handle all possible spawn failures

Best Practices

Use absolute paths: For both executable and file actions
Clean environments: Pass only necessary environment variables
Resource limits: Set appropriate limits via spawn attributes
Signal management: Explicitly set signal handling
Error checking: Validate all inputs before spawning

Source References

Author

My name is Jan Bodnar, and I am a passionate programmer with extensive programming experience. I have been writing programming articles since 2007. To date, I have authored over 1,400 articles and 8 e-books. I possess more than ten years of experience in teaching programming.

List all Python tutorials.

Ebooks