Python os.cpu_count Function
Last modified April 11, 2025
This comprehensive guide explores Python's os.cpu_count function,
which detects the number of available CPU cores. We'll cover basic usage,
multiprocessing applications, and practical examples.
Basic Definitions
The os.cpu_count function returns the number of CPUs in the system.
If undeterminable, it returns None. This helps optimize parallel processing.
The count includes both physical and logical cores. On systems with hyper-threading, logical cores may be double the physical core count.
Basic CPU Count Detection
The simplest use of os.cpu_count retrieves the total number of
available CPU cores. This provides system information for resource allocation.
import os
# Get CPU count
cpu_count = os.cpu_count()
if cpu_count is not None:
print(f"Number of CPU cores: {cpu_count}")
else:
print("Could not determine CPU count")
This example shows the basic usage pattern. The function may return None if the count cannot be determined, so always check for this case.
The result varies by system configuration and may include hyper-threaded cores as separate CPUs on supported hardware.
Setting Multiprocessing Pool Size
os.cpu_count is commonly used to set optimal pool sizes for
multiprocessing. This example demonstrates creating a pool with one worker
per CPU core.
import os
import multiprocessing
def worker(num):
return num * num
if __name__ == '__main__':
# Use all available cores
pool_size = os.cpu_count() or 1
with multiprocessing.Pool(pool_size) as pool:
results = pool.map(worker, range(10))
print(results)
This creates a multiprocessing pool with one worker per CPU core. The 'or 1' fallback ensures at least one worker if cpu_count returns None.
Using all available cores maximizes parallel processing efficiency for CPU-bound tasks.
Adjusting for Hyper-Threading
Some workloads benefit from using only physical cores. This example shows how to estimate physical core count by halving the logical count.
import os
def get_physical_cores():
logical_cores = os.cpu_count() or 1
# Assume hyper-threading doubles core count
return max(1, logical_cores // 2)
print(f"Logical cores: {os.cpu_count()}")
print(f"Estimated physical cores: {get_physical_cores()}")
This provides a rough estimate of physical cores by halving the logical count. The actual physical core count may differ on some systems.
For precise physical core detection, platform-specific tools like lscpu on Linux may be needed.
Thread Pool Executor Configuration
os.cpu_count helps configure ThreadPoolExecutor for optimal
parallelism. This example demonstrates CPU-bound task distribution.
import os
from concurrent.futures import ThreadPoolExecutor
def cpu_intensive(n):
return sum(i*i for i in range(n))
if __name__ == '__main__':
max_workers = (os.cpu_count() or 1) * 2 # I/O bound multiplier
with ThreadPoolExecutor(max_workers) as executor:
results = list(executor.map(cpu_intensive, [10**6]*10))
print(f"Completed {len(results)} tasks with {max_workers} workers")
For CPU-bound tasks, worker count typically matches CPU cores. For I/O-bound tasks, multiplying by a factor (like 2-4x) may improve throughput.
The optimal worker count depends on task characteristics and system resources.
Custom Resource Allocation
This example shows reserving CPU cores for other processes while using the remaining cores for your application.
import os
def allocate_cores(reserve=1):
total = os.cpu_count() or 1
available = max(1, total - reserve)
print(f"Total cores: {total}")
print(f"Allocating {available} cores (reserving {reserve})")
return available
# Reserve 2 cores for system processes
cores_to_use = allocate_cores(2)
print(f"Using {cores_to_use} cores for processing")
This pattern is useful when running alongside other critical processes. The reserve parameter specifies how many cores to leave available.
Always ensure at least one core remains allocated to prevent zero-worker scenarios in edge cases.
Cross-Platform CPU Detection
This example demonstrates platform-specific CPU count detection as a fallback
when os.cpu_count returns None.
import os
import platform
import subprocess
def get_cpu_count():
count = os.cpu_count()
if count is not None:
return count
system = platform.system()
try:
if system == "Linux":
return int(subprocess.check_output("nproc", shell=True))
elif system == "Windows":
return int(os.environ["NUMBER_OF_PROCESSORS"])
elif system == "Darwin":
return int(subprocess.check_output("sysctl -n hw.ncpu", shell=True))
except:
return 1
print(f"Detected CPU cores: {get_cpu_count()}")
This provides fallback methods for Linux (nproc), Windows (environment var), and macOS (sysctl) when os.cpu_count fails. The try-except ensures a minimum of 1 core is returned.
Platform-specific methods may provide more accurate counts in some edge cases.
Performance Benchmarking
This example uses os.cpu_count to configure a performance
benchmark that scales with available CPU resources.
import os
import time
import multiprocessing
def stress_test(duration):
end = time.time() + duration
while time.time() < end:
pass
if __name__ == '__main__':
duration = 5 # seconds per core
cores = os.cpu_count() or 1
print(f"Running stress test on {cores} cores for {duration} seconds each")
processes = []
for _ in range(cores):
p = multiprocessing.Process(target=stress_test, args=(duration,))
p.start()
processes.append(p)
for p in processes:
p.join()
print("Benchmark completed")
This creates one CPU-intensive process per core for a specified duration. The benchmark workload automatically scales with available CPU resources.
Such benchmarks help evaluate system performance under full CPU load conditions.
Security Considerations
- Resource limits: Containers/VMs may report host CPU count
- Dynamic scaling: Cloud environments may change CPU count
- Affinity masks: Process may be limited to subset of CPUs
- Power saving: Some cores may be offline for energy efficiency
- Virtual cores: Count may not reflect actual performance
Best Practices
- Always check for None: Provide fallback value (usually 1)
- Consider workload: Adjust worker count based on task type
- Reserve resources: Leave cores for system processes
- Monitor utilization: Adjust based on actual performance
- Document assumptions: Note CPU count dependencies
Source References
Author
List all Python tutorials.