← Back to issue list

project: use os.sched_getaffinity instead of multiprocessing.cpu_count

View original Github issue

Metadata

Project
snapcraft
Number
#2732
Type
pull request
State
merged
Author
kyrofa
Labels
—
Created
Updated
Closed

Current evaluation

Merged as commit 332df0f97 (approved by 2 reviewers, +4/-11 across 3 files), replacing multiprocessing.cpu_count() with os.sched_getaffinity() in project code so parallel build counts honour cgroup/affinity limits such as LXD containers.

Resolution: closed resolved

Reason: The PR itself was merged into the snapcraft mainline as commit 332df0f97, landing the os.sched_getaffinity-based CPU count in get_parallel_build_count. A follow-up PR #2963 (commit fc9799b26) later added fallbacks for platforms lacking os.sched_getaffinity (e.g. macOS), but the original change shipped as merged.

Evaluated by: mr-cal-bot

No scores available.

Issue body

- [x] Have you followed the [guidelines for contributing](https://github.com/snapcore/snapcraft/blob/master/CONTRIBUTING.md)? - [x] Have you signed the [CLA](http://www.ubuntu.com/legal/contributors/)? - [x] Have you successfully run `./runtests.sh static`? - [x] Have you successfully run `./runtests.sh tests/unit`? ----- `multiprocessing.cpu_count` always returns the number of raw hardware CPUs available. However, in the context snapcraft is using it, what is actually desired is the number of _usable_ CPUs (taking cgroups, affinity, etc. into account). For example, if you limit a LXD container to a single CPU, `multiprocessing.cpu_count` will continue to detect the number of CPUs available on the host, not 1. This PR moves snapcraft away from this, and starts using `os.sched_getaffinity` instead, which detects the CPUs actually available for use by the process.

Evaluation history

Date Model Scores Action Summary
stealth/space-bunny-alpha — closed resolved Merged as commit 332df0f97 (approved by 2 reviewers, +4/-11 across 3 files), replacing multiprocessing.cpu_count() with os.sched_getaffinity() in project code so parallel build counts honour cgroup/affinity limits such as LXD containers.
stealth/space-bunny-alpha — closed resolved Merged as 332df0f97, replacing multiprocessing.cpu_count() with os.sched_getaffinity() so project builds respect process CPU affinity and container limits.
qwen/qwen3.6-35b-a3b — — Merged PR replaces multiprocessing.cpu_count with os.sched_getaffinity to accurately detect usable CPUs within cgroups and affinity constraints. Approved by reviewers and passed CI.
qwen3.6-35b-a3b-mtp-q6 — — Merged replacement of multiprocessing.cpu_count with os.sched_getaffinity to accurately detect usable CPUs by respecting cgroups and process affinity, fixing incorrect CPU detection in containerized environments.

Update history

No update history recorded yet.

Related work

  • Related To: canonical/snapcraft#2963 (confidence 88%)

    Follow-up merged PR (commit fc9799b26) adding multiprocessing.cpu_count()/1 fallbacks when os.sched_getaffinity is unavailable, fixing a regression introduced by 332df0f97

  • Related To: snapcraft#354ad897c (confidence 45%)

    Later commit 'parts: fix parallel build count' touching the same parallel-build-count logic

Related issues

Issue Project State Summary Similarity
#2963 project: add fallbacks for os.sched_getaffinity snapcraft merged Merged #2963 (commit fc9799b26): get_parallel_build_count now falls back to multiprocessing.cpu_count(), then 1, when os.sched_getaffinity() is missing (e.g. macOS), fixing the regression from 332df0f9; unit tests added.
80%