Premature parts environment variable expansion
Metadata
Current evaluation
Fixed premature CRAFT_ARCH_TRIPLET_BUILD_FOR expansion by sorting variable replacements; merged in craft-parts#554 (438fa857).
Resolution: closed resolved
Reason: craft-parts#554 was merged as commit 438fa857, ordering project-variable substitutions so CRAFT_ARCH_TRIPLET is replaced after longer names. The change also added regression coverage for the overlapping architecture triplet variables.
Evaluated by: mr-cal-bot
No scores available.
Issue body
### Bug Description
I am trying to create a ROCK for [High-Performance Linpack (HPL)](https://netlib.org/benchmark/hpl/), a Fortran-based toolkit used to benchmark the Top 500 supercomputers in the world. HPL uses the OpenBLAS library under the hood. Since there are multiple OpenBLAS implementations available in the Ubuntu universe package repository, the Debian alternatives mechanism is used in the OpenBLAS package's postinst script to configure the system-wide OpenBLAS installation. However, since *craft's do not execute Debian package postinst scripts in part staging, I need to create the symlinks that would be created by `update-alternatives` manually in override-stage. Same situation for libpsm-infinipath1, another dependency of HPL.
HPL expects _libopenblas.so.0_ and _libpsm_infinipath.so.1_ to be available under _usr/lib/x86_64-linux-gnu_ so I want to use the [CRAFT_ARCH_TRIPLET](https://snapcraft.io/docs/parts-environment-variables#heading--snapcraft-configuration-core22) environment variable, but it is deprecated. I am using `$CRAFT_ARCH_TRIPLET_BUILD_FOR` instead, but it looks like craft-parts is instead expanding `$CRAFT_ARCH_TRIPLET_BUILD_FOR` to `x86_64-linux-gnu_BUILD_FOR` instead. My ROCK packing is failing since `x86_64-linux-gnu_BUILD_FOR` does not exist under _stage/_.
### To Reproduce
1. `sudo snap install rockcraft --classic --edge`
2. Copy & paste attached _rockcraft.yaml_ file to local directory
3. `wget https://netlib.org/benchmark/hpl/hpl-2.3.tar.gz`
4. `rockcraft -v`
5. See error after `craftctl default` completes in the _override-stage_ step.
### part yaml
```shell
---
name: linpack
base: ubuntu:22.04
version: "0.1.0"
summary: A ROCK for the High-Performance LINPACK benchmark
description: |
High-Performance LINPACK (HPL) is a benchmark used to measure the
performance of the Top 500 supercomputers. This ROCK contains
the necessary MPI libraries and LINPACK binaries to run the
benchmark on a Charmed HPC cluster. Libfabric is included
to assist with MPI bindings to the underlying machine host.
license: Apache-2.0
platforms:
amd64:
parts:
linpack:
source: ./hpl-2.3.tar.gz
plugin: autotools
build-packages:
- gfortran
- libmpich-dev
- libatlas-base-dev
- libopenblas-dev
stage-packages:
- libmpich-dev
- libatlas-base-dev
- libopenblas-dev
- libfabric1
override-stage: |
set -eux
craftctl default
ln --force --verbose \
usr/lib/$CRAFT_ARCH_TRIPLET_BUILD_FOR/openblas-pthread/libopenblas.so.0 \
usr/lib/$CRAFT_ARCH_TRIPLET_BUILD_FOR/libopenblas.so.0
ln --force --verbose \
usr/lib/libpsm1/libpsm_infinipath.so.1.16 \
usr/lib/$CRAFT_ARCH_TRIPLET_BUILD_FOR/libpsm_infinipath.so.1
```
### Relevant log output
```shell
Starting Rockcraft 0.0.1.dev1
Logging execution to '/tmp/rockcraft.log'
CRAFT_ARCH_TRIPLET is deprecated, use CRAFT_ARCH_TRIPLET_BUILD_{ON|FOR}
...
2023-09-06 14:53:44.691 :: 2023-09-06 14:53:38.849 :: + set -eux
2023-09-06 14:53:44.691 :: 2023-09-06 14:53:38.850 :: + craftctl default
2023-09-06 14:53:44.691 :: 2023-09-06 14:53:42.868 For pkg-config file /root/stage/usr/lib/x86_64-linux-gnu/openblas-pthread/pkgconfig/blas-openblas.pc, prefix was changed from prefix=/root/parts/linpack/install/usr
2023-09-06 14:53:44.691 :: to prefix=/root/stage/usr
2023-09-06 14:53:44.691 :: 2023-09-06 14:53:42.869 For pkg-config file /root/stage/usr/lib/x86_64-linux-gnu/openblas-pthread/pkgconfig/lapack-openblas.pc, prefix was changed from prefix=/root/parts/linpack/install/usr
2023-09-06 14:53:44.691 :: to prefix=/root/stage/usr
2023-09-06 14:53:44.691 :: 2023-09-06 14:53:42.939 For pkg-config file /root/stage/usr/lib/x86_64-linux-gnu/pkgconfig/blas-atlas.pc, prefix was changed from prefix=/root/parts/linpack/install/usr
2023-09-06 14:53:44.691 :: to prefix=/root/stage/usr
2023-09-06 14:53:44.691 :: 2023-09-06 14:53:42.940 For pkg-config file /root/stage/usr/lib/x86_64-linux-gnu/pkgconfig/lapack-atlas.pc, prefix was changed from prefix=/root/parts/linpack/install/usr
2023-09-06 14:53:44.691 :: to prefix=/root/stage/usr
2023-09-06 14:53:44.691 :: 2023-09-06 14:53:42.941 For pkg-config file /root/stage/usr/lib/x86_64-linux-gnu/pkgconfig/libnsl.pc, prefix was changed from prefix=/root/parts/linpack/install/usr
2023-09-06 14:53:44.691 :: to prefix=/root/stage/usr
2023-09-06 14:53:44.691 :: 2023-09-06 14:53:42.941 For pkg-config file /root/stage/usr/lib/x86_64-linux-gnu/pkgconfig/libtirpc.pc, prefix was changed from prefix=/root/parts/linpack/install/usr
2023-09-06 14:53:44.691 :: to prefix=/root/stage/usr
2023-09-06 14:53:44.691 :: 2023-09-06 14:53:42.942 For pkg-config file /root/stage/usr/lib/x86_64-linux-gnu/pkgconfig/libxcrypt.pc, prefix was changed from prefix=/root/parts/linpack/install/usr
2023-09-06 14:53:44.691 :: to prefix=/root/stage/usr
2023-09-06 14:53:44.691 :: 2023-09-06 14:53:42.943 For pkg-config file /root/stage/usr/lib/x86_64-linux-gnu/pkgconfig/mpich.pc, prefix was changed from prefix=/root/parts/linpack/install/usr
2023-09-06 14:53:44.691 :: to prefix=/root/stage/usr
2023-09-06 14:53:44.691 :: 2023-09-06 14:53:43.271 :: + ln --force --verbose usr/lib/x86_64-linux-gnu_BUILD_FOR/openblas-pthread/libopenblas.so.0 usr/lib/x86_64-linux-gnu_BUILD_FOR/libopenblas.so.0
2023-09-06 14:53:44.691 :: 2023-09-06 14:53:43.275 :: ln: failed to access 'usr/lib/x86_64-linux-gnu_BUILD_FOR/openblas-pthread/libopenblas.so.0': No such file or directory
2023-09-06 14:53:44.691 :: 2023-09-06 14:53:43.428 'override-stage' in part 'linpack' failed with code 1.
```
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| stealth/space-bunny-alpha | — | closed resolved | Fixed premature CRAFT_ARCH_TRIPLET_BUILD_FOR expansion by sorting variable replacements; merged in craft-parts#554 (438fa857). | |
| stealth/space-bunny-alpha | — | closed resolved | Fixed by sorting project variables before expansion in #554 (commit 438fa857), preventing CRAFT_ARCH_TRIPLET from partially rewriting CRAFT_ARCH_TRIPLET_BUILD_FOR; regression coverage was added. | |
| stealth/space-bunny-alpha | — | closed resolved | Fixed premature CRAFT_ARCH_TRIPLET_BUILD_FOR expansion by sorting project variables before replacement, adding comments and a regression test, in craft-parts #554 (438fa857). | |
| qwen/qwen3.6-35b-a3b | — | — | Fixed premature expansion of $CRAFT_ARCH_TRIPLET_BUILD_FOR caused by a naming conflict with deprecated $CRAFT_ARCH_TRIPLET. Resolved by reordering the environment variable dictionary in craft-parts, adding comments, and a regression test. | |
| qwen/qwen3.6-35b-a3b | — | — | Closed. CRAFT_ARCH_TRIPLET_BUILD_FOR expanded prematurely due to deprecated CRAFT_ARCH_TRIPLET overlap. A curly brace workaround was provided. The craft-parts environment dictionary requires reordering and a regression test. | |
| qwen/qwen3.6-35b-a3b | — | — | Closed. Fixed premature expansion of CRAFT_ARCH_TRIPLET_BUILD_FOR due to overlap with deprecated CRAFT_ARCH_TRIPLET. Resolution reorders the environment dictionary to prevent partial matching and adds a regression test. Bracket workaround provided. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Resolved premature expansion of CRAFT_ARCH_TRIPLET_BUILD_FOR caused by overlap with deprecated CRAFT_ARCH_TRIPLET. Fixed by reordering the environment variable dictionary in craft-parts and adding a regression test. |
Update history
No update history recorded yet.
Related work
-
Likely Fixed By:
canonical/craft-parts#554
(confidence 100%)
Merged fix that reordered environment-variable replacement and added a regression test.
-
Caused By:
canonical/craft-parts#514
(confidence 95%)
Its build-on/build-for triplet variables overlapped with deprecated CRAFT_ARCH_TRIPLET expansion; implemented by PR #532.
Related issues
| Issue | Project | State | Summary | Similarity |
|---|---|---|---|---|
| #571 fix(environment): sort project variables before replacement | craft-parts | merged | Merged in canonical/craft-parts#571 as bdc12ec4, sorting project variables so specific build-on/build-for names are replaced before shorter prefixes. | |
| #4366 fix(environment): sort core20 project variables before replacement | snapcraft | merged | Merged PR #4366 in commit ae891d1d, sorting core20 project variables so BUILD_ON/BUILD_FOR values expand before SNAPCRAFT_ARCH_TRIPLET and fixing #4363. | |
| #226 environment: expand global variables | craft-parts | merged | Merged #226, adding expansion of global environment variables in parts values for CRAFT-1073 in commit 5b5fbdc5 (merge f8738e42). |