← Back to issue list

Unwanted source files copy in `build`

View original Github issue

Metadata

Project
craft-parts
Number
#1609
Type
issue
State
open
Author
imatrisciano
Labels
Created
Updated
Closed

Current evaluation

nil plugin copies source files into build directory unnecessarily, wasting disk space for large files. Maintainer proposed static-assets plugin or hardlink-source attribute; neither implemented yet.

Suggested action: keep open

Reason: Maintainer @mr-cal has actively engaged, acknowledging the issue and proposing concrete solutions (static-assets plugin or hardlink-source build attribute). The author responded positively to the proposed approach. No implementation has begun yet, but the issue is triaged and under active discussion.

Impact: 75 Quick Win: 33.75 Staleness: 35 Complexity: 55 Confidence: 75 Support Request: 10

Issue body

### Bug Description At the beginning of the **build** stage, a part's source files are copied into the `build` directory (see the [relevant source code](https://github.com/canonical/craft-parts/blob/139cbce69c1ef9e62c0ecf36b7567061299f3444/craft_parts/executor/part_handler.py#L463)). This happens if the build is not an update and if the plugin is not **out-of-source**. Out-of-source plugins are those which are known to not modify source files during build (e.g. `cmake`, `qmake`, `go-use`, `meson`). The issue here is that the `nil` plugin is supposed to be an empty plugin, where every instruction is explicitly set by the developer. Instead, if the `nil` plugin is used, the copy operation mentioned above is performed anyway as part of the **build** stage, and there is nothing a developer can do to prevent that. This is an issue if the project packs large files, like [inference snaps](https://documentation.ubuntu.com/inference-snaps/) do when packing larger models. This behavior is unexpected and can leave a developer wondering why their build takes more space than anticipated. The same could be said about the `dump` plugin, whose purpose is to move files from **src** to **install**, there's arguably no need for it to copy source files into **build**. ### Workaround A **workaround** for this issue is to use an **out-of-source** plugin (such as `cmake`) so the copy in **build** is skipped at the cost of confusion in the _snapcraft.yaml_ file, and unneeded build dependencies. Dropping the `source` keyword in the part description also prevents the copy from happening, but will also prevent the build system from noticing if the files on disk change to trigger an update. ### To Reproduce Setup an example project with a 1 GiB data file: ```bash mkdir -p ~/source/craft_size_playground cd ~/source/craft_size_playground mkdir data # Create large file (1 GiB) dd if=/dev/random of=data/my_file bs=1G count=1 mkdir snap touch snap/snapcraft.yaml ``` Use the following as your snapcraft.yaml: ```yaml name: craft-size-playground base: core24 version: '0.1' summary: Size test snap description: | Test snap. grade: devel confinement: strict compression: lzo parts: my-file: plugin: nil source: data override-build: | cp -rf --archive --link --no-dereference ${CRAFT_PART_SRC}/* ${CRAFT_PART_INSTALL} organize: my_file: (component/my-component) components: my-component: type: standard summary: Test component description: Just a file size test ``` Then, pack it with `snapcraft pack`. If doing multiple tests, make sure to `snapcraft clean` in between. Finally, check the build environment disk usage: ```bash sudo su cd /var/snap/lxd/common/lxd/storage-pools/default/containers/snapcraft_snapcraft-craft-size-playground-amd64-*/rootfs/root/parts du . -d 3 -h ``` The disk usage report will look like this: ``` 28K ./my-file/state 4.0K ./my-file/install 4.0K ./my-file/export 1.1G ./my-file/src 4.0K ./my-file/layer 1.1G ./my-file/build 4.0K ./my-file/run 2.1G ./my-file 2.1G . ``` The report above shows that the large file was copied two times: once in the **src** directory and another time in the **build** directory. The first copy is expected, because we are not using destructive build, and an hard-link to **src** from the project directory is not possible because it would be a cross-device link; but the second copy in **build** is unexpected and, in this example, unwanted. ### part yaml ```shell ``` ### Relevant log output ```shell Starting snapcraft, version 9.0.0.post19+dirty Logging execution to '/home/ubuntu/.local/state/snapcraft/log/snapcraft-20260605-161006.529105.log' Launching managed ubuntu 24.04 instance... Creating instance from base instance Starting instance Starting snapcraft, version 9.0.0.post19+dirty Logging execution to '/tmp/snapcraft.log' Initializing lifecycle Installing build-packages Installing build-snaps Pulling my-file Building my-file :: + cp -rf --archive --link --no-dereference /root/parts/my-file/src/my_file /root/parts/my-file/install/ Staging my-file Priming my-file Packing... Reading snap metadata... Running linters... Running linter: classic Running linter: gpu Running linter: library Running linter: metadata Creating snap package... Packed: craft-size-playground_0.1_amd64.snap, craft-size-playground+my-component.comp ```

Evaluation history

Date Model Scores Action Summary
qwen3.6-35b-a3b-mtp-q6
Impact: 75
Quick Win: 33.75
Staleness: 35
Complexity: 55
Confidence: 75
Support Request: 10
keep open nil plugin copies source files into build directory unnecessarily, wasting disk space for large files. Maintainer proposed static-assets plugin or hardlink-source attribute; neither implemented yet.
qwen/qwen3.6-35b-a3b
Staleness: 10
Complexity: 40
Confidence: 90
Support Request: 0
keep open Bug report: unnecessary source file copying in build directory with nil plugin, causing high disk usage. Actively discussed with a proposed new plugin solution.
qwen3.6-35b-a3b-mtp-q6
Staleness: 20
Complexity: 65
Confidence: 90
Support Request: 10
keep open Maintainer proposed a new static-assets plugin to avoid copying large source files into the build directory when using nil or dump plugins. Currently under discussion with the author regarding LXD hardlink limitations.
qwen3.6-35b-a3b-mtp-q6
Staleness: 5
Complexity: 65
Confidence: 90
Support Request: 5
keep open Unwanted source file copying to the build directory with nil/dump plugins causes excessive disk usage. Maintainer acknowledged the issue, proposed a new static-assets plugin, and is actively discussing implementation details with the reporter.

Update history

No update history recorded yet.

Related issues

No related issues found above the similarity threshold.