elf: add a class to check for dependency problems in a group of files
Metadata
Current evaluation
Closed unmerged: core22 linting covered missing and unused libraries, while duplicate-base-library checking moved to snapcraft#4149.
Resolution: closed superseded
Reason: Maintainer @sergiusens closed the unmerged PR and created #4149 for its remaining check after @mr-cal confirmed core22 library linting already covered missing and unused libraries.
Evaluated by: mr-cal-bot
No scores available.
Issue body
- [ ] Have you followed the [guidelines for contributing](https://github.com/snapcore/snapcraft/blob/master/CONTRIBUTING.md)?
- [ ] Have you signed the [CLA](http://www.ubuntu.com/legal/contributors/)?
- [ ] Have you successfully run `./runtests.sh static`?
- [ ] Have you successfully run `./runtests.sh tests/unit`?
-----
This is still a work in progress, and somewhat duplicates the current `ldd` based dependency checking code. It is currently not hooked up to anything, and I'm not quite sure where it would make most sense to run the check.
With a single pass over all files in the snap, it performs the following checks:
1. check for unused shared libraries, using the non-libraries as a starting point and recursively following dependencies.
2. check that all dependencies in the set of used elf files are met by either the snap itself or its base. This includes checking that required symbol versions are defined (e.g. to catch executables requiring newer glibc).
3. check whether libraries in the snap duplicate those in the base snap. We only generate a warning if the library from the base snap defines all the symbol versions required by the elf files in the snap.
For (1), I'm defining non-libraries as ELF files without a soname. I intended this to cover both executables and "plugin" type shared object files. Unfortunately my testing showed up a number of cases where plugins have sonames, so perhaps it needs to take the file path into account too.
Here is an example of the current output:
```
>>> from snapcraft.internal import elf
>>> dep = elf.DependencyChecker()
>>> dep.add_base_libs('/snap/core18/current')
>>> for w in dep.check('/snap/lxd/current'): print(w)
...
library lib/liblxcfs.so in snap is unused
library lib/libmnl.so.0.2.0 in snap duplicates /snap/core18/current/lib/x86_64-linux-gnu/libmnl.so.0.2.0 from base snap
library lib/libseccomp.so.2.5.1 in snap duplicates /snap/core18/current/lib/x86_64-linux-gnu/libseccomp.so.2.4.3 from base snap
library lib/libsqlite3.so.0.8.6 in snap duplicates /snap/core18/current/usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6 from base snap
library lib/x86_64-linux-gnu/ceph/compressor/libceph_snappy.so.2.0.0 in snap is unused
library lib/x86_64-linux-gnu/ceph/compressor/libceph_zlib.so.2.0.0 in snap is unused
library lib/x86_64-linux-gnu/ceph/compressor/libceph_zstd.so.2.0.0 in snap is unused
library lib/x86_64-linux-gnu/ceph/crypto/libceph_crypto_isal.so.1.0.0 in snap is unused
library lib/x86_64-linux-gnu/device-mapper/libdevmapper-event-lvm2mirror.so in snap is unused
library lib/x86_64-linux-gnu/device-mapper/libdevmapper-event-lvm2raid.so in snap is unused
library lib/x86_64-linux-gnu/device-mapper/libdevmapper-event-lvm2snapshot.so in snap is unused
library lib/x86_64-linux-gnu/device-mapper/libdevmapper-event-lvm2thin.so in snap is unused
library lib/x86_64-linux-gnu/libdb-5.3.so in snap duplicates /snap/core18/current/usr/lib/x86_64-linux-gnu/libdb-5.3.so from base snap
library lib/x86_64-linux-gnu/libdevmapper-event-lvm2.so.2.02 in snap is unused
library lib/x86_64-linux-gnu/liblvm2app.so.2.2 in snap is unused
library lib/x86_64-linux-gnu/liblvm2cmd.so.2.02 in snap is unused
library lib/x86_64-linux-gnu/libsnappy.so.1.1.7 in snap is unused
library lib/x86_64-linux-gnu/libusbredirhost.so.1.0.0 in snap is unused
library lib/x86_64-linux-gnu/nss/libfreeblpriv3.so in snap is unused
library lib/x86_64-linux-gnu/nss/libsoftokn3.so in snap is unused
library libebt_802_3.so required by bin/ebtables is missing
library libebt_among.so required by bin/ebtables is missing
library libebt_arp.so required by bin/ebtables is missing
library libebt_arpreply.so required by bin/ebtables is missing
library libebt_ip.so required by bin/ebtables is missing
library libebt_ip6.so required by bin/ebtables is missing
library libebt_limit.so required by bin/ebtables is missing
library libebt_log.so required by bin/ebtables is missing
library libebt_mark.so required by bin/ebtables is missing
library libebt_mark_m.so required by bin/ebtables is missing
library libebt_nat.so required by bin/ebtables is missing
library libebt_nflog.so required by bin/ebtables is missing
library libebt_pkttype.so required by bin/ebtables is missing
library libebt_redirect.so required by bin/ebtables is missing
library libebt_standard.so required by bin/ebtables is missing
library libebt_stp.so required by bin/ebtables is missing
library libebt_ulog.so required by bin/ebtables is missing
library libebt_vlan.so required by bin/ebtables is missing
library libebtable_broute.so required by bin/ebtables is missing
library libebtable_filter.so required by bin/ebtables is missing
library libebtable_nat.so required by bin/ebtables is missing
library zfs-2.0/lib/libzfsbootenv.so.1.0.0 in snap is unused
library zfs-2.0/lib/libzpool.so.4.0.0 in snap is unused
```
While this shows up a few libraries that could be removed from the snap, there it also exposes a few problems with the checks:
* the ceph and device-mapper plugins are listed as unused libraries, which has knock-on effects for other libraries in the snap. They should be treated as non-libraries despite having a soname.
* the `libebt_*.so` libraries are reported as missing. It turns out the files exist but have no soname set so were incorrectly excluded from the library set.
So perhaps doing a path based check is a better approach for distinguishing libraries from executables/plugins. Perhaps checking that the file's dirname ends with `/lib` or `/lib/${any_arch_triplet}` would be appropriate?
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| stealth/space-bunny-alpha | — | closed superseded | Closed unmerged: core22 linting covered missing and unused libraries, while duplicate-base-library checking moved to snapcraft#4149. | |
| stealth/space-bunny-alpha | — | closed superseded | Closed unmerged: core22 library linting covered missing/unused libraries; duplicate-base-library detection was deferred to a follow-up issue. | |
| stealth/space-bunny-alpha | — | closed superseded | PR #3413 (elf DependencyChecker, +134/-2, 1 file) closed unmerged by sergiusens on 2023-05-17: core22 library linting already covered missing/unused libs; duplicate-base-lib check moved to follow-up issue #4149. | |
| stealth/space-bunny-alpha | — | closed superseded | Closed unmerged: core22 library linting covered missing and unused libraries; the duplicate-base-library check moved to snapcraft#4149. | |
| stealth/space-bunny-alpha | — | closed superseded | Closed unmerged; core22 linting covered missing and unused libraries, while the remaining duplicate-base-library check was moved to a follow-up issue. | |
| qwen/qwen3.6-35b-a3b | — | — | Closed without merging. The DependencyChecker class inspired existing core22 library linting, but the contribution was abandoned. Maintainers closed it to track the remaining duplicate-base-library check in a new issue. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Superseded by core22 library linting, which implements the first two dependency checks. The PR was closed after maintainers adopted the reference code and opened a separate issue to implement the remaining duplicate base library check. |
Update history
No update history recorded yet.
Related work
-
Superseded By:
snapcraft#4149
(confidence 100%)
Follow-up issue created to implement detection of staged libraries also supplied by the base snap.
Related issues
No related issues found above the similarity threshold.