← Back to issue list

Create a workflow for testing dev environments

View original Github issue

Metadata

Project
starflow
Number
#85
Type
issue
State
open
Author
mr-cal
Labels
Type: Enhancement Status: Triaged
Created
Updated
Closed
—

Current evaluation

Triaged enhancement (JIRA CRAFT-4739) to add test-dev-environment.yaml and self-test-dev-environment.yaml running make setup/lint/format/test/docs in clean LXD across 3 distros x amd64/arm64. Still unimplemented: no such workflow files exist under .github/workflows.

Suggested action: keep open

Reason: Maintainer-authored enhancement with Status: Triaged, a Jira ticket (CRAFT-4739), and recent maintainer activity (2026-02-11 comment about a `ty` versioning edge case during `make setup` that this workflow would surface), so it is live and relevant. I verified the work is still outstanding: `.github/workflows/` contains no test-dev-environment.yaml or self-test-dev-environment.yaml (git log search for "dev-environment"/LXD workflow returned no commits), and the root Makefile defines setup, lint, format and docs but has no `test` target, so the spec's command list needs a small addition. Scope is real work though: cross-arch arm64 via LXD means QEMU emulation, which is slow and flaky.

Evaluated by: mr-cal-bot

Impact: 55 Quick Win: 19 Complexity: 60 Confidence: 85 Actionability: 85

Issue body

## Overview Our development environment should be tested across a variety of distros and architectures. ## Rationale Project maintainers rarely set up pristine development environments, so we rarely experience what it's like for new contributors to follow our [getting started guides](https://github.com/canonical/snapcraft/blob/main/CONTRIBUTING.md#set-up-for-development). ## Specification * Create a `test-dev-environment.yaml` workflow that runs: ```bash make setup make lint make format make test make docs ``` * Run the workflow in clean LXD environments. * Note that a github runner isn't a good environment, since it has tons of pre-installed software. * By default, the workflow should run 6 tests with the matrix: * distros * Ubuntu 22.04 * Ubuntu 24.04 * Debian Trixie * architectures: * amd64 * arm64 * The distros and architectures should be configurable inputs, similar to other workflows ([example](https://github.com/canonical/starflow/blob/b447744488c725ad99f00498b85ddec3edce2f4a/.github/workflows/test-python.yaml#L4)) * Create a `self-test-dev-environment.yaml` workflow that clones starbase and runs the `test-dev-environment.yaml` workflow * Use this workflow in starbase (https://github.com/canonical/starbase/issues/438)

Evaluation history

Date Model Scores Action Summary
stealth/space-bunny-alpha
Impact: 55
Quick Win: 19
Complexity: 60
Confidence: 85
Actionability: 85
keep open Triaged enhancement (JIRA CRAFT-4739) to add test-dev-environment.yaml and self-test-dev-environment.yaml running make setup/lint/format/test/docs in clean LXD across 3 distros x amd64/arm64. Still unimplemented: no such workflow files exist under .github/workflows.
qwen/qwen3.6-35b-a3b
Impact: 40
Quick Win: 14
Complexity: 50
Confidence: 75
Actionability: 70
keep open Enhancement to create a GitHub workflow testing dev environments across multiple distros and architectures using LXD. Triaged but not yet implemented.
qwen/qwen3.8-27b
Impact: 45
Quick Win: 20
Staleness: 55
Complexity: 55
Confidence: 55
Support Request: 5
keep open Enhancement to add a test-dev-environment.yaml GitHub workflow running make setup/lint/format/test/docs in clean LXD environments across a distro/arch matrix, plus a self-test workflow for starbase. Triaged and open; last activity 201 days ago noted a ty versioning edge case in make setup.
qwen/qwen3.6-35b-a3b
Staleness: 45
Complexity: 55
Confidence: 85
Support Request: 10
keep open Request to create GitHub Actions workflows testing development environments across multiple Linux distributions and architectures using LXD. Currently triaged, with recent discussion on dependency versioning edge cases.
qwen3.6-35b-a3b-mtp-q6
Staleness: 55
Complexity: 40
Confidence: 75
Support Request: 10
keep open Enhancement to create a GitHub Actions workflow testing dev environments across distros and architectures using LXD. Currently triaged but inactive for ~5 months, with a recent comment noting a ty versioning edge case in make setup.

Update history

No update history recorded yet.

Related work

  • Related To: canonical/starbase#438 (confidence 95%)

    Explicitly cited in the issue body as the consumer of the self-test workflow ("Use this workflow in starbase").

  • Related To: starflow#18 (confidence 70%)

    Commit cd64717 'feat: python test, lint and policy check workflows (#18)' introduced the existing test-python.yaml/lint-python.yaml pattern this issue explicitly reuses for its configurable inputs.

Related issues

Issue Project State Summary Similarity
#127 feat: Create workflow for testing dev environments starflow open Adds a 61-line GitHub Actions workflow spinning up LXD containers to exercise makefile commands. WIP for #85; author confirms it fails on missing packages, Canonical CLA check failing, no reviewers and no activity for 174 days.
71%