← Back to issue list

ci: Add starbase smoketest workflow

View original Github issue

Metadata

Project
starflow
Number
#189
Type
pull request
State
open
Author
lengau
Labels
—
Created
Updated
Closed
—

Current evaluation

Adds .github/workflows/smoketest.yaml (130 lines) that pushes bot/smoketest/starflow-<n> to canonical/starbase, opens a draft PR there and comments the link; needs a STARBASE_TOKEN fine-grained PAT. All CI green (incl. zizmor), verified live via starbase#607-609, 1 unresolved comment, awaiting maintainer review.

Suggested action: needs review

Reason: All listed checks pass — including 'GitHub Actions security audit' and 'zizmor', which matter here given the PR pushes a branch and opens PRs in canonical/starbase with a PAT — and the workflow demonstrably works: the three comments from @steinbro/@github-actions ('Smoke test: canonical/starbase#607/#608/#609') are its own output, and all three starbase PRs were closed as throwaway test PRs. It is not landable yet: 'Review status: pending (0 reviewers)' with 1 unresolved review comment, and this repo has a history of tightening CI permissions (e.g. 'ci: prevent template expansion attacks (#178)', 'fix: widen permissions on filter step (#184)') so a second pair of eyes should confirm the token scope and the untrusted-PR-input handling. Confidence is limited because repo file access was unavailable in this round, so the diff body itself could not be inspected.

Evaluated by: mr-cal-bot

Impact: 38 Quick Win: 13 Complexity: 50 Confidence: 70 Actionability: 68

Issue body

Adds .github/workflows/smoketest.yaml: on PR open, points starbase starflow refs at the PR branch, pushes bot/smoketest/starflow-<n> to canonical/starbase, opens a draft PR there, and comments back with the link. On synchronize, pushes an empty commit to re-run starbase CI. Requires a STARBASE_TOKEN secret (fine-grained PAT). Setup: 1. GitHub avatar > Settings > Developer settings > Personal access tokens > Fine-grained tokens > Generate new token. 2. Token name: starflow-smoketest-starbase. Set expiration (e.g. 1 year, rotate before expiry). 3. Resource owner: canonical. 4. Repository access: Only select repositories > starbase, starflow. 5. Repository permissions: Contents = Read and write, Pull requests = Read and write (Metadata read-only is automatic). Everything else No access. 6. Generate token, copy value (shown once). Complete org SSO/approval if asked. 7. This repo > Settings > Secrets and variables > Actions > New repository secret. Name STARBASE_TOKEN, paste value, save.

Evaluation history

Date Model Scores Action Summary
stealth/space-bunny-alpha
Impact: 38
Quick Win: 13
Complexity: 50
Confidence: 70
Actionability: 68
needs review Adds .github/workflows/smoketest.yaml (130 lines) that pushes bot/smoketest/starflow-<n> to canonical/starbase, opens a draft PR there and comments the link; needs a STARBASE_TOKEN fine-grained PAT. All CI green (incl. zizmor), verified live via starbase#607-609, 1 unresolved comment, awaiting maintainer review.
qwen/qwen3.8-27b
Impact: 35
Quick Win: 15
Complexity: 30
Confidence: 80
Actionability: 60
needs review Adds .github/workflows/smoketest.yaml (+130/-0): on PR open, pushes a bot branch to canonical/starbase, opens a draft smoke-test PR there, and comments back with the link; on synchronize, pushes an empty commit to re-run starbase CI. Requires a STARBASE_TOKEN secret. CI fully green; workflow verified live via starbase#607-609; 1 unresolved review comment, no formal reviewer assigned.
qwen/qwen3.8-27b
Impact: 40
Quick Win: 14
Complexity: 40
Confidence: 85
Actionability: 60
needs review Adds .github/workflows/smoketest.yaml to starflow: on PR open it pushes a bot/smoketest branch to canonical/starbase, opens a draft PR there, and comments back with the link; on synchronize it re-runs starbase CI. CI fully green; workflow validated by three live smoke-test runs; awaiting maintainer review with 1 unresolved comment.
qwen/qwen3.8-27b
Impact: 35
Quick Win: 22
Complexity: 20
Confidence: 85
Actionability: 80
needs review Adds .github/workflows/smoketest.yaml: on PR open it pushes a bot/smoketest branch to canonical/starbase, opens a draft PR there, and comments back with the link; on synchronize it pushes an empty commit to re-run starbase CI. Requires a STARBASE_TOKEN secret. CI fully green, workflow validated by two smoke-test runs, awaiting maintainer review.
qwen/qwen3.8-27b
Impact: 45
Quick Win: 20
Complexity: 30
Confidence: 75
Actionability: 65
needs review Adds smoketest.yaml: on PR open, points starbase starflow refs at the PR branch, pushes bot/smoketest/starflow-<n> to canonical/starbase, opens a draft PR, comments back with the link; on synchronize, pushes an empty commit to re-run starbase CI. Review pending.
qwen/qwen3.8-27b
Impact: 35
Quick Win: 18
Complexity: 35
Confidence: 85
Actionability: 80
needs review Adds .github/workflows/smoketest.yaml (+121 lines): on PR open, pushes a bot branch to canonical/starbase, opens a draft PR there, and comments back with the link; on synchronize, pushes an empty commit to re-run starbase CI. Requires STARBASE_TOKEN secret. All CI green; live smoke run confirmed via starbase#607. Awaiting maintainer review.
qwen/qwen3.8-27b
Impact: 35
Quick Win: 21
Complexity: 25
Confidence: 85
Actionability: 80
needs review Adds .github/workflows/smoketest.yaml: on PR open, points starbase starflow refs at the PR branch, pushes bot/smoketest/starflow-<n> to canonical/starbase, opens a draft PR there, and comments back with the link. CI green; smoke test run confirmed.
qwen/qwen3.8-27b
Impact: 45
Quick Win: 12
Complexity: 25
Confidence: 65
Actionability: 35
keep open Adds smoketest.yaml: on PR open, points starbase starflow refs at the PR branch, pushes a bot branch to canonical/starbase, opens a draft PR; on synchronize, pushes an empty commit. Needs STARBASE_TOKEN secret. New, no review; 'smoke' check failing.
qwen/qwen3.8-27b
Impact: 40
Quick Win: 10
Complexity: 35
Confidence: 60
Actionability: 40
keep open Adds smoketest.yaml workflow: on PR open, points starbase's starflow ref at the PR branch, pushes a bot/smoketest branch to canonical/starbase, opens a draft PR, and posts the link. Needs STARBASE_TOKEN secret. New, 0 reviews, smoke check failing.
qwen/qwen3.8-27b
Impact: 35
Quick Win: 18
Complexity: 35
Confidence: 70
Actionability: 80
needs review Adds smoketest.yaml: on PR open, pushes a bot/smoketest/starflow-<n> branch to canonical/starbase, opens a draft PR there and comments back; on synchronize, pushes an empty commit to re-run starbase CI. Requires STARBASE_TOKEN; CI green, pending review.
qwen/qwen3.8-27b
Impact: 40
Quick Win: 12
Complexity: 35
Confidence: 65
Actionability: 45
needs review Adds smoketest.yaml: on PR open, pushes a branch to canonical/starbase, opens a draft PR there, and comments the link; on synchronize, pushes an empty commit to re-run starbase CI. Requires STARBASE_TOKEN. Review pending, 6 unresolved comments, CI green.

Update history

Date Change
updated
updated
updated
updated
updated
updated
updated
created

Related work

  • Related To: starbase#607 (confidence 95%)

    Throwaway smoke-test PR opened by this workflow; closed after the run confirmed it works.

  • Related To: starbase#608 (confidence 95%)

    Second workflow-generated smoke-test PR, closed by the author.

  • Related To: starbase#609 (confidence 95%)

    Third workflow-generated smoke-test PR; its summary states the workflow was already verified via #607-609.

Related issues

Issue Project State Summary Similarity
#627 ci: test starbase main with starflow's split lint workflow starbase closed Test PR verifying starflow's split lint workflow compatibility; not intended to merge. Closed after testing purpose was served.
72%
#607 Smoke test for starflow#189 starbase closed Closed unmerged after successfully exercising the smoketest workflow from canonical/starflow#189; #607-609 served as throwaway validation PRs.
71%
#608 Smoke test for starflow#189 starbase closed Throwaway smoke-test PR by steinbro (3 files, +5/-5) to exercise the smoketest workflow from canonical/starflow#189. Tests and CLA passed; Conventional Commits policy failed as expected. Author closed it unmerged after the workflow ran.
71%
#60 ci: use starflow policy and QA workflows craft-platforms merged Merged into main as commit ab5f54e, replacing project-specific CI configuration with Canonical Starflow policy and QA workflows across eight files.
71%
#442 ci: test the starflow lint workflow split craft-store closed Closed as integration test for canonical/starflow#199; status tracked in that PR.
71%
#5915 Test qa.yaml with starflow PR #101 changes snapcraft closed Temporary qa.yaml test PR pinning starflow workflows to branch `copilot/use-pristine-ubuntu-action` for starflow#101 was closed unmerged after ~298 days with no reviews or comments; main still uses @main.
70%
#146 Migrate to starflow craft-archives closed CI was migrated to canonical/starflow by merged PR #272 (commit a411e24); commit 00d3e30 also replaced the remaining legacy test workflows.
70%
#272 ci: use starflow CI actions starbase merged Merged as a411e24, switching Starbase CI to Starflow actions after canonical/starflow#18 landed.
70%