FileSourceHandler is slow for large sources
Metadata
Current evaluation
FileSourceHandler downloads/extracts large sources very slowly (requests+tarfile,1KB chunks, 3600s timeout): ~6h vs ~1.5h with wget on RISC-V. Triaged enhancement, root cause identified, optimizations (piped download+extract, threaded decompression) not yet implemented.
Suggested action: keep open
Reason: Labels 'Status: Triaged'/'Type: Enhancement' plus substantive maintainer input from @steinbro (asking about the 1-hour timeout rationale and an `extract-patterns` key) and @lengau (scoping the work to 'download and begin extraction simultaneously', 'multi-threaded decompression where feasible', 'separate threads') confirm a live, valid enhancement tracked as CRAFT-4689. Code inspection shows the slow path is unchanged: `craft_parts/sources/base.py:313-314` still uses `requests.get(..., timeout=3600)` with `url_utils.download_request` writing 1024-byte chunks, and `craft_parts/sources/tar_source.py` still extracts via single-threaded `tarfile.extractall`; `git log_path` on both files shows no performance work since the original `81263be5 lint: set a timeout in requests.get() (#270)`. No commit/PR resolves it, and415 days of inactivity is not grounds for closing. Impact is moderate (severe build-time regressions only on slow/emulated builders like LP riscv64); complexity is high because the proposed pipelining changes checksum, caching, resume, and error-handling semantics and may need new schema keys.
Evaluated by: mr-cal-bot
Issue body
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| stealth/space-bunny-alpha |
Impact:
52
Quick Win:
11
Complexity:
68
Confidence:
84
Actionability:
64
|
keep open | FileSourceHandler downloads/extracts large sources very slowly (requests+tarfile,1KB chunks, 3600s timeout): ~6h vs ~1.5h with wget on RISC-V. Triaged enhancement, root cause identified, optimizations (piped download+extract, threaded decompression) not yet implemented. | |
| qwen/qwen3.6-35b-a3b |
Impact:
70
Quick Win:
34
Complexity:
40
Confidence:
80
Actionability:
80
|
keep open | FileSourceHandler download performance is slow for large sources on RISC-V, taking ~6 hours vs ~1.5h with wget workaround. Issue is triaged and under active discussion. | |
| qwen/qwen3.8-27b |
Impact:
45
Quick Win:
20
Staleness:
55
Complexity:
55
Confidence:
78
Support Request:
5
|
keep open | FileSourceHandler download/extraction is CPU-bound and slow on resource-constrained hardware (riscv64). Triaged as enhancement with Jira ticket CRAFT-4689. Discussion identified multi-threaded decompression and piped download as solutions. No fix committed; code still uses requests.get with 1-hour timeout and single-threaded tarfile. | |
| qwen/qwen3.6-35b-a3b |
Staleness:
80
Complexity:
65
Confidence:
85
Support Request:
10
|
keep open | Performance issue with FileSourceHandler causing slow downloads and extraction on resource-constrained hardware. Currently under discussion with maintainers exploring multi-threading and piping optimizations. Internal ticket CRAFT-4689 created. | |
| qwen3.6-35b-a3b-mtp-q6 |
Staleness:
45
Complexity:
75
Confidence:
85
Support Request:
0
|
keep open | Performance enhancement for FileSourceHandler to speed up downloading and extracting large archives on constrained hardware. Currently triaged with an internal Jira ticket and under discussion for parallel processing and multi-threaded decompression. |
Update history
No update history recorded yet.
Related issues
No related issues found above the similarity threshold.