IR-0011: Testing Distribution Binary Download 400 Error - FWD¶
- Status: Resolved
- Date: 2025-12-19
- Severity: SEV-1 - Critical
- Duration: ~4-6 hours
- Affected: FWD (Cloud) - Testing Distribution portal binary downloads (iOS + Android)
- Owners: @enver, @burako
Summary¶
Testers received intermittent HTTP 400 errors when downloading iOS and Android binaries from the Testing Distribution portal. Roughly half of download attempts failed across 4+ profiles, blocking testing for several hours.
Timeline¶
All times UTC+3 (Istanbul).
| Time | Date | Event |
|---|---|---|
| 07:58 | Dec 19, 2025 | FWD reports download failures on the TD portal; business side escalating. |
| 08:22 | Dec 19, 2025 | Customer details: ~50% of downloads fail with error 400 across multiple profile links. |
| 08:31 | Dec 19, 2025 | Appcircle acknowledges, treating as urgent priority. |
| 08:46 | Dec 19, 2025 | Team reproduces the issue. |
| 09:35 | Dec 19, 2025 | Backend hotfix released to production; customer asked to re-verify. |
| 09:50 | Dec 19, 2025 | Customer verifies all 4 applications; no further issues. |
Root Cause¶
A TesterWeb release regression: a fix present in dev was missed when merging to prep/prod. The defect was related to the Redis instance removal feature and was occasional (did not reproduce on every request), making it hard to catch in tests. The result was intermittent 400 errors on binary download requests.
Impact¶
- Customers affected: FWD (Cloud)
- Services degraded: Testing Distribution binary download (iOS + Android), 4+ profiles
- Data loss: No
Resolution¶
After the customer report, Appcircle reproduced the issue in follow-up tests. The backend team coordinated and prepared a hotfix within ~1.5 hours and deployed it to production. UAT tests were performed first, then the customer was informed; the customer confirmed within ~30 minutes that the issue was resolved across 4+ profiles.
Action Items¶
| # | Action | Owner | Due Date | Status |
|---|---|---|---|---|
| 1 | Ensure dev fixes are not dropped during prep/prod merges (release-branch parity check) | Backend / DevOps | Pending | |
| 2 | Add regression coverage for occasional TD download failures tied to the Redis-instance-removal change | Backend / QA | Pending |
Links¶
- Linear issue(s): BE-7928
- Related: TesterWeb release regression (Redis instance removal)