Skip to content

ADR-0006: Evaluate Replacing MinIO Object Storage

  • Status: 📝 Proposed (evaluation)
  • Date: 2026-02-13
  • Owners: @osmank, @team/devops, @team/architecture, @team/security

Context

Appcircle uses MinIO as the default S3-compatible object storage layer.

We are evaluating a replacement because MinIO’s license and distribution model changes have introduced increased risk and uncertainty for production use, especially across both SaaS (Appcircle-operated) and self-hosted (customer-operated) deployments.

Timeline of relevant MinIO changes

  • 2021 (May): MinIO changed the server/client/gateway licensing from Apache 2.0 to GNU AGPLv3.
  • 2025 (May–June): The Community Edition web-based management experience was significantly reduced, and several management/enterprise-grade capabilities (policy management, monitoring, replication controls, etc.) became aligned with paid offerings.
  • 2025 (Oct): Community Edition moved to “source-only distribution” (no regular prebuilt binaries / container images; users are expected to build).
  • 2025 (Dec): The public GitHub repository entered “maintenance mode” signaling limited/no feature development and reduced community contribution flow.

Resulting risks we want to mitigate

  • License/commercial risk: AGPLv3 obligations and commercial licensing expectations can be incompatible or unclear depending on how Appcircle is deployed and redistributed.
  • Operational risk: Maintenance-mode posture and source-only distribution can increase operational burden (build/publish images, patch cadence control) and raise security maintenance concerns.
  • Self-hosted adoption risk: Customers expect a turnkey install/upgrade experience; source-only distribution and reduced tooling increase friction and support burden.
  • Vendor/trust risk: Single-vendor governance and sudden product packaging changes increase platform dependency risk.

Decision

TBD.

This ADR documents the evaluation and the option space. Once we select an approach, the status will be updated to ✅ Accepted.

Options considered

Option A — Ceph Object Gateway (RGW) (often via Rook-Ceph)

Pros

  • Mature ecosystem; widely deployed.
  • Strong durability/HA patterns; scales well for large clusters.
  • S3-compatible gateway with established operational knowledge in the industry.

Cons

  • Higher operational complexity and resource footprint.
  • Requires storage expertise (tuning, upgrades, failure domains, capacity planning).

Option B — SeaweedFS (S3 Gateway)

Pros

  • Lighter operational footprint than Ceph; Kubernetes-friendly.
  • Supports replication and erasure coding.
  • Commonly used as a pragmatic “self-hosted S3” alternative.

Cons

  • S3 compatibility depth must be validated against Appcircle usage.
  • Production maturity for our workload patterns requires POC evidence.

Option C — Garage (S3-compatible)

Pros

  • Designed with self-hosting in mind; distributed object storage focus.
  • S3-compatible API surface aimed at common workloads.

Cons

  • Feature set and operational maturity must be proven for Appcircle requirements.
  • S3 edge-case compatibility must be validated (multipart, presigned URLs, etc.).

Option D — RustFS (S3-compatible)

Pros

  • Apache-2.0 licensed (business-friendly), potentially reducing copyleft-related risk.
  • Rust-based implementation (memory safety focus).
  • Positioning as a MinIO-alternative with S3 compatibility.

Cons

  • Ecosystem maturity and production track record must be validated.
  • S3 compatibility depth and operational tooling must be proven via POC.

Evaluation criteria

  1. License/compliance fit (SaaS + redistributed/self-hosted)
  2. S3 compatibility depth (APIs used today)
  3. Operational complexity (install/upgrade/scale, day-2 burden)
  4. HA/DR capabilities (replication, erasure coding, failover patterns)
  5. Security features (TLS, auth/policies, encryption, auditability)
  6. Performance & scalability (throughput, latency, large object handling)
  7. Kubernetes friendliness (operators, Helm maturity)
  8. Supportability (community health, vendor options)
  9. Cost model (infra + ops + licensing)

Consequences

Positive

  • Clear risk framing for why MinIO replacement is being evaluated.
  • Produces a traceable comparison and reduces ad-hoc decision-making.
  • Helps define a storage strategy that can differ between SaaS and self-hosted.

Negative

  • POC and evaluation effort (engineering + ops + security/compliance).
  • Potential migration complexity (data move, compatibility gaps, downtime planning).

Follow-ups / migrations

  • Inventory Appcircle’s current S3 usage by service (operations + features).
  • Shortlist 2–3 candidates for POC among: Ceph RGW, SeaweedFS, Garage, RustFS.
  • Run functional + performance + operational tests.
  • Produce a final recommendation and update this ADR to ✅ Accepted.