Skip to content

GCS to Cloudflare R2 Sync Disaster Recovery Guide

This guide explains how to restore the GCS to Cloudflare R2 synchronization pipeline in the event of an infrastructure failure, accidental deletion, or cross-project migration.

It specifically focuses on restoring the internal service account roles and permissions required for Eventarc and Pub/Sub to function correctly.

Prerequisites

  • Target GCS source bucket should be created/restored.
  • Target Cloudflare R2 bucket should be available and API credentials generated.
  • GCP Project Number and Project ID must be known.

Required Permissions

To successfully rebuild the sync flow, specific IAM roles must be granted to both custom and Google-managed internal service accounts.

1. Custom Sync Service Account

This is the account attached to the Cloud Run service (r2-sync-service-account).

  • Role: roles/storage.objectViewer (To read files from GCS)
  • Role: roles/run.invoker (To be triggered by Eventarc)

2. Internal Google-Managed Service Accounts

Eventarc relies on underlying Pub/Sub infrastructure. The following internal accounts must have their permissions explicitly restored:

  • Google Storage Project Account: service-<PROJECT_NUMBER>@gs-project-accounts.iam.gserviceaccount.com
    • Role: roles/pubsub.publisher
  • Google Pub/Sub Service Account: service-<PROJECT_NUMBER>@gcp-sa-pubsub.iam.gserviceaccount.com
    • Role: roles/iam.serviceAccountTokenCreator

Step-by-Step Process

1. Define Environment Variables

First, define your project number and project ID to ensure the IAM bindings are applied to the correct project:

export PROJECT_ID="your-project-id"
export PROJECT_NUMBER="$(gcloud projects describe $PROJECT_ID --format='value(projectNumber)')"

2. Restore Internal Service Account Roles

Restore the missing IAM requirements for the Google-managed service accounts. This is the most common point of failure during a disaster recovery or project migration scenario.

# Grant pubsub.publisher to the Google Storage project account
gcloud projects add-iam-policy-binding $PROJECT_NUMBER \
  --member="serviceAccount:service-${PROJECT_NUMBER}@gs-project-accounts.iam.gserviceaccount.com" \
  --role="roles/pubsub.publisher"

# Grant iam.serviceAccountTokenCreator to the Pub/Sub service account
gcloud projects add-iam-policy-binding $PROJECT_NUMBER \
  --member="serviceAccount:service-${PROJECT_NUMBER}@gcp-sa-pubsub.iam.gserviceaccount.com" \
  --role="roles/iam.serviceAccountTokenCreator"

Example (using hardcoded project number 625774899042):

gcloud projects add-iam-policy-binding 625774899042 \
  --member="serviceAccount:service-625774899042@gs-project-accounts.iam.gserviceaccount.com" \
  --role="roles/pubsub.publisher"

gcloud projects add-iam-policy-binding 625774899042 \
  --member="serviceAccount:service-625774899042@gcp-sa-pubsub.iam.gserviceaccount.com" \
  --role="roles/iam.serviceAccountTokenCreator"

3. Re-create and Configure the Sync Service Account

If the custom service account was deleted, recreate it and bind the necessary roles:

# Create the service account
gcloud iam service-accounts create r2-sync-service-account \
  --display-name="R2 Sync Service Account"

# Grant Storage Object Viewer on the specific bucket
gcloud storage buckets add-iam-policy-binding gs://YOUR_GCS_BUCKET_NAME \
  --member="serviceAccount:r2-sync-service-account@${PROJECT_ID}.iam.gserviceaccount.com" \
  --role="roles/storage.objectViewer"

4. Restore Eventarc Triggers

Once the IAM permissions are restored, you must recreate the Eventarc triggers for the Cloud Run service to resume syncing new objects:

# Restore Upload Trigger
gcloud eventarc triggers create sync-upload-trigger \
  --location=global \
  --destination-run-service=YOUR_CLOUD_RUN_SERVICE_NAME \
  --destination-run-region=YOUR_REGION \
  --event-filters="type=google.cloud.storage.object.v1.finalized" \
  --event-filters="bucket=YOUR_GCS_BUCKET_NAME" \
  --service-account="r2-sync-service-account@${PROJECT_ID}.iam.gserviceaccount.com"

# Restore Deletion Trigger
gcloud eventarc triggers create sync-delete-trigger \
  --location=global \
  --destination-run-service=YOUR_CLOUD_RUN_SERVICE_NAME \
  --destination-run-region=YOUR_REGION \
  --event-filters="type=google.cloud.storage.object.v1.deleted" \
  --event-filters="bucket=YOUR_GCS_BUCKET_NAME" \
  --service-account="r2-sync-service-account@${PROJECT_ID}.iam.gserviceaccount.com"

5. Catch Up on Missed Data

Any files uploaded to GCS while the sync service was down will not be automatically synced when the service is restored.

You must use the Cloudflare R2 Data Migration Tool (in the Cloudflare Dashboard) to pull the missed files from GCS to R2.

Important Notes

  • Eventarc will silently fail to deliver messages if the internal Google-managed service accounts lack the pubsub.publisher and iam.serviceAccountTokenCreator roles.
  • The IAM binding commands require the numeric Project Number, not the Project ID string.
  • If you are restoring to a completely new GCP project, the Google-managed service accounts might not exist until you activate the Pub/Sub and Eventarc APIs. Ensure APIs are enabled before running the IAM restore scripts.