Overview

Migrating data from AWS S3 to Google Cloud Storage (GCS) is a common requirement for organizations adopting multi-cloud strategies or transitioning to Google Cloud Platform. This guide provides a comprehensive walkthrough of the migration process using Cloney.

Migration Benefits

Google Cloud Storage offers competitive pricing, strong integration with BigQuery and other GCP services, and excellent global performance.

Prerequisites

  • A Cloney account with an active subscription
  • AWS S3 bucket with data to migrate
  • Google Cloud Storage bucket (destination)
  • Appropriate credentials for both services

Setup Access Credentials

Before proceeding, ensure you have configured least privilege access for both services:

AWS S3: Least Privilege Access SetupGoogle Cloud: Least Privilege Access Setup

Step 1: Add Your AWS S3 Source

  1. Log in to your Cloney dashboard
  2. Navigate to Jobs → Create New Job
  3. Select AWS S3 as your source provider
  4. Enter your AWS credentials:
    • Access Key ID: Your IAM user access key
    • Secret Access Key: Your IAM user secret key
    • Region: The AWS region where your bucket is located
    • Bucket Name: The name of your source S3 bucket
  5. Click Validate Connection to verify access

Step 2: Configure Your GCS Destination

  1. Select Google Cloud Storage as your destination provider
  2. Upload your GCP service account JSON key file or enter credentials manually:
    • Project ID: Your GCP project identifier
    • Service Account Email: The service account email address
    • Private Key: The service account private key
    • Bucket Name: The name of your destination GCS bucket
  3. Click Validate Connection to verify access

Step 3: Configure Migration Options

Customize your migration with these options:

OptionDescriptionRecommendation
Prefix/Path FilterMigrate only objects matching a specific prefixUse for partial migrations
Preserve MetadataCopy object metadata along with dataEnable for full fidelity
Storage ClassSet the GCS storage class for migrated objectsStandard for frequent access
Overwrite ExistingReplace objects that already exist in destinationEnable for sync operations

Step 4: Start the Migration

  1. Review your migration configuration
  2. Click Start Migration
  3. Monitor progress in the Jobs dashboard
  4. Cloney will provide real-time updates on:
    • Objects transferred
    • Data volume migrated
    • Transfer speed
    • Any errors encountered
Migration Complete

Once finished, verify your data in the GCS console and update your applications to use the new bucket.

Storage Class Mapping

AWS S3 storage classes map to GCS as follows:

AWS S3 ClassRecommended GCS Class
S3 StandardStandard
S3 Intelligent-TieringStandard (with Autoclass)
S3 Standard-IANearline
S3 One Zone-IANearline
S3 Glacier Instant RetrievalNearline
S3 Glacier Flexible RetrievalColdline
S3 Glacier Deep ArchiveArchive

Best Practices

  • Test with a subset first: Run a pilot migration with a small subset of data to validate your configuration.
  • Schedule during off-peak hours: For large migrations, consider running during periods of low traffic.
  • Verify data integrity: Compare object counts and sizes between source and destination after migration.
  • Plan for DNS/application updates: Update your applications to point to the new GCS bucket after successful migration.
  • Keep source data temporarily: Don't delete your S3 data immediately; retain it until you've fully validated the migration.