Overview

Migrating data from Google Cloud Storage (GCS) to Amazon S3 is common for organizations consolidating on AWS or implementing multi-cloud strategies. This guide provides a comprehensive walkthrough of the migration process using Cloney.

Migration Benefits

AWS S3 offers extensive service integrations, mature tooling ecosystem, and broad global availability across regions.

Prerequisites

  • A Cloney account with an active subscription
  • Google Cloud Storage bucket with data to migrate
  • AWS S3 bucket (destination)
  • Appropriate credentials for both services

Setup Access Credentials

Before proceeding, ensure you have configured least privilege access for both services:

Google Cloud: Least Privilege Access SetupAWS S3: Least Privilege Access Setup

Step 1: Add Your GCS Source

  1. Log in to your Cloney dashboard
  2. Navigate to Jobs → Create New Job
  3. Select Google Cloud Storage as your source provider
  4. Upload your GCP service account JSON key file or enter credentials manually:
    • Project ID: Your GCP project identifier
    • Service Account Email: The service account email address
    • Private Key: The service account private key
    • Bucket Name: The name of your source GCS bucket
  5. Click Validate Connection to verify access

Step 2: Configure Your AWS S3 Destination

  1. Select AWS S3 as your destination provider
  2. Enter your AWS credentials:
    • Access Key ID: Your IAM user access key
    • Secret Access Key: Your IAM user secret key
    • Region: The AWS region for your destination bucket
    • Bucket Name: The name of your destination S3 bucket
  3. Click Validate Connection to verify access

Step 3: Configure Migration Options

Customize your migration with these options:

OptionDescriptionRecommendation
Prefix/Path FilterMigrate only objects matching a specific prefixUse for partial migrations
Preserve MetadataCopy object metadata along with dataEnable for full fidelity
Storage ClassSet the S3 storage class for migrated objectsStandard for frequent access
Overwrite ExistingReplace objects that already exist in destinationEnable for sync operations

Step 4: Start the Migration

  1. Review your migration configuration
  2. Click Start Migration
  3. Monitor progress in the Jobs dashboard
  4. Cloney will provide real-time updates on:
    • Objects transferred
    • Data volume migrated
    • Transfer speed
    • Any errors encountered
Migration Complete

Once finished, verify your data in the AWS S3 console and update your applications to use the new bucket.

Storage Class Mapping

GCS storage classes map to AWS S3 as follows:

GCS ClassRecommended S3 Class
StandardS3 Standard
NearlineS3 Standard-IA
ColdlineS3 Glacier Instant Retrieval
ArchiveS3 Glacier Deep Archive

Best Practices

  • Test with a subset first: Run a pilot migration with a small subset of data to validate your configuration.
  • Choose the right S3 region: Select an AWS region close to your users or other AWS resources.
  • Enable versioning: Consider enabling S3 versioning for additional data protection.
  • Set up lifecycle policies: Configure S3 lifecycle rules to optimize storage costs over time.
  • Keep source data temporarily: Don't delete your GCS data immediately; retain it until you've fully validated the migration.