Configuring 1space Migration

Migrate data from the Public Cloud to SwiftStack Cluster

The 1space migration functionality can be used to execute a live data migration for object data stored in a public cloud storage location. Even before the data is moved from the public cloud to a SwiftStack cluster, it is immediately visible in the 1space namespace. This means that applications can access it as if the data was already migrated.

The migration is distributed across multiple nodes within the SwiftStack cluster to maximize use of available network resources.

SwiftStack supports live migrations from a remote AWS S3 or Swift API object storage.

Configuration Overview

  1. Create the appropriate cloud credential for the target storage sytem. (see 1space Cloud Credentials)
  2. Create a 1space migration the SwiftStack storage location to the remote storage location.
  3. Deploy cluster configuration. (see: Deploying Changes to the Cluster)

Configuration Example

  1. Start with client applications accessing data in an Amazon or Google bucket that contains at least a few files.
  2. Configure a migration policy to move data from Amazon or Google to your on-premises SwiftStack Cluster; as soon as the new migration policy is pushed to the SwiftStack cluster, the migration will start.
  3. View migration progress in the SwiftStack Controller user interface.
  4. Observe that an object not yet migrated is accessible to a client configured to access the SwiftStack API endpoint.
  5. Once the migration is complete, observe in the SwiftStack Client that all objects are now located in SwiftStack containers.

Details

The following steps provide an example of configuring a Multi Cloud Migrate policy.

  1. Integration with AWS S3

    • Add some files to a logical folder (e.g., “pal”) in your Amazon bucket (e.g.,  “swiftstack-anup”).

      ../_images/folder_aws_migration.png
    • Configure a migration policy to migrate data from Amazon to your on-premises SwiftStack Cluster.

      ../_images/aws_migration_policy.png
    • Push the configuration updates to your SwiftStack cluster to make the migration policy effective; as soon as the migration policy is pushed, migration will start.

    • Migration status can be observed in the SwiftStack Controller user interface.

      ../_images/migration_aws_status.png
    • During migration, you can validate that all objects are accessible—even if they have not been migrated.  Note that if you are migrating a very small number of files, you may not have much time to observe this, as the migration can be completed quickly!

    • Once migration completes, objects will appear under your local container (e.g., “swiftstack-anup”) in the Swiftstack client.

      ../_images/migration_local_container.png
    • The objects should appear in both the remote (e.g., AWS S3) and local (SwiftStack) locations.

      ../_images/migration_local_aws.png
      ../_images/migration_aws_remote.png
  2. Integration with Google

    • Add some files to a bucket (e.g., “cloudmigration-anup”) in your Google account.

      ../_images/migration_google_bucket.png
      ../_images/migration_google_folder.png
    • Configure a migration policy to migrate data from Google to your on-premises SwiftStack Cluster.

      ../_images/migration_google_policy.png
    • Push the configuration updates to your SwiftStack cluster to make the migration policy effective. As soon as the migration policy is pushed, migration will start.

    • Migration status can be observed in the SwiftStack Controller user interface.

      ../_images/migration_google_status.png
    • During migration, you can validate that all objects are accessible—even if they have not been migrated.  Note that if you are migrating a very small number of files, you may not have much time to observe this, as the migration can be completed quickly!

    • Once migration completes, objects will appear under your local container (e.g., “cloudmigration-anup”) in the Swiftstack client.

    • The objects should appear in both the remote (e.g., Google) and local (SwiftStack) locations.

      ../_images/migration_google_remote.png