Configuring 1space Lifecycle¶

Archiving with 1space Lifecycle Profile¶

1space Lifecycle can be used to enable an archive strategy by automatically moving on-premises data in SwiftStack storage location another storage location. Those locations must support either an AWS S3 or Swfit API. Those locations may be the public cloud such as AWS S3 or Google Cloud Storage, another SwiftStack Cluster, or another storage location in the same cluster that uses an archival storage policy such as erasure codes.

Data is moved after a period of time, configurable in days. This can be used to free physical storage capacity in on-premises or in a specific storage policy.

Archiving can be configured either on a storage account or on a single individual bucket/container.

Configuration Overview¶

Create the appropriate cloud credential for the target storage sytem. (see 1space Cloud Credentials)
Create a 1space Lifecycle Profile (see: Setting up 1space Profiles to move data older than the desired number of days from SwiftStack to the target storage location.
Create a mapping from the SwiftStack storage location to the remote storage location. (see: 1space Mappings)
Deploy cluster configuration. (see: Deploying Changes to the Cluster)

Archiving Behavior¶

Data will be archived after the number of days set in the Lifecycle Profile.
The archived data can be accessed directly from the SwiftStack namespace even though the data is physically in the remote storage location.
Bucket/container listings can be observed through Swift and S3 APIs.
The SwiftStack Client lists storage location for a given bucket/container.
Any archived data is available directly in its native format by directly accessing the data in the remote storage location. For example, data archived to AWS S3 or Google Cloud Storage is directly accessible using their APIs or UI.
Metadata keys / values can provide addiitonal triggers for the Lifecycle Profile.

Restore-on-GET Configuration Option¶

“Restore on GET” is configured on the storage profile that enables data "promotion" if an archived copy of the data is accessed. This feature is only used for an object that is requested that is not in the SwiftStack cluster, but does exist in the remote location.

When enabled, an additional copy will be made in the SwiftStack cluster when the data is accessed and the lifecycle will begin again. When the lifecycle period occurs for the object and the archived object in the remote archive location has not changed, the local object will be removed.

When disabled, the data will be returned for the remote object, but a local copy will not be made.

Configuration Walk Through¶

The following steps provide an example of configuring a Lifecycle Archive profile.

Create Archive Profile¶

Under Cluster Manage, navigate to 1space. Select Profiles, and click New.

For the Profile type select "lifecyele". Set the days before archiving objects to the desired number of days that works for your use case. "Restore on GET" configuration option can be set here.

Label your profile appropriately and click Save Profile.

Configure Remote Location¶

The remote storage location is where the data will be archived to. Once the location is determined, create an appropriate cloud credential for the target storage sytem.

Under Cluster, navigate to 1space and then Credentials. Create a credential for the archive target. (see 1space Cloud Credentials)

Below are common archive target locations.

Configure AWS S3 Storage location¶

Create a new bucket, or select the desired AWS S3 bucket under the desired AWS account. Ensure that appropriate IAM policies are enabled on the bucket. See Configuring Amazon S3 IAM Policy

Configure Google Cloud Storage location¶

Create a new bucket, or select the desired Google Cloud Storage bucket under the desired Google Cloud account. Ensure that S3 Interoperability has been enabled. See Configuring S3 Interoperability in Google Cloud Storage

Configure SwiftStack location¶

A SwiftStack storage location may also be used as the archive destination. Just as public cloud providers offer storage with various levels of durability, location and performance, a SwiftStack cluster may be configured with various storage policies. (see: SwiftStack Storage Policies)

If the SwiftStack cluster is configured with mutiple storage policies, the SwiftStack Client (see: SwiftStack Client) or other tools such as the python swift client (see: python-swiftclient), can be created with the desired container/bucket storage policy.

The archive target may be the same SwiftStack cluster, or another SwiftStack cluster.

Create Mapping¶

To create a mapping, navigate to Cluster select Manage, then 1space and click on Mappings

Mapping SwiftStack Storage Location¶

For Account, enter the desired storage location in your SwiftStack cluster to be archived. Be sure to include the account prefix such as ``AUTH_`` or ``LDAP_``.

Note

With a SwiftStack cluster, users and storage accounts are distinct entities in order to provide more flexibility in configuration. Be sure to include the account prefix, such as AUTH_ or LDAP_. (see: Storage Users)

Archiving may be performed on an entire account or a specific bucket/container may be archived.

Enter the single bucket/container to be archived, otherwise select 'sync all containers'.

Mapping Remote Location¶

Enter the remote bucket location that data is to be lifecycled to. Also select the credential for the remote bucket location.

This bucket location along with the credential will be verified when the 'Verify' button is selected.

By default, 1space will generate a hashed prefix to ensure that remote bucket namespaces do not overlap. A custom storage location may also be specified. See S3 Specific Information

Select 1space Profile¶

Select the 1space profile you created in previous steps that specifies the lifecycle rules for this mapping.

Verify & Save Configuration¶

A verification step ensures that the credentials selected can access the specified remote storage location.

After the verification passes, save the configuration as a new mapping.

Deploy Configuration¶

The last configuration step is to deploy the configuration to the cluster.

Click the button to navigate to the deploy configuration page and then deploy the configuration to the cluster.

Example Lifecycle Behavior¶

After configuring your lifecycle storage policy the archiving should behave as follows.

Data stored in the configured SwiftStack storage location are archived automatically after the number of days set in the 1space profile. Any archived data remains accessible to clients.

Note the object path in the remote location. If the default prefix was selected, then a prefix will be automatically generated before the account name and bucket/container name. If custom prefix was selected, the hash prefix, account name and bucket/container name will not be present. See S3 Specific Information

AWS S3 Console .. image:: /images/multi_cloud/folder_aws_zero.png

Google Cloud Storage .. image:: /images/multi_cloud/folder_google_one.png

Note the object location is visible via the SwiftStack Client .. image:: /images/multi_cloud/archive_aws_zero.png

If an archived copy of an object is requested and "Restore on GET" is enabled on the 1space profile, a local copy of the object will be made. This local copy will be in addition to the remote copy.

After the specified lifecycle period, the local copy of the object will be removed.

Moving data with 1space Lifecycle Profile¶

Lifecycle can also be be used to move on-premises data to the public cloud in a cloud-native format. Any object stored in SwiftStack on-premises can immediately be moved to the public cloud by the network and hardware infrastructure available to SwiftStack.

Configuration to Move Data¶

Follow the steps outlined in the Lifecycle Archive section; be sure to choose a 0-day archive period to initiate movement of data from on-premises to the public cloud immediately upon storing the data in SwiftStack.