Migrating to S3

You can migrate data to S3 storage from another provider's storage (for example, AWS S3, Azure, GCP) using Rclone.

1. Prepare the site for migration.
2. Perform data migration from the source storage to Servercore S3 (target).

1. Prepare the site for migration

1. Create a bucket in S3.
2. Connect to the storage via Rclone.
3. Create two new configurations via the S3 API — for the target and source storage. Use the parameters of the current provider for the source storage.

2. Data migration

1. Perform the copying of objects from the source storage to the target:

   rclone copy -v -P --log-file=rclone.log --transfers=<transfers> <source_provider>:<bucket_name> <destination_provider>:<bucket_name>

   Specify:

   • <transfers> — the number of parallel copy threads. The recommended parameter value is 10. If a large number of timeout errors occur, reduce the number of threads;
   • <source_provider> — the name of the source storage configuration;
   • <destination_provider> — the name of the target storage configuration;
   • <bucket_name> — the name of the bucket in the target storage.

2. Run the synchronization. Only changed or new objects from the source storage will be copied:

   rclone sync -v -P --log-file=rclone.log --transfers=<transfers> <source_provider>:<bucket_name> <destination_provider>:<bucket_name>

3. Stop the services that use the source storage. This is necessary for data consistency.

4. Disable data modification in the source storage; leave read-only access only.

5. Synchronize the storages (previously copied objects will be skipped):

   rclone sync -v -P --log-file=rclone.log --transfers=<transfers> <source_provider>:<bucket_name> <destination_provider>:<bucket_name>

6. When the synchronization is complete, switch the services to S3.

7. Optional: delete the data from the source storage.