Example of migration to S3 via Rclone

You can migrate data to S3 storage from another provider's storage (e.g., AWS S3, Azure, GCP) using Rclone.

1. Prepare the environment for migration.
2. Migrate data from the source storage to Servercore S3 (destination).

1. Prepare the environment for migration

1. Create a bucket in S3.
2. Connect to the storage using Rclone.
3. Create two new configurations via the S3 API: one for the destination storage and one for the source storage. For the source storage, use the settings of your current provider.

2. Data migration

1. Copy objects from the source storage to the destination storage:

   rclone copy -v -P --log-file=rclone.log --transfers=<transfers> <source_provider>:<bucket_name> <destination_provider>:<bucket_name>

   Specify:

   • <transfers> — the number of parallel transfer threads. The recommended value is 10. If a large number of timeout errors occur, reduce the number of threads;
   • <source_provider> — the configuration name of the source storage;
   • <destination_provider> — the configuration name of the destination storage;
   • <bucket_name> — the bucket name in the destination storage.

2. Start synchronization. Only modified or new objects from the source storage will be copied:

   rclone sync -v -P --log-file=rclone.log --transfers=<transfers> <source_provider>:<bucket_name> <destination_provider>:<bucket_name>

3. Stop the services that use the source storage. This is necessary for data consistency.

4. Disallow data modification in the source storage, leaving only read access.

5. Synchronize the storages (previously copied objects will be skipped):

   rclone sync -v -P --log-file=rclone.log --transfers=<transfers> <source_provider>:<bucket_name> <destination_provider>:<bucket_name>

6. Once synchronization is complete, switch the services to S3.

7. Optional: delete data from the source storage.