How to Migrate Data to Stook Using Rclone

Step-by-step guide to migrate data to Stook using Rclone.

This guide explains how to migrate data from any S3-compatible storage provider to Medianova’s Stook Object Storage using Rclone. You will learn how to configure Rclone, copy data, monitor the transfer, and verify the integrity of migrated files.

Prerequisites

• Installed Rclone (download from https://rclone.org/downloads/)

• Access credentials (Access Key, Secret Key) for both source and Stook buckets

• Endpoint URL of your source and target storage

Step-by-Step Instructions

Step 1: Configure Rclone

1. Open your terminal or command prompt.

2. Run the following command:

   rclone config

3. Follow the interactive wizard:

   • Choose n for a new remote.

   • Enter a name for your remote (e.g., s3-source).

   • Select your S3-compatible provider.

   • Enter configuration details (endpoint URL, access key, secret key).

4. Repeat the process to configure your Stook target bucket (e.g., stook-target).

   • When asked for S3 type, choose Ceph Object Storage.

---

Step 2: Copy Data

To copy all data from the source bucket to the target bucket:

rclone copy s3-source:source-bucket-name stook-target:target-bucket-name

To copy specific files or directories:

rclone copy s3-source:source-bucket-name/path/to/source-data \
stook-target:target-bucket-name/path/to/target-location

---

Step 3: Monitor Progress

Add the --progress flag to monitor transfer status in real time:

rclone copy --progress s3-source:source-bucket-name stook-target:target-bucket-name

---

Step 4: Verify Data

After transfer, verify integrity using the rclone check command:

rclone check s3-source:source-bucket-name stook-target:target-bucket-name

This compares file sizes and hashes (MD5 or SHA1) and reports mismatches.

---

Troubleshooting / FAQ

• Problem: Files missing in target bucket.

  • Solution: Re-run the rclone copy command with --progress.

• Problem: Authentication error.

  • Solution: Verify your access key, secret key, and endpoint URL.

• Problem: Slow transfer speed.

  • Solution: Use --transfers=N option to increase parallel transfers.

---

References

• Rclone Downloads