Amazon S3

Amazon S3 is the cloud storage solution offered by Amazon Web Services, and you can use set up an integration in Refuel to read from S3. This is commonly used for reading documents and images, but also for uploading and ingesting CSVs and other files formats.

​

Setup guide

​

Step 0: Get Refuel AWS Account ID and External ID

Contact us at support@refuel.ai to get Refuel’s AWS Account ID and external ID that you can use in the later steps.

​

Step 1: Create IAM Policy for S3 Access

Navigate to the IAM Console in AWS, click on Policies in the left navbar, and then hit the “Create Policy” button. Click on the JSON tab, copy the policy below (substitute your bucket name, add multiple buckets, restrict to specific keys in a bucket) and then hit Next. Click Next again, give the policy a name, such as “RefuelS3ReadAccessPolicy” and then hit “Create policy”.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::YourBucketName/*",
                "arn:aws:s3:::YourBucketName"
            ]
        }
    ]
}

​

Step 2: Create IAM Role for S3 Access

Follow the steps below to create an IAM role that Refuel can assume to access your S3 bucket.

• In the IAM Console, click on Roles in the left navbar and click on “Create role”.
• In “Trusted entity type”, click on “AWS account” and then click on “Another AWS account”.
• Paste in the Refuel AWS account ID from Step 0.
• Click “Require external ID” and paste in the external ID from step 0, and click Next.
• Search for, and add the policy you just created: “RefuelS3ReadAcccessPolicy” and hit Next.
• Give this role a name, let’s say “RefuelS3ReadAccessRole” and click “Create role”.

​

Step 3: Send Refuel the ARN of the newly created role

Share the role ARN with Refuel via email or Slack.

​

Step 4: Start sending Refuel your data!

That’s it! When you send Refuel any pieces of content that sit in S3, all you need to send is the path in S3, and Refuel will be able to securely access it.