Files & storage
Set up Amazon S3
An Amazon S3 bucket, or a prefix within one — or any S3-compatible store such as MinIO, Wasabi or Backblaze B2. Every object becomes a document, and a later sync re-downloads only the objects whose ETag changed.
Before you start
- An IAM user (or role) with programmatic access, and a policy allowing s3:ListBucket on the bucket and s3:GetObject on its objects — scoped to your prefix if you're setting one.
- That's all it can ever do: with nothing beyond List and Get attached, the key can't write, delete or overwrite anything in the bucket.
- In the AWS IAM console create a user (or role) with programmatic access and attach that policy.
- Create an access key for the user — an Access key ID and a Secret access key, shown once.
- Documents → Amazon S3: enter the Bucket and its Region (or a Custom endpoint instead, for an S3-compatible store), then paste the two keys.
- Optionally set a Prefix to limit to a sub-folder, choose access groups, Test, then Sync.
What comes in
- Every object under the bucket (or prefix) becomes one document, named from the tail of its key.
- Objects in Glacier or Deep Archive storage are skipped, not errored on — reading them back needs an hours-long restore request first.
| Prefix | Optional key prefix (e.g. policies/) — S3 has no real folders, so this is the folder-like start of object names to sync. Blank = the whole bucket. |
|---|---|
| Custom endpoint | For MinIO, Wasabi, Backblaze B2 and other S3-compatible stores. Blank = Amazon S3 itself. |
| Session token | Only for temporary credentials (key IDs starting ASIA…) from STS, SSO or an assumed role — these expire within hours, and the source stops syncing when they do. |
Keys are write-only here — once saved they're never shown again, so create a fresh pair in IAM if one is lost. A wrong Region fails every request outright, and the error names the right one. Very large buckets stop at 20,000 objects per sync — narrow the prefix to reach the rest.
Was this page helpful?
Last updated 20 Sep 2026