How to Upload a File to S3: A Step-by-Step Guide
Introduction
Amazon S3 (Simple Storage Service) is a scalable object storage service used widely for various data storage needs. This guide will walk you through the process of uploading a file to an S3 bucket, from creating the bucket and setting up the necessary permissions, to writing and testing the code that performs the upload. Let's get started!
Create an S3 Bucket
First we need to create a bucket in S3.
Create a Policy
Next, we need to create a policy that grants access to this bucket.
- Navigate to the IAM section of AWS.
- Go to the policy section in the left sidebar.
- Create a new policy.
- Set the resource to S3 and check "All S3 actions" in the "actions allowed" section.
- In the Resources section, input the ARN (Amazon Resource Name) of your S3 bucket opposite the bucket label. You can find the ARN in the properties section of the bucket.
- Scroll down and select "Any" for the object label, which grants permission on any object inside the bucket.
- Provide a name and description for your policy.
Create an IAM User
After creating the policy, we need to create an IAM user and assign this policy to the user.
- Go to the user section in the IAM Dashboard.
- Enter a name for the IAM user.
- Skip the AWS Management console access since we only need programmatic access for this user. Attach the policy created earlier to this user.
- Review the details and create the user.
- Create access keys for the user so that our API can access the bucket on behalf of the created user. Go to the "Security Credentials" section.
- Scroll down to "Access Keys" and create a new access key.
- Optionally, add a description to the access key. Make sure to save the secret access key as it will not be retrievable later.
Create a web server
To handle file uploads, we will create a web server using the Go programming language and the go-fiber package.
- Download the required packages:
go get github.com/gofiber/fiber/v2
go get github.com/aws/aws-sdk-go/aws github.com/aws/aws-sdk-go/aws/credentials github.com/aws/aws-sdk-go/aws/session github.com/aws/aws-sdk-go/service/s3/s3manager
- Create a web server listening on port 3000:
- Initialize the AWS SDK:
import (
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/s3/s3manager"
)
var Uploader *s3manager.Uploader
func NewAWS() {
var region string = "" // AWS Region
var accessKey string = "" // access key
var secretKey string = "" // secret access key
awsSession, err := session.NewSessionWithOptions(
session.Options{
Config: aws.Config{
Region: aws.String(region),
Credentials: credentials.NewStaticCredentials(
accessKey,
secretKey,
"",
),
},
})
if err != nil {
panic(err)
}
Uploader = s3manager.NewUploader(awsSession)
}
Initialize this function in main.go:
package main
import (
"github.com/Xebec19/psychic-enigma/internal"
"github.com/gofiber/fiber/v2"
)
func main() {
internal.NewAWS() // initialize AWS SDK
app := fiber.New()
app.Get("/", func(c *fiber.Ctx) error {
return c.SendString("Hello, World!")
})
app.Listen(":3000")
}
- Create a function to upload a file to S3:
type Result struct {
Value string
Err error
}
func UploadImage(file *multipart.FileHeader) <-chan Result {
ch := make(chan Result)
go func() {
defer close(ch)
src, err := file.Open()
if err != nil {
return
}
defer src.Close()
var bucketName string = "" // bucket name
_, err = Uploader.Upload(&s3manager.UploadInput{
Bucket: aws.String(bucketName),
Key: aws.String(file.Filename),
Body: src, // add file body here
})
if err != nil {
ch <- Result{Value: "", Err: err}
return
}
url := fmt.Sprintf("https://%s.s3.amazonaws.com/%s", bucketName, file.Filename)
ch <- Result{Value: url, Err: nil}
}()
return ch
}
- Create an API endpoint to handle file uploads:
app.Post("/upload", func(c *fiber.Ctx) error {
form, err := c.MultipartForm()
if err != nil {
return err
}
ch := internal.UploadImage(form.File["image"][0])
response := <-ch
if response.Err != nil {
return response.Err
}
c.SendString(response.Value)
return nil
})
Testing the Code
- Use Postman to test the file upload endpoint:
- Verify the file has been uploaded to the S3 bucket:
Congratulations! You have successfully uploaded a file to S3. You can find the source code https://github.com/Xebec19/psychic-enigma.
Conclusion
Uploading files to S3 involves creating a bucket, setting up the appropriate IAM policies and user, and writing code to handle the file upload. By following this guide, you should now be able to upload files to S3 using Go and the AWS SDK. Happy coding!
