Skip to main content

One post tagged with "s3"

View All Tags

How to Upload a File to S3: A Step-by-Step Guide

· 4 min read

Introduction

Amazon S3 (Simple Storage Service) is a scalable object storage service used widely for various data storage needs. This guide will walk you through the process of uploading a file to an S3 bucket, from creating the bucket and setting up the necessary permissions, to writing and testing the code that performs the upload. Let's get started!

Create an S3 Bucket

First we need to create a bucket in S3.

create a bucket

Create a Policy

Next, we need to create a policy that grants access to this bucket.

  1. Navigate to the IAM section of AWS.

iam dashboard

  1. Go to the policy section in the left sidebar.

policy dashboard

  1. Create a new policy.

create policy

  1. Set the resource to S3 and check "All S3 actions" in the "actions allowed" section.

set resource and actions

  1. In the Resources section, input the ARN (Amazon Resource Name) of your S3 bucket opposite the bucket label. You can find the ARN in the properties section of the bucket.

arn

resource section

add bucket arn

added bucket arn

  1. Scroll down and select "Any" for the object label, which grants permission on any object inside the bucket.

set object label to any

  1. Provide a name and description for your policy.

add details in policy

Create an IAM User

After creating the policy, we need to create an IAM user and assign this policy to the user.

  1. Go to the user section in the IAM Dashboard.

create iam user

  1. Enter a name for the IAM user.

enter name for iam user

  1. Skip the AWS Management console access since we only need programmatic access for this user. Attach the policy created earlier to this user.

attach policy

attach policy

  1. Review the details and create the user.

review iam user

  1. Create access keys for the user so that our API can access the bucket on behalf of the created user. Go to the "Security Credentials" section.

move to security credentials section

  1. Scroll down to "Access Keys" and create a new access key.

scroll down to access key

create access key

  1. Optionally, add a description to the access key. Make sure to save the secret access key as it will not be retrievable later.

add description to access key

access key creds

Create a web server

To handle file uploads, we will create a web server using the Go programming language and the go-fiber package.

  1. Download the required packages:
go get github.com/gofiber/fiber/v2
go get github.com/aws/aws-sdk-go/aws github.com/aws/aws-sdk-go/aws/credentials github.com/aws/aws-sdk-go/aws/session github.com/aws/aws-sdk-go/service/s3/s3manager

download packages

  1. Create a web server listening on port 3000:

create web server

  1. Initialize the AWS SDK:

import (
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/s3/s3manager"
)

var Uploader *s3manager.Uploader

func NewAWS() {

var region string = "" // AWS Region
var accessKey string = "" // access key
var secretKey string = "" // secret access key

awsSession, err := session.NewSessionWithOptions(
session.Options{
Config: aws.Config{
Region: aws.String(region),
Credentials: credentials.NewStaticCredentials(
accessKey,
secretKey,
"",
),
},
})

if err != nil {
panic(err)
}

Uploader = s3manager.NewUploader(awsSession)
}

Initialize this function in main.go:

package main

import (
"github.com/Xebec19/psychic-enigma/internal"
"github.com/gofiber/fiber/v2"
)

func main() {

internal.NewAWS() // initialize AWS SDK

app := fiber.New()

app.Get("/", func(c *fiber.Ctx) error {
return c.SendString("Hello, World!")
})

app.Listen(":3000")
}

  1. Create a function to upload a file to S3:

type Result struct {
Value string
Err error
}

func UploadImage(file *multipart.FileHeader) <-chan Result {
ch := make(chan Result)

go func() {
defer close(ch)
src, err := file.Open()
if err != nil {
return
}
defer src.Close()

var bucketName string = "" // bucket name

_, err = Uploader.Upload(&s3manager.UploadInput{
Bucket: aws.String(bucketName),
Key: aws.String(file.Filename),
Body: src, // add file body here
})
if err != nil {
ch <- Result{Value: "", Err: err}
return
}

url := fmt.Sprintf("https://%s.s3.amazonaws.com/%s", bucketName, file.Filename)

ch <- Result{Value: url, Err: nil}
}()

return ch
}

  1. Create an API endpoint to handle file uploads:
app.Post("/upload", func(c *fiber.Ctx) error {

form, err := c.MultipartForm()

if err != nil {
return err
}

ch := internal.UploadImage(form.File["image"][0])

response := <-ch
if response.Err != nil {
return response.Err
}

c.SendString(response.Value)
return nil
})

Testing the Code

  1. Use Postman to test the file upload endpoint:

postman request

  1. Verify the file has been uploaded to the S3 bucket:

s3 files

Congratulations! You have successfully uploaded a file to S3. You can find the source code https://github.com/Xebec19/psychic-enigma.

Conclusion

Uploading files to S3 involves creating a bucket, setting up the appropriate IAM policies and user, and writing code to handle the file upload. By following this guide, you should now be able to upload files to S3 using Go and the AWS SDK. Happy coding!