May 24, 2022

How to transfer files between AWS S3 and AWS EC2

A simple and quick walkthrough of how you can transfer files between AWS EC2 and AWS S3.

Table of Contents

Introduction

Transfer files between AWS S3 and AWS EC2

     Prerequisites

     Adding Instance Profile to grant permissions to EC2 instances

     Adding required S3 permissions to the IAM Instance Profile role     

     Copying files from the S3 bucket to the EC2 instance

          Notes

Conclusion

Introduction

EC2 and S3 are regarded as some of the most extensively used services that AWS offers. At times, situations arise where we need to transfer files between the EC2 instance and the S3 bucket.

This article will be a quick primer on how to configure an EC2 instance to communicate with an S3 bucket to transfer files between the two resources. In this article we will use an Instance Profile role for EC2 instance to grant permissions to access our S3 bucket.

Transfer files between AWS S3 and AWS EC2

Prerequisites

  1. An EC2 instance should exist where we would be performing actions on the S3 bucket (Documentation)
  2. Ensure that AWS CLI is installed on the EC2 instance (Documentation)
  3. An S3 bucket with some files to use for the transfer (Documentation)

Adding Instance Profile to grant permissions to EC2 instances

Perform the following steps to add an instance profile to the EC2 instance to grant required permissions to the instance:

  1. Log in to the AWS console
  2. Navigate to the EC2 details page
  3. Select the instance we want to add the instance profile (IAM role) to, click on the Actions drop-down menu, and click on the “Modify IAM role” option
Modify IAM role
  1. Select an existing IAM role from the drop-down menu (or create a new one if needed) and click on the “Save” button
Save

Adding required S3 permissions to the IAM Instance Profile role

Perform the following steps to add required permission to the IAM instance profile role to allow the instance to perform actions on the S3 bucket

  1. Navigate to the IAM details page and click on the roles menu and select the role that was attached to the EC2 instance

Navigate to the IAM details page and click on the roles menu and select the role that was attached to the EC2 instance

  1. Click on the “Add permissions” button and select the “Attach policies” option

Add permissions

  1. Filter and attach the “AmazonS3ReadOnlyAccess” policy from the list and click on the “Attach policies” button

Copying files from the S3 bucket to the EC2 instance

Perform the following steps to copy a file from an S3 bucket to the EC2 instance:

  1. SSH into the EC2 instance
  2. Run `aws sts get-caller-identity` to confirm that the EC2 instance has the correct role attached and AWS CLI is working properly
confirm that the EC2 instance has the correct role attached and AWS CLI is working properly
  1. Run `aws s3 cp <S3_Object_URI> <Local_File_Path>` to copy files from S3 bucket to the EC2 instance
run the command

Notes

We have added permission to the instance profile, `AmazonS3ReadOnlyAccess`, which only allows the instance to read files from an S3 bucket and copy them to the instance. The instance can also write objects to S3 buckets, as well as perform other S3 management tasks such as creating new buckets, deleting buckets, etc. by editing the permission attached to the IAM Instance Profile Role.

Additionally, we could have used IAM user credentials with AWS CLI from the instance to perform the same operations on the S3 bucket. While that is another potential method when working with AWS EC2 instances, using IAM Instance Profile Roles is the best practice to grant permissions to the EC2 instance to be able to perform actions on AWS resources, not just limited to S3 buckets.

Conclusion

In this article we saw how we can configure an EC2 instance to use an IAM Instance Profile role, granting it permissions to fetch objects from an S3 bucket to the instance. We looked at how to add the appropriate policy to the instance profile role for the EC2 instance to be able to perform actions on the S3 bucket.

To reiterate, using instance profiles is the best practice method to grant permissions to the EC2 instance instead of using IAM user CLI credentials. Lastly, the actions that are permitted to the EC2 instance via the profile role can be extended by adding additional permissions by either adding existing or custom-managed IAM policies or adding an inline policy to the role.

***

This article is brought to you by Kloudle Academy, a free e-resource compilation, created and curated by Kloudle. Kloudle is a cloud security management platform that uses the power of automation and simplifies human requirements in cloud security. Receive alerts for Academy by subscribing here.

Written by:
Ayush Priya

Ayush Priya

Cloud Security Specialist

Ayush is a Cyber Security Engineer specialising in Cloud and Data Security and DevSecOps practices. He loves to develop automation for security controls and process. He has delivered talks at various conferences and security communities like GrayHat'20, Cyber Security Global Summit to name a few. Ayush also contributes to the OWASP Ranchi chapter as the Chapter Lead.

Read more