~ 4 min read
How to transfer files between AWS S3 and AWS EC2
EC2 and S3 are regarded as some of the most extensively used services that AWS offers. At times, situations arise where we need to transfer files between the EC2 instance and the S3 bucket.
This article will be a quick primer on how to configure an EC2 instance to communicate with an S3 bucket to transfer files between the two resources. In this article we will use an Instance Profile role for EC2 instance to grant permissions to access our S3 bucket.
Transfer files between AWS S3 and AWS EC2
- An EC2 instance should exist where we would be performing actions on the S3 bucket (Documentation)
- Ensure that AWS CLI is installed on the EC2 instance (Documentation)
- An S3 bucket with some files to use for the transfer (Documentation)
Adding Instance Profile to grant permissions to EC2 instances
Perform the following steps to add an instance profile to the EC2 instance to grant required permissions to the instance:
Log in to the AWS console
Navigate to the EC2 details page
Select the instance we want to add the instance profile (IAM role) to, click on the Actions drop-down menu, and click on the “Modify IAM role” option
Select an existing IAM role from the drop-down menu (or create a new one if needed) and click on the “Save” button
Adding required S3 permissions to the IAM Instance Profile role
Perform the following steps to add required permission to the IAM instance profile role to allow the instance to perform actions on the S3 bucket
Navigate to the IAM details page and click on the roles menu and select the role that was attached to the EC2 instance
Click on the “Add permissions” button and select the “Attach policies” option
Filter and attach the “AmazonS3ReadOnlyAccess” policy from the list and click on the “Attach policies” button
Copying files from the S3 bucket to the EC2 instance
Perform the following steps to copy a file from an S3 bucket to the EC2 instance:
SSH into the EC2 instance
aws sts get-caller-identityto confirm that the EC2 instance has the correct role attached and AWS CLI is working properly
aws s3 <S3_Object_URI> <Local_File_Path>to copy files from S3 bucket to the EC2 instance
We have added permission to the instance profile,
AmazonS3ReadOnlyAccess, which only allows the instance to read files from an S3 bucket and copy them to the instance. The instance can also write objects to S3 buckets, as well as perform other S3 management tasks such as creating new buckets, deleting buckets, etc. by editing the permission attached to the IAM Instance Profile Role.
Additionally, we could have used IAM user credentials with AWS CLI from the instance to perform the same operations on the S3 bucket. While that is another potential method when working with AWS EC2 instances, using IAM Instance Profile Roles is the best practice to grant permissions to the EC2 instance to be able to perform actions on AWS resources, not just limited to S3 buckets.
In this article we saw how we can configure an EC2 instance to use an IAM Instance Profile role, granting it permissions to fetch objects from an S3 bucket to the instance. We looked at how to add the appropriate policy to the instance profile role for the EC2 instance to be able to perform actions on the S3 bucket.
To reiterate, using instance profiles is the best practice method to grant permissions to the EC2 instance instead of using IAM user CLI credentials. Lastly, the actions that are permitted to the EC2 instance via the profile role can be extended by adding additional permissions by either adding existing or custom-managed IAM policies or adding an inline policy to the role.