S3 Programmatic access with AWS-CLI

What is AWS S3?

AWS S3 (Amazon Simple Storage Service) is a cloud-based storage service provided by Amazon Web Services (AWS). It allows individuals and businesses to store and retrieve large amounts of data, such as files, images, videos, backups, and more, in a highly scalable and durable manner. S3 provides a simple and intuitive interface for managing data, and it offers features like data encryption, versioning, access control, and the ability to host static websites. It's widely used for various purposes, including data backup, content distribution, application data storage, and more.

What is AWS CLI?

AWS CLI (Amazon Web Services Command Line Interface) is a tool that allows users to interact with various AWS services through the command line of a terminal or command prompt. It provides a convenient way to manage and automate AWS resources and services without needing to use the AWS Management Console web interface. With AWS CLI, users can perform tasks such as creating and managing EC2 instances, S3 buckets, IAM users, and more, by issuing commands directly in the terminal. It's particularly useful for scripting, automation, and managing AWS resources in a programmatic and efficient manner.

Task-01

Launch an EC2 instance using the AWS Management Console and connect to it using Secure Shell (SSH).

Step-01: Launch an EC2 instance using the AWS Management Console and select OS as Ubuntu.

• Step-02: Select instance type ,key pair and security group.

Step-03: Launch the instance via SSH so copy the command and paste it in your terminal.

Step-04: Now you can see your instance is running.

Before creating an S3 bucket we need to install AWS CLI in our EC2 instance.

Step-01: Install AWS CLI using the following command

sudo apt update
sudo apt install unzip
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install

• Step-02: Verify the installation using the following command

aws --version

• Step-03: Configure the AWS CLI using the following command in which we will provide the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY which we have created in the previous task and also provide the region we want to work and the output format.

aws configure

• Step-04: Verify the configuration using the following command which will show the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY which we have provided in the previous step.

aws configure list

Create an S3 bucket and upload a file to it using the AWS Management Console.

Step-01: Go to S3 service and click on Create bucket.

• Step-02: Give a name to your bucket as 'mizan-s3-bucket'.

Step-03: Now click on Create bucket.

Step-04: Now click on Upload and upload a file.

Step-05: Now click on Upload.

---

Access the file from the EC2 instance using the AWS Command Line Interface (AWS CLI).

aws s3 ls - This command lists all of the S3 buckets in your AWS account.

aws s3 cp <filename> s3://<bucketname>

• Thus we have uploaded our file to our S3 bucket.

• Check the file in the S3 bucket.

• Now we will see how to download a file from S3 bucket to our EC2 instance.

• aws s3 cp s3://bucket-name/file.txt . - This command downloads a file from an S3 bucket to your local file system.

aws s3 cp s3://mizan-s3-bucket/mizan.txt /home/ubuntu/storage

Task-02

Create a snapshot of the EC2 instance and use it to launch a new EC2 instance.

Step-01: Go to EC2 service and click on snapshot.

• Step-02: In Snapshot click on resource type and select instance with appropriate instance id.

• Step-03: After creating the snapshot,we will create image from snapshot.

• Step-04: Now we are creating image from snapshot.

• Step-05: Now we will select the snapshot from the Amazon Machine Image(AMI).

• Select (Owned by me) and click on Launch instance from AMI.

• Launch snapshot by AMI instance.

• Step-06: Now we will launch the instance which is a SnapShot instance of the first instance.

• Step-07: Now we will check the key pair login and Network Setting of the new instance and create it.

• Step-08: After creating the instance we will ssh into the instance.

Please login as the user "ubuntu" rather than the user "root" - This error occurs because we are trying to login as root user.

• So follow this command to login as ubuntu user.

ssh -i "s3bucket.pem" ubuntu@ec2-13-234-204-102.ap-south-1.compute.amazonaws.com

Now we will check the content of Instance-one (MyS3Bucket) is snapped into instance-two (rohit-s3bucket-snapshot-server)

• Verify that the contents of the file are the same on both EC2 instances.

Here are some commonly used AWS CLI commands for Amazon S3:

• aws s3 ls - This command lists all of the S3 buckets in your AWS account.

• aws s3 mb s3://bucket-name - This command creates a new S3 bucket with the specified name.

• aws s3 rb s3://bucket-name - This command deletes the specified S3 bucket.

• aws s3 cp file.txt s3://bucket-name - This command uploads a file to an S3 bucket.

• aws s3 cp s3://bucket-name/file.txt . - This command downloads a file from an S3 bucket to your local file system.

• aws s3 sync local-folder s3://bucket-name - This command syncs the contents of a local folder with an S3 bucket.

• aws s3 ls s3://bucket-name - This command lists the objects in an S3 bucket.

• aws s3 rm s3://bucket-name/file.txt - This command deletes an object from an S3 bucket.

• aws s3 presign s3://bucket-name/file.txt - This command generates a pre-signed URL for an S3 object, which can be used to grant temporary access to the object.

• aws s3api list-buckets - This command retrieves a list of all S3 buckets in your AWS account, using the S3 API.