Services used: IAM, VPC, S3, AWS Batch, Fargate, ECR, ECS, Step Functions, CloudWatch
Languages used: Python, Terraform (HCL), JSON
Objective: Orchestrate a scalable, fault-tolerant ETL pipeline that ingests Common Crawl WET files into S3, extracts/filters high-quality text, deduplicates, applies toxicity screening, normalizes and tokenizes, and writes AI-ready shards back to S3. Containerize stages with Docker and publish images to ECR. Automate orchestration using Step Functions (Map) to fan-out per-file jobs and submit to AWS Batch running on ECS Fargate in private subnets. Provide egress via NAT for ECR pulls, use an S3 gateway endpoint for data IO, stream logs to CloudWatch, and secure access with least-privilege IAM. The result is an automated, resumable pipeline that prepares large-scale datasets for AI model development.
Services used: Route 53, S3, CloudFront, ACM
Languages used: JSON
Objective: Create a domain name with Route 53 for the portfolio website. Create an S3 bucket to store all of the website data. Enable the bucket to be publicly accessed and allow it to host a static website. Test the static website using the HTTP protocol. Disable the bucket to be publicly accessed and use CloudFront to deliver the content globally instead. Create a certificate with ACM to create secure HTTPS connections to the portfolio website. Create another S3 bucket and redirect users to the subdomain. Create A records in Route 53 to the CloudFront distributions for both IPv4 and IPv6.
Services used: ECR, ECS, Fargate, Cloud9, VPC
Languages used: Python
Objective: Use Amazon ECS, Amazon ECR, and AWS Fargate to deploy a sample application. Create a docker image for an application. Create an ECR repository, and then push the docker image to it. Deploy an application with ECS and Fargate by using an image from ECR. Deploy a second application in Fargate by using the image from ECR. Validate access to the second application.
Services used: Kinesis, Lambda, DynamoDB, S3, Glue, Athena, EC2
Languages used: N/A
Objective: Create an Amazon KDF (Kinesis Data Firehose) delivery stream. Ingest and store clickstream data in an Amazon S3 bucket. Transform ingested data by using AWS Lambda functions. Create real-time queries by using AWS Glue and Amazon Athena. Edit the KDF delivery stream configuration to send real-time analytics to an Amazon DynamoDB table. Change the S3 buffer hints, buffer interval setting to 60 seconds.
Services used: Config, S3, EC2, KMS
Languages used: N/A
Objective: Use AWS Config rules to enforce standards and ensure architectural consistency. Configure an AWS Config managed rule for S3 KMS encryption. Configure an AWS Config managed rule for tagging resources. Add an additional rule to verify that the S3 buckets have versioning enabled.
Services used: DynamoDB, API Gateway, S3, Lambda, CloudWatch
Languages used: JSON
Objective: Deploy a single-page app by using Amazon S3, Amazon API Gateway, and AWS Lambda. Use Amazon CloudWatch to troubleshoot the deployment. Configure S3 to host a single-page application. Locate the API Gateway invoke URL. Troubleshoot resource connections from API Gateway. Troubleshoot a Lambda function by using CloudWatch Logs. Ensure that the Details button in the app returns the relevant details.
Services used: VPC, S3, Network ACLs
Languages used: N/A
Objective: Detect and deny network traffic from a specific IP address to increase security. Set up VPC Flow Logs in an Amazon S3 bucket. Read and analyze the flow logs. Create network ACL rules to block traffic. Reject outbound traffic by using a network ACL.
Services used: CloudFormation, EC2, S3
Languages used: YAML, JSON
Objective: Help reduce human error and standardize deployment infrastructure. Determine how to use AWS CloudFormation to automate environment creation. Demonstrate how to read and write basic CloudFormation templates in YAML and JSON. Demonstrate how to run the automation to create different environments.
Services used: Route 53, EC2, VPC
Languages used: N/A
Objective: Use Amazon Route 53 to map an IP address to a domain name, and create a CNAME record to point one domain to another. Configure local hosted zones to provide domain names internal to the VPC.
Services used: Lambda, CloudWatch
Languages used: Python, JSON
Objective: Use AWS Lambda to run code without provisioning a server. Use Python to create an AWS Lambda function. Deploy and test the Lambda function. Modify the Lambda function code to display different feeling values based on the emoji_type value in the JSON element.
Services used: API Gateway, Lambda
Languages used: Python, JSON
Objective: Use Amazon API Gateway and an AWS Lambda function to create and deploy an API. Create an Amazon API Gateway REST API. Deploy the REST API. Invoke an AWS Lambda function by using API Gateway proxy integration. Use a sample Python file to create a new Lambda function. Create a new API Gateway endpoint and integrate it with the new Lambda function.
Services used: DynamoDB, API Gateway, Lambda
Languages used: Python, JSON
Objective: Add a NoSQL database to the “vehicle rental” application, and integrate the app with Amazon API Gateway and AWS Lambda. Create an Amazon DynamoDB table and define the table schema. Create an AWS Lambda function to create, update, and query table items. Expose an Amazon API Gateway RESTful API to the Lambda function.
Services used: AWS Backup, EC2
Languages used: N/A
Objective: Use AWS Backup to centralize and automate data protection across AWS services. Create a custom backup vault. Configure an on-demand backup of EBS volumes with a resource ID. Create automated backup plans for EC2 attached EBS volumes using tags. Configure an AWS Backup plan for an Amazon Aurora serverless cluster by using the default backup server. Assign the Amazon Aurora serverless cluster as a resource using tags.
Services used: CloudWatch, EC2, SNS
Languages used: JSON
Objective: Build a central monitoring dashboard that immediately and automatically notifies engineers of a system failure. Investigate default monitoring of EC2 instances. Create an Amazon CloudWatch dashboard and add a widget. Create a CloudWatch alarm for CPU utilization. Create a CloudWatch action to send an email notification by using Amazon SNS. Add a CloudWatch dashboard for memory utilization. Create a CloudWatch alarm for memory utilization over 500M. Add an EC2 action to reboot the server if the alarm is invoked.
Services used: CloudFront, EC2, S3
Languages used: N/A
Objective: Decrease latency by deploying a CDN. Set up a CDN using Amazon CloudFront. Configure an EC2 instance as the origin for a CloudFront distribution. Configure an S3 hosted website as the origin for a CloudFront distribution. Reconfigure a CloudFront distribution to use an S3 website as the origin, with index.html as the main page.
Services used: SQS, SNS, EC2
Languages used: JSON
Objective: Improve availability and fault tolerance by decoupling microservices with messages sent to Amazon SQS queues and an Amazon SNS topic. Create a SNS topic and a SQS queue. Subscribe the SQS queue to the SNS topic. Create an additional SQS queue and subscribe it to the existing SNS topic.