DevOps.Academy
CoursesCurriculumPricing

    Loading…

    ↑ ↓ move · ↵ open · Ctrl K anywhere

    ◔My learningSign in
    DevOps.Academy

    Free, hands-on DevOps lessons. Made with care in India.

    Learn

    All coursesLinux roadmapCurriculumMy learning

    Academy

    PricingCertificates

    Legal

    PrivacyTermsContact

    devops.rajeev.pro

    AWS for DevOpsLesson 5 of 17
    Your progress · 0%
    On this page
    1. From one server to a fleet
    2. One server, described once
    3. The Auto Scaling group
    4. The load balancer
    5. Kill a server
    6. Scaling on load
    7. ALB or NLB
    8. Clean up
    9. Listener rules choose the target
    10. Sticky sessions and safe refreshes
    ↖ Course roadmap
    Lesson 5 · Scaling & load balancing

    Scaling and load balancing

    Put two servers in two AZs behind an Application Load Balancer, let an Auto Scaling group keep them alive and scale on CPU, and watch it replace a server you kill.

    40 min read · intermediate · hands-on

    By the end of this lesson you'll be able to

    0 of 6 complete

    From one server to a fleet

    The server from lesson 3 had two problems: if it died, the site died, and if traffic tripled, it couldn't grow. This lesson builds the standard fix, in the two-AZ VPC from lesson 4, in a real account (all of it deleted afterwards):

    internet → Application Load Balancer (public subnets, 2 AZs)
                 → listener HTTP:80 → target group (health check GET /)
                    → Auto Scaling group: 2–4 × t3.micro from a launch template, spread across AZs
    

    Two security groups lock it down:

    GroupInbound
    academy-alb-sgport 80 from 0.0.0.0/0
    academy-web-sgport 80 only from academy-alb-sg
    $ aws ec2 authorize-security-group-ingress --group-id $WEBSG --protocol tcp --port 80 --source-group $ALBSG
    
    aws ec2 authorize-security-group-ingress --group-id $WEBSG --protocol tcp --port 80 --source-group $ALBSG
    • awsthe AWS command line

    A security group can use another security group as its source, so nobody can skip the load balancer and hit a server directly.

    One server, described once

    A launch template holds everything from lesson 3's wizard (AMI, instance type, security group, user data, IMDSv2) so a machine can start servers without you:

    (1) The template's name. (2) Its instance type. No key pair: these servers are never logged into by hand.© Screenshot: DevOps Academy

    (1) The template's name. (2) Its instance type. No key pair: these servers are never logged into by hand.© Screenshot: DevOps Academy

    The user data this time also asks the metadata service which AZ the server is in, so you can see the load balancer spreading requests:

    #!/bin/bash
    dnf install -y nginx
    TOKEN=$(curl -s -X PUT http://169.254.169.254/latest/api/token -H "X-aws-ec2-metadata-token-ttl-seconds: 60")
    AZ=$(curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/placement/availability-zone)
    echo "<h1>Hello from $(hostname) in $AZ</h1>" > /usr/share/nginx/html/index.html
    systemctl enable --now nginx
    

    Templates are versioned: change the AMI, create version 2, and roll it out (an instance refresh replaces the servers a few at a time).

    The Auto Scaling group

    (1) Desired capacity 2. (2) Scaling limits: never fewer than 2, never more than 4.© Screenshot: DevOps Academy

    (1) Desired capacity 2. (2) Scaling limits: never fewer than 2, never more than 4.© Screenshot: DevOps Academy
    SettingMeaning
    Minthe floor: the group never goes below it, even if every server dies
    Desiredhow many it runs right now; scaling policies change this number
    Maxthe ceiling, which also caps your bill

    Give the group subnets in two or more AZs and it spreads instances evenly, so losing a whole AZ leaves half the fleet running:

    Two instances, two AZs, both InService and Healthy.© Screenshot: DevOps Academy

    Two instances, two AZs, both InService and Healthy.© Screenshot: DevOps Academy

    The load balancer

    An Application Load Balancer (ALB) takes requests on a listener (here HTTP:80), checks its rules and forwards to a target group. The target group health-checks every server (GET / every 10 seconds) and only sends traffic to the healthy ones:

    (1) The DNS name clients use; an ALB has no fixed IP. (2) The HTTP:80 listener.© Screenshot: DevOps Academy

    (1) The DNS name clients use; an ALB has no fixed IP. (2) The HTTP:80 listener.© Screenshot: DevOps Academy

    (1) Listener → rule → (2) target group → (3) targets: the whole request path, and the health of every hop.© Screenshot: DevOps Academy

    (1) Listener → rule → (2) target group → (3) targets: the whole request path, and the health of every hop.© Screenshot: DevOps Academy

    Both targets healthy. The Auto Scaling group registers new instances here by itself.© Screenshot: DevOps Academy

    Both targets healthy. The Auto Scaling group registers new instances here by itself.© Screenshot: DevOps Academy

    Requests take turns between the AZs:

    $ for i in 1 2 3 4; do curl -s http://academy-alb-2056515812.ap-south-1.elb.amazonaws.com; done
    <h1>Hello from ip-10-0-23-40.ap-south-1.compute.internal in ap-south-1b</h1>
    <h1>Hello from ip-10-0-23-40.ap-south-1.compute.internal in ap-south-1b</h1>
    <h1>Hello from ip-10-0-10-240.ap-south-1.compute.internal in ap-south-1a</h1>
    <h1>Hello from ip-10-0-10-240.ap-south-1.compute.internal in ap-south-1a</h1>
    
    for i in 1 2 3 4; do curl -s http://academy-alb-2056515812.ap-south-1.elb.amazonaws.com; done
    • forloop over a list
    • dostart of what the loop runs each time
    • curlmake an HTTP request
    • -ssilent: no progress bar
    • doneend of the loop

    Kill a server

    Terminate one instance by hand, as if its hardware failed:

    $ aws ec2 terminate-instances --instance-ids i-0dca3be571b8075be
    
    aws ec2 terminate-instances --instance-ids i-0dca3be571b8075be
    • awsthe AWS command line

    (1) The dead instance is taken out of service and drained. (2) A replacement is launched the same second, with no one paged.© Screenshot: DevOps Academy

    (1) The dead instance is taken out of service and drained. (2) A replacement is launched the same second, with no one paged.© Screenshot: DevOps Academy

    The load balancer stops sending traffic to the dying server (connection draining lets in-flight requests finish), the group launches a replacement from the template, and once its health check passes, traffic returns to two servers. The curl output above already includes the new server, ip-10-0-10-240.

    Scaling on load

    A target tracking policy: keep average CPU near 50% by changing desired capacity between min and max.© Screenshot: DevOps Academy

    A target tracking policy: keep average CPU near 50% by changing desired capacity between min and max.© Screenshot: DevOps Academy
    PolicyHow it worksUse for
    Target trackingkeep a metric near a target, like a thermostatmost apps: CPU, requests per target
    Step scalingadd or remove N instances at alarm thresholdswhen you need precise steps
    Scheduledchange capacity at set timesknown patterns: office hours, a sale at 9 a.m.

    Scaling takes minutes (boot, user data, health checks), so it handles growth, not a sudden spike. Keep a little headroom in min.

    ALB or NLB

    Application Load BalancerNetwork Load Balancer
    Layer7: HTTP and HTTPS4: TCP, UDP, TLS
    Routingby path, host, headerby port
    IPDNS name onlystatic IPs per AZ
    Use forweb apps and APIsvery high throughput, non-HTTP protocols, fixed IPs

    For HTTPS, request a free public certificate from AWS Certificate Manager (ACM) for your domain, add an HTTPS:443 listener with it, and redirect HTTP:80 to HTTPS. ACM renews the certificate automatically.

    Clean up

    Listener rules choose the target

    An ALB listener evaluates rules from the lowest priority number upward. A path condition can route /images/* to one target group and a host condition can route api.example.com to another. The default action is the fallback when neither condition matches.

    $ aws elbv2 create-rule --listener-arn "$LISTENER" --priority 10 \
        --conditions Field=path-pattern,Values='/images/*' \
        --actions Type=forward,TargetGroupArn="$IMAGES_TG"
    $ aws elbv2 describe-rules --listener-arn "$LISTENER" \
        --query 'Rules[].{Priority:Priority,Conditions:Conditions}' --output table
    
    aws elbv2 create-rule --listener-arn "$LISTENER" --priority 10 \
    • awsthe AWS command line

    Sticky sessions and safe refreshes

    Stickiness keeps a browser on one target, which can help a legacy in-memory session but causes uneven load. Prefer an external session store and stateless targets. Instance refresh replaces an Auto Scaling group gradually after a launch-template change; lifecycle hooks pause launch or termination so work can drain, and warm pools keep pre-initialised instances ready for bursts.

    $ aws elbv2 modify-target-group-attributes --target-group-arn "$TG" \
        --attributes Key=stickiness.enabled,Value=true Key=stickiness.lb_cookie.duration_seconds,Value=300
    $ aws autoscaling start-instance-refresh --auto-scaling-group-name "$ASG" \
        --preferences MinHealthyPercentage=90,InstanceWarmup=120
    
    aws elbv2 modify-target-group-attributes --target-group-arn "$TG" \
    • awsthe AWS command line

    The load balancer is billed per hour even with no traffic, so delete everything, in order:

    $ aws autoscaling delete-auto-scaling-group --auto-scaling-group-name academy-asg --force-delete
    $ aws elbv2 delete-load-balancer --load-balancer-arn $ALB && aws elbv2 wait load-balancers-deleted --load-balancer-arns $ALB
    $ aws elbv2 delete-target-group --target-group-arn $TG
    $ aws ec2 delete-launch-template --launch-template-name academy-web-lt
    $ aws ec2 delete-security-group --group-id $WEBSG && aws ec2 delete-security-group --group-id $ALBSG
    
    aws autoscaling delete-auto-scaling-group --auto-scaling-group-name academy-asg --force-delete
    • awsthe AWS command line
    aws elbv2 delete-load-balancer --load-balancer-arn $ALB && aws elbv2 wait load-balancers-deleted --load-balancer-arns $ALB
    • &&run the next command only if this one worked

    --force-delete terminates the group's instances too. The web security group can only be deleted once those instances are gone.

    In production

    Servers go in private subnets with a NAT gateway or endpoints, only the ALB is public, HTTPS uses an ACM certificate, and the whole stack is in Terraform (lesson 16). Use instance refresh to roll out new AMIs, CloudWatch alarms on unhealthy targets and 5xx errors (lesson 13), and at least two AZs, or three for anything critical.

    ⌘
    Mini mission

    A self-healing web tier

    Behind an Application Load Balancer, run an Auto Scaling group of 2 t3.micro instances across two AZs, reachable only through the load balancer. Show requests answered from both AZs, kill one instance, watch it be replaced, then delete everything.

    Input: An AWS account, the AWS CLI, and a VPC with two public subnets (the lesson 4 VPC, or the default VPC)Output: curl answers from two AZs, an Activity entry replacing the instance you killed, and no load balancer or group left
    Reveal one safe solution
    $ V=vpc-…; SUBS="subnet-… subnet-…"                       # two public subnets in different AZs
    $ ALBSG=$(aws ec2 create-security-group --vpc-id $V --group-name academy-alb-sg --description "HTTP to the ALB" --query GroupId --output text)
    $ aws ec2 authorize-security-group-ingress --group-id $ALBSG --protocol tcp --port 80 --cidr 0.0.0.0/0
    $ WEBSG=$(aws ec2 create-security-group --vpc-id $V --group-name academy-web-sg --description "HTTP from the ALB only" --query GroupId --output text)
    $ aws ec2 authorize-security-group-ingress --group-id $WEBSG --protocol tcp --port 80 --source-group $ALBSG
    $ AMI=$(aws ssm get-parameter --name /aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64 --query Parameter.Value --output text)
    $ aws ec2 create-launch-template --launch-template-name academy-web-lt --launch-template-data \
        "{\"ImageId\":\"$AMI\",\"InstanceType\":\"t3.micro\",\"SecurityGroupIds\":[\"$WEBSG\"],\"UserData\":\"$(base64 -w0 user-data.sh)\",\"MetadataOptions\":{\"HttpTokens\":\"required\"}}"
    $ TG=$(aws elbv2 create-target-group --name academy-web-tg --protocol HTTP --port 80 --vpc-id $V --health-check-path / --query 'TargetGroups[0].TargetGroupArn' --output text)
    $ ALB=$(aws elbv2 create-load-balancer --name academy-alb --type application --subnets $SUBS --security-groups $ALBSG --query 'LoadBalancers[0].LoadBalancerArn' --output text)
    $ aws elbv2 create-listener --load-balancer-arn $ALB --protocol HTTP --port 80 --default-actions Type=forward,TargetGroupArn=$TG
    $ aws autoscaling create-auto-scaling-group --auto-scaling-group-name academy-asg --launch-template LaunchTemplateName=academy-web-lt,Version='$Latest' \
        --min-size 2 --max-size 4 --desired-capacity 2 --vpc-zone-identifier "$(echo $SUBS | tr ' ' ,)" \
        --target-group-arns $TG --health-check-type ELB --health-check-grace-period 60
    $ DNS=$(aws elbv2 describe-load-balancers --load-balancer-arns $ALB --query 'LoadBalancers[0].DNSName' --output text)
    $ for i in 1 2 3 4; do curl -s http://$DNS; done            # after a few minutes: both AZs answer
    $ aws ec2 terminate-instances --instance-ids <one of the group's instances>
    $ aws autoscaling describe-scaling-activities --auto-scaling-group-name academy-asg --max-items 2 --query 'Activities[].Description'
    
    aws ec2 authorize-security-group-ingress --group-id $ALBSG --protocol tcp --port 80 --cidr 0.0.0.0/0
    • awsthe AWS command line
    for i in 1 2 3 4; do curl -s http://$DNS; done
    • forloop over a list
    • dostart of what the loop runs each time
    • curlmake an HTTP request
    • -ssilent: no progress bar
    • doneend of the loop
    aws autoscaling describe-scaling-activities --auto-scaling-group-name academy-asg --max-items 2 --query 'Activities[].Description'
    • --querypick fields from the JSON response

    user-data.sh is the script from this lesson; the subnets must give instances public IPs so they can download nginx (if not, turn it on: aws ec2 modify-subnet-attribute --subnet-id … --map-public-ip-on-launch). Clean up with the commands above. An ALB plus two t3.micro for half an hour costs a few cents.

    Try it yourself

    0 of 6 steps done

    Run each task in your own account or terminal, then tick it off.

    Knowledge check

    Quick check

    Question 1 of 5

    What does an Auto Scaling group do when one of its instances fails its health check?

    Go deeper

    Official reading for this lesson

    • DocsAmazon EC2 Auto Scaling user guideAWS Docs ↗
    • DocsTarget tracking scaling policiesAWS Docs ↗
    • DocsApplication Load BalancersAWS Docs ↗
    • DocsNetwork Load BalancersAWS Docs ↗
    • DocsAWS Certificate ManagerAWS Docs ↗

    Or get every quiz answer right and it completes itself.

    ← PreviousVPC: your own private networkNext lesson →S3: storage for everything