Currently all of our PHP (Laravel) applications are running on one EC2 instance. To deploy code, we SSH onto the live box and pull from CodeCommit.
The first thing you need to decide on is a Continuous Deploy strategy. There's a lot of servers out there with different use cases, you don't have to pick the most popular, just pick the one that fits your use case.
I guess we want to end up with a load balancer and application servers which are created on demand.
AWS has some great reference architectures but overall I would say start simple. At a base level you'll be using Route 53, Certificate Manager, ALB (ELBv2), VPC, and EC2. I would get used to the options you have with each of these and play with them. AWS CSA will not teach you about reference architecture, rather it will go over what each product is used for.
Before you go building anything for production I would also recommend using an infrastructure deployment tool. AWS provides CloudFormation out of the box but that can be difficult to interface with. Some others are Sceptre and Terraform. These allow you to store your infrastructure as code in a repository and deploy updates via a CD server.
Also keep in mind if you don't have Continuous Integration then you'll need to work on that, followed by Continuous Delivery, then by Continuous Deployment. Each of these is a critical step in the right direction.
Note on "Created on Demand"
Please keep in mind that many people come to the cloud thinking that "scaling" is instantaneous. It's not. Depending on your provisioning it can take some time. You will not scale yourself out of burst traffic. Your traffic will need to be profiled, you will need to determine your instance capacity (for web servers I measure instances in requests per second with a load testing tool), and plan accordingly.
Scaling is primarily designed for a gradual rise in traffic over time, such as if your website traffic grows gradually larger around noon (peak) and then settles late in the day.