I want to deploy a public facing web project on aws for which I have setup ec2 for front end and another ec2 for backend (leave discussion about hpa and other services etc for now). I have to integrate jenkin ci/cd pipeline with this. The setup have 3 environment (staging, production and development with 3 individual databases). My query is how to setup these 3 individual env? Do I need to create 3 different individual projects in aws including frontend ec2, backend ec2 and cicd pipeline? code will be committed in all the env by developer manually and respective cicd pipeline will take care of rest in the respective environment? OR there would be one infrastructure with separate databases per env and common cicd.
This question is very valid, as the setup you create today will stay with you for longer. And believe me, it become more tough to change every single day later :)
So, what options do you have?
In the simplest but "ok-ish" solution. Create VPCs per environment in your account. All resources must be properly tagged (for example key: environment, value: production"). In this way you will have logical separation withing one account, you can set boundaries, who can access what (using the tag I mentioned) and also you will have reasonably good way of understand your costs.
There is a better solution. Every environment goes to its own account. Now, you may think "omg, this will be overcomplicated". I know, it looks like this on the beginning. So I will go further, and create fourth account :D Let's call it master. This account is the one where you will create organization and SSO.
But let's forget now about this fourth one.
With accounts dedicated to environment you have full separation and you use full power of AWS.
Now CI/CD. Infrastructure is a beautiful but treacherous thing :) You will learn in time, that to work with IaC in CI/CD you have to have a slightly different approach than for your applications. Generic CI/CD tools, however powerful, don't ofer full control over the whole life cycle of infrastructure. Well, yes, you can do it, but you need to invest a lot into it.
I suppose you use IaC (terraform, cloudformation, etc) for the rest of my answer.
You need to stage your Jenkins pipelines. First, I recommend to do some SAST test of the quality of your IaC. Checkov, tfsec, etc are your friends here. After that you can create stages for all your environments. Example:
tfsec -> deploy_on_dev -> deploy_on_staging -> deploy_on_prod
You can have some tests between stages:
tfsec -> deploy_on_dev -> tests_on_dev -> deploy_on_staging -> tests_on_staging -> deploy_on_prod -> tests_on_prod
You can force the manual acceptance, something like:
... -> tests_on_staging -> manual accept -> deploy_on_prod ...
And so on. These are like the generic approaches.
The point is, you should separate your environments. On AWS and in your pipelines. Of course, this requires more work related to authentication, but hey, you want easy things or secure things? ;)
I hope this helps. If you have any more questions, ping me!
On the end, the best way of manage IaC is to use dedicated tools, like Spacelift. I mention it, because I am Developer Advocate there :) But orchestration tools plays very important role in proper management of huge environments. You will be there someday (I mean, with huge infrastructure :) )