If you are unit testing, you should be mocking external dependencies like Spark and AWS Glue. Look into Python mocking frameworks like unittest.mock, monkeypatch and pytest-mock. Your unit tests should just test your code and how it interacts with Spark and AWS Glue. Your unit tests should not test Spark and AWS Glue functionality. Do that in your ...
The AWS answer would be to use appropriate services for each component.
E.g. Put the database on an RDS - they have multiple choices. Deploy the front-end using Fargate and Lambda.
There are 2 issues to consider:
1. The learning curve.
Getting up to speed on AWS technology - and keeping up with them as they make minor tweaks that break lagacy deployments - ...
Cancelling spot requests is good if you have created a 'persistent' spot instance. If you try terminating a spot instance in the EC2 console when its persistent a new spot instance will spin up in its place in a few minutes.
However, if you cancel the spot request this will terminate the spot instance and also cancel any future rebuilds of this instance. So ...
This lab from AWS Well-Architected will show you how to implement an AWS AZ failure simulation using Bash, Java, Python, C#, or PowerShell
Which flavor of Linux are you using? (eg Ubuntu/Debian, RHEL/CentOS etc)
Also which version? (eg Ubuntu started using the "netplan" configuration by default in 20.04)
At first glance, it looks like your DNS nameservers are hosed somehow.
And can you post the results for doing a: netstat -nutpl?
Look at which PID/Program Name is listening on port 53 ...
For RDS, the simplest would probably be to have a template RDS snapshot that you create instances off of, just like you would an AMI.
However if later updates to the database structure will be frequent/complex enough that the added complexity of a CD migration tool will be worthwhile, then you may be better off starting with that for the initial deployment ...
I have not tried it but I think if you change your deny to a list of elements it will work.