What's a good devops approach for a single developer writing python web applications?

Question

I'm guessing this question will seem incredibly trivial for some readers, but as someone who is a developer but with little experience of deploying apps in anything other than a manual, hit and hope sort of a way, I hope you will understand that it's quite daunting to see the number of different approaches and tools there are, so I could do with a bit of advice to get me started in the right direction.

I am a developer, now only in my spare time, which is limited. Up to now I have worked with Java, building webapps, and have been reasonably happy with deploying a war file to a Tomcat environment which keeps things nicely encapsulated.

I am now working in Python and Django, but as I get closer to the point where I need to deploy, I want to set up a solid devops workflow to automate as much as I can and ensure I can deploy reliably, but given that my use case is relatively simple, I want to avoid learning a big fat toolset which is over-engineered for my needs and which requires a big investment of time I would rather use coding my app.

So I am looking for a middle ground which allows me reliably to deploy and manage my app(s) without investing a huge amount of time setting up and learning a big devops ecosystem.

Some more details...

Context

1. I develop on a Mac, using PyCharm to build Django 2, Python 3.
2. I use git (but not on github) to manage software versioning.
3. I am comfortable with other languages and scripting languages and have written a few (probably fairly amateurish) bash scripts, although I don't enjoy bash. I've also dabbled with Perl, which I realised isn't really a language for dabbling (i.e. you need to spend time learning it properly)
4. I intend to deploy on a VPS environment, probably DigitalOcean.
5. My app isn't mission critical but it is important that I know if the site goes down, and need to have a way of reliably recovering if it does, whether this be restarting the app, restarting the server, or moving to another server (or other).

Specific Requirements

1. Ability to set up a new environment to receive the app.

   Up to now while I am learning, this has been manual, and every time I have done it I have started from scratch with a new Droplet. I would like this to be much simpler (automated) so that if I have to set up a new environment in an emergency I can do so reliably.

2. Ability to deploy the app to a staging environment which is as identical to the live as possible, ideally as an automated process triggered by a git push using a continuous integration approach (which I have never done before).

3. Ability to "press the button" when I am happy with the app in the staging environment to push to a live environment ideally automatically.

4. Way to monitor the site (just a poll to a page will do)

5. Way to switch live site to another server if I need to recover from an app or server failure on the live site. I think maybe a Blue-Green approach would work for me?

What have I tried or considered?

• Manual set up of live environment with Django app, then manually copy new codebase to it when there is a change. This feels prone to human error and I fear making a mistake in a deploy causing an un-recoverable failure.

• Docker. I admit when I found out about Docker it seemed like a dream come true but after a bit of experimenting and research I am daunted by how much I need to learn and know to get this up and running and to manage it. It may be that this is worth it because once it's working it is very low risk but at the moment it feels like a larger investment of my time than I am hoping for.

• Bash scripts. Use them to set up the original environment and for specific tasks like updating the app. My worry about this is that the scripts will be code which needs testing and I fear it would take a lot of time to build a reliable toolset this way.

• I've looked at Digital Ocean's options for floating IP addresses and the ability to have two servers for a "blue green" approach which seems quite sensible. If I go this route I still need to be able to automate the deployment.

So... I am looking for advice on a devops approach which finds the right balance between minimising risk (e.g. the risk of breaking the live app with an update, or the risk of being unable to recover from a failure) and minimising the time investment I need to make to set up the environments and workflow.

Answer 1

I am not familiar with Python development nor DigitalOcean, so I'll just offer a few pointers:

• The goal is to automate. Everything. How you achieve that is really up to you, and creating your own tooling is not far-fetched, many do it that way. One concrete and pretty low(ish) hanging fruit is to get a git post-receive hook running which deploys and restarts your test environment. If you have that, the rest should be simple.
• "My worry about this is that the scripts will be code which needs testing" - that worry is unfounded. You are testing those scripts every time you deploy to your test environment, after all. Especially coupled with a blue-green approach it will be fine to have bash scripts.
• "I don't enjoy bash." - then find another scripting language that you enjoy. Maybe try Ruby? The language and core libraries are very clean and well documented, and I'd say, rather easy to learn. Or, just for fun, Go(lang), which seems to be well-suited to devops tooling tasks. And finally, as you seem to like Python, you certainly can do installation tasks with that as well. From these, Go has the benefit that it creates standalone binaries and does not need a complex environment installed first, itself, so bootstrapping may be easier.
• "a staging environment which is as identical to the live as possible" - if you have a script that spins up an environment from scratch, i.e. from a more or less empty base image, then your environments will be identical, save for deltas encoded in your script. That's the point of all of this.
• "Way to switch live site to another server" - the only thing to ponder is what happens with your persistent data. I.e., you will want to make it so you can link your applications with different persistent volumes/stores on the fly, to be able to switch back and forth.
• "Docker - daunted" - to be honest, it should not be that bad. If you know how to build an environment from scratch with command line tools (no GUI tools), then placing those in a Dockerfile should be rather easy. The worrisome details appear when it's time to tune (i.e., reduce image sizes), but apart from that it should not be too bad. First get it to work somehow, then find out how to make it beautiful. The good thing is that the knowledge you gain will transfer to lots of other environments.

Good luck!

Answer 2

Thanks for the great question. Nothing is really trivial the first time you do it and we all were new to something once.

My first recommendation is to revisit docker. Try some different guides and tutorials. It's really simple. You have a docker file that gets "built", literally just commands you want ran on the "container" or "image". You push that image to a registry which can be public or private. You then run that image on a host machine. Docker is really important with node.js and python where you have lots of dependencies and it can really be hard to manage them sometimes. If you are using pip, and you should be, you can generate a requirements.txt file to feed to your docker container.

Now you said you are using git, so I would use local git hooks. You can use these to build the docker container, run automated tests and then deploy your container. You can look up lots of different guides and tutorials on this subject.

For managing your infrastructure I would you use Terraform. Terraform is great because you can spin up an environment on demand and delete it when done. My recommendation would be to start out simple and once you mastered docker and terraform you can try blue/green deployments.

Now if you are using Gitlab or willing to switch, it also offers a free ci/cd service. This includes so many cool features and is really easy to use. I use it personally for all my apps. You could completely skip the local git hooks and test in the gitlab pipeline or keep them for testing each commit locally and using gitlab to build and deploy.

I hope this was somewhat helpful.

Answer 3

The posted answers have been very helpful in allowing me to re-think my problem and various approaches. I haven't implemented a solution yet but I have decided on an approach so I am documenting that and selecting it as the answer. In summary it is this:

My Chosen Approach

• For the live environment I will use two Virtual Machines (probably using DigitalOcean droplets) running Ubuntu and configured exactly the same.
• I will employ a Blue-Green approach using the Floating IP facility within DO to maintain my two identical server as Live and Pre-Prod/Backup.
• I will create a VM (probably using VirtualBox) in my development set up for use as a staging environment. This VM will be set up exactly the same as my two live servers.
• I will write a single common script in Python for setting up an environment from scratch and I will use this to configure my staging environment and my live/pre-prod pair.
• I will employ git hooks to trigger updates to the environments (probably manually triggered).

Considerations Which Drove This Approach

• Docker: I have decided against it. Although I take seriously the responses (thank you @Levi and @Dan) which say I should re-visit and it shouldn't be that bad, I have had too many experiences in the past of embarking on something new and realise I have fallen down a rabbit hole which eats up time and takes an age to get going. I think it would be different if I was even working with one other person but as someone working completely alone every minute is precious.

• Virtual Machines: I hadn't really considered this until I started working with some of the Docker tutorials which use VMs to demonstrate the Swarm functionality. The idea of being able to create a brand new environment which I have complete control of is very appealing.

• Scripting: Prompted by @AnoE's helpful reply I have done a bit more digging and it seems that Python is recognised as a viable option for scripting and as that is what I am writing my app in it seems there should be some synergy (If I need to learn something new for my script it will be knowledge I may use in writing my app)

I will update once I have made some progress with this and if it goes horribly wrong I will acknowledge I maybe made the wrong choice!).

Update on 20th October 2018.

I set out to write a Python script but this often involved invoking a bash command from Python and then getting the response back and I found this added to the development time quite a lot. After a couple of weeks of slow progress I looked elsewhere. I admit I may have been approaching it wrongly but I needed something which was going to be faster.

I eventually settled on Vagrant / Ansible / VirtualBox and after more months than I like to admit got something that worked well, but after a lot of work learning some new skills (Vagrant and Ansible were completely new to me). I then applied the Ansible script to provision a DigitalOcean Droplet and found this worked really well. I have become a fan of Ansible but even though I agree (with reviewers) that it is relatively easy to use, it's still a new paradigm and quite a steep learning curve.

At the time of writing, I have provisioned two separate Droplets on DigitalOcean in a blue-green configuration, using DO Floating IP addresses to switch between the two, and on each the application is within a git working copy so I just need to refresh the Master to update an environment.

I am having a problem getting Floating IPs to work as I was expecting but I expect to resolve that soon and then I will have a working DevOps environment.

The big advantage of this approach is that the way Ansible works, once you have something working it's relatively easy to make it work in a different environment and this may not be so easy to achieve with a python script (or at least you have to build this in which is extra work).

I think the big lesson is that things take longer than I expect them to and learning a new technology always brings unknown unknowns. This shouldn't be a surprise to me - but it always is and working as a lone developer, this happens to me a lot.