Methodology for software development and deploy in fast-paced agency

Question

Background:

Three months ago I was called to do a freelance job for a rather big agency, with 40~50 employees.

Things were chaotic when I got here. The entire development team had been hired by another company one month ago, and there were a lot of overdue jobs. I freelanced here for about three weeks, until they hired me. Since then, I had been working hard to fix things, and there were so many things to do that I had to let go of the process to focus on delivering the software.

Now, three months later, the dust have finally settled down, and now I'm looking forward to implement good development practices and methodologies, mixing development, deployment and project management.

Scenario:

We have several different clients, most of them with their own servers. Some of them have their own infrastructure team to handle their servers. Projects are in different languages, and I have little control over the client's servers.

Throughout the day, there might pop up several demands for different clients, and I must deliver them as quick as possible.

Conclusion:

I have to deliver things quickly, but I feel a urge to employ a methodology to do it, because I know that if I just keep doing things without proper control, some thing will certainly go wrong sooner than later.

My two cents:

One of the first things I plan to do, is to elect a preferred development language and framework, so we can specialize in something. Other than that, I'm not sure what to do.

Answer 1

In terms of development procedure, you should consider using the agile method. This is the most widely used method of software development and has proven itself. Within the scope of the agile method Pair programming will allow you to spool up another developer to assist with your projects when you are ready and find that you have more work than you can handle. You can then move into a standard agile method like scrum/kanban/scrumban (or continue the pair programming if you find you like it!)

One particular reason for choosing this method is that is lends itself well to DevOps and Continuous Integration. In fact, one could describe DevOps as a specific way of implementing the agile method. The Iterative, incremental and evolutionary philosophy of agile and the very short feedback loop and adaptation cycle marry beautifully with DevOps/Continuous Integration and provide a solid foundation to build upon.

Here in the agency we have a infrastructure team as well, with the possibility to set-up our own internal or external servers to handle things.

Make good friends with the infrastructure team. Involve them early and often and drive them towards a Infrastructure/Configuration as Code philosophy which will allow you to collaborate closely with them. While it might seem like it is faster to just do it manually, doing so is creating technical debt. In a few years when it comes time to upgrade, no one will remember how the thing is built. It will also allow the infrastructure team to look good. Never again will they make a mistake or forget a step during deployment. Either the deployment code works or it doesn't and it will deploy exactly the same way it deployed in test/staging - making deployments and upgrades easy and horizontal scaling a breeze. Ignoring these techniques also slows down your development. If you can just fire up a new server, have it pull down (and compile) your latest git commit, run through an automated testing suite and then E-mail you the results while you go do something else it is a huge time-saver.

If you can, utilize the cloud - with Amazon for instance, a lot of the tools are already built for you. While this might make your Infrastructure team leery, just remind them that the cloud is just someone elses computer. All the cloud does is provide you with hardware. It will still require someone from the infrastructure team to install maintain and operate it. And it will allow you to avoid scalability issues. The database problems, storage and backup issues and headaches have now been outsourced.

and I have little control over the client's servers.

See what you can do about this. Consider leveraging a tool like The Forman's Katello scenario (I like to deploy it with Salt Stack personally - I don't like ruby) which will allow you to manage and deploy your VMs on nearly any iron or cloud (including bare metal), manage the configuration of that hosts using Salt Stack, Chef, Puppet or Ansible, manage your docker containers and manage updates to your machines. This will allow you to then easily provide the code to your client to either A) automate or B) manually deploy their updates and deployments. Once you have proven the value of the speed and efficiency this method gives you, you may be able to simply use these same tools to spawn your client a copy of this stack for managing their environment or you may be able to talk/upsell your clients into letting you manage these deployments - The Foreman will allow you to configure sites and organizations (as well as mange users if paired with FreeIPA) so you can configure multi-tenancy. And the project is backed and supported by Red Hat.

If you have any doubts about the above, I recommend you pick up a copy of "The Phoenix Project" of the Audiobook which illustrates very well the value and benefit of the above.

One of the first things I plan to do, is to elect a preferred development language and framework

If you go this route, just make sure to standardize on a language that plays well with REST most DevOps stuff uses this interface.

And finally,

The entire dev team had been hired by another company - and they didn't have a solid process as well, only a few gits here and there

Do what you can to merge these together into one place. It will make your life a heck of a lot easier.