Portable Development Environments with Vagrant and Ansible

Hello! It looks like I'm back with another blog post!

Unlike the previous one, which was more of a self-introduction and reflection upon my transition from school into different work environments - a bit of a ‘story so far’ - in this post, I will reflect on why we introduced Vagrant in our development platform, our experience with it and how it has improved our whole development and deployment cycle.

Reality Check

Imaginary Cloud as been around for a while and you can notice it. We're about to celebrate our sixth anniversary in less than a month away from this post. In six years we've watched with special attention the evolution of the web ecosystem to Ruby and Ruby on Rails, our go-to tool in the development of backend infrastructure to support both web and mobile applications.

Six years is a long time in the life of any technology on a never-ending evolution process. For that reason, Imaginary Cloud has seen projects built in the almost forgotten Rails 3.1 on top of Ruby 1.9, without keyword arguments or safe navigation operators, to the most recent Rails 5 on Ruby 2.3. We are not limited to Ruby on Rails though, as we have also dangled with different iterations of Node, Python, Java, Objective-C, Swift, SQL and NoSQL databases, and even Wordpress, just to name a few variables in our projects. Now, add several different versions of gems and a dash of package’s dependencies - on top of different operating systems - and we get a myriad of project environments.

Imaginary Cloud's clients tend to stick with us for a long time and some of them have been our partners since our foundation, requiring us to actively support their applications. Occasionally those applications need to be patched, changes on third party API - which are quite common - need to be addressed, new bugs that show up in a more recent version of a web browser require hot-fixing, and we even end up extending applications to introduce new features that fit our clients new course of action.

In this scenario, setting up a development environment that allows replication of the conditions to reproduce a bug, or even having a quick way of checking the current state of the project in order to estimate the effort of introducing a new feature, became a critical and urgent task. Switching between projects with different configurations and environments became harder, and transitioning to old projects in order to provide a hot-fix, turned into a dreadful and frustrating task. This is because the setup of the old project takes longer than the time to reproduce, fix and deploy a new hot-fix.

Scary

That’s not fun.
That’s not healthy.
That’s not efficient.

It became evident that there was a need to preserve the development environment in a portable and confined manner, allowing a smoother transition between projects. That’s when Vagrant stepped into action.

In Vagrant We Trust

Vagrant is a popular open source virtual machine declaration and management tool from Hashicorps. Its popularity comes from the simple Ruby syntax that describes a resource, from base image, resource allocation and minor provision configurations, such has how to mount a shared folder or hook provision scripts to prepare the development environment.

It also comes with a variety of providers that can support Vagrant in multiple platforms, with the creation of local virtual machine on Virtual Box, VMware and Hyper-V. It was even extended by the community to support popular hosting services such as Amazon Web Services or Digital Ocean, allowing them to work as providers and abstract all the creation and management process of Droplets and EC2 instances. This way, developers can simulate and coordinate multiple node architectures through a familiar interface.

The best part? All the specification is done on a single text file, which can be easily bundled into a project and kept under version control, ready to checkout.

Digging further with Ansible

Vagrant alone isn’t enough. It offers a fast and clean way of interacting with a development environment and using an operating system similar to our staging and production environments. This increases the resilience of our applications and makes them less prone to failures that stay hidden on development environments. But it’s not the best tool to provision a virtual machine by itself.

Goingdeeper with Ansible

Nevertheless, it offers hooks to integrated third party provision scripts which range from simple shell scripts that perform all the required actions, to more high level provisioning tools such as Chef, Puppet or Ansible.

Ansible was our weapon of choice to provision the virtual machines, because it doesn’t require any kind of agent running on the virtual machine, like Chef and Puppet would, and does most of its work through Push via SSH. Its specification is done in YAML, making it quite readable and easy to understand, so that anyone could quickly entail each instruction and change a provision script to suit their new needs, such as adding any extra system libraries for a new gem. And, with Vagrant, Ansible’s scripts are just plain text that can be stored with the project, under version code.

The Vagrant experience

Like every new little thing, Vagrant and Ansible's introduction met different degrees of resistance on its adoption. Developers that are allocated on just one project for longer periods of time didn't fully appreciate this necessity or value its portability. It's understandable.

I'm on the other side of the spectrum. I actively support Vagrant and its usage, as I hop from project to project, tinkering with different stuff on a daily basis. It might come as a shock that I actually do some work around here, besides writing blog posts. It's not uncommon for me to fix something on an old Python project in the morning, switch to some Ruby 2.0 maintenance in the afternoon and tinker with Node.js on the next day.
Vagrant allows me to efficiently switch project's context.

Our Future with Vagrant and Ansible

As big fan of Vagrant and Ansible, I look forward to the opportunity of reusing this technology and apply it to new challenges. Ansible alone is a powerful provisioning tool that effectively interacts with multiple VPS and can be used to deploy changes in large clusters.

While we don't foresee a massive usage of Ansible to manage large clusters in the near future, the idea is to improve its usage, as we can reuse the already built Ansible scripts to setup our development virtual machines, provisioning the VPS that will host staging and production. And that's something we are trying to enforce: build good and flexible provision scripts that can fulfil more than just provisioning a virtual machine, and reuse them when trying to deploy the application to multiple environments.

To infinity and beyond!

Whilst Vagrant can be used as a tool to create and manage VPS for development, or even to create staging and production environments, I feel like there are better tools for the job, such as Terraform. But that's a whole new story for another day...