SocialCops, when it was first started over two years ago, stood at a place very different from where it is today. At the beginning, SocialCops was mainly focused on building a mobile application for collecting data. Now we’re working on a powerful platform — with services for data collection, accessing a massive secondary data repository, transforming and cleaning data, and creating beautiful data visualizations — to help data-driven decisions making easy.
In the last two years, everything has become more complex at SocialCops.
The number of functionalities on our platform has increased. The number of technologies we use has increased with each new functionality and team member we add. As the number of our users grew, so did the size and types of their requirements. The number of issues we juggle at any given time has increased.
Everyone who has ever studied scalable technologies has heard the stories of the challenges that some of the biggest companies faced when they switched from a hairball of software code to a well managed, well maintained, modular code base.
SocialCops, however, has always been different when it comes to this domain. During my time at the company, I realized that we are not one of those companies who would first have to fall before rising again. Call it a coincidence or a call of requirements, SocialCops has naturally adapted to the size and complexity of our software systems with a modular and better maintained software. Yes, we had our fair share of problems, but the modularity of our back-end software has always been a lucky part of the company.
The beginning of polyglot microservices
Of the many problems faced by young startups, one of the greatest is that of hiring people and getting them to contribute to the company as quickly as possible. Another challenge is that each new person comes with their own set of skills and technologies with which they are comfortable.
Time has taught us that the best way to get fast results is to use an agile and modular workflow. This means that, instead of spending several days on planning and perfecting software, we build software in pieces and improve each piece in several iterations. At the end of everything, these small pieces communicating with one another are served as a platform to the world.
These small services (often called microservices) were written in different programming languages including Python, JavaScript, Shell Scripts, and Scala, making the system truly polyglot.
The challenges of polyglot microservices
Working with many pieces of software written in different languages does help reduce your development time. However, the biggest challenge with this approach is managing all the small pieces of software and making sure they are playing well with each other.
If, for example, a service which sends emails to people crashes, all the other services which use the first service to send emails would either crash or stop working until someone noticed and fixed the email service.
This problem was fairly easy to solve. We just needed some simple configurations on our servers that would restart a service every time it crashed. This way, if an event made a service crash, we would at least have the service restart and work well until the crash-causing event occurred again.
On its face, this looks really simple, but it quickly becomes tedious when your requirements grow and you have many services running on many servers.
Once we got in this loop, we found ourselves writing software to run and manage other software. Even that sounds fine until you realize that you are spending more time on managing your software than writing it. If you follow an approach like this, you realize that you are back at exactly where you did not want to be when you started.
Solving service management with orchestration software
When we looked at other technology companies, we realized that SocialCops is not the only one using polyglot microservices. The world is filled with teams using the same concept and managing it well. After a little research and some effort, we realized the power of orchestration software.
Orchestration software is a software that is used to initialize, run, and manage other software and assign them proper hardware resources and do a hundred other things that you would otherwise have to do manually. It was important, however, that we went with a solution which has stood the test of time, or at least has survived enough tests for it to be considered reliable.
After analyzing various alternatives, we decided to use Mesos — a software that is used and trusted by companies like Twitter where software infrastructure had always been a huge challenge.
After spending a little bit of time on other technologies in the ecosystem, we started feeling confident with Mesos and decided to use it. So far, this has been immensely useful for us and has helped us make sure that our software does not crash, even when it is abused by our self-proclaimed testers.
How to do orchestration software right
There are several benefits of using orchestration software, but the entire process can easily backfire if not done right. Here is what we have learned so far from our journey towards automating software infrastructure.
System is important
In an environment that is built by several people and made of pieces of different technologies, it is important that you develop a system that works for everyone in the team.
It is easy for everyone in the team to write their own software and deploy it on the server. However, it quickly becomes tedious to keep track of all the running services, even when you have powerful orchestration software in place. Thus, we have automated a major part of this process.
Every time someone makes a change to the software, they have to trigger the request to start the deployment with a simple command on their computer. The system then automatically tests and builds the software for production. Once this is done, you can be more confident that your software will run on the cloud — a process that you can trigger with a few clicks.
You have to do a little to make sure that you don’t end up doing a lot
I understand that fiddling with things to make them work is interesting. But it is much easier to stare at your screen while an automated software takes care of something which would have taken you much longer to do on your own.
Automation is helpful when you respect the methodology required to maintain it. By automating things, we now have to write a few additional lines of code for instructing the system to test and build our code. This takes time, but you only have to do it once.
Not all automation is good
Despite the fact that I love automating every thing that I have to do more than once, it is important to understand that not all automation is good. Automation software, like all other software, is software. When you use lots of software, you lose the simplicity. Without simplicity, automation loses its purpose of making life easy.
A simple rule of thumb is to make sure that you are automating processes and not random activities. When you are automating a process, things usually tend to naturally fit together without many complications. For random automated tools, not so much.
When it’s not a good idea to be a polyglot
It is not an easy decision to switch to a polyglot development environment, much less one with microservices. It is, on the other hand, very easy to know when you do not want a such a system.
The most important thing to know before moving to micro-services is that you should not complicate things more than they need to be. When you are working on a platform with several features, modular micro-services are inevitable. But, for most use cases, it is more work than necessary to have such a complex system hanging over your head.
We are still learning new things every day and make changes to our software and infrastructure depending on requirements and lessons. It is important that you learn from your mistakes and adapt and remember that there is no one-size-fits-all solution when it comes to managing software.