In today’s era of ‘divide and conquer’, software development has been redefined multiple times to suit the need of the hour. Typically with small, young product teams — like the SocialCops team was in the summer of 2014 — quick prototyping is the dominant and most pressing need. The focus is almost entirely on getting out a minimum viable product.

That is exactly how we started, sprinting to get out the prototype of our first mobile data collection tool, Collect. We managed to do that in just about two months — starting with defining the need, learning new tools, technologies and frameworks on the go, then building the core features and getting the first users to test it.

MBaaS lets you concentrate on the problem you’re solving, while it’s someone else’s headache to handle the infrastructure — behind the scenes — that takes you to the solution.

It’s been challenging for sure. Today’s ‘get it right’ mantra involves a seamless user experience, amazing onboarding, and third-party services and integrations. Adding better functionality is considered a game-changer.

That is exactly where Mobile Backend as a Service (MBaaS) tools really come in handy. Explained by tech analysts as outsourced on-tap infrastructure, MBaaS lets the team focus single-mindedly on developing the product and user experience, rather than getting bogged down thinking about the infrastructure and dev-ops needed to power the product behind the scenes. MBaaS lets you concentrate on the problem you’re solving, while it’s someone else’s headache to handle the infrastructure — behind the scenes — that takes you to the solution.

That’s exactly how we used one of the MBaaS platforms for Collect — to fight the right battles at the right time.

A Quick Background on MBaaS

After the rise of cloud computing platforms like Heroku, the Amazons and Googles of the world heated up the market for on-demand web infrastructure services. Currently, Google App Engine and AWS Elastic Beanstalk are currently the favorites among the developer community. At this point, it made sense to take the game to the next level — orchestrate and set up everything, and make it a one-click go for developers!

MBaaS is a model that conveniently connects a web and mobile application with a cloud backend through APIs, servicing almost every need — storage, services, and integrations managed and orchestrated by the service provider.

This is how MBaaS caught its initial traction. MBaaS is a model that conveniently connects a web and mobile application with a cloud backend through APIs, servicing almost every need — storage, services, and integrations managed and orchestrated by the service provider.

This model saw a huge rise in the early 2010s, soon after “platform as a service” caught on in a big way to speed up the development processes of products. The biggest players in the industry were Parse (now open-source), Kinvey (now a Progress company), and Stackmob (now closed), among many others.

Our Experience With MBaaS

Thanks to MBaaS, we were able to work on just evolving the product experience and enriching it with features. This meant we didn’t have to care about setting up a MongoDB cluster in the right way and creating the right indices (the platform did predictive indexing using the most frequent queries), going through the hassles of deploying and maintaining the servers on the cloud, or dealing with critical aspects like user authentication and access controls. These were taken care of by the MBaaS service based on the pre-configured rules we set up. Third-party integrations like sending out emails, SMSes, and push notifications; providing API(s) to be exposed; and integration with common SaaS platforms came as an added bonus.

Then there was the elephant in the room — compliance and data security. This too was also well addressed. We could write any extra product-specific business logic in a matter of minutes on Node.js based interfaces, and deploy them with the click of a button or a single command line argument. Nothing could go wrong with MBaaS, not just yet.

We had started clocking a million API calls a day. That is exactly when a black-box backend started to worry us.

By the end of 2015, our second prototype was about a year old, and it soon needed to handle 100x heavier traffic and be much more reliable. That’s when we started to realize there was another side to the coin too.

We had started clocking a million API calls a day, supporting a user base of 50,000 users, and we had almost 100GBs of user-specific data in our databases. This was just the tip of the iceberg. The heavy lifting got more onerous, with new tasks for data reporting and quality checks becoming ever more resource-intensive.

That is exactly when the limited ability to understand a black-box backend started to worry us. Most MBaaS platforms — especially non-enterprise plans with a multi-tenant hosting approach — come with certain constraints and limitations. Business logic time-outs of ‘x’ seconds (whereas some reports needed hours to generate), failure of MongoDB queries (i.e. simple GET API requests, internally db.collection.find() on MongoDB) due to non-availability of indices as our data size grew, and no access to historical logs to access and fix problems right at the root — all of this meant huge turnaround times to fix critical application-specific bugs.

We could vouch to fix any feature that broke on Collect’s web frontend or the Android application, but we were not so confident about the backend, since we often didn’t even have control over the issue.

That’s not all. Third-party services, when taken through a stress test, also started failing. One such huge problem was right at the entry point to our product — we needed to send emails to confirm users’ email addresses, and many times the emails would just not go out. As a result, we ended up losing a lot of prospective clients. At other times, new systemic protocols would be imposed without any notice. For example, due to the sudden introduction of rate limits on the APIs (like 30 requests a second using sliding window protocol), a lot of requests had to be rejected with a 429 error code returned by the APIs.

One of the biggest deal-breakers was corruption of data due to encoding issues. Being a data collection app that guarantees the highest resilience levels, we couldn’t just tell our customers that we lost data.

But one of the biggest deal-breakers was corruption of data due to encoding issues. We realized that in approximately 5% of the data feeding into one of our MongoDB collections, the data was getting corrupted and replaced with gibberish characters. This happened because the data was multi-lingual, and the default encoding at the API handling layer wasn’t being set as UTF-8.

Being a data collection app that guarantees highest resilience levels, we were determined to go to the extra mile to make things right for our users. As a backup, we used to cache every request that came into our platform before it even hit the backend as a service; and we could turn to these to restore corrupted data. However, this took huge amounts of development and testing time from our small team, wherein ensuring eventual consistency with zero data duplication after the rerun of the cached requests was critical.

When to Use MBaaS

By now I think the message here is pretty clear — it’s mostly about picking the right battles to fight at the right time.

MBaaS is a great starting point to those who want a first prototype of an application out quickly and test market fit. It also is the right choice when the application being developed is not highly write-intensive, and has a static dataset to read from and serve.

Choosing MBaaS is about picking the right battles to fight at the right time. It’s a great starting point to get an application out quickly and test market fit.

Spending hours on an algorithm and overly optimizing it to work on a multi-core machine is an effort worth if, and only if, its impact is quantitatively justifiable — like saving users an hour each day or helping database administrators maintain consistency.

Scale and development are battles that you can take on at a later stage, but only after you have proof of concept and your users really need that scale.

From the first prototype developed over MBaaS, we got amazing validations of Collect’s powerful capabilities, and amazing insights on how to design a robust backend infrastructure. Now, as we continue to build our own backend for Collect’s next iteration, we are using the same learnings to fight massive battles like data security, controlled data ingestion for scale, micro-service communication and managing a database in production.