My Journey to SocialCops

“At its heart, engineering is about using science to find creative, practical solutions. It is a noble profession.” -Queen Elizabeth II

Being a self-proclaimed maths geek and computer lover from my school days, I wonder what would have happened if I hadn’t become an engineer. I knew early on that it was the only profession I could see myself pursuing. I always had a keen interest in making end-user solutions — software that could actually impact people’s lives.

The Android platform was the easiest and cheapest place to start during my college days. Thus I became an Android developer, and found myself at SocialCops, a data intelligence company trying to solve critical problems with data. It sounded fascinating, and as it turns out, it does not disappoint. Within a week of applying, I had all my interview rounds and I scored an internship.

My Hack Week

My first week at SocialCops was quite challenging. I got to know my team and get used to the culture. Every intern is given a task of core importance that will be crucial for the product development overall. You are told to build a feature that is going to be used by actual users, and that you are going to have complete ownership. No pressure, right?

I was tasked with creating a peer-to-peer offline sync feature, to enable users of the same organization to share data without internet connectivity.

Basically, the team member with the latest data set had to act as a sender (a temporary server) to allow the other team member to connect to him/her and access the same data.

Sounds exciting? I was feeling the same but it involved a lot of challenges. Let’s explore.

Finding a Solution

Half of knowledge is arguably having no knowledge at all. When you have no proper context, things go haywire. Such is the case with data collection. You need a complete set of data, else the results you get are mere estimates of the actual problem.

Incomplete data collection in India occurs mostly due to no internet connectivity; that is, regions that are technologically isolated (not connected) from the world. This lead to the loss of valuable data.

“Our data collection model need not fail when there is no internet connectivity,” said my teammate when he presented the problem to me.

I had to make a completely offline peer-to-peer data transfer solution for people collecting data in regions with no internet connectivity. My perspective completely changed from being a person who says, “Internet is not a problem anymore” to “Internet is a big problem!”.

I had to find a way to allow two people in a remote area to share the forms without the ability to sync to a server. I also had to find a simpler way to allow users to connect with each other via hotspot, thus simplifying the user experience as well.

I started off by brainstorming with my teammates and searched all over the internet to see if someone else had solved this problem. I got a good idea of the feature I had to build, but it appeared that the problem was I was trying to solve was a problem no one had ever touched before: sure, there are apps like ShareIt and SuperBeam, which provide a somewhat similar data-sharing offline experience, but the kind of user base we were targeting for our Collect platform is completely different. Our users include people who are not quite familiar with the internet and how it works. Explaining the concept of peer-to-peer sync and creating a user-friendly experience with our particular demographic in mind was challenging and way out of my league.

After few more days of research, I found out about the JAVA Reflection API and its role in providing a way to modify device settings — something that is not directly accessible by the Android framework. This API allowed me to connect two devices without a server or internet! Now I was on to the second part of the problem: sharing data. I solved this using NanoHttpd library, allowing me to make one of the devices act as a server. From here, things started moving really fast.

The real confidence boost came when I was able to send a text file from one device to another.

Now it came time to build the complete module. It had two main functions to perform:

  1. Connect the two devices without internet, and prepare data to be sent.
  2. Authenticate the receiver and allow the successful transfer of data.

Using the hidden Android APIs via Java Reflection to control the device hotspot and wifi state, and the HTTP layer (acting as a mini server) to create protocols to authenticate and send data to receiving device, the peer-to-peer sharing module was born.

The Impact

At the end of the Hack Week, I was proudly standing with my team, watching the feature I built ship and release on the Google Play Store. I learned a lot about user experience and how to keep a feature’s core principle simple and easily understandable.

Collect users could now access and share data without internet connectivity, which was the base premise of our app. It felt amazing to solve for India.

At SocialCops, you get to solve problems that no one else is solving. We push our limits every day, create and break things. This is how we are shaping the data collection by transforming Collect into the world’s best data collection tool.