Building for the “next billion” is a challenge, thanks to difficulties posed by the lack of internet, electricity, and low-cost devices. At SocialCops, we set out to build an Android data collection application that could work in the most remote, rugged parts of the world. We have been successful in building an Android application, Collect, that works without internet, on phones that cost INR 2,000 (USD 30), on 512 MB memory, and 1 GB storage. We still managed to include features such as voice to text, media capture, and location-based verification. Here’s how we did it.
Only 1 out of 3 people can access the Internet today. In a country like India, an estimated 75% of the population doesn’t have access to Internet facilities. Today, most of the data that we “mine” in the big data revolution comes from the 30% of the world’s population that can access the internet. So much for #bigdata. At SocialCops, we aim to drive the most important decisions facing mankind through data – in most cases, these decisions heavily impact the 70% non-internet-accessing, non-Facebook-using population.
We set out to map the remotest of areas – along with the people living in them – by providing a free data collection and monitoring tool for nonprofits around the world. By building a kickstarter for data collection, we aimed to empower nonprofits and public sector organizations to effectively collect and manage their data. We decided to build the tool on Android. Given the Android revolution and the decreasing cost of phones, it was the only logical option. It wasn’t the easiest task though.
Challenge 1: Working with memory constraints in low-cost Android devices
Most low-cost Android devices have a limited memory (RAM), usually 512 MB or 1 GB of RAM. This makes developing a “data collection device” for Android a difficult task. The data structure needs to be designed with the memory consumption of the application in mind. No user enjoys an “OUT OF MEMORY” warning message flashing on their screens.
To optimize memory consumption, we defined objects for each entity like surveys, questions, and other parameters. We carefully controlled each object to prevent memory leaks and to ensure the object was deleted while not in use. This saved us tons of memory and increased the speed of the application.
These are some tips to ensure your Android application can function with low memory:
1. Avoid memory leaks: Memory leaks are usually caused by holding on to object references in global members. Remember to release any reference objects at the appropriate time.
2. Use services sparingly: If your app needs a service to perform work in the background, do not keep it running unless it’s actively performing a job.
3. Release memory as it becomes tight: The onTrimMemory callback can be used to check when the overall device memory is getting low.
4. Check how much memory you should use: Call getMemoryClass to get an estimate of your app’s available heap in megabytes.
5. Release memory when your user interface is hidden: Another cool trick is to release any resources that are used only by your UI when the user navigates to a different app and your UI is no longer visible.
6. Avoid wasting memory with bitmaps: When you load a bitmap, keep only the resolution you need for the current device’s screen.
7. Use optimized data containers: Use SparseArrray instead of generic HashMap.
8. Stay aware of memory overheads: Make sure you have sufficient knowledge about the cost and overhead of the language and libraries you are using, then keep this in mind at every step when you design your app. The less the RAM you use, the greater the efficiency of your Android application. This allows more data to be processed.
Memory is especially important while making an app usable without internet. You will need to store and process all your data at the same time, since you can’t fetch the required data and process it.
Challenge 2: Working with storage constraints in low-cost Android devices
Given the limited storage on Android devices (e.g. 1 GB, no internal storage space, or 1 GB – 16 GB external memory), we had to leverage innovative solutions to ensure that field workers could collect data and store it locally until they could access internet at district centers. There are many alternatives available to store data, which you can choose from depending on your needs. Here are some storage parameters that you can use in Android:
1. Shared Preferences: This is a key/value store where you can save data under a certain key and easily access it by entering the key. This is very useful when you need to store a small amount of data, but storing and reading large structured data is extremely difficult as you need to define a key for every single data point. Furthermore, you can’t search within the data except when you have a certain concept for naming the keys. This is perfect for the NOSQL concept (JSON).
2. SQLite: Large amounts of similarly structured data should be stored in a SQLite database. Since the data is structured and managed by the database, the data can be queried using a query language like SQL. This makes it possible to search within the data and get a subset of the data that matches a certain criteria. Managing and searching large sets of data influences performance, so reading data from a database can be slower than reading data from Shared Preferences.
3. Files: Android uses a file system that is similar to disk-based file systems on other platforms. File objects are well-suited to reading or writing large amounts of data in start-to-finish order without skipping around. For example, Android files are perfect for image files or anything exchanged over a network, and these can be stored in internal or external storage before they are uploaded.
4. Realm Database: Realm provides an awesome database to manage JSON-formatted data. Realm is an open-source project with great issue tracking and support. Read more about our experience with Realm.
In the first version of Collect, we used Shared Preferences to store JSON data and Files to store media files until they can be uploaded to the server. For the next version, we are exploring Realm for storing JSON-formatted data.
Note: Don’t forget to delete local data from storage once it has been uploaded on the server.
Challenge 3: Dealing with performance and speed issues in low-cost Android devices
The speed of your application is one of your users’ top priorities, so you should be sure to optimize speed wherever possible. One classic way of improving speed is to fetch information in one go and restrict updates to when there is a worthy change.
Some of the other ways to optimize for speed are:
- Avoid creating unnecessary objects
- Prefer static over virtual
- Use enhanced for Loop Syntax
- Avoid using floating point
- Know and use the Libraries
- Use Native methods carefully
Using these simple methods, you can optimize performance and increase speed. This will ensure that the application runs in the same way both online and offline.
Challenge 4: Designing a user interface for a population that has never been online
Applications for rural India require a simple, intuitive user experience to ensure that rural users can actually use the platform. Designing for the next billion poses a whole new level of challenges, which we explored in depth in a separate blog post.
On the mobile front – we had to build the mobile application keeping in view the erratic internet connectivity in remote areas. The one mantra we religiously followed was to keep things simple and show users information up front.
P.S. SocialCops is hiring full time UI/UX designers. If you think designing for the next billion while changing millions of lives excites you, apply!
Challenge 5: Optimizing battery use
The key limitation to using a phone in the rural world is battery — once a phone runs out of battery, data collection is done for the day. This is why it’s essential to be careful about how much battery an app uses.
While developing an Android application for the offline world, you need to be specific about everything you’re storing offline, the space you’re providing to every module, and the processes you’re running. Your user will most likely be using a low-cost phone, so make sure you know how to store information, change the frequency of storage, and notify the user if they need more storage space.
To optimize battery, optimize the way you fetch locations for the use case of your application. For example, if the application is navigation based, then the location has to be fetched faster. If we need the location at one point, you can use a single fetch. You can also decrease the location accuracy, since higher accuracy uses more battery.
We use Battery Historian (a tool in the Android studio) to observe an application’s battery consumption.
Most Android developers love the thrill of coding for the latest gadgets in the market, with rich graphics and lots of heavyweight features. However, this changed when I visited Mewat in Haryana to observe a field pilot with our partner NGO. I saw my product in the hands of people who had never used technology before and the sheer joy they derived from simply tapping on answers. I experienced a strange high for having conquered the challenge of developing for a world that hasn’t seen the internet age. Trust me — that feeling of building something that could potentially change someone’s life is indescribable.
Collecting data in the field is always easier with a mobile app. We’ve helped over 150 partners collect over 20 million data points through our mobile data collection app Collect.
We’re always looking for new team members to join our Product and Engineering team. Think you’d be a good fit? Check out our open positions here!