How a Team of Grad Students Built a Mobile App for Entomologists

Entomology Today August 19, 2019 Leave a Comment

LeafByte is a free, open-source mobile app for quickly measuring leaf damage caused by herbivorous insect pests.

By Zoe Getman-Pickering

Every year, chewing insects do billions of dollars of damage to the crop plants that produce our food, medicine, flowers, and wood products. Every plant that we grow for human use has insect herbivores that try to consume it. While trying to find ways to combat these chewing herbivores, scientists often need to measure the damage these insects inflict. This can help assess economic impact, test methods of pest control, or ask basic questions about the evolutionary and ecological relationships between plants and insects.

Zoe Getman-Pickering

Measuring the amount of damage an insect does can be hard. Many of the techniques that exist are slow, difficult, or prohibitively expensive. Our lab at Cornell University used the most common: ImageJ, a powerful and general image processing software. While ImageJ is free and incredibly versatile, it isn’t optimized for the specific use case of measuring leaves and herbivory, so each leaf required an involved process to scan and upload the leaves, select the areas of damage, and reformat the data. That’s why my collaborators and I decided to make LeafByte, a mobile app to measure leaf area and levels of herbivory.

In my optimism, I thought we would be able to make the app in two weeks—no problemo. I was only off by a year. I thought it would be simple: You figure out what you want it to do, program it, and then put it on the app store. We had already sorted out step one, so we were one-third done. However, an app needs far more thought to design than I ever imagined. The user interface has to be pretty, intuitive, and accessible, which requires forcing every friend, family member, and acquaintance to test it out. Like writing a manuscript, the process is iterative. Huge pieces of the app were built, reorganized, slashed, improved, and rebuilt. And, like designing a field experiment, it is impossible to foresee all the challenges, mishaps, and bugs (pun intended). We had to test, revise, and re-test the app in every conceivable situation.

LeafByte is an image analysis tool; while created to measure insect damage to plant leaves, the app’s creator notes that “as long as the thing you are measuring has high contrast and a defined area, you can measure it with LeafByte.” (Image courtesy of Zoe Getman-Pickering)

LeafByte is a free, open-source mobile app for quickly measuring leaf damage caused by herbivorous insect pests, developed by Zoe Getman-Pickering, a Ph.D. candidate in entomology at Cornell University, and Adam Campbell, a Cornell computer science graduate.

Most developers start with Android, for good reason. More than 75 percent of smartphone users globally have Androids, so the potential audience is much larger. Most countries other than the U.S., United Kingdom, and Australia predominantly use Androids. They are cheaper and more accessible, which can make them a better choice for messy field work. Beyond that, the Apple system has quirks that can complicate programming complex apps, particularly those that require image processing. But, despite all that, I had an iPhone, so that’s what we went with.

Making the app is just the start. Effectively communicating and disseminating the app is critical. We needed to design and make a website with useful information on how the app works, how to use it, tips and tricks, and more. It’s important to anticipate the questions your users will have and the challenges they might face using the app. Beyond that, a video can be helpful to show potential users the product. Making a video requires sound and video editing skills and quite a bit of time. I also needed to learn how to advertise, a skill that is woefully under-taught in STEM graduate programs. I emailed every scientist that I knew, but that was only a small fraction of the folks the app might help. To reach a broader audience, I worked with the Cornell press office to write an article on LeafByte’s impacts. I disseminated articles though various listservs, posted on social media and research forums, and worked with my scientific societies like ESA to spread the word.

If you read all of that and still feel inspired to make your own app, you may be wondering how to go about the programing. As a biologist, I wasn’t equipped to do all the programing that an app like LeafByte entails. The secret trick is to find and collaborate with a computer scientist. Many computer scientists are deeply interested in finding and working on projects that make a real-world difference. You can find wonderful computer scientists by reaching out to the undergraduate computer science classes and advertising in the computer science buildings, especially if you can communicate the importance of your research. However, the method that worked for me was to make friends across disciplines. Pick up hobbies, take exercise classes, go to seminars outside your field, and talk to all the new people you meet. (And even if you don’t find awesome collaborators like I did, you will find yourself with a good support network, a healthy body, and a life outside of research.) If you can’t find programmers the hard way, you can always hang around on computer-science and engineering quads with a sweep net and try to catch any student that passes by.

Adam Campbell

I met Adam Campbell at a Phillipino stick fighting class at Cornell, and we became fast friends and then collaborators. Adam is now a software engineer at a data analytics company, but he was kind enough to help make an app in his free time. While we both deal with “bugs” in our jobs, that’s where the similarities in our expertise end. Working together, we quickly realized we were speaking different languages: leaf area index, general linear models, and margins on one side, and endianness, connected components, and planar homography on the other. It was important for me to clearly communicate the science that would be done with the app, what was needed, and why, while Adam needed to clearly communicate what different technical options could offer and why one option might be preferable to another. This struggle also added to the iterative nature of the app, as each iteration brought us closer to a product that was both technically viable and scientifically useful.

When the app was done, the benefits were immediate. Measurements that took days wrestling with ImageJ could be done in minutes or hours. My lab was ecstatic. As the weeks and months went by, I began hearing from researchers around the world who were using or wanted to use our app. A major agrochemical company reached out to us and asked us to present the app to one of their team leaders. Six months later, and the app has been downloaded by more than 1,000 people.

One of the coolest things about making the app was hearing all the alternate uses people have found for it. Of course, folks are measuring leaf area and herbivory levels, but I also talked to a man at a conference who wanted to use the app to measure fly feces on filter paper. Another researcher was measuring damage to butterfly wings, and a third used it to measure fungal growth on petri dishes. One of my committee members uses the app for decay of filter paper in different soil conditions. As long as the thing you are measuring has high contrast and a defined area, you can measure it with LeafByte.

Making LeafByte has been an exciting experience for us, and we’re excited to see it being used. If you think it might be useful for you, check out the website. We’d love to hear from you if you have questions or suggestions for improvement, and don’t forget to rate LeafByte on the App Store!