Outcomes and Next Steps of Satellite Imagery Innovation Challenge Version 1

Summary:

• Late last year, we launched an Innovation Challenge aimed at developing remote sensing technology to help us cost-effectively scale our programming. We offered financial rewards to interested parties who could successfully develop models that would allow us to utilise remote sensing for our programming. This blog post summarises the key outcomes of the Innovation Challenge, most specifically, summarizing how models fared when their predicted values were compared against ground-truthed data. Unfortunately, no model met our minimum requirements. 

• Despite this disappointment, we will continue to try to develop this technology, given that success could be transformative for our programming. We are initiating two parallel tracks to maximize our chance of success:

  • 1: We are launching a follow-up Innovation Challenge Version 2.0, with a 10X increased financial reward.

  • 2: We are now hiring for a Remote Sensing Lead to develop these models in-house.

Background

In order to increase the scalability, impact, and cost-effectiveness of our farm program, the Alliance for Responsible Aquaculture (ARA), we are interested in leveraging satellites to detect water quality issues remotely, which would lessen our current reliance on visiting farms individually. We launched an Innovation Challenge on December 13th, 2024, seeking interested parties to develop new models that would allow us to remotely monitor key water quality parameters at aquaculture farms in India through satellite data analysis or share existing models that can be utilized for our purposes. We offered financial rewards based on the outcomes of a validation process. 

Overview of Process

As part of this Innovation Challenge (full details here), we requested interested parties to notify us of their interest in participating in this challenge by March 14th, 2025. After the notification deadline, the parties were requested to use their models to determine values for key water quality parameters relevant to their models (dissolved oxygen, ammonia, pH and/or chlorophyll-a) from 20 ponds in the Eluru region of Andhra Pradesh, at which we had collected ground-truthed data. Once we received the values determined by the various models, we conducted a validation exercise for each model, whereby we compared the predicted values with ground-truthed data we had collected from the same 20 ponds in Eluru. The validation of models—and the awarding of financial prizes—was based on predetermined minimum criteria for each water quality parameter. 

FWI Senior Program Associate Chandu takes a water sample at a partner ARA farm. Such samples are a key part of the ground-truth data that is used to train and validate remote sensing models.

Outcomes

Summary of models submitted

Twenty-six parties notified us of their intention to participate in the Innovation Challenge. Of the 26 parties that notified us of their intention to participate, only 10 followed through with using their models to determine water quality at the study ponds. 

Of the 10 parties that followed through, some developed models for all four of the water quality parameters of focus, while others developed models for one to three of the water quality parameters. Overall, 33 different models were tested as part of the Innovation Challenge:

• Models for dissolved oxygen: 9 

• Model for ammonia:  6 

• Model for pH: 8 

• Model for chlorophyll-a: 10 

Summary of validation of models

Unfortunately, none of the 33 models met our minimum criteria. Even the very few models that performed relatively well (compared to the other models) on our criteria had very little predictive value in real-world terms.

Path Forward

We recognise that we asked a lot of the parties in this Innovation Challenge. We recognise that our water quality parameters of focus represent a significant challenge for satellite-based remote sensing, particularly dissolved oxygen, pH, and ammonia—these are optically inactive parameters which present challenges for their direct measurement through analysis of satellite imagery. We also recognise that we did not provide parties with ground-truthed data to train their models; rather, the hope was that they would use existing datasets for training their models. 

Despite the challenge and the setbacks, we continue to believe that the potential of satellite-based remote sensing is sufficiently exciting for our programming that it warrants further investigation. Given the potential for this strategy to be transformative for our programming—and for helping millions of fishes—we don’t want to give up just yet! We are going to explore two parallel tracks as a form of a “last throw of the dice” to try to crack this problem. 

Follow-up Innovation Challenge

Despite the Innovation Challenge not resulting in useful models, we remain excited about this technology and believe the Innovation Challenge is worth one more try. We are relaunching the Innovation Challenge, with some changes. One key difference this time around is that we are significantly increasing (by 10X) the financial rewards on offer as we believe this will serve to incentivise a bigger pool of parties with more experience in this space, and/or to incentivise the parties that participated in the original Innovation Challenge to commit more time and resources to cracking this challenging problem. While this may sound repetitive of a project that previously failed, we believe that this strategy is a low-risk, high-reward approach for FWI. Another key change is that this time around, we will offer historical data from the ARA to parties interested in using this data. 

Recruitment of an in-house expert

Our efforts to date to develop models have relied on part-time, external expertise. We do not have in-house expertise in satellite-based remote sensing technology and developing machine learning models, but we have decided that it’s time to change that. We are planning to recruit a full-time expert for approximately six months to help us develop models (we have just published the job description).