Outcomes and Next Steps of Satellite Imagery Innovation Challenge Version 2

Summary

To test whether satellite imagery could help us remotely monitor water quality, we conducted a second Innovation Challenge. Interest was high: 90 groups engaged and 39 submitted models. However, unfortunately, none met the accuracy thresholds needed for operational use.

Given the difficulty of predicting welfare-relevant water quality parameters with current satellite data, we are now focusing on the second track of our original dual approach: in-house model development.

To support it, we launched a targeted data campaign on November 20, 2025, collecting time-aligned, within-pond measurements—including continuous dissolved oxygen, temperature, and pH—across five regions in Andhra Pradesh. We expect 200+ samples for DO, pH, temperature, and ammonia, and 100+ for chlorophyll-a.

If, after using this improved dataset, remote sensing still proves unreliable, we will likely conclude that the technology isn’t yet viable for the ARA program. For now, we believe this intervention deserves this final test.

A single satellite image processed two ways: the left half highlights water bodies using a water-detection filter, while the right half applies our improved chlorophyll-a filter to show differences across ponds. This was captured during the first morning of our data campaign, where we’re collecting ground-truth measurements to calibrate remote sensing models for water quality monitoring.

Background

To increase the scalability, impact, and cost-effectiveness of our Alliance for Responsible Aquaculture (ARA) program, we’re exploring whether satellite imagery can help us detect water quality issues remotely—reducing our current reliance on visiting farms individually.

In December 2024, we launched an Innovation Challenge inviting people to either develop new models or share existing ones that could estimate key aquaculture water quality parameters in India using satellite data. We offered financial rewards based on performance in a validation process. That first round didn’t produce models suitable for our needs, but we believed the approach still held promise.

So in June 2025, we launched a “version 2” of the Innovation Challenge, this time with two major changes:

• A significantly larger prize pool—up to USD 100,000.

• Access to historical ground-truthed data from the ARA program, collected from over 200 farms in Andhra Pradesh since June 2021, for participants who wanted to train their models.

FWI staffer Ravi Teja collecting a water quality sample as part of the ARA. Our current methodology requires us to visit all 180+ program farms in person, limiting scalability.

Overview of Innovation Challenge V2 Process

For our Innovation Challenge V2, participants were asked to register their interest by August 20, 2025. After that deadline, they were given a fixed task: use their models to estimate values for one or more key water quality parameters—dissolved oxygen, ammonia, pH, and/or chlorophyll-a—for 20 ponds in the Eluru region of Andhra Pradesh on specific dates for which we already had ground-truthed measurements.

Once submissions were received, we validated each model by comparing its predicted values against our ground-truthed data from those same ponds and dates. Eligibility for financial prizes depended on meeting predetermined minimum accuracy criteria for each parameter. 

Results

Summary of Models Submitted

Innovation Challenge V2 drew far more engagement than the first round. Ninety groups expressed interest (up from 26 previously), and 39 ultimately submitted model-generated values (compared to 10 in V1).

Participants varied in scope: some built models for all four water quality parameters, while others focused on one to three. Several parties also submitted multiple models for the same parameter. In total, we evaluated 169 models (versus 33 in V1):

• Dissolved oxygen: 52

• Ammonia: 43

• pH: 39

• Chlorophyll-a: 35

A summary of the number of parties and models in Version 1 and Version 2 of our Innovation Challenge.

Summary of Validation

Each model’s predictions were compared against ground-truthed measurements from 20 ponds, allowing us to calculate R², root mean square error (RMSE), and mean absolute error (MAE). These metrics determined whether a model was sufficiently accurate for operational use.

Unfortunately, none of the 169 models met the minimum accuracy thresholds required to qualify for financial prizes.

One ammonia model reached the target R² but fell well short on RMSE and MAE. Beyond that, only nine additional models—two for dissolved oxygen, five for ammonia, and two for pH—achieved R² values above 0.3, a low but still noteworthy threshold. Even these relatively better-performing models offered limited real-world predictive value.

We also considered whether any model might merit further collaboration or refinement. However, based on performance against the validation dataset, we did not identify any that were sufficiently promising to pursue at this stage.

Full Validation Report

A full report of our analysis, including all of our data (with farm GPS coordinates and model owners anonymized), may be found here:

Innovation Challenge Version 2 Full Report

Ground truth data compared to predicted data for DO values (mg/L) for one of the submitted models. This was one of the more accurate models for DO, but unfortunately not accurate enough to be operationally useful. More information can be found in our full report.

Path Forward: In-House Development and Data Campaign

While these results fell short of what we had hoped for, they are not entirely surprising. Developing accurate satellite-based models for the specific water systems and welfare-relevant parameters we monitor is an exceptionally difficult technical challenge—something underscored by the 202 models submitted across two Innovation Challenge rounds. It remains possible that this simply isn’t achievable with current satellite technology.

Given these severe challenges, to mitigate the risk of failure, we intentionally pursued two parallel development tracks:

• The external Innovation Challenge V2

• In-House Model Development

We now turn our efforts to the second track—our in-house model development effort—which likely represents our final planned attempt to determine whether remote sensing is currently viable for the ARA program.

In-House Model Development

The second track is an internal effort led by our recently recruited Remote Sensing Lead, Dr. Solomon (Sol) White. Since joining in August, Sol has evaluated the suitability of our existing ARA dataset for training models. Although the dataset is large—spanning more than 200 farms since 2021—two limitations reduce its usefulness for remote sensing:

• Sampling location: Most measurements were taken from pond edges (for logistical reasons). Edge sampling introduces land–water mixing effects, complicating satellite analysis. (We recently published results on how within-pond and edge sampling differ.)

  
  

• Timing mismatch: Sentinel-2 satellites pass over Andhra Pradesh at roughly 10:30 a.m., whereas ARA sampling typically occurs early morning or evening. This is especially problematic for dissolved oxygen, which fluctuates substantially throughout the day and requires close time alignment.

  
  

These issues do not diminish the operational value of the dataset, but they do constrain its suitability for model training.

The Data Campaign

To address the limitations of our historical dataset, we have launched a dedicated data campaign designed specifically for remote sensing model development. This effort began on November 20, 2025, and is structured to generate the kind of high-quality, time-aligned data that satellite-based modeling requires.

The campaign includes several major upgrades:

• Within-pond sampling, rather than pond-edge measurements

• Field sampling coordinated with Sentinel-2 flyovers, improving temporal alignment

• Continuous monitors for dissolved oxygen, temperature, and pH, enabling constant data collection rather than single daily readings

• Data collection across five regions in Andhra Pradesh, increasing geographic representativeness

  
  

Across the campaign period, we expect to collect:

• 200+ samples each for dissolved oxygen, pH, temperature, and ammonia

• 100+ samples for chlorophyll-a

  
  

Executing this campaign will require tight coordination across teams over the coming 2–3 months, but we believe it is a necessary investment. If, after incorporating this improved dataset into our in-house modeling effort, remote sensing still fails to produce operationally useful models, we will likely conclude that the technology is not yet viable for our purposes. For now, however, we remain optimistic that this campaign will give us the clearest possible answer.

Two continuous water-quality monitors from different suppliers during our two-week accuracy trial. The evaluation is now complete, and we’re working with the selected company to procure the remaining 15 units for the data campaign.

Acknowledgements

We’d like to express our sincere gratitude to the 39 individuals/groups who participated in this Innovation Challenge. Building models for this setting is a significant technical challenge, and we deeply appreciate the time, expertise, and care you invested. Your contributions helped us better understand the current feasibility of remote sensing for aquaculture—thank you.