When Rigorous M&E Meets Real-World Readiness: Piloting Digital M&E in Stunting Reduction

In the development sector, rigorous monitoring and evaluation is often treated as non-negotiable. Donors expect measurable impact. Governments demand accountability. And with the surge of digital tools—chatbots, apps, dashboards—the promise of real-time, high-frequency data has never felt more within reach.

But what happens when a digital measurement system outpaces real-world readiness?

Over the past year, our experience piloting SAKTI chatbot helped us examine this gap. SAKTI is a WhatsApp-based AI chatbot used within the ZeroStunting initiative to track egg consumption and deliver behavioral nudges. Rather than offering a single conclusion, the pilots highlighted how even best-practice digital M&E designs need to be tested, adapted, and sometimes scaled back before they can be scaled up.

The Promise: Digital Tool That Does It All

SAKTI was initially built to address a clear operational bottleneck. Field offices and Posyandu cadres (kader) were spending significant time tracking daily egg consumption. Every day, caregivers had to report via WhatsApp, kader had to verify photos, sent it to our field officers and then had to manually compile data into spreadsheets. It was time-consuming and unsustainable.

SAKTI automated this process, saving over 500 staff hours in our first six-month pilot. But we wanted to go further. If we already had a digital channel reaching caregivers, why not use it for more, to track learning and behavior change?

So, we expanded the tool. We piloted weekly educational nudges: short messages with links to caregiving content, and monthly quizzes with pre- and post-tests to assess knowledge increase and prove our digital education component was working. In theory, if effective, this would provide a scalable and efficient M&E approach. Testing this assumption was part of the pilot’s purpose.

The Reality Check: What Piloting Revealed

Results varied across sites. In Kalipare, Malang, quiz participation was high in a one-month-testing, reaching 82%, alongside increased reporting of egg consumption. In Turen, another sub-district in Malang, when we tried to do 6-month pilot, participation declined over time: by the second month, only 34% of participants completed both pre- and post-tests, with completion more common among caregivers with higher baseline knowledge. On average, daily egg consumption report completion reached ~74%.

Follow-up surveys with non-compliance participants pointed to familiar constraints. Caregivers cited limited time, connectivity and data limitations, and message fatigue. These patterns were consistent with broader challenges of digital engagement in low-resource settings, rather than issues with access to smartphones per se, as smartphone ownership and social media use were already widespread in the areas.

At the same time, face-to-face workshops produced substantially knowledge gains than the digital quizzes. Yet the digital channel still showed signs of engagement: among

caregivers who did not complete quizzes, many reported read the messages and click through the link sent, even if they did not actively respond.

The Learning: What Digital Tools are Actually Good For

Taken together, the pilots points to SAKTI functioning more reliably as a low-barrier reminder and improve than an instrument to measure knowledge change. This does not imply digital or AI-enabled approaches are ineffective for nutrition education; rather its competitive advantage lies in reinforcing behavior and enabling timely action. Additionally, the short quizzes promote curiosity of parents to click to articles and use AI-enabled function in the WhatsApp chatbot.

This was reflected in operational outcomes. Posyandu attendance increased from 88% to 97% in Kalipare, in part because cadres could use SAKTI dashboards to identify families needing follow-up. Reducing operational burden and improving targeting for frontline workers could deliver comparable, and in some cases greater, gains than expanding data collection requirements for caregivers.

These findings prompted a clearer matching of tools to purpose:

- Knowledge measurement is better suited to face-to-face settings where participation can be supported and misunderstandings addressed.

- Behavioral nudging can be effectively delivered through digital channels that do not require frequent active responses.

- Impact measurement should prioritize observed behaviors and health outcomes, rather than digital completion metrics alone.

We also moved away from requiring digital-only compliance, allowing paper-based records to complement chatbot data. While this reduced data uniformity, it improved representativeness.

What This Means Beyond One Program

As digital tools become increasingly common in development programs, the key question is less about “Can we build this?” , and more about

“Does this tool reduce burden—or quietly create new ones? And is it flexible enough to survive real-world messiness?”.

From our piloting, three considerations stand out:

1. Digital M&E systems need to function despite lapses in connectivity, attention, and participation.

2. The ability to collect data should not be confused with the necessity of collecting it.

3. Pilots should be designed with adaptation in mind; mixed or declining engagement is not failure, but information.

The future of digital M&E isn't about building one perfect system. It's about building adaptive systems that meet communities where they are—in their digital literacy, their bandwidth constraints, and their daily realities. And that requires piloting with humility, measuring what matters, and accepting that sometimes "imperfect" data gives you a more honest picture of impact.

This article is written by Melinda Mastan

Back To List