The Key to Credible Impact Measurement?

Imagine a world where every data point tells a story—of impact, progress, and change. But how do we know these stories are real? How can impact-driven organisations prove their efforts are making a difference? What if the numbers they rely on are flawed? Can we trust impact measurement when the data behind it is inconsistent or incomplete? And if data quality isn’t guaranteed, what does that mean for the future of meaningful, measurable change? Let’s dive into this, trying to answer the question

Is data quality just optional or existential for impact measurement and consequently for achieving sustainable change?

We interviewed xy impact investing funds about their data quality issues. Here is what they told us

The main feedback in conversations with impact investors is that data quality and availability is the biggest obstacle in impact measurement and reporting. After we heard the complaints over and over again we started a learning session as part of our R&D work to understand what that actually means. We talked to x investment funds that manage a total of x assets under management in impact investments.We asked openly “We often heard that data quality is bad. What exactly is the problem with data quality?” Here summary of the findings (anonymised), add a quote from our notes

What Makes Data “High-Quality”? A summary on what science has to say about it.

In general, data quality is multidimensional. According to multiple studies (e.g. Batini et al. (2009)), it encompasses six dimensions: Completeness, Accuracy, Consistency, Validity, Uniqueness, Timeliness.
Building up on those, we started research on data quality criteria especially with regards to impact measurement. Based on our results we propose the following criteria as the most relevant for collecting high-quality data:
‍

1. Data Consistency: Enabling Comparability & Scalability
   Standardised definitions and collection methods enable organisations to track progress accurately and compare impact across regions. Without consistency, data becomes fragmented and misleading.
   Example: A social enterprise focusing on women’s empowerment uses surveys with inconsistent definitions of “employment” across regions. In one region, beneficiaries report any paid work, while in another, stable jobs ≥3 months are counted. Mismatched data makes cross-regional impact comparisons impossible.
   Solution: Standardise survey questions (e.g., by using acknowledged impact indicators) and train staff to apply them uniformly.
   
   
2. Data Representativity: Ensuring holistic and inclusive Decision-Making
   Representative data captures diverse populations, preventing biases that exclude marginalised groups. Without it, critical issues go unnoticed, reinforcing inequalities.
   Example: A climate resilience survey by a GreenTech-Startup is distributed via mobile app to farmers, excluding those without smartphones (e.g., elderly, low-income groups). Data overrepresents tech-savvy farmers, skewing funding toward digital tools instead of low-tech irrigation needs.
   Solution: Use hybrid surveys (mobile + in-person interviews) to include offline populations to ensure that all voices are heard.
   ‍
3. Data Integrity: Building Trust & Accountability: 
   Maintaining data accuracy and security prevents errors, fraud, and misinformation. Reliable data strengthens stakeholder confidence, reduces the risk of misallocated aid, and ensures compliance with privacy regulations. Safeguarding integrity is essential for ethical and transparent decision-making.
   Example: A nonprofit conducts a digital survey on program satisfaction, but many participants rush through it, selecting the first or most positive option without reading the questions. This leads to overly optimistic data, masking real issues and resulting in misguided program improvements.
   Solution: Use randomised answer orders and attention-check questions to ensure participants engage thoughtfully, reducing the risk of rushed or biased responses.

Why Does Data Quality Matter for Impact Measurement?

Impact measurement is only as credible as the data fueling it. Poor-quality data isn’t just a technical hiccup—it’s a threat to mission success, donor trust, and societal progress. We recognise that data quality is the foundation of holism, comparability, trust, and accountability in impact measurement. Without accurate data, efforts to assess social and ecological outcomes become unreliable, leading to misguided decisions and wasted resources.

How Data Quality Drives Social Impact

Reliable data ensures that organisations can measure progress effectively, adapt or pivot strategies, and maximise impact. Poor data, on the other hand, distorts reality, perpetuates inequities, and undermines stakeholder trust. Robust data governance, continuous monitoring, and advanced analytics enable organisations to uncover actionable insights, ensuring that social initiatives are truly impactful.

In practice, transforming your organisation’s data quality begins with clear, actionable steps. For social enterprises, consider these four concrete measures:
‍

1. Establish a Solid Foundation:
   Utilise standardised and widely acknowledged indicators to ensure comparability across datasets. Concentrate on identifying and tracking the most important key performance indicators (KPIs) that align with your mission.
   ‍
2. Implement Robust Data Collection Processes:
   Integrate comprehensive data collection methods into your day-to-day operations and customer success management. This approach ensures that every interaction is captured accurately, minimising manual errors and enhancing data consistency.
   ‍
3. Utilise the Right Tools for Data Processing:
   Invest in a robust digital data pipeline and adopt best practices in data management. Employ automated tools and establish rigorous quality checks to detect and correct inconsistencies in real time, ensuring your data remains reliable.
   ‍‍
4. Incorporate Data into Management Meetings:
   Regularly bring data insights into your strategic discussions. By reviewing results frequently, you can make actionable decisions, promptly address any discrepancies, and continually refine your approach for improved outcomes.

New technologies like machine learning can predict, audit, and resolve data quality risks, but human oversight is essential to prevent inequities. Let’s have a look at how, through continuous R&D, we have developed a great solution for our customers.

‍

Leonardo’s automated data quality audit: Using AI to Ensure Data Quality

At Leonardo, we use Deep Impact Intelligence to run AI-powered audits, ensuring data quality and reporting credibility.
Our data science team under the lead of Alan Sicart has developed the first version of leonardos data quality audit.

‍

Conclusion

Impact measurement isn’t just about collecting data—it’s about ensuring that data reflects reality. By prioritising data quality, social organisations can unlock funding, strengthen donor trust, and influence policy, transforming raw data into powerful insights that drive sustainable, scalable change. High-quality data empowers organisations to align actions with values, inspire collective action, and drive systemic change. Thus it is existential for impact measurement and consequently for achieving sustainable change.