Unlocking Environmental Insights

A Policy-Driven Data Platform Transforming Access to NDFF’s Biodiversity Data

 

The Dutch Database of Flora and Fauna, Nationale Databank Flora & Fauna, (NDFF) manages one of the Netherlands’ most comprehensive collections of biodiversity data. This is a vast resource, compiled from countless disparate sources – researchers, nature organizations, provinces, companies, and more – resulting in a dataset with inherent complexity, varying formats, structures, and sensitivities. Facing a recent governmental mandate to make key portions of this data publicly accessible, NDFF encountered a significant challenge: how to transform this intricate landscape of information into a secure, compliant, and easily usable public resource? We partnered with NDFF to design and implement the sophisticated data platform required to navigate these technical, policy, and stakeholder complexities, effectively unlocking the immense potential of their data to fuel environmental insights and inform decision-making across the nation.

Partnership & Process: Working Together for Success

Delivering a complex data platform under public policy requirements necessitates synthesizing knowledge from many experts. Beyond our own technical expertise, success hinged on effectively collaborating with NDFF’s internal team and a range of external stakeholders – those deeply familiar with the nuances of the biodiversity data itself and those planning to build critical applications utilizing the platform. Our process was built around actively engaging these different groups from the outset. We established iterative feedback loops, transforming valuable input on data structure, platform requirements, and validation needs into concrete development steps. By creating a structured environment for these interactions, we ensured that diverse expertises were taken into account for the platform’s design and build, mitigating potential misunderstandings and driving toward a solution that met real-world data generation, usage, and validation demands. This inclusive and responsive approach was key to navigating the project’s complexities.

 

Building the Data Platform: Technical Implementation

Translating policy requirements and diverse data sources into a functional, public-facing resource demanded a robust technical core. We designed and implemented a scalable data platform infrastructure capable of handling the substantial volume and variety of NDFF’s biodiversity observations. The architecture centers around a reliable data pipeline: securely ingesting data from multiple sources, performing necessary transformations and harmonization based on established data engineering best practices, and storing the processed data in a query-optimized format. This foundation provides the backend for applications like the NDFF Verspreidingsatlas. Key technical considerations included ensuring the platform’s security, maintaining high data quality through automated checks, and building for scalability to accommodate future growth in data volume and user access – establishing a solid technical base for current and future use cases.

Measurable Results and Policy Compliance

Meeting the governmental mandate for public data access was the key objective, but the true success lies in how this project transformed NDFF’s data capabilities and the value unlocked for the public. We delivered a high-quality, robust data platform that fundamentally changes how NDFF manages and shares its critical biodiversity information.

For the public, this means the large, nationally important dataset is now genuinely accessible and reliable. The platform directly enables the powering of key public data interfaces, fulfilling the promise of increased transparency and enabling informed decision-making across research and policy based on credible, governed data.

For NDFF, the transformation is equally significant. What was once complex data management has become a streamlined process backed by:

  • Operational Efficiency: Achieved through embedded automation and improved data transparency.
  • Data Authority: Guaranteed through rigorous version controlling and comprehensive data governance, building confidence in data quality.
  • Strategic Capacity: Providing a scalable, dependable foundation built for handling complex transformations and equipping NDFF to proactively pursue numerous future data initiatives.

The resulting platform is a testament to effective implementation, delivering immediate compliance and public benefit while providing NDFF with a lasting asset for future data strategies.

Screenshot User Interface of the web application

“Aurai’s data engineers make an essential contribution to the further development of the NDFF. Thanks to their expertise, we are structurally improving data quality and availability.” – Ivo Eggink, adviseur Informatievoorziening

LET’S TALK!
MORE CASES