This repository provides access to the Checks by Monitoring (CbM) project code and technical documentation developed by the Joint Research Centre (JRC) to support Member States (MS). Here the Guidance and Tools for CAP (GTCAP) JRC group shares a CbM system layout and code examples to demonstrate how Paying Agencies (PA) can process and use Sentinel data to check aid applications for the common agricultural policy (CAP). The JRC CbM is based on a cloud infrastructure solution that is modular and built exclusively on open source components.
The description of the system components and the code examples is available in the JRC CbM TECHNICAL DOCUMENTATION. In this documentation you can also find instruction on how to set up a CbM system.
An introduction to the project, with an overview of its architecture and scope, is provided in the JRC CbM GENERAL DOCUMENTATION.
JRC CbM ARCHITECTURE is made of two layers: a BACKEND SERVER that provides the end-points to retrieve data and includes the physical infrastructure and the routines that generates the information used by the analysts and the decision makers; and a FRONTEND COMPONENT that is manipulated by the user and provides access to the data generated by the backend through standard Application Programming Interface (API).
In this repository we share the code developed for both the backend and the frontend, structured according to three levels:
- Setting up the infrastructure (target users: system administrators)
- Develop analytical functionalities (target users: analysts)
- Apply the functionalities (target users: final users)
The scripts and documentation are organized in the following sections/subfolders:
- api: Modules to build a RESTful API for CbM
- cbm: CbM Python library
- docker: Docker image files
- docs: Documentation pages with example codes
- ipynb: Jupyter Notebook examples
- scripts: Python scripts for parcel extraction routines and time series calendar view
- tests: Test scripts for generic functionalities
In the framework of the Outreach project, a cloud infrastructure (based on CreoDIAS) to experiment the functionalities of the system has been created by GTCAP with the backend component developed and managed by JRC. MS can use dedicated API to explore and analyse Sentinel data extracted for they declared parcels. In this project, users do not have to install anything and can run the example code for data retrieval and manipulation stored in this repository, particularly using Python and Jupiter Notebooks. Instead, to start a dedicated CbM system for a PA is essential to have:
- Computing resources
- Copernicus Analysis Ready Data (CARD) (Sentinel 1 and 2 data for the study area)
- Agricultural parcel data (typically, declared parcels from the Land Parcel Identification System (LPIS) and the Geospatial Aid Application (GSAA))
The first two requisites can be achieved using one of the five Copernicus Data and Information Access Services (DIAS) available (CREODIAS, WEKEO, SOBLOO, MUNDI, ONDA).
There are several steps to set up the core components for CbM that require different types of technical expertise.
- Setup server applications
- Docker (containerization system)
- Postgres database with PostGIS extension
- Jupyter (interactive analysis and visualization environment)
- Restful API (intermediate layer to access and use Copernicus data and the database)
- Adding data to the database
- Parcels data
- CARD Metadata and other setting data
- Process Sentinel data to derive relevant information
- Parcel stats extraction routines
- Machine learning algorithms
- Analytical routines (e.g. markers detection)
- Analyzing and reporting
- Using Jupyter Notebooks to explore and analyze data
- Generate reports and other outputs to classify aid applications
This repository is open for contribution, particularly from PA experts. Please read "Creating a pull request" for details on the process for submitting pull requests.
- Guido Lemoine - glemoine62
- Konstantinos Anastasakis - konanast
See also the list of contributors who participated in this project.
This project is licensed under the 3-Clause BSD license - see the LICENSE file for details.
Copyright (c) 2021, European Commission, Joint Research Centre. All rights reserved.