Speaker
Description
Recent advances in satellite technology have led to a regular, frequent, and high-resolution monitoring of Earth at global scale, providing an unprecedented amount of Earth observation (EO) data. The growing operational capability of global Earth monitoring from space provides a wealth of information on the state of our planet Earth that waits to be mined for several different EO applications, e.g., climate change analysis, urban area studies, forestry applications, risk and damage assessment, water quality assessment, crop monitoring, etc. A growing number of cloud-based EO data access and processing resources (such as the five Copernicus Data and Information Access Services (DIAS) platforms) have become available. These platforms allow users (e.g., EO application and service developers, space agencies, space industry, the science community and the general public) to search for satellite images required for the EO applications of interest, by using keywords/tags in terms of sensor type, geographical location and data acquisition time of the satellite images stored in the archives. Thus, a growing need for accurate and scalable techniques for satellite EO images understanding, search and retrieval from the massive archives (e.g., Copernicus archives) has appeared. However, in the era of big data, the semantic content of the satellite data is much more relevant than the keywords/tags. To keep up with the growing need of automatization, image search engines that extract and exploit the content of the satellite images are necessary. In other words, it is emerging the need of being able to go beyond the traditional query of EO data catalogues based on technical image metadata (location, time of acquisition, technical parameters), and enrich the semantic content of image catalogues enabling a brand new class of query possibilities powered by the combination of Natural Language Processing (NLP)to understand the query and to describe the content of the data, and Computer Vision (CV) to massively annotate data and implement multi-modal text-to-image and image-to-image searches.
In such a context, the ESA funded Digital Assistant for Digital Twin Earth project aims at developing a precursor of a Digital Assistant able to address those needs through NLP and CV cutting-edge AI technologies applied to EO. Through the Digital Assistant the user will be able to search satellite images using natural language processing (both describing semantic features of the image or giving information about location/time) or search semantically similar images. Furthermore, using NLP the user will be also able to ask direct questions on images or ask for features extraction, in addition to task a future acquisition. In this way, the Digital Assistant will simplify satellite data search and retrieval, and open the use of satellite data also to non-expert users that could benefit from it. Furthermore, it will enable the extraction of additional value information from the satellite data for every type of user, through AI models to extract features of interest, using a simple NLP query. Finally, the Digital Assistant will have a conversational AI able to keep the conversation with the user in order to simplify even more the use.
