Speaker
Description
One of the core databases in the European Central Bank is the RIAD (Register of Institutions and Affiliates Data) register, a massive repository of 16 million counterparties, covering companies, institutions, funds and much more.
One of our main task is to make sure that every entity refers to a different counterparty and no duplicates are generated. While not every record is a duplicate, every new entry must be checked against a huge dataset to ensure data integrity. Legacy string-matching algorithms and pre-defined checks based on the reported attributes served us well so far, but are becoming somehow obsolete in the face of modern scale and semantic complexity.
Key challenges in this domain include semantic ambiguity and logical consistency. For instance, how do you teach a system that "Ferrari S.p.A" and "Ferrari F.Lli Lunelli S.P.A (In Breve Ferrari S.P.A.)" may sound similar, but one is a famous car producer while the other one produces wines and they are very distinct companies? And how do you then make it realize that "HP" and "Hewlett-Packard" refer to the same company despite looking different? Furthermore, how do you prevent "chain reactions" of bad matches, where a single error bridges two distinct corporate groups into one large incorrect cluster? A modern approach to address these is Deep Entity Matching (DEM) combined with Graph Theory.
In this lecture, we will explore how to move from simple text comparison to a "Two-Pass" Neural Architecture. We will cover the use of efficient blocking strategies to narrow the search space, Cross-Encoders for high-precision semantic checks, and the application of Graph Topology to detect and cut illogical links in the data. The aim of this session is to demonstrate how High-Performance Computing can be used to solve critical data infrastructure problems in a wide variety of contexts.
| Number of lecture hours | 1 |
|---|---|
| Number of exercise hours | 0 (no exercises) |
| Attended school | tCSC 2025 Security (Abingdon) |