Hands-on Entity Resolution with Python-v3.png

dockerfile.tar.gz

conda-main

A typical business stores data across multiple systems, including ERPs for operations, a CRM for marketing, files, notebooks, and BI apps for other purposes. Records of the same customer (entity) exist in multiple places, likely not in sync across nor unique within sources. This inconsistent situation generates an opportunity for us to drive business value by cross-referencing and deduplicating records with entity resolution.

This course covers business acumen and hands-on coding. It starts with several business cases and a quick introduction to entity resolution in Python. Then, it explores semantic-preserving preprocessing, similarity feature engineering, graph clustering, weak supervision, confident learning, and integration.

As a developer, you’ll increase your company’s business value by developing and deploying entity resolution pipelines. As a decision-maker, you’ll know which solution best suits your business cases and how to negotiate the best value for your money.

An Introduction to Entity Resolution in Python

It reduces the number of extreme outliers in record attributes.

It reduces the number of missing or incorrect join-key relations.

It reduces the number of wrong attributes.

How does a pairwise matching model directly contribute to data quality?

How does a pairwise matching model directly contribute to data quality?

A perfect score of 1 because the case does not matter.

A score of 0.5 because of different cases but the same letters score with half the possible similarity.

A score of 0 because none of the characters from the first match with the second.

It depends on the optional case penalty parameter.

What is the similarity between "HELLO" and "hello" measured with Damerau-Levenshtein?

What is the similarity between “HELLO” and “hello” measured with Damerau-Levenshtein?

The accuracy takes into account only a subset of all labeled predictions.

Optimizing by accuracy in scenarios with a strong class imbalance can mislead into very accurate but useless models.

None of the measures are practicable because real-world scenarios have no true labels.

The accuracy is not problematic when the index—all candidate pairs— has a balanced share of match and no-match pairs.

The accuracy of a classifier is the number of correct predictions divided by the number of all predictions. What is problematic about using accuracy to evaluate the performance of entity resolution models?

The accuracy of a classifier is the number of correct predictions divided by the number of all predictions. What is problematic about using accuracy to evaluate the performance of entity resolution models?

Precision is the number of correct match predictions divided by the number of actual matches.

Recall is the number of correct no-match predictions divided by the number of actual no-matches.

Recall is the number of correct match predictions plus wrong no-match predictions.

Precision is the number of correct predictions divided by the number of all predictions.

Which definition of model performance measures is correct where we define the positive class as a match and the negative as a no-match?

Which definition of model performance measures is correct where we define the positive class as a match and the negative as a no-match?

The index consists of pairs of record keys, indicating all known duplicates.

The index length is always larger than the total number of the $n$ available records.

There is an upper limit for the length of the index.

To avoid duplicate computing efforts, the index length should be no more than $k=\frac{n\times(n - 1)}{2}$.

Which statement about the index of a deduplication task with the $n$ records is true?

Which statement about the index of a deduplication task with the <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>n</mi></mrow><annotation encoding="application/x-tex">n</annotation></semantics></math>n records is true?

Evaluate your acquired knowledge about entity resolution fundamentals.

Introduction to Entity Resolution and Applications

A Quickstart Guide Using the RecordLinkage Package

Preprocessing

Indexing

Feature Engineering

Pairwise Matching

Clustering

Integration

Entity Resolution Fundamentals

Matching Products Across Two Online Shops

Conclusion

Appendix

Quiz on Entity Resolution Fundamentals