Gain insights into entity resolution in Python, covering business use cases, semantic preprocessing, graph clustering, and weak supervision. Increase business value through hands-on coding and strategic decision-making.

Hands-on Entity Resolution with Python-v3.png

dockerfile.tar.gz

conda-main

A typical business stores data across multiple systems, including ERPs for operations, a CRM for marketing, files, notebooks, and BI apps for other purposes. Records of the same customer (entity) exist in multiple places, likely not in sync across nor unique within sources. This inconsistent situation generates an opportunity for us to drive business value by cross-referencing and deduplicating records with entity resolution.

This course covers business acumen and hands-on coding. It starts with several business cases and a quick introduction to entity resolution in Python. Then, it explores semantic-preserving preprocessing, similarity feature engineering, graph clustering, weak supervision, confident learning, and integration.

As a developer, you’ll increase your company’s business value by developing and deploying entity resolution pipelines. As a decision-maker, you’ll know which solution best suits your business cases and how to negotiate the best value for your money.

An Introduction to Entity Resolution in Python

It replaces the "ä," "â," and "à" characters with a plain "a."

It removes text wrapped in open and closing brackets of the same kind, including the brackets.

It removes all nonalphanumeric characters, excluding single spaces, by an empty string.

It detects and truncates outliers of numeric columns.

The default behavior of the `preprocessing.clean` function of `recordlinkage` takes care of the following:

The default behavior of the <code>preprocessing.clean</code> function of <code>recordlinkage</code> takes care of the following:

The `s.str.contains('[a-zA-Z0-9]{3,}')` method returns a series of booleans: it is `True` if the corresponding string contains a sequence of three alphanumeric characters, otherwise it is `False`.

The `s.str.replace('-')` method replaces all dashes with an `''` empty string.

The `s.str.extract('[a-zA-Z0-9]{3}')` method returns a series—the sequence of three alphanumeric characters appearing first in the corresponding string or a missing value if there is no such sequence.

The `s.str.match('[a-zA-Z0-9]{3,}')` method returns a series of booleans: it is `True` if the corresponding string contains a sequence of three alphanumeric characters, otherwise it is `False`.

The `pandas` library uses the `re` module from Python’s standard library to deal with regular expressions. Which statement is true when applied to an `s` series of strings?

The <code>pandas</code> library uses the <code>re</code> module from Python’s standard library to deal with regular expressions. Which statement is true when applied to an <code>s</code> series of strings?

Removing stopwords before computing TF-IDF scores is irrelevant because stopword scores are down-weighted by IDF to practically zero.

The dimensions of the matrix of numeric scores correspond to the texts (rows) and the tokens (columns). We can set tokens as words—substrings separated by space—or n-grams, fixed-length sequences of characters.

The score of a text-token tuple is an increasing function of the term frequency of that token in that text and a decreasing function of the document frequency of the token across all texts.

TF-IDF is based on token counts. It can only consider tokens for scoring covered by the training dataset.

The scikit-learn library's `TfidfVectorizer` transforms an array of texts into a matrix of numeric scores. Which statement is false?

The scikit-learn library’s <code>TfidfVectorizer</code> transforms an array of texts into a matrix of numeric scores. Which statement is false?

OSM's service level agreement ensures the service is reachable for a guaranteed percentage of the time.

The open-source Nominatim geocoder is best-in-class, using Elasticsearch under the hood.

The huge community contributes countless data points each day. OSM is the most comprehensive source for locations across all regions and domains.

If we attribute correctly, we can store, process, and share OSM data freely.

What is a key benefit of using OSM based geocoders compared to its proprietary competitors like Google Maps?

What is a key benefit of using OSM based geocoders compared to its proprietary competitors like Google Maps?

Metaphone is built on rules for encoding English pronunciation.

Double Metaphone and Beider-Morse account for pronunciation ambiguity for words of foreign origin.

Double Metaphone works particularly well for Eastern European languages.

Beider-Morse covers more languages than Double Metaphone.

Which statement about phonetic algorithms is wrong?

Which statement about phonetic algorithms is wrong?

Evaluate your learnings about preprocessing for entity resolution.

Introduction to Entity Resolution and Applications

A Quickstart Guide Using the RecordLinkage Package

Preprocessing

Indexing

Feature Engineering

Pairwise Matching

Clustering

Integration

Entity Resolution Fundamentals

Matching Products Across Two Online Shops

Conclusion

Appendix

Quiz on Preprocessing