How would you implement address normalization to canonicalize abbreviations and remove diacritics?

How would you implement address normalization to canonicalize abbreviations and remove diacritics?

• A. Build a normalization pipeline: Unicode normalization, diacritic removal, standardize abbreviations via a mapping, and then store both normalized and original forms for search.
• B. Only convert to lowercase and trim whitespace.
• C. Remove all non-letter characters and convert to uppercase.
• D. Do not perform normalization; rely on fuzzy matching only.

Answer: (A)

Focusing on a robust normalization pipeline that makes addresses comparable across variations is the key. Start with Unicode normalization to ensure characters that can be represented in multiple ways are treated consistently, then remove diacritics so accented characters don’t block matches from sources that don’t use them. Next, standardize abbreviations with a mapping, turning abbreviated forms like St., Ave., Blvd. into a single canonical form such as Saint, Avenue, Boulevard. Storing both the normalized form and the original form gives you reliable searchability without losing the original, human-friendly address for display.

This approach is best because it handles both diacritic differences and abbreviation variants in a controlled, extensible way, improving deduplication and lookup accuracy while preserving readability. Other options miss important pieces: lowering and trimming alone doesn’t address diacritics or abbreviations; removing non-letter characters and uppercasing is too destructive and can erase meaningful structure; and not performing normalization at all leaves many variations unmatched.