Netrics HD

In the recent New York Times article 6 Considered Threats Kept Licenses for Aviation, Matthew Wald reported on how the TSA failed to match suspected terrorists against their watch lists even though two of them were on the Federal Bureau of Investigation’s Ten Most Wanted List.

A small database company called Safe Banking Systems was able to easily identify these suspected terrorists using “fuzzy logic” in their data matching techniques against publicly available watch lists used by banks to scrub lists of customers for potential links to terrorism.

David M. Schiffer, the president of Safe Banking Systems explained that exact matching techniques are ineffective because:

“This data’s dirty.  People have typos, misspellings, and the data gets truncated or entered in the wrong field.”

This is the Universal Rule of Structured Data (URoSD) – data is never perfect, and it never can be! So we must build solutions that can function correctly given this immutable fact.

Effective data matching in the context of URoSD faces the challenging possibilities of both false positives (records identified as a match that do not represent the same entity) and false negatives (records that do represent the same entity but were not identified as a match).

Considerable concerns are often raised about the possibilities of too many false positive matches against terrorist watch lists.

This concern is also often cited in data matching involving data and applications not critical to national security where “under-matching” is often implemented because of the perceived higher negative impact of a false positive.

However, when the false negatives are potential terrorists planning an attack, this is simply unacceptable.

Data matching in the context of imperfect data is critically important and stories like this one demonstrate how vital data matching truly is.

Related Posts

TSA “Secure Flight” will require more demographic information

A Sisyphean Task…

What’s in a Name?

What’s in a Name? (Part II)

For those of you who think data matching is an esoteric topic only considered interesting by the techno-geeks with pocket protectors and taped glasses wandering the hallways of your IT department, a recent article in The New York Times begs to differ.

In Flying? Don’t Book Under a Nickname, Susan Stellin reports on the Secure Flight program of the Transportation Security Administration (TSA) and its struggles with checking airline passenger names against terrorist watch lists.

TSA is not only recommending that you book your ticket using your full name but also make sure that your name is consistently represented across all your identifications (driver’s license, passport, frequent-flier account). And the TSA will begin requiring airlines to collect birth date and gender for all passengers.

The goal is to improve the process of clearing airline passengers for travel (i.e. matching against terrorist watch lists) while properly identifying individuals that require additional screening.

If you have ever been bored sitting in a meeting about how data matching differentiates potential matches and confirmed matches, then consider the fact that potential matches against a TSA watch list will be kept on file for 7 years and confirmed matches against a TSA watch list will be kept on file for 99 years.

Does data matching suddenly sound much more interesting?

Although it is encouraging to hear that the TSA Secure Flight program is attempting to improve both the data quality of the watch lists and the data matching against those lists, it is disappointing to read that the approach appears to be get airline passengers to standardize their names so that exact matching can be used.

Moreover, does anyone believe that these new requirements will thwart determined terrorists? I certainly don’t… if they can plan an attack, they can certainly make sure their (false) identities are consistent.

What do you think?

Related Posts

A Sisyphean Task…

What’s in a Name?

What’s in a Name? (Part II)