The Monk Factor


I came across this recent article in BtoB Magazine

BtoB Magazine: “Marketers: Clean customer data a priority in 2008
By Carol Krol
March 17, 2008

It’s great to see that simply collecting customer data is no longer good enough - the data needs to be usable to benefit the business.

Refining customer data quality and access to customer data have emerged as two of the top marketing investment priorities of b-to-b CMOs this year. Half of b-to-b marketers plan to put more resources against creating marketing databases, cleaning up customer data, improving sales force automation and CRM integration, according to Forrester Research in its “B2B CMO Investment Priorities for 2008” report.

The article references a recent Forrester Research report, a survey released by Alterian, and quotes several marketing executives at large well-known enterprises - all agree that something must be done. Dealing with imperfect customer data is now a top priority for 2008. Finally!

Now that we all agree on the problem, the question is how should this problem be solved? Of course, it’s often a multi-facetted, cross-organizational solution that will always involve Technology, Process, and People (TPP). But all three components of TPP depend on each other - and any one of the three can be the weak link in the chain.

But in most cases, the weakest link is Technology - which is why more People are thrown at a problem, which in turn requires more Process to keep everyone working together efficiently. Then you get the snowball effect and the solution grows out of control and takes on a life (and expense) of its own.

So, the argument is if we can improve the core Technology, then we can reduce the People part and streamline the Process part as well. The TPP solution will stay nicely under control and perform as it’s expected to perform - while keeping costs in check.

Which leads to this wonderful quote from the article:

“Data always degrade, get dirty, become obsolete or old. You need a department of 20 people like Adrian Monk [from the TV show “Monk”] who live to keep things organized.”

Brett Butler
Director of Global Sales and Marketing Practices, Lexmark International

Yes, I agree whole-heartedly - the nature of customer data (in fact, almost all database data) is that it is (1) never perfect and (2) constantly changing in ways that change.

But needing a department of twenty Adrian Monks? This is the result of Technology failing to delivery and thus requiring the Monk department to compensate.

So I’m curious, what is your organization’s Monk Factor?

The point is that with Technology pulling its weight, the Monk Factor can be very low - but it’ll never be zero. Moreover, the People part shouldn’t require that everyone have those extra-special Monk skills. After all, Adrian Monk is a genius - you can’t build a successful solution to a business problem if you require twenty Monks here and twenty Monks there… the solution won’t scale and neither will your organization. And by the way, don’t expect your Adrian Monks to complete Sisyphean Tasks either!

Use your Monks strategically and leverage the heck out of them - that approach will bring the most benefit to your organization.

The right innovative Technology can help you do just that.

A Sisyphean Task…


It never ceases to amaze me just how much (successful) brainwashing is out there… a common misconception is that the way to deal with the pervasive nature of “imperfect data” is to somehow magically keep all of the data “perfect” all of the time.

Like Sisyphus, we’d up end never ever reaching our goal.

Instead, what we need to focus on is enabling software systems that can handle imperfect data - while still doing what we can to keep the data in reasonable shape. This is the solution to strive for - and it really is independent of the software that Netrics makes. Of course, I believe that our software delivers the best such solution but that’s beside the point.

High Fidelity Data

Some folks asked if we watch too much TV here at Netrics… why did we name our blog “Netrics HD”?  Are we launching a high-def channel? Alas, no we aren’t.

We chose the name because we believe in viewing database data – regardless of the application – though a “High Definition” lens. It’s this High Definition lens that enables us to deliver High Fidelity data, despite the fact that the data itself is not perfect (moreover it can never be perfect).

This High Definition lens happens to be powerful mathematics that learns about data. As opposed to rules-based (probabilistic and deterministic) solutions that require a lot of guessing. 

Yes, Business AND Technology

Recently a friend of mine asked exactly how I was planning to blog about both business and technology. He pointed out the well known (but flawed) theory that says business folks are bored with technical details and technical folks are bored with business details.I love both business and technology and am particularly fond of their intersection!

I hope to offer meaningful insight on both subjects and, of course, how they need each other.

Stage Name

All these butcherings of my last name remind me of something that happened a few years ago.

We were meeting with a large hospital near the west coast and, as I often do, I gave a few examples of my own experiences having “Damianakis” butchered (how can you not butcher such a name?) and how that affected my experience with various organizations. And of course, the flip side, how the organizations were impacted as well (added expense and risk, etc).

At one point during the meeting I was fielding questions about the advantages of our mathematical modeling approach when the CIO raised his hand and asked me “is Damianakis your stage name?” I was caught off guard, “No, I’m not involved in the performing arts” I responded. “No”, he said, “is that your real name?” Then it finally registered…

It’s good to know that I have the right name for my chosen profession.

Surprisingly though, most people suffer the same problem with their names as well. At least that’s the result of my informal, unscientific poll. Even people with short common names often end up having their names being butchered.  Common responses in my poll were “they never get my name right” and “they often make a mistake,” etc…

Unfortunately computers are equal opportunity offenders, so the problem isn’t confined to just names of people, it happens with every kind of data they store.

Lots of Worms…

Alas, this core matching problem is not limited to people names… it basically applies to anything and everything that is stored in your DBMS. For example, product names, vendor names, claims, applications, and the list goes on and on. No data is immune.

This fundamental data matching limitation then spreads to every application built on top of a DBMS.

So this seemingly small functional limitation – the inability to intelligently compare and match imperfect data – can, at best, render enterprise applications ineffective and wrong, or worse still, result in their failure to perform as required.

Very scary… but fear not, all is not lost! More details on this is on the way… mixed in with other commentary of course.

Root Cause

We often get asked what creates the business-data problems that Netrics solves… Why is database data not perfect? How can databases make mistakes?

These business problems arise because of computers’ well-defined limitations - after all, they’re just fancy calculators. Calculators that can perform billions of operations every second - quite amazing when you think about it. Built into every CPU is the capability to compare data - but these data comparisons are limited to “equals” or “not equals” - that’s all.

These same limitations propagate up to the applications (i.e. databases) that run on these CPUs. So, if you ask any commercially available DBMS (e.g., Oracle, IBM DB2, MS SQLServer, MySQL, Cache, etc) to retrieve a record where the last name is “Damianakis” then that’s exactly what it’s going to do.

But what happens if “Damianakis” is actually in the database as “Damanakos” and also as “Dananakis? Easy, the DBMS will find neither - at least not with built-in database operators. Someone needs to build an abstraction layer that uses the built-in database operators to address this issue.

And thus the can of worms is opened…

Welcome!

Finally! I’m excited about the launch of our new blog. So, what’s this blog all about? Well, here at Netrics we’re passionate about the business and technology of data matching. An esoteric topic you might think, but as we’ll explore over the coming blogs-to-be, a technology that’s mission-critical for virtually every IT solution deployment.

The goal of this blog is to educate and evangelize while offering a “high-definition” view of data matching. This means we won’t sweep details under the rug, or obscure them with marketing-speak – we feel it’s important to see and understand how data matching works and the broad impact is has on IT solutions that businesses depend on.

I’ll certainly get ideas, suggestions and help from everyone here at Netrics… Here is a partial list of topics that I’ll cover in upcoming posts:

  • how does inferior matching performance adversely affect a company’s bottom line?
  • real-world examples of organizations’ failures to make effective use of structured data due to matching issues, and the impact of these failures on the ability to conduct business
  • how ineffective data matching hampers organizational compliance efforts, resulting in increased regulatory issues with government agencies
  • why is real-time error-tolerant matching computationally expensive?
  • what is deterministic matching? what is probabilistic matching? how are they different?

And of course, I’ll look to you, the readers, for comments, feedback and topics of interest. I’m very excited to have a medium to talk about these mission-critical business problems and the innovative mathematics that Netrics developed to solve them.


About Netrics HD

Data matching is a fundamental operation in many applications, from improving data quality to implementing master data management. Stef Damianakis, CEO of Netrics, a world leader in matching technology, shares his thoughts on the state of the technology and business of data matching.

Brought to you by...

Netrics Logo