Netrics HD

“One of the bigger trends to come down the pike lately,” explained Jim Ericson in his recent Information Management blog post The API is the New Network, “is the proliferation of Web-based application programming interfaces, or APIs, and how network traffic is growing exponentially through APIs.”

More and more organizations continue to look to innovations in cloud computing, software-as-a-service (SaaS), and information as a service, as a new paradigm for enterprise applications.  In a recent press release, Gartner Research identified the Top 10 Strategic Technologies for 2010 and the list includes both cloud computing and client computing.

This is an almost stark contrast to the traditional approach taken by large technology vendors, who tend to innovate via acquisition in order to offer consolidated enterprise application development platforms with seamlessly integrated components for data quality, data integration, master data management and business intelligence.  This allows the large technology vendors to offer end-to-end solutions and the convenience of one-vendor information technology shopping.

However, does buying everything from one large vendor guarantee a best of breed solution for each individual component?

An API-oriented approach enables a plug-and-play enterprise application strategy.  Under this model, enterprise applications are assembled from best of breed individual components that are loosely coupled via a network of API calls.

Historically, many of the most powerful new trends in technology originated from small entrepreneurial ventures.  Small technology vendors tend to be specialists with a narrow focus that can provide a great source of rapid innovation.

Perhaps we are witnessing the beginning of the reversal of the recent trend of vendor consolidation, and a return to the earlier industry landscape where smaller vendors remained focused on enhancing and improving their highly specialized components.

If the API is indeed the new network, then the innovation of enterprise applications is to be found in collaboration and not consolidation.

Related Posts

Innovation Recession?

Innovation – Do More with Less

The Cloud brings Commoditization

Master Data Management (MDM) is the logical extension of a 20 year evolution in data management practice.  The strategic goal for MDM is to provide a single, “golden” repository of mission-critical data that assures all systems, organizations, and users are getting consistent, accurate information to support their needs.

Today, a number of vendors are positioning themselves to take on this challenge with new technologies that purport to make MDM feasible.  Once implemented, MDM promises to maintain real-time, clean, and consistent 360° views of prospects, customers, and products.

However, in her recent IT World Canada article Data quality vendors missing the mark, Kathleen Lau reported on a study by Andy Hayler, President and CEO of the analyst firm The Information Difference that shows:

“The issue for lack of attention to data quality by MDM vendors is that traditionally these vendors have focused on building systems that digest data quickly, only to later realize such systems were useless if the data being input was bad.”

Amassing poor quality data would appear to be what many MDM “solutions” are actually delivering.  The technology behind many of these systems is powerful and their functionality is impressively robust.

However, simply assuming the underlying data is “good enough” to support the MDM system, will only transform a “golden” repository of mission-critical data into an enterprise database of “fool’s gold.”

In the recent New York Times article Training to Climb an Everest of Digital Data, Ashlee Vance reported on the challenges associated with managing – and deriving value from – massive repositories of data.

“Researchers and workers in fields as diverse as bio-technology, astronomy and computer science,” reports Vance, “will soon find themselves overwhelmed with information.  The next generation of computer scientists has to think in terms of what could be described as Internet scale.  Facebook, for example, uses more than 1 petabyte of storage space to manage its users’ 40 billion photos.  (A petabyte is about 1,000 times as large as a terabyte, and could store about 500 billion pages of text).”

According to Gartner Research, the volume of enterprise data is doubling every 18 months.  This rapid data proliferation is causing day-to-day business challenges to evolve faster than the existing applications (or new applications under development) can react.

“Science these days has basically turned into a data-management problem,” said Jimmy Lin, an associate professor at the University of Maryland, at a recent technology conference.

From the beginning of civilization, mathematics (the language of science) has been central to our advancement.  But our relatively new found ability to collect massive amounts of digital data has ushered in a new era for leveraging and benefiting from mathematics.

Advancements in machine learning technology using sophisticated mathematical algorithms are providing the capability to not only rapidly process large volumes of data, but more importantly, enable enterprises to make better data-driven business decisions.

According to Vance, companies large and small, as well as universities and government agencies, are “looking for big data experts” capable of scaling today’s digital data mountains.

Perhaps tomorrow we will even see a listing in the classifieds (or more likely in a Twitter status update) that simply reads:

Data Sherpas Needed

Related Posts

The Growing Importance of Mathematics

Adaptive Software

Drowning in Imperfect Data

A Sisyphean Task…

In the recent Federal Computer Week article Stimulus spenders race to the finish line, Alice Lipowicz reported on the efforts of state and local recipients to prepare detailed reports on how they are spending federal stimulus money.

Among the common concerns cited in the article was dealing with the sheer volume of data, and more importantly, the quality of the data.

“Transparency advocates predict that data quality,” reports Lipowicz, “will be as much of a concern at Recovery.gov as it is at USAspending.gov, which Congress established in 2006 to provide visibility into federal spending.  That site has been plagued with problems such as errors, missing data and mislabeled data.”

Data transparency is definitely a laudable goal, and not just for the government.  Organizations in every industry and of every size need to do a better job of making available for review, the data that was used to drive critical business decisions, especially financial decisions.

Missing or incomplete data is a common problem, but transparency can not simply mean a massive dump of all available data.

Completeness without any regard for accuracy could possibly do more harm than good.  Data frequently contains numerous variations caused by different conventions, lack of standards, omissions, and other inconsistencies.

An excellent question raised in the article was:

“Data quality has been a problem for years, so why do we keep getting [more data] instead of addressing these priorities?”

I think that this question represents one of the most significant challenges for data transparency.

Should data be concealed until it has been verified to be of sufficient quality?  Or should data be provided as soon as it becomes available without regard for quality?

Please share your thoughts.

Related Posts

Drowning in Imperfect Data

A Sisyphean Task…