Netrics HD

In his recent ebizQ.net article SOA, Phase 2: Toward a Loosely Coupled World, Joe McKendrick declared:

“I am a passionate believer in the power of technology, as an enabler of entrepreneurship and organizational transformation. I have long advocated flattening the organizational hierarchy, and pushing decision-making down to the managers and employees who deal with customers and production on a day-to-day basis.”

I couldn’t agree more.  Nothing has a more powerful effect on an organization’s ability to succeed than putting the right technology into the hands of front line employees.

There is an unstoppable industry trend gaining daily momentum where organizations are increasingly looking for solutions with cloud computing and software-as-a-service (SaaS) as the new paradigm for enterprise architecture.

“Cloud computing is pushing some software vendors to change their models to component delivery,” explains McKendrick.  “This makes plenty of room not only for small start-ups, but also for development shops within traditional enterprises that have great ideas.”

Historically, many of the most powerful new trends in technology originated from small entrepreneurial vendors.  By focusing on enhancing their highly specialized components, they can provide a great source of rapid innovation.  Therefore, small software vendors, whose solutions are designed for deployment using a loosely coupled service-oriented architecture (SOA), may be the industry’s small giants upon whose broad shoulders we will all be standing in the not-to-distant future.

And according to Mohan Sawhney, professor at Northwestern’s Kellogg School of Management:

“The best-run companies are becoming orchestrators of networks of services.  Five years from now, the concept of an application will be obsolete.  They will all be services, combined, mixed, matched and reused as needed.”

Therefore, when it comes to enterprise architecture — service-oriented is future-oriented.

Related Posts

The API and the Innovation of Enterprise Applications

Innovation Recession?

Innovation – Do More with Less

The Cloud brings Commoditization

“One of those issues that is always a source of frustration in the enterprise,” explained Michael Vizard in his recent IT Business Edge blog post, The Never Ending War for Data Quality, “is the quality of the data we spend so much time and money processing.  The quest to make sure we have high quality data is nothing short of a never-ending battle between the forces of order and the chaos that envelopes every attempt to organize anything.”

I have to admit that this is one of my pet peeves.  A remarkably common misconception is that the only way to deal with the pervasive nature of “imperfect data” is to somehow magically keep all of the data “perfect” all of the time.

Data frequently contains numerous variations caused by different conventions, lack of standards, omissions, and other inconsistencies.  The traditional approach to data quality is to heavily rely on standardization and other data cleansing efforts in order to prepare data before it can be effectively used for making business decisions.  These preparation activities attempt to create a consistent format of parsed attributes with standardized values.

“Alas, the war over data quality can never really be won,” explains Vizard.  “What can be done is that the number of instances where we have conflicting data and outright errors can be sharply reduced.  There’s no shame in having bad data; everybody does.  The only real sin is not trying to do anything about it.”

I agree with Vizard on the points that everybody has bad data and that we do need to do something about it.

However, the time is long overdue for us to stop depending on outdated approaches to data quality.

Perfection (especially in data) is impossible to achieve.  Intelligent business decisions can be made using imperfect data – without extensive data cleansing.  Instead of trying to make the data perfect, we need to focus on enabling enterprise applications to handle the unavoidable reality of imperfect data, which is something that humans do naturally.

Advancements in mathematics and machine learning algorithms provide the capability to adapt to (and overcome) data’s inherent chaos, and enable enterprises to make better data-driven business decisions.

I call this approach the Chaos Theory of Data Quality.

Related Posts

The Growing Importance of the Algorithm

The Growing Importance of Mathematics

Adaptive Software

Drowning in Imperfect Data

A Sisyphean Task…

“One of the bigger trends to come down the pike lately,” explained Jim Ericson in his recent Information Management blog post The API is the New Network, “is the proliferation of Web-based application programming interfaces, or APIs, and how network traffic is growing exponentially through APIs.”

More and more organizations continue to look to innovations in cloud computing, software-as-a-service (SaaS), and information as a service, as a new paradigm for enterprise applications.  In a recent press release, Gartner Research identified the Top 10 Strategic Technologies for 2010 and the list includes both cloud computing and client computing.

This is an almost stark contrast to the traditional approach taken by large technology vendors, who tend to innovate via acquisition in order to offer consolidated enterprise application development platforms with seamlessly integrated components for data quality, data integration, master data management and business intelligence.  This allows the large technology vendors to offer end-to-end solutions and the convenience of one-vendor information technology shopping.

However, does buying everything from one large vendor guarantee a best of breed solution for each individual component?

An API-oriented approach enables a plug-and-play enterprise application strategy.  Under this model, enterprise applications are assembled from best of breed individual components that are loosely coupled via a network of API calls.

Historically, many of the most powerful new trends in technology originated from small entrepreneurial ventures.  Small technology vendors tend to be specialists with a narrow focus that can provide a great source of rapid innovation.

Perhaps we are witnessing the beginning of the reversal of the recent trend of vendor consolidation, and a return to the earlier industry landscape where smaller vendors remained focused on enhancing and improving their highly specialized components.

If the API is indeed the new network, then the innovation of enterprise applications is to be found in collaboration and not consolidation.

Related Posts

Innovation Recession?

Innovation – Do More with Less

The Cloud brings Commoditization

In the recent New York Times article Training to Climb an Everest of Digital Data, Ashlee Vance reported on the challenges associated with managing – and deriving value from – massive repositories of data.

“Researchers and workers in fields as diverse as bio-technology, astronomy and computer science,” reports Vance, “will soon find themselves overwhelmed with information.  The next generation of computer scientists has to think in terms of what could be described as Internet scale.  Facebook, for example, uses more than 1 petabyte of storage space to manage its users’ 40 billion photos.  (A petabyte is about 1,000 times as large as a terabyte, and could store about 500 billion pages of text).”

According to Gartner Research, the volume of enterprise data is doubling every 18 months.  This rapid data proliferation is causing day-to-day business challenges to evolve faster than the existing applications (or new applications under development) can react.

“Science these days has basically turned into a data-management problem,” said Jimmy Lin, an associate professor at the University of Maryland, at a recent technology conference.

From the beginning of civilization, mathematics (the language of science) has been central to our advancement.  But our relatively new found ability to collect massive amounts of digital data has ushered in a new era for leveraging and benefiting from mathematics.

Advancements in machine learning technology using sophisticated mathematical algorithms are providing the capability to not only rapidly process large volumes of data, but more importantly, enable enterprises to make better data-driven business decisions.

According to Vance, companies large and small, as well as universities and government agencies, are “looking for big data experts” capable of scaling today’s digital data mountains.

Perhaps tomorrow we will even see a listing in the classifieds (or more likely in a Twitter status update) that simply reads:

Data Sherpas Needed

Related Posts

The Growing Importance of Mathematics

Adaptive Software

Drowning in Imperfect Data

A Sisyphean Task…

In his absolutely fantastic 2006 Princeton University essay The Algorithm: Idiom of Modern Science, Bernard Chazelle pondered the Holy Grail quest of computer science:

“How to unleash the full computing and modeling power of the Algorithm.”

Chazelle describes how Moore’s Law, which states that computing power doubles every two years, has delayed the rise to prominence of the algorithm, in much the same way that an abundance of relatively cheap oil has delayed the emergence of alternative energy sources.

The Triumph of Mathematics

“To make sense of the world, we have math,” explains Chazelle, and therefore, some might ask: Who needs algorithms?

“It is beyond dispute,” continues Chazelle, “that the dizzying success of 20th century science is, to a large degree, the triumph of mathematics.  A page’s worth of math formulas is enough to explain most of the physical phenomena around us: why things fly, fall, float, gravitate, radiate, blow up, etc.”

As Albert Einstein said:

“The most incomprehensible thing about the universe is that it is comprehensible.”

“Granted,” says Chazelle, “Einstein’s assurance that something is comprehensible might not necessarily reassure everyone, but all would agree that the universe speaks in one tongue and one tongue only: mathematics.”

The New Language of Science

“The Algorithm’s coming-of-age as the new language of science,” declares Chazelle, “promises to be the most disruptive scientific development since quantum mechanics.”

Algorithms are thought by some to be simply a way to automate the rapid execution of a task.  Although speed is important and the exponential growth of computing power has allowed algorithms to execute faster, it is the quality of the work performed by the algorithm that is vastly more important, especially algorithms used for complex data analysis in support of critical business decisions.

“The algorithmic paradigm,” explains Chazelle, “is not about what but how to think.  Self-reference is associated mostly with self-replication.  In the algorithmic world, by contrast, it is the engine powering the complex recursive designs that give abstraction its amazing richness: it is, in fact, the very essence of computing.  Should even a fraction of that power be harnessed for modeling purposes, there’s no telling what might happen.”

For example, using graph theory (a branch of theoretical mathematics), algorithms can construct mathematical models for the ways that humans recognize patterns in data.  The goal of these algorithms is not to replace human decision makers.

These algorithmically constructed models can be used to automate the rapid execution of analytical tasks providing true decision support for humans to use while navigating today’s challenging business environment, which faces daunting data volumes and a constantly evolving marketplace.

“Some say the Algorithm is poised to become the new New Math, the idiom of modern science,” explains Chazelle.  “I say The Sciences They Are A-Changin’ and the Algorithm is Here to Stay.  One thing is certain, Moore’s Law has put computing on the map: the Algorithm will now unleash its true potential.”

I completely agree and wholeheartedly echo the closing remark of Chazelle’s essay:

“May the Algorithm’s Force be with you.”

Related Posts

The Growing Importance of Mathematics

Drowning in Imperfect Data

Matches Created

A more precise, but less certain world

Narrative Fallacy and Data Matching

In the 1970s, spell checking programs debuted on mainframe computers.  In the 1980s, personal computers arrived and spell checkers soon followed, especially embedded within word processors.  In the 1990s, spell checkers joined the Internet revolution and debuted in e-mail programs and web browsers.

Today, spell checkers are virtually everywhere, from mobile text messaging to social networking websites.  In fact, nowadays you would be surprised (and probably annoyed) if spell checking wasn’t available whenever and wherever you were entering text.

Spell checkers are far from perfect.  However, their greatest feature is not just telling you that you have typed a word it can’t find an exact match for in its dictionary – it also provides a list of potential matches that may contain the word you were trying to spell.

In my previous post Apples and Oranges, I asked how many times you have looked up a contact in your e-mail program only to get zero results back.  Now, this might be understandable if you were looking up the e-mail address for Apple by typing in Orange.  However, most of the time what you typed was off by just a single letter (i.e. Aple).

Practically all of the devices and applications we use on a daily basis provide only exact matching.  Isn’t this just as surprising and annoying as not providing a spell checker?

For goodness sake, it’s 2009. Why does this problem persist – especially since the core technology to virtually eradicate this problem exists. Shouldn’t we demand the standard functionality of a Data Checker?

A Data Checker wouldn’t be clairvoyant.  However, if it couldn’t find an exact match for your lookup, it would provide a ranked list of potential matches.

Shouldn’t all of our devices and applications provide the capability of overcoming data variations, lack of standards, omissions, and other inconsistencies in our lookups and searches?

Related Posts

Apples and Oranges

All industries are feeling the effects of the current economic recession.  All industries are interested in the many benefits of innovation.  However, in the technology industry, not only is it a significant interest, innovation is the driving force and lifeblood of a rapidly evolving sector.

Historically, many of the most powerful new trends in technology originated from small entrepreneurial ventures.  Small technology vendors tend to be specialists with a narrow focus that can provide a great source of innovation.  Small vendors are usually less financially stable and rely mainly on funding for their early survival.  However, the current economy is limiting venture capital opportunities.

Large technology vendors tend to innovate via acquisition of these smaller vendors.  A continuing trend in information technology is the consolidation of acquired functionality into enterprise class application development platforms with integrated components for data quality, data integration, master data management and business intelligence.  This allows large vendors to offer end-to-end solutions and the convenience of one-vendor information technology shopping.

However, further innovation is typically delayed while the vendor prioritizes integrating the acquired technology into the existing suite of products and integrating the acquired people into their existing staff.  Additionally, training programs and sales strategies must be adjusted to reflect the updated platform and product offerings.  All of this requires significant time and effort.  The collateral damage is innovation can lose momentum or become stagnant.

Even before the economic recession, many in the information technology industry expressed concern about the effect of this vendor consolidation on innovation.  Now with the economy starting to claim some promising start-ups, are we looking at an innovation recession?

Of course, the situation is more complicated than small vendors vs. large vendors.  Vendors of all sizes are struggling with finding viable business models to pursue possibilities such as cloud computing and software-as-a-service (SaaS).

The trend with most innovations is that early adopters often spend more than they earn in these pursuits.  Innovation is often high-risk with no guarantee of high-reward. Entrepreneurial start-ups are usually more willing to go “all-in” and risk everything, but again venture capital is currently hard to come by.  Large vendors have more financial stability, but in a down economy it is often better even for them to play it safe.

Everyone wants to do more with less.  But some are settling to simply do less – long term (or even medium term) this is a losing strategy.  Innovation often stimulates the economy but the paradox is that an economic recession both spurs and suppresses innovation.

With many information business technology professionals feeling forced to find a way to survive, has do more with less become do without innovation or is it truly do with innovation?

My belief is that it’s the latter.

Related Posts

Innovation Video from BusinessWeek

Innovation – Do More with Less

The Cloud brings Commoditization

A lot has been written (and rightfully so) about the need to better align and to foster collaboration between the Business and Information Technology (IT). Traditionally this has been necessary because of the typically disparate skills and backgrounds of the people working on the business and technical sides of the enterprise.

In the early days of information technology when computers represented a brave new world that had such people in it that were typically electrical engineers and mathematicians, they spoke a language that the business world didn’t understand. They were so consumed with the difficulties and intricacies of the emerging technology that they had no time to dedicate to day to day business matters.

Today, rapid advancements in technology are becoming so commonplace that the next generation of college graduates will never have lived in a world without the Internet, laptop computers and cell phones that have more processing power and functionality than even the best desktop computers had less than two decades ago.

Our brave new world is seeing a dramatic increase in number of people with skills and knowledge in both business and technology and I believe that in the near future Information Tecnology (IT) will transition simply to Business Technology (BT).

If you ask me, it’s long overdue.