Anyone that works in data analytics and is excited it’s Halloween could appreciate a sign Teradata had displayed near the press room at the PARTNERS event last week in Anaheim, California. The copy proclaimed “The original big data company,” and the image displayed a candy store, with each toffee, lollipop and taffy flavor neatly compiled into its own little jar. Since confectionary stores popped up at the turn of the last century, shop owners had a perfect visual for just how popular each of its types of candy were. It was easy to tell just how valuable each set of candy was to the store, because its sales were visualized in real-time based on how full or empty each jar was.
Getting real-time, visualized analytics is nowhere near as easy today as it was for a candy shop owner 100 years ago, and the problem is unstructured data. In fact, it remains one of the last great impediments of analytics.
Today’s data is more like that giant pillowcase full of a random array of candies that children amass trick or treating. The initial goal was just to collect as much of as possible. But later on, organizations realized it takes a long time to sort through it all and, even after that arduous process, some of it doesn’t even hold value to an organization.
The companies that currently are able to leverage large amounts of data and turn it into insights — even predictions — are using mostly structured or semi-structured data, fueled by AI and machine learning crunching through this information in real-time. As Oliver Ratzesberger said at last week’s “The Sentient Enterprise” breakfast, AI needs a foundation. Companies can’t just hope that whatever they’ve throw into a data lake is going to turn into insights. And AI is proving extremely useful in helping businesses blend their many disparate data silos to gain a better understanding of the complex data ecosystem that now inhabits every major company. They have thousands of those candy jars and need to understand and instantly analyze what that information is doing to their business. But what about all the unstructured data sitting around that’s too time-consuming to sift though?
Eventually, as neural networks and deep learning progress, the processes will be in place to leverage fully unstructured data too. And in the interim, processes like semantic smoothing makes sense of that data, including speech. The problem was clear from some of the presentations at Partners that mentioned chatbots or digital personal assistants gone awry.
So while it may seem like a hindrance to fast, inexpensive insights right now, businesses are already dreaming up how machine learning and advanced analytics will turn unstructured data from what seems like a trick into a treat.
For more on how Teradata helps its customers apply machine learning to structured, unstructured and semi-structured data check out our Teradata Analytics Platform.