Big Data, Little Data…

“Big data, little data…” Sounds like a Dr. Seuss book. Hadoop…Splunk…sound like characters in one of his books. Data is everywhere. Given Big Data is all anyone can talk about today, I have spent the better part of the past year immersed in many forms of data, trying to sort out where we might consider investing.

At its core these platforms are attempting to make sense of unstructured, often times, machine generated data to provide unique actionable insights – and not just for the Head of the Analytics Department, but for all employees. The desire to re-use, share and store this unstructured data has opened up enormous market opportunities up and down the IT stack. My particular focus is around the applications which are creating “smart tools” to drive innovation in the enterprise – and it is clear that every market vertical will be impacted. My most recent investment is in the Big Data analytics space for health plans – and it is very cool – more to come.

The IT stack today involves the following hierarchy: collection > ingestion and storage > discovery and cleansing > integration > analysis > delivery. I am most focused on the right side of this equation (some of my partners have made some very compelling investments on the left side of that equation such as 10gen, Nasuni, Crashlytics, Tracelytics, InfoBright). Out of my exploration there have been a number of interesting insights and funny sound bites which inform some of our Big Data investment themes:

  • Big Data will democratize the enterprise, that is, all employees will become analytics experts who will drive work flow and productivity improvements – move the battlefield to front line employees
  • The “3 V’s” – velocity, variety and volume – are not going away, in fact they are only getting more severe
  • Movement to real-time analytics from batch processing is very powerful particularly in industries which process transactions where insights can now be moved from post-pay to pre-pay and pre-settlement (so rather than detecting fraud after the fact, fraud can now be readily detected prior to the transaction)
  • Real demand driven supply chains
  • Real need to drive insights from legacy IT architectures, particularly in the small to medium end of the market, who will be reluctant to overall existing infrastructures
  • Make Big Data small data or useable data through adaptive algorithms
  • In early innings of hyper-targeting across every industry
  • “Social sensing” will overhaul product development, stocking decisions, better forecasting and alerting, etc
  • Love the comment that “we are not looking to build more dashboards, but instead, cockpits”

We will undoubtedly be more refined and precise over time in how we look at the Big Data investment opportunity set. As part of this evolution, Flybridge hosted a Big Data CEO dinner a few weeks ago in Boston to identify how best to galvanize the community and where the greatest opportunities lie. In addition to great wines, there was a lot of enthusiasm for the new tools and architectures which are coming into the market. A follow-up dinner is being planned for the near future to better frame the opportunities – I welcome any suggestions for that agenda.

For me right now I am fascinated by the Big Data opportunities across the healthcare delivery system; as the FDA has become prohibitively hostile towards therapeutics and medical device companies, healthcare analytics is an area where profound benefits will be derived. As I mentioned earlier, my most recent investment is an analytics company focused on health plans – and the insights they are already demonstrating have enormous cost and revenue impacts for every health plan. Stay tuned – more to come.

7 Comments

Filed under Uncategorized

7 responses to “Big Data, Little Data…

  1. Ken Hillman

    Brilliant piece…and something I have been speaking about for years on the consumer side (the difference between data, information and knowledge in category management)…but the reality is that the key driving factor is the presence of enlightened leadership and management willing to turn insights into “out sights”…meaningful changes in strategy, tactics operations or organization that moves away from the “tried and true” and into meaningful new direction.

  2. Haana

    Very insightful article Liuz! I think it is worth mentioning HPCC System which provides a single platform that is easy to install, manage and code too. Their built-in analytics libraries for Machine Learning and integrations with open source tools like Pentaho provide you with an end to end solution for ETL, Data Mining and Reporting. More info at http://hpccsystems.com

  3. Great piece. And cool to see the industry finally adopting the “3V”s of big data over 11 years after Gartner first defined them. For future reference, and a copy of the original article I wrote in 2001, see: http://blogs.gartner.com/doug-laney/deja-vvvue-others-claiming-gartners-volume-velocity-variety-construct-for-big-data/. –Doug Laney, VP Research, Gartner, @doug_laney

  4. I really like the idea of “adaptive algorithms” to make big data small or useable. I wrote an article about making data useable by employing better business metadata. I thought it was a good idea anyway. I am curious your opinion on the space as it relates to the ability of analysts to actually use data and perform analysis. I think the current vendors are missing the boat on joining metadata to the data iteself. http://joshrutstein.files.wordpress.com/2011/10/rutstein_-is-it-really-bad-data-final.pdf

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s