The evolution of big data analytics platforms is continuing at an unprecedented rate. The last quarter has seen major announcements from IBM on its investment in support of the Apache Spark platform, from SAP with a new big data-focused release of its HANA database and platform, and from Oracle with the addition of its new Big Data Discovery, Spatial, and Graph capabilities to its platform. Exciting though this innovation is, opening up new analytical possibilities, organizations mustn’t lose sight of the fact that gaining value from these insights depends on understanding the data available, and its effective application to business problems.
Existing approaches to business intelligence are well entrenched, having matured over many years, but the new opportunities created by analytics and big data require more lateral thinking on how business processes can be improved, and new value propositions created. Despite the “big” designation, this is equally applicable to SMEs, which have access both to their own sources of internal data, and to the large data sets that are now frequently available in the public domain.
Explosive data growth from websites, enterprise IT systems, sensor networks, audio and video streams, and mobile devices, combined with data technologies including in-memory analysis, non-relational databases, distributed data analysis, and real-time data integration, has generated innovative use cases. The opportunity is not, however, just for the bleeding-edge, or simply about petabyte-scale databases, and organizations must also consider the available sources of data and the speed of analysis.
This requires detailed exploration that involves examining the data sources and information flows associated with key business processes and considering how that information can generate new insights. Often there is data available that is not currently collected at all, not analyzed, or not analyzed sufficiently quickly to do anything other than post-hoc analysis.
