We are witnessing an explosive and unprecedented growth in the volume of information around us, fuelled largely by the “internet of things”, where every device is connected in some way to the web and by social media. Network sensors are creating intelligent: buildings, devices, clothing, roads and much more, all of which consume and generate data with valuable interrelationships that can predict outcomes. Winning businesses are adapting rapidly to swim in this new ocean and grab the opportunities for competitive advantage, whilst the losers are already lagging behind and risk drowning. Which one are you?
Not so long ago, business information was predominantly internally produced within organisations, but now external data flows are massive. Business intelligence also used to be held securely on virtual private networks whereas now, with advances in mobile devices, especially BYOD (where employees bring their own device to work) and cloud applications, valuable company intellectual property is everywhere and most of the time, organisations no longer know where it is. This is a Board level agenda item, bringing risks as well as opportunities.
The term Big Data describes a collection of data sets which are so large that they can’t easily be analysed by traditional relational databases. It’s characteristics are Volume, Velocity (speed of change) and Variety. Structured data is typically numerical and fits into traditional data models. Unstructured data, which now represents ca. 80 per cent of all the information out there and where most of the growth has come from, is more likely to be in text or bitmap (audio, video) formats. Handling these vast amounts of information ideally requires cheaper storage, faster processing and smarter software with sophisticated algorithms and pattern recognition in order to capitalise on it.
The commercial opportunities are massive. A good example is online retail where Amazon and others have the advantage of not just knowing in real time what has been purchased by tightly defined customer segments, but also what else they have looked at, what people in their segment have viewed and bought as well as sophisticated models to test pricing/elasticity of demand and inventory and logistics management.
Many “bricks and mortar” competitors can only dream of having this information (although even this is changing will data from self-service tills) but its interpretation is key and that requires totally new skill sets. These opportunities are replicated in many sectors: financial services, pharmaceuticals, energy, motor manufacturing to name just a few. Massive potential cost savings are also available through improved labour productivity, reduced overheads and faster time to market.
Yet none of this can readily be achieved without the ability to “crunch” and “mash” massive volumes of information from multiple sources and present them in a way that management can make informed analytical decisions. This also requires new open data standards (as recently announced by the White House) and introduces a whole host of new regulatory, legal and moral challenges across multi geographies.
Technology is allowing smart SMEs to compete with much larger rivals in a way they never could before, but it is also worrying that many businesses are not even at the starting line. On too many occasions, I’ve come across situations where businesses have data in multiple unconnected databases and/or the data they have on their customers let alone their prospects is incomplete and/or erroneous. There is no point investing money in marketing campaigns and expecting impressive results if this is where you are currently at. Instead, invest in integrated databases, data cleansing and analysis first to provide a firm foundation for effective marketing.
As we have seen very recently with Bloomberg accidentally providing journalists with access to confidential data, when your information is everywhere, leakage is a major risk, which can cause serious brand stress along with the visible tangible costs. This is not a random occurrence. Every week another major organisation is affected. Businesses need to know where their information is and use sophisticated software to classify their valuable IP, track its movement, stop it getting into the wrong hands and provide meaningful reports to senior management so they have better visibility. This requires a much more sophisticated approach to policy generation and content inspection, particularly as recent studies have shown that the majority of breaches come, often accidentally, from within.
Like the story of King Canute, this new wave cannot be held back, so far better to be a leader rather than a laggard and address it now by building the capability to capture the massive commercial growth opportunities whilst effectively managing the risks.
Martin Leuw is a serial entrepreneur and former CEO of IRIS, the UK’s largest private software house.
