IT infrastructure spending has experienced a rough patch over the last few years, with many large Australian businesses slowing down their investments in new technologies. Cloud services have made inroads into many enterprise and government organisations precisely because of this drying up of internal investment in both skills and infrastructure.
Big Data is one of the latest technology bandwagons. The Australian federal government has even recently issued a round of consultation around the potential use of Big Data within Federal agencies. Big Data is essentially a philosophy, where an organisation systematically performs widespread continuous data collection and analysis, measuring and examining actions across an organisation.
A media frenzy has already emerged, as it is easy for journalists to draw links between the use of Big Data within businesses, the now widespread collection and sharing of sensitive personal information on websites like Facebook, and the proliferation of smartphones that can surreptitiously measure and report almost every aspect of our lives.
So it should hardly be surprising that the Australian federal government feels the need to consult on this topic. There are existing legal protections for the privacy of individual citizens, and restrictions covering the sharing of information between government departments. The timing is however quite ironic, coming within days of technology consulting firm Gartner finally proclaiming that the Big Data technology emperor has no clothes.
While there are clearly some elements of new technology to be found amongst the latest Big Data hype, even the most casual look back through archives of computer marketing material from the 1950s and 1960s reveals a simple truth. Large scale data analysis (Big Data) has always been one of the primary selling points used to justify investment in business computers. Even the ill fated high profile release of the Apple III in 1980 used data analysis as a primary selling point.
At many large businesses, the board and senior executive teams started investing heavily in these visionary “Big Data” style analysis capabilities in the 1980s. A particularly interesting example of Big Data hype repeating itself is this 1986 marketing brochure for a pattern recognition computer that “scans whole databases” and “handles words and pictures as easily as numbers”. The specific technologies being spruiked may differ, but the claims are extremely similar to those being touted for the latest Big Data technologies in 2013 by vendors such as IBM, Oracle, and EMC.
These strategic concepts are not new at an executive level, and the business outcomes being touted by today’s Big Data spruikers promise little more than the technology visions of the 80s. At a board level, every time a CIO goes back to the piggy bank for a new investment to solve the same old problem, the focus immediately shifts to the CIOs credibility.
At a very fundamental level, all Big Data initiatives are supposed to uncover a stream of fact based insights to inspire new efficiencies, improved business processes, or product innovations. In their most recent research into Australian business innovation, the Australian Bureau of Statistics found that only 16.4% of Australian businesses introduced new operational processes compared to the year before.
In the same period, 17.3% of businesses introduced new products, and a total of 39.1% of all Australian businesses claimed to have made an innovation or process change of any type.
The report found that the top Australian industries carrying out innovation were Wholesale Trade (51%), Retail (42%), and Professional Services (40%). This should not be surprising, as these industries are all facing intense competitive pressures caused by the widespread adoption of e-commerce, and changes in the Australian dollar exchange rate.
Research and development is of course a separate issue, with the ABS reporting that private sector R&D is currently 1.3% of GDP. While the mining industry gets a lot of media attention, the largest sector for R&D investment has actually been manufacturing. 80% of all private sector R&D dollars spent in Australia came from businesses in manufacturing, mining, financial services, and professional services. The government also spends very significantly on direct R&D, with most of that R&D spend going directly to universities.
By dealing with Big Data as a technology issue, CIOs and the organisations they work for are setting themselves up for failure.
When examining impediments to innovation, the most recent ABS report found that amongst companies that actively innovated, lack of funds was the primary roadblock to innovation, followed by lack of skilled labour.
Data insights need to be translated into actionable change that is implemented, resulting in modified behaviour and improved operations.
In an idealised scenario it becomes a continuous process of iterative innovation, sparked by data based insights and experimentation. The differences between the 1950s and 2013 simply boil down to the sources of the data points, the level of automation, and the amount of information that can be realistically stored and analysed.
In my experience, most large organisations struggle to build and maintain a culture of innovation. Primarily because they have complex supply chains, well defined business processes, and a large hierarchy of stakeholders. Put bluntly, change usually takes a lot of time and leadership attention.
For an insight to be useful, it must be rapidly turned into change. Large organisations that successfully innovate often chose to focus on small discrete projects, which are limited within a single division, or focused on a vertical slice of workflow. By limiting the scope, risks can be better understood, more focus placed on customer needs, and decision making processes sped up.
This is precisely why decades of data analytics projects have been focused in narrow areas of interest within organisations. It was not a technical limitation, but a consequence of the way people interact within large organisations.
Leaders need to consider how much capacity there is for innovation within their organisations. While the internal workplace culture of the organisation can have a huge impact on innovation, supplier contracts and technology infrastructure may also limit the ability to rapidly make meaningful changes.
The returns from Big Data projects can be expected to flow from the innovations and minor changes that are actually implemented. Fostering a culture of innovation within an organisation is a complex long term project. If there is little opportunity to quickly adapt products, change workflows, or respond to a weekly flow of insights, then there will be little return from a Big Data investment.