Big data a catalyst for change Corporate IT, BI and enterprise search Finding the gems Social, collaborative common to BI and search
Business intelligence (BI) and enterprise search have been converging slowly in corporate IT organisations. But this convergence has been frustrated by organisational politics and competing technology legacies, according to experts canvassed in recent months.
The enterprise search software market was worth $1.3bn in 2011 according to Gartner, and the business intelligence software market was worth $10.5bn by the end of 2010. This suggests that BI is more likely to absorb enterprise search, if the technologies do converge.
Nick Millman, senior director of Accenture Information Management Services, believes that "though these technologies have grown from different bases, we will start to see convergence. There has been some already.
"BI has been focused around OLAP cubes, and dashboards and reporting," Nick Millman said. "But there is now more in the way of the development of visualisation technologies, using in-memory techniques that allow the user to explore the data. That is not search, as in a Google search, but it is mining data in real time.
"On the search tools front, typically they started out as Google-like, but they are getting more sophisticated in terms of presentation. And so the answer you get back looks more like a BI tool," he said.
Big data a catalyst for change?
Has big data, with its major concentration on unstructured data, been a catalyst for convergence between enterprise search and BI?
"They are happening in parallel, but because of the vast volumes of unstructured content, there is a move to discard the irrelevant and architect an end-to-end solution," said Millman.
"One approach to that is to perform a map reduce type operation, take a data subset, combine that into a data warehouse and make that available for more structured analytics. But the key thing is to avoid technology duplication, so that value is not being destroyed".
For more on enterprise search and business intelligence Five myths of business intelligence search IBM brings enterprise search, business intelligence to Super Bowl coverage What effect will Enterprise Search have on BI?
He confirmed that in the information management practice at Accenture, enterprise search is included, along with BI and data warehousing.
"It is more about being holistic. The tools should be secondary to that," said Millman.
But he conceded that in corporate organisations, "'search' has been grouped with knowledge management and collaboration people; then there are the data warehousing and BI people and, beyond that, information consumers in the business."
In response to this fragmentation, he said: "We see a strong drive to make BI reporting more contextualised and visual. But the way the business asks the question is not: 'can you integrate BI and enterprise search?' It is more: 'I want context wrapped around my BI reports to see hotspots more easily'. So, for example, a trading report, overlaid with weather or other seasonal information."
Corporate IT, BI and enterprise search
Ollie Ross, head of research at the Corporate IT Forum, notes a convergence of business intelligence and enterprise search interests among members of the blue chip user group, at least to the extent of the holding of a member-driven 'Enterprise Search as a Business Intelligence Tool' workshop in December 2011.
Ross said that when enterprise search was first broached inside the forum, some two years ago, it was intensive from some, but not general.
"It was an activity on the back of freedom of information requests and other legal discovery," said Ross.
But, over time, it has come to be seen as potentially a matter of business advantage, given sufficient usage. And that has been very much a ground up, collaborative phenomenon.
Corporate IT has got "used to appreciating the wisdom of crowds", she said.
As for business intelligence, that has grown in interest over the course of the past 18 months. "It has gone from being an interest to a problem needing to be solved for business benefit," said Ross.
Finding the gems
She echoed Millman's remark about how business people are looking at what both enterprise search technologies and business intelligence tools are addressing, without getting hung up on the technologies as such. And, she says, "even if you find the tools, it's about filtering out the noise to find the gems, and where they are. And that is down to people.
"And there is a tools crisis".
Corporate IT users, even those with job titles like head of data analytics or senior BI manager find the BI and enterprise search tools on the market "bewildering", Ross said.
Workshop findings, within the forum, do indicate a pressure to deal with the problem of new types of data growth, "often from outside the organisation, not transactional, often qualitative and where there are quality issues. The rudiments [of solutions] are there".
And it is appreciated that getting it right "could make a business difference.
"But 'next generation data', tangible and valuable, is better than unfocused talk about 'big data' for our members," said Ross.
Social, collaborative common to BI and search
On the supplier side of the house, the chairman and chief executive officer of open source BI company Jaspersoft, Brian Gentile, said the social, collaborative aspects to both these strains of technology could prove the ground of convergence.
"There is an intersection between search and analytics and reporting. Over time, we will see a richer display of search technology. And when the cloud is more established, we will see more of a move to collaboration within BI tools. Predictive analytics will also have a big future, too, in this context. But there are wonderful open source search technologies, such as Lucene, and we'd probably connect with those."
Erik de Muinck Keizer, head of enterprise search for Google, EMEA, thinks that enterprise search and business intelligence "will slowly grow towards each other. Enterprise search will be the most important part of BI and vice versa. When there will be one solution I don't know, but our definition of search – making information useful – is also a definition of BI.
But, it would seem, within corporate organisations, there are differing constituencies, rooted in data warehousing, BI, and knowledge management. Convergence is a while off.
