Between April and August 2013, Intersect360 Research held a survey among end users where they discuss their environments, challenges, solutions and 'satisfaction gaps' in addressing their Big Data challenges.
Their is a different mindset between technical and enterprise computing, according to Addison Snell. Technical computing on the one hand is driven by price/performance. It involves a fast adoption of new technologies, algorithms, and approaches. Enterprise computing on the other keeps the business running and is used for communicating and collaborating; marketing and selling products; accounting, HR and finance. It is drives by reliability, availability and serviceability (RAS). Because of this, there is a slow adoption of new
technologies, algorithms, and approaches in enterprise computing.
Addison Snell told the audience that Big Data constitutes a big opportunity. A lot of money is being spent on Big Data. 60% of enterprises that responded to the survey will spend more than 10% of their IT budget on technology relating to Big Data. However, we should use caution in describing 'the Big Data market'.
In 2012 in a earlier survey, only 17% of the respondents mentioned Hadoop when describing their Big Data applications. In 2013, this went down, driven by the enterprise respondents. Deployments might be based on Hadoop, but the majority of Big Data implementations are on in-house applications and algorithms. The most common source of data is also 'in-house'. ISV software for Big Data is thinly scattered. So there is more to Big Data than just Hadoop, Addison Snell showed.
Metrics of performance show up as key factors in enterprise as well as technical computing. Big Data will be a driver for expanded usage of HPC, if HPC developers can still meet enterprise requirements.
For the vendor panel, the following companies were invited: SAS, Intel, SAP, SGI, and Quantum.
Addison Snell had three main questions ready for them, accompanied by some secondary questions:
1. What is your company's view of Big Data, and how do you approach providing solutions for Big Data problems?
a. How pervasive is Hadoop in providing real Big Data solutions? When is Hadoop appropriate or inappropriate?
b. Where are we today on the hype-versus-reality curve for Big Data?
c. Most Big Data deployments rely on in-house applications or algorithms. Is this a good thing? What tools can help organisations create and scale their Big Data software environments?
2. Big Data is an enterprise IT problem with the performance and scalability challenges of HPC applications. To what extent are Big Data deployments standard IT infrastructure, versus a special investment?
a. Are there any high-performance technologies that are not currently enterprise standards that will gain greater adoption because of the requirements of Big Data?
b. What do you see as the areas of overlap for Cloud and Big Data?
3. Does Big Data change how we think about enterprise IT, enterprise computing, and enterprise storage?
a. What are the specific areas of development your company is making because of Big Data trends?
b. How will Big Data continue to evolve?
The vendors faced a tough challenge facing all the different facets of the Big Data topic but managed to come up with interesting answers that no doubt will give rise to new questions.