A growing number of organisations today recognize Apache Hadoop as a powerful technology for collecting and economically storing a very large set of highly variable data, and yet they struggle to realize its full potential in every part of their business. For example, a business analyst needs to quickly find relevant information, the data scientist needs to make sense of the data with statistical modelling, and the whole environment needs to be easy for IT to manage and deploy for everyone to use.
As the size and complexity of Hadoop applications continues to grow, data science has become a critical function for unlocking meaningful insights and identifying actions that could optimize outcomes across the entire business. IBM BigInsights for Apache Hadoop includes a broad data science toolset to query data, visualize, explore and conduct distributed machine learning at scale.
"There's so much data out there that it's often difficult for companies to find the information that really matters", stated Anand Mahurkar, CEO of Findability Sciences. "IBM BigInsights helps connect data elements that often go undiscovered to bring context to the customer relationship. Some of our clients have improved customer retention by up to 25 percent with this enhanced understanding. They can now focus efforts and resources on where they'll have the greatest effect, helping win and retain more business and drive more profitable operations."
IBM BigInsights for Apache Hadoop will introduce three new modules:
Also launched is IBM Open Platform with Apache Hadoop. Based on open source software, the platform will provide the necessary data access controls and authentication for an enterprise. The company is also adding support for Apache Spark to enable new computing engines to facilitate interactive analytic applications.
IBM is also sponsoring Big Data University and its new courses for Programming for Analytics and Data Science that cover machine learning. Big Data University addresses today's lack of Big Data skills by delivering free online courses to a community of more than 230,000 registered participants around the world.
These Hadoop solutions fit cohesively into IBM's broader analytics platform to help deliver insight from data, when, where and how it is needed. The platform provides a full range of analytics and integrates Hadoop solutions as part of a warehousing and data architecture. In particular, the predictive capabilities of SPSS can build the predictive models, exercise machine learning and R in Hadoop, apply business optimization, and deploy these models into real-time business processes.
"In our fast paced world, the ability to turn data into insight is the difference between success and failure", stated Beth Smith GM of IBM Analytics Platform. "With the announcements today, our ability to address the sophisticated needs of data scientists, to improve access for a broader community of analysts, and the surety of production scale will set in motion the next wave of value our clients expect from Big Data."
The new capabilities in IBM BigInsights for Apache Hadoop expand what is already the industry's deepest portfolio of Big Data solutions, spanning software, services, research and hardware. IBM Analytics combines traditional data warehouse technologies with new Big Data techniques, such as Apache Hadoop, stream computing, data exploration, advanced analytics, enterprise integration and IBM Watson cognitive computing to create an integrated solution to embed the opportunity of Big Data & Analytics into every organisation.
IBM is also a founding member of the Open Data Platform (ODP) Initiative, a new industry association to help drive collaboration, innovation, and standardization across Hadoop and Big Data technologies.