Big Data can obscure security and operational threats and, at the same time, it can reveal opportunities to increase operational efficiency and unlock enormous business potential. Red Lambda is the first to combine leading-edge Big Data technologies to create an ultra-scalable, purpose-built software platform designed to keep pace with the increasing volume, variety, and velocity of IT operational data and find the threats and opportunities buried within it.
The first four customers currently deployed under Red Lambda's Early Release Programme are prominent players in network and Cloud infrastructure, social web, and government contractor arenas. Although they are very different businesses, they all share the need to gain unified situational awareness across their operations and network security domains, uncover and respond to threats and anomalies on-the-fly, and find actionable insights that can help transform their business and improve bottom and top-line performance.
"The data explosion is just as real in IT as elsewhere. And just as with other aspects of the
intelligence-driven enterprise, "Big Data" offers new challenges - and new opportunities", stated Scott Crawford, Managing Research Director with Enterprise Management Associates. "Much more information is available than ever before that can help organizations identify previously unrecognized threats, sharpen defenses, and acquire the awareness needed to develop more effective risk management programmes and improve IT efficiency."
IDC's 2011 Digital Universe Study forecasts that data creation and storage will reach 1.8 zettabytes (trillion gigabytes) in 2011 and grow to an astounding 35 zettabytes by the end of the decade. Moreover, 80% of data will traverse enterprise IT systems at some point during its lifecycle. In the wake will be an unprecedented amount of operational data﻿﻿ - structured and unstructured data that emanates from or affects IT infrastructure. Operational data sources can include telemetry, traffic, device, sensor, events, and transient application data from IT systems as well as external contextual data such as news, closed caption, weather, social, geographical, and global threat feeds.
"Security, as is most aspects of IT operations, has been a Big Data problem for years", stated Robert Bird, Red Lambda's Founder and Chief Technology Officer. "AppIron and MetaGrid fuse massively scalable Grid computing, relational stream processing, and breakthrough artificial intelligence into a single, cohesive solution that transcends the capabilities of conventional approaches and delivers true situational awareness."
MetaGrid is a new software application that runs on the company's AppIron Grid Platform and is designed to unify operational silos and situational awareness, visualize and analyze network security and operational anomalies, and automate IT operations.
Big Data creates big challenges when it comes to unification of operational data. Centralized and appliance-based solutions fall short in their ability to process petabytes a day of operational data. As a result, salient data is often siloed, filtered-out or excluded all together in order to reduce the load and "backhaul" requirements. MetaGrid leverages AppIron's Grid architecture and GridStream relational stream processing to move the computation to the data and not the other way around.
Collecting more data without better visualization and analysis can leave security operations teams overwhelmed rather than informed. Todays signature-based analysis only looks for threats that are known or anticipated - but what about the unknown unknowns? Red Lambdas patent-pending Neural Foam represents a breakthrough in knowledge discovery analytics. It's based on a lifelong-learning neural network algorithm that can analyze and reduce billions of data records - on-the-fly - into a manageable set of clusters, and provides bit-level universal anomaly detection on a timescale that can be years in duration. MetaGrid's data visualization and exploration capabilities make it easy for security operation teams to identify threats and further correlate, classify, drill-down and search data for intelligence or forensic purposes. Neural Foam increases IT operations effectiveness by providing complete situational awareness, dramatically reduced false positives, and an efficient way to zero-in on anomalies that represent real risk to the organisation.
Capitalizing on actionable insights requires the ability to take action quickly. MetaGrid incorporates incident management and a workflow and autonomics engine to provide intelligent automation of threat notification, response, mitigation, and remediation actions.
Leading up to our Early Release Programme we have met with over 20 of the worlds most sophisticated IT and service delivery organizations representing cloud and service providers, financial services, web properties, and government organizations, and we have been elated with the response to our MetaGrid offering", stated Bahram Yusefzadeh, Chairman and CEO of Red Lambda. "These customers have a vision for how they can harness their IT Big Data to improve their security posture, streamline IT operations and unlock business value; they have been waiting for a security and operational intelligence platform built for Big Data that can make it a reality."