Big data applications represent a fast-growing category of high-value applications that are increasingly employed by business and technical computing users. However, they have exposed an inconvenient dichotomy in the way resources are utilized in data centers.
Conventional enterprise and web-based applications can be executed efficiently in virtualized server environments, where resource management and scheduling is generally confined to a single server. By contrast, data-intensive analytics and technical simulations demand large aggregated resources, necessitating intelligent scheduling and resource management that spans a computer cluster, cloud, or entire data center. Although these tools exist in isolation, they are not available in a general purpose framework that allows them to interoperate easily and automatically within existing IT infrastructure.
A new approach, known as Big Workflow, is being created by Adaptive Computing to address the needs of these applications. It is designed to unify public clouds, private clouds, Map Reduce-type clusters, and technical computing clusters. Specifically Big Workflow will:
Schedule, optimize and enforce policies across the data center Enable data-aware workflow coordination across storage and compute silos Integrate with external workflow automation tools
Such a solution will provide a much-needed toolset for managing big data applications, shortening timelines, simplifying operations, and maximizing resource utilization, and preserving existing investments.
DatacenterDynamics is a brand of DCD Group, a global B2B media and publishing company that develops products to help senior professionals in the world's most ICT dependent organizations make risk-based infrastructure and capacity decisions.
Our portfolio of live events, online and print publishing, business intelligence and professional development brands are centred on the complexities of technology convergence. Operating in 42 different countries, we have developed a unique global knowledge and networking platform, which is trusted by over 30,000 ICT, engineering and technology professionals.
Data Centre Dynamics Ltd.
102-108 Clifton Street
London EC2A 4HW