Intel has unveiled a raft of new innovations and software tools to tap the power of Big Data at the Supercomputing Conference 2013 held in Denver, Colorado.
At the 25th conference for the HPC community, the company revealed how the next generation Intel Xeon Phi product (codenamed “Knights’ Landing”), will fit into standard rack architectures and run applications entirely natively, instead of requiring data to be offloaded to the coprocessor.
According to Intel, this will significantly reduce programming complexity and eliminate “offloading” of the data, thus improving performance and decreasing the latencies caused by memory, PCIe and networking.
Knights’ Landing will also offer developers three memory options to optimise performance.
Unlike other Exascale computing concepts, which require programmers to develop code specific to one machine, the new processors will provide standard memory programming models.
Intel and Fujitsu also recently announced an initiative that could potentially replace a computer’s electrical wiring with fibre-optic links to carry ethernet or PCI Express traffic over an Intel Silicon Photonics link.
This enables Xeon Phi coprocessors to be installed in an expansion box, separated from host Intel Xeon processors, but function as if they were still located on the motherboard.
It also allows for a much higher density of installed coprocessors and the ability to scale the computer’s capacity without affecting host server operations.
However, with unstructured data accounting for 80 percent of all known data, and growing at a rate 15 times faster than other data, the industry is looking to tap into all of this information to uncover valuable insight.
The company is attempting to address this need with the Intel HPC Distribution for Apache Hadoop, which combines the Intel Distribution for Apache Hadoop with the Intel Enterprise Edition of Lustre software to deliver an enterprise-grade solution for storing and processing large data sets.
This powerful combination allows users to run MapReduce applications, without change, directly on shared, fast Lustre-powered storage, making it fast, scalable and easy to manage.
Cloud Edition for Lustre software is a scalable, parallel file system that is available through Amazon Web Services and allows users to pay-as-you go to maximise storage performance and cost effectiveness.
The software is ideally suited for dynamic applications, including rapid simulation and prototyping.
Intel vice president Raj Hazra said, in the last decade, the high-performance computing community had created a vision of a parallel universe where the most vexing problems of society, industry, government and research are solved through modernised applications.
“Intel technology has helped HPC evolve from a technology reserved for an elite few to an essential and broadly available tool for discovery,” he said.
“The solutions we enable for ecosystem partners for the second half of this decade will drive the next level of insight from HPC.
