Tackling the SKA Big Data challenge


The data that will flow into the Square Kilometre Array (SKA) radio telescope when it is fully operational poses its own set of challenges to those tasked with collecting data dating from celestial events dating back more than 13 billion years.

South Africa’s National Research Foundation (NRF); The Netherlands Institute for Radio Astronomy (ASTRON) and international computing giant IBM have teamed up in a research collaboration aimed at researching extremely fast, but low power, computer systems aimed at developing advanced technologies to handle the massive amount of data emanating from one of the world’s most ambitious science projects.

The companies are looking at exascale computing (one exaflop is a quintillion floating point operations a second – a thousand times more than the first petascale computer than began operation in 2008).
“The project constitutes the ultimate Big Data challenge. Scientists must produce major advances in computing to deal with it. The advances will be felt far beyond SKA, helping usher in a new era of computing which IBM has christened the era of cognitive systems,” said Simon Ratcliffe, technical co-ordinator of DOME in South Africa.

As part of the global effort to solve this unprecedented challenge, ASTRON and IBM last year launched a public-private partnership called DOME to develop a fundamental IT roadmap for SKA. Through its SKA South Africa unit, the NRF is now a user platform partner in DOME.
“The DOME collaboration brings together a dream team of scientists and engineers in a partnership of public and private institutions. This project lays the foundation to help the scientific community solve other data challenges such as climate change, genetic information and personal medical data,” Ratcliffe added.

SKA is an international effort to build the world’s largest and most sensitive radio telescope, located jointly in Southern Africa and Australia to help better understand the history of the universe. Its nerve centre will be at Carnarvon in the Northern Cape.

When the SKA is completed it will collect data from deep space containing information dating back to the big bang more than 13 billion years ago. The aperture arrays and dishes of the SKA will produce 10 times the global internet traffic, but the power to process all of this data as it is collected far exceeds the capabilities of current state-of-the-art technology.

Scientists from all three organisations will collaborate remotely and at the newly established ASTRON & IBM Centre for Exascale Technology in Drenthe, the Netherlands.

SKA South Africa scientists from SKA South Africa will focus on four research themes. These include data technology, desert proofing hardware and creating analytical software.
“The DOME research has implications far beyond astronomy. These scientific advances will help build the foundation for a new era of computing, providing technologies that learn and reason. Ultimately, these cognitive technologies will help to transform entire industries, including healthcare and finance,” said Dr Ton Engbersen, DOME project leader, IBM Research.
“DOME is not only innovating in the laboratory, but our user platform is setting a new standard in open collaboration,” said Dr Albert-Jan Boonstra, DOME project leader, ASTRON. “In addition to SKA South Africa, four additional organisations are expected to join in the coming weeks including universities and small and medium-sized businesses located in the Netherlands.”

The initial five-year DOME collaboration has financial support from the Dutch Ministry of Economic Affairs, Agriculture and Innovation (EL&I) and the Province of Drenthe.

In May last year it was announced that South Africa had been awarded the lion’s share of the internationally-funded €1.5 billion (R15.6 billion) SKA radio telescope project. The decision means the bulk of the up to 3000 dishes, each 15 meters wide, will be built and installed at Klerefontein, near Carnarvon, in the Northern Cape. The remainder will be shared between Australia and New Zealand, the other contenders in the long-running race to host and gain from the mega-project.

South Africa, with its eight partner countries – Botswana, Ghana, Kenya, Madagascar, Mauritius, Mozambique, Namibia and Zambia – have been working on the bid to host the SKA since 2003.