Currently, the computational technologies to handle such massive data volumes and rates are non-existent, and the roadmap to exascale computing is termed "disruptive" - unlike anything seen before. As the disruption propagates, perhaps for the very first time in the history of science, the research will be rendered unachievable without the essential computational backbone critically needed to process massive data rates and volumes.
Director of eScience Technology Rob van Nieuwpoort has been invited to speak at the Exascale Radio Astronomy conference in Monterey, USA, held 30 March to 4 April. The ERA conference provides a dynamic and interactive platform for international experts and enthusiasts from academic and industry backgrounds to introduce, explore, and discuss the scope and challenges of exascale radio astronomy.
Rob van Nieuwpoort is an expert in the field of high performance computing for radio astronomy; as a researcher at ASTRON he designed and developed software for the real-time data processing of the LOFAR software telescope, the largest radio telescope in the world. At the conference Rob van Nieuwpoort will present results from our eScience project Big Data for the Big Bang, in which a new development model for astronomical processing software is explored.
The research involves the following:
Radio Frequency Interference (RFI) mitigation is extremely important to take advantage of the vastly improved bandwidth, sensitivity, and field-of-view of exascale telescopes. For current instruments, RFI mitigation is typically done offline, and in some cases (partially) manually. At the same time, it is clear that due to the high bandwidth requirements, RFI mitigation will have to be done automatically, and in real-time, for exascale instruments.
In general, real-time RFI mitigation will be less precise than offline approaches. Due to memory constraints, there is much less data to work with, typically only in the order of one second or less, as opposed to the entire observation. In addition, the researchers can record only limited statistics of the past. Moreover, the scientists will typically have only few frequency channels locally available at each compute core. Finally, the amount of processing that can be spent on RFI mitigation is extremely limited due to computing and power constraints.
Nevertheless, there are potential benefits as well, which include the possibility of working on higher time and frequency resolutions before any integration is done, leading to more accurate results. Most importantly, the researchers can remove RFI before beam forming, which combines data from all receivers. The RFI that is present in the data streams from the separate receivers is also combined, effectively taking the union of all RFI. Thus, the RFI from all receivers pollutes all beams. Therefore, it is essential to do real-time RFI mitigation before the beam former. This is particularly important for pulsar surveys, for instance.
Although the techniques are generic, the research describes how the scientists implemented real-time RFI mitigation for one of the SKA pathfinders: The Low Frequency Array (LOFAR). The RFI mitigation algorithms and operations the scientists introduce here are extremely fast, and the computational requirements scale linearly in the number of samples and frequency channels. The researchers evaluate the quality of the algorithms with real LOFAR pulsar observations. By comparing the signal-to-noise ratios of the folded pulse profiles, the scientists can quantitatively compare the impact of real-time RFI mitigation, and compare different algorithms.