8 Apr 2016 Amsterdam - At the EGI Conference 2016 in Amsterdam, The Netherlands,Primeur Magazinehad the opportunity to talk with Dugan O'Neil from Compute Canada. In Canada there is one national provider in what is called advanced research computing. This includes high performance computing, high throughput computing, and Cloud. Essentially, if it is something that researchers from an academic institution want to do and it is beyond what they are comfortably doing themselves on desktop machines and small clusters, they come to Compute Canada. Compute Canada has a wide variety of different services to offer and a wide variety of different hardware platforms that it needs to support. It is an international organisation. Canada is a very wide country, supporting researchers from the Atlantic to the Pacific. Right now, there are about 10,000 active researchers, 3,000 different research projects and 10,000 researcher groups.
The organisation itself was first formed around 2006 but was reformed as a really national organisation in 2012. Depending on which way you look at it, Compute Canada has been around for a long time, almost ten years, but is still in a kind of start-up mode, being about three to four years old in its new form.
Prior to 2012, Compute Canada was a loose academic consortium, basically a consortium of consortia. There were different regions in Canada that each had formed their own advanced research computing organisations. Compute Canada was an umbrella organisation over these organisations. From 2012 onward, there was an central office with people trying to do the job of better integrating resources from these regional consortia, creating national services that make it easier for researchers to use the resources that are out there.
The tradition was that Compute Canada was mainly supporting people who were very self-sufficient on resources to give them access to a big machine. They just go and use it. They don't really need your help. Compute Canada was helping with optimizing codes for those machines but would not providing in-depth help. Today, Compute Canada supports people from all disciplines with all kinds of different experience levels and really needs to spend more time in effort, training and making services that are more accessible to a broader base.
Compute Canada is still a federated organisation. The funding model in Canada is a combined federal/provincial funding. The provinces are very strong. They deliver a lot of the services. In Canada, even in health care, the money for health care comes from the federal government but the delivery of the service is the responsibility of each province. It is somewhat similar with Compute Canada except that it is co-funded. Compute Canada is responsible for getting the federal funding which is about 40 percent of the total. Provincial governments chip in some part. Institutions and universities chip in another part to make the whole, to fill up the 100 percent.
Compute Canada is the infrastructure provider. Right now, it is not purchasing any infrastructure from Cloud providers. Compute Canada directly addresses vendors and builds its own clusters. In the future, it may strike up these kinds of arrangements with commercial Cloud providers. Up until now, it has not been the highest priority to give an interface but Dugan O'Neil assumes that it is coming.
Right now, Compute Canada is right in the middle of moving to the beginning of a major technology refresh. Compute Canada has built a national platform with the last major investment being in 2012. Then there has been a bit of a gap. Now, Compute Canada has started taking delivery of the first major equipment since 2012. It is installing four new major computer centres and the national data infrastructure, all over the next year. Compute Canada is going from a bit of a gap in terms of technology funding - it had good support for operations but a gap in technology funding - to suddenly deploying a whole lot of systems at once. The technology changes include a bigger emphasis on Cloud. Compute Canada has a great expansion of its OpenStack capabilities but also a big emphasis on storage and on data services compared to what it had before.
Compute Canada was very computer-focused when it started and has become more and more data-focused as time has gone on. This means it should be buying different kinds of infrastructure than it did in the past. This is reflected in the approach that it has taken this time where the very first purchase as a national data infrastructure has taken place across all of its sites. Later, it will be deploying the compute to go with that.
International collaboration is super important for Compute Canada. The community Compute Canada serves is very collaborative internationally. There are people from particle physics, astronomy, neuroscience, and genomics. They do not limit themselves to the borders of Canada. They have direct collaborations with researchers in Australia, in the US, and in Europe. They need to be able to exchange data and they need to be able to run common tools on that data across borders.
This is one of the reasons that Compute Canada signed a Memory of Understanding with EGI, late last year, to work together on interoperability, starting with a few specific projects that are really needed today but then trying to make a much more broad alliance so that users from Canada can move easily to EGI resources and EGI users or European users can move easily to Canadian resources. Especially when it comes to experimental data that has to be shared and processed together, it really enables a lot of different research that is just not possible when you have individual silos communicating by e-mail instead of communicating by actually doing the work.