Who's afraid of supercomputing?
When you have a supercomputer, you can ask it almost anything - as long as you have the right model ...…
Who’s afraid of supercomputing
A new type of “user-friendly” supercomputer installed in the centre of Glasgow is helping to prove to industry and academic researchers – in Scotland and beyond – that greater quantity can equal greater quality...
Maxim Fedorov is a man on a mission – to promote the use of supercomputing; not only because it's a smart way to solve complex problems in industry and scientific research but also to create new jobs and boost the economy.
According to Fedorov, the Director of the West of Scotland Academia–Industry Supercomputer Centre (ARCHIE-WeSt, www.archie-west.ac.uk), based in the University of Strathclyde, Scotland has more supercomputer “cores” (the equivalent of CPUs, or central processing units) per head of population than most other countries. “The infrastructure and the people are already in place,” he explains, “and this presents fantastic opportunities for Scotland. The only limit is our own imagination.”
ARCHIE-WeSt was installed in 2012 after Strathclyde won a bid (led by Professor David Littlejohn) for £1.6 million in funding from the UK's Engineering and Physical Sciences Research Council (EPSRC), as part of a plan to establish a network of regional centres which would serve as hubs for supercomputing and encourage collaboration between industry and academics. The process of winning the bid and installing the system (made by Dell) in only three months, to meet the EPSRC deadline, was a stellar achievement of the project team, says Fedorov – such projects typically take at least 12 months.
Today, the multi-faculty centre is being used to model everything from molecules interacting in fluids to armies fighting each other in battle (see sidebar). “ARCHIE-WeSt is relatively easy to manage,” says Fedorov, requiring a support team of only five people, compared to other supercomputers which need dozens of technicians. Another major selling point is access to a pool of different specialists right on the doorstep – for example, engineers who understand car manufacturing or biologists who know about producing pharmaceuticals. ARCHIE-WeSt also operates as a consortium of five universities in the west of Scotland – Glasgow, Glasgow Caledonian, Stirling, West of Scotland and Strathclyde.
There is a “quantum leap” from personal computers to supercomputers, says Fedorov, but ARCHIE-WeSt is tailored to the needs of smaller organisations, including small to medium-sized enterprises (SMEs), who can be trained to use it relatively quickly without going too deeply into the more esoteric technical details.
Fedorov speaks from experience, having been a major user of supercomputers in various countries worldwide because of his own work in molecule modelling or “solvation science.” And now he is in charge of ARCHIE-WeSt, he is keen to spread the word about supercomputing so potential users get a new perspective on what it can do for their organisation.
Academics and industry have very different time scales when it comes to research, with university departments sometimes focusing on the same project for years, while companies often demand a result within weeks, but ARCHIE-WeSt can cope with this, says Fedorov, because it adapts very quickly to changes in use.
Collaboration makes all the difference
For Fedorov, a specialist in computational chemical physics and molecular biophysics, coming to Glasgow was an easy decision to make. Before Strathclyde, he worked at several leading institutes, including the Max Planck Institute in Leipzig and the University of Cambridge, after graduating from the Russian Academy of Sciences in 2002; but his current job presented an opportunity to broaden his horizons and advance his career – and his personal research – in ways that had never been possible elsewhere. “Strathclyde enabled me to diversify and do more experimental research,” he explains. In the past, to do his research in solvation (modelling solvent molecules to study how they interact with solute ions, macromolecules and nanobjects), Fedorov had to fly thousands of miles to do a simple experiment (e.g. dissolving a compound in water) or rent CPU time, but now he only needs to walk a few yards down the corridor to get all the computational and experimental resources he needs.
Having a “local” supercomputer is not always the answer, however. In Fedorov's opinion, the optimal use of resources means making it easy to share all the hardware and software as well as ideas. “What I like most about Strathclyde is the spirit of collaboration among all the different departments,” he says. “In many places, it is hard to collaborate, even within a department, and different departments like Physics and Chemistry often seem like different planets. But here in Strathclyde, and in Scotland as a whole, there are no barriers to collaboration, with other academics or with industry. The ARCHIE-WeSt team itself is a good example of such interdisciplinary collaboration: Paul Mulheran, our Operational Director, is from Chemical Engineering; Richard Martin, our High Performance Computing (HPC) Manager, has a PhD in Physics but then became interested in supercomputers. Our User Support Officer, Karina Kubiak-Ossowska, and our HPC System Administrator, Jonathan Buzzard, also have interdisciplinary backgrounds. And, together, this contributes to the overall success of ARCHIE-WeSt, as most of us are not only HPC facility administrators but also active researchers, which helps us to understand better our users’ needs.”
Trends in supercomputing
Configuring the right type of supercomputer means striking a balance between different user requirements. Having your own system can offer greater control over security, scheduling and architecture, but it’s much more expensive; while using remote machines is also expensive and wastes lots of time – and puts you at the mercy of system administrators.
ARCHIE-WeSt (with 3,408 cores capable of almost 38 Teraflops, plus 150TB of data storage) may not meet the needs of a large corporation, but Fedorov points out that some large industrial users simply should buy their own systems. If he can steer users in that direction, he feels he is doing his job to promote the use of supercomputers per se – with his centre serving the role of a hub as well as providing a service to regional and international organisations. “We’re not here just to sell CPU hours,” says Fedorov, “but to serve as a platform or entry point for supercomputing.”
In addition, ARCHIE-WeSt avoids many problems with training and security issues because it uses industry-standard components. “Good standards and practices not only make the system easier to manage,” says Fedorov, “but also make it easier to transfer skills, share knowledge and collaborate.”
Currently, there are basically three types of supercomputers:
1 very large national centres
2 industry-oriented systems for specific research
3 local clusters with 100 to 1,000 cores
Over the last few years, supercomputers and their applications have also gone through different phases or “fashions,” as part of a trend towards more computational science. For example, computational molecular scientists can “play around” with molecules ad infinitum, but this can have its down side, too, because the interactions between molecules in condensed molecular systems (of which our body is a good example) are so complex and involve “astronomical numbers.” As a result, he continues, “there was a time when academic supercomputers spent up to 40 per cent of their time simulating moving molecules around, or spent a year answering one simple question.”
In the early days, the big pharmaceutical companies thought that their supercomputers “would solve all their problems,” but Fedorov believes this underestimated the complexity involved – for example, how the body digests (or metabolises) drugs, as well as problems with toxicity. The journey from conceiving the first theoretical model of drug action to getting a drug on the market also involves many stages, so because of the complexities faced by the industry, at one time the big pharmaceutical companies “fell out of love” with their giant machines.
However, interest in supercomputing applications in the pharmaceutical sector is rapidly increasing again. The industry now recognises that it has to study not only the biological activity of drug candidates but also other issues such as solvation, to improve the physical–chemical properties of existing bioactive compounds (e.g. solubility, stability and storability) before going on to develop new drugs. Solvation is therefore a critical issue, and Fedorov describes it as “the science where nanotechnology meets chemical engineering, physics and chemistry,” with supercomputers providing the power to do the research.
During his own career, Fedorov – like many other biophysicists and chemists – has dreamed of developing a blockbuster product or at least playing a key role in its development, stressing the importance of the need for more sustainable and people-friendly products.
He also stresses the need to build supercomputers which are very good at handling a specific job as well as more diverse research needs. ARCHIE-WeSt meets that mid-scale requirement, he adds, while the big pharmaceutical companies are falling in love again with very large supercomputers, because they are so useful in genome research and ground-breaking areas such as synthetic biology and personalised medicine.
“The challenge for researchers is complexity,” says Fedorov. “We have a fundamental knowledge of the laws of physics which govern our world, but not of the complexity of interactions – for example, the synergy of interacting molecules. We can use a so-called ‘brute force approach’ and try out every possible combination, but a model is only an approximation of reality – so we must be more selective in identifying candidates for modelling.” The stakes are increasingly high, he explains. For example, a developer may start off with 1,000 to 10,000 possible bioactive compounds for testing, and reduce this to about ten final candidates for clinical trials, and even then fail to make money. Many molecules may look extremely promising in terms of their biological activity but may not be very soluble, or have low chemical stability or unpredictable side-effects. And no matter what the challenges are, computer models come up with the answers much quicker than doing lots of experiments (which is not to say the computational techniques avoid the hazards of experimental research such as, for example, the unknown toxicity of new molecules).
Fedorov also strongly believes that “science should not be conducted in ivory towers,” but should meet practical needs. “There are so many ‘simple’ questions of practical interest which we still cannot answer,” he says, “but when you have a supercomputer, you can ask it almost anything – as long as you have the right model.”
Promotion of supercomputing
To attract potential industrial and academic users in Scotland, including small to medium-sized enterprises (SMEs), ARCHIE-WeSt is promoted in three different ways. As well as tapping academic networks and personal contacts, the centre reaches out to the community through advertising and workshops, guided by the centre's industrial advisory board. The centre also puts academics together with industrial partners to encourage collaboration and leverage funding.
“The opportunities are endless,” says Fedorov. “Companies could solve a lot of very complex problems with facilities like ARCHIE-WeSt, and one of my jobs is to make sure they are not scared of supercomputers. Just a few years ago, you had to be more expert to take full advantage of supercomputing, and know how to write lines of code, but there are not so many technical barriers now, thanks to the development of new tools and protocols. You can even run a supercomputer program from your mobile phone.”
As time goes by, new factors will be introduced to supercomputing, including social media and ethics. For example, by analysing comments on Twitter, a large corporation may change its approach to its business, while ethical considerations may also have an increasing impact on business decisions.
The real innovation does not lie in hardware and software, says Fedorov, or even in exploring new dimensions of computing, but in bringing different parts of the system together, including different scientific disciplines and people. “And larger quantity can equal better quality,” Fedorov comments.
The future of supercomputing in Scotland is something that is very much on Fedorov's mind: “The economic opportunities for supercomputing in Scotland are underdeveloped,” he says. “For example, we know less about the interior of Earth than we do about outer space, and our experimental capabilities in this area are (and will presumably continue to be) very limited. However, in terms of modelling, our capabilities are limited only by our own imaginations, so major industries like oil and gas could benefit greatly from supercomputing – like every other industry in Scotland – by modelling and visualising what they cannot see.”
Google on steroids
To illustrate how ARCHIE-WeSt and supercomputing can do more than people imagine, Fedorov cites the example of one recent user, a company which wanted to analyse its manufacturing methods and global supply chain, to see if it was worth making more components in-house or finding better sources.
The supercomputer facility then became what Fedorov describes as “Google on steroids,” searching for suppliers and identifying possible suppliers. As a result, the company was able to make a more intelligent decision about future production. “This proves how supercomputing can change how a business is run,” says Fedorov, “including its investment decisions.”
Where do all the drugs go?
Fedorov’s group plans to use ARCHIE-WeSt in collaboration with other Strathclyde academics and their colleagues at Glasgow Caledonian University to model what happens to all the pollutants – including pharmaceuticals such as antibiotics – we flush down the toilet.
The battle never ends
To celebrate the 700th anniversary of the battle of Bannockburn (1314), the Digital Design Studio of the Glasgow School of Art in conjunction with Historic Scotland used ARCHIE-WeSt as the computational workhorse to render 160,000 frames of animation for state-of-the-art motion capture technology to create a 270o immersive 3D re-enactment of the battle for the visitor centre near Stirling – consuming over 2.5 million core-hours of compute time.
The team’s Production Manager, Jared Benjamin, said: “We currently have access to 100 fast nodes at ARCHIE-WeSt. The pricing is competitive and remote access has been simple and fast, as has the upload of supporting files and download of produced rendered files. Consequently, we can use ARCHIE as an extension of our own in-house rendering capability from our desktops. We would recommend its use to anyone.”
Modelling underwater noise
The underwater noise (hydroacoustics) caused by industrial activities, in particular shipping, is an area of growing concern, because of its potential effects on marine life. The group of Professor Osman Turan in the Department of Naval Architecture and Marine Engineering at the University of Strathclyde is using ARCHIE-WeSt and computational fluid dynamics (CFD) software to predict the underwater noise radiated by ships in a realistic environment, “within a time frame which would not otherwise be possible.”