Skip to navigation Skip to content


In search of the invisible computer

Professor Jon Oberlander talks about his experience as the founding director of SICSA and how it mirrors his own academic career – bringing diverse disciplines together so the end result is greater than the sum of its parts...…

In search of the invisible computer

Article by Peter Barr

Professor Jon Oberlander talks about his experience as the founding director of SICSA and how it mirrors his own academic career – bringing diverse disciplines together so the end result is greater than the sum of its parts...

In informatics, everybody talks about solutions, but for Jon Oberlander, it isn't so much about answering questions as “asking new questions and finding new ways to ask questions.” Oberlander's first degree was in philosophy, and listening to him today, more concerned with human dialogue and ethics than hardware and software, the former SICSA director seems to have travelled full circle. But his career so far is just as much a spiral as a circle, returning to subjects again and again, to see them from different perspectives, gained through his experience with different researchers and projects. And the last three years as founding director of SICSA have exposed him to a more diverse range of projects than ever before, some of which may seem at first more science fiction than science.

Oberlander's “spiralling” career path also reflects the way that informatics has evolved in recent years, drawing together researchers from the arts and social sciences as well as computing. Now SICSA's director of knowledge exchange and Professor of Epistemics in the University of Edinburgh's School of Informatics, as a young man he wanted to study artificial intelligence but was persuaded to explore more general studies and specialise later. After graduating in philosophy from Pembroke College, Cambridge, he received his PhD in cognitive science at the University of Edinburgh, in 1987. Later on, he started to focus on natural language processing and “the intersection of computational linguistics and cognitive science” – how human beings interact with each other as well as with machines, with the ultimate aim of creating more “human” machines.

Communication is the key to Oberlander's research – whether it is words, graphics, icons or gestures.  For example, he recently analysed thousands of blogs by comparing the results of personality tests with the style of the writing (as opposed to the content), analysing the typical characteristics of extroverts and introverts and the different expressions they use – even the way people use very simple devices such as exclamation marks, full stops and parenthesis. By mapping such characteristics, it should be possible to “psychoanalyse” people simply by reading their emails – interpreting the “secret” signals all of us send to each other.   

Oberlander also talks enthusiastically about the challenges faced in developing robots which have “social graces” and are able to empathise with human beings, recognising how we feel and making the appropriate response. For example, SICSA researchers are currently engaged in a joint research project with colleagues in Greece and Germany to develop an “intelligent barman” called James. The idea is not to programme social skills into the robot but to make it capable of learning them, Oberlander explains. And what interests him most about projects like this is that in the process of developing the robot, we are forced to ask ourselves what makes us human. For example, human beings “align” with each other when we talk to each other – echoing each other's words and mirroring our body language (crossing arms and legs, etc.). To build a “realistic” robot, therefore, it has to be able to learn for itself how to align with the people it meets. Similarly, if we want to build a robot which is capable of making decisions, then we have to understand not just how human beings make decisions by building a model of the decision-making process, but also how they learn to make decisions.

These intelligent machines are not simply clever devices but “models of real things.” Some researchers, says Oberlander, are even beginning to create “digital animals” (e.g. ants), not for its own sake but in order to learn what hardware and software is needed to be a “successful” animal, and how the same techniques may be employed for more practical applications.

Oberlander's interest in human–machine interaction dates back to his work at the Human Communication Research Centre (HCRC), an interdisciplinary research centre at the Universities of Edinburgh and Glasgow, founded in 1989. As soon as HCRC researchers started building up large collections of human conversations and  extending techniques for computational linguistics, they started to accumulate a huge amount of data – and that was how, for Oberlander, one discipline led to another. 

Scotland, especially Glasgow, has traditional strengths in information retrieval, but as Oberlander’s colleagues got to grips with the masses of information linked to human conversations, they developed a new strand of expertise in information extraction, at the same time finding common cause with researchers in more mainstream areas of computing.

Over the years, collaboration has become second nature to Oberlander and his fellow researchers. “To stretch yourself,” he says, “you need to find new challenges, and tackling these new challenges demands  collaboration.If disciplines look inward, they become less interesting. If they look outward, they become much more healthy.”

For Oberlander, social challenges are also of central importance, including how to make complex systems more usable and improve our quality of life by developing more advanced systems and tools in tune with individual requirements – including non-technical users. 

Easy access to technology is also important. “We are at a transformational point in society,” says Oberlander, “and informatics is the glue which keeps us together. The social effects of new communication tools are really just beginning to make themselves felt. These tools could pull existing institutions apart, but they can also pull us together.” 

The Internet may be an increasingly social place, but Oberlander questions whether the Web as we know it today is as smart as it should be. For example, researchers have long been predicting the advent of the “Semantic Web” – a term coined by World Wide Web pioneer Tim Berners-Lee to describe methods and technologies which allow machines to understand the meaning of information. In this more advanced “linked data” Web, the intelligent agents are smart enough to put information in context, act on our behalf, search, shop or negotiate, not simply follow the obvious path to an answer – in other words, “think” for themselves, as if they are extensions of the people who use them.  

Technology, however, is not yet as advanced or pervasive as we sometimes believe, he continues. Broadband coverage does not yet cover every part of Scotland, and bandwidth is limited. There are also many privacy issues to deal with. At the same time, increased coverage brings with it deeper questions about privacy, as people start to question their relationship with digital media such as social networks and use smart devices which not only know where we are all the time but also gather masses of personal data, as if they know much more about us than we do.

 Intelligent systems which help to improve life (or “technologies for assisted living”) are coming closer to reality, says Oberlander, and a major focus of SICSA research. For example, researchers who are working on social inclusion at the University of Dundee are developing a new AI (artificial intelligence) system which uses visual tools and synthesised speech to provide subtle prompts to people with cognitive or memory problems (e.g. Alzheimer's sufferers), combining perception and reason to know when it's appropriate to intervene and talk to the person in need, trying not to be obtrusive or annoying while the person is doing an everyday task such as boiling a kettle. 

It may be hard to draw the line between surveillance and monitoring or observation, but technologies are already being developed that blur the distinction. For example, Oberlander draws attention to the “meeting room” created in the Informatics Forum – an intelligent room which listens to discussions, using speech recognition to search the Internet for relevant data.

Although this may seem rather scary, like having someone read our minds, Oberlander takes an optimistic view of future technologies, stressing the benefits of personalisation – including the idea that we can be more in control of the data and systems we use, with computing tools which learn from our experience rather than simply processing masses of data which we may or may not find useful. Oberlander himself has helped to develop a “personalised interaction” solution for use in museums which allows the visitor to experience different exhibits and discover a web of connections between them, building up new data in the process, rather than simply moving from exhibit to exhibit in sequence, and hearing the same script as everyone else.

Oberlander also cites the example of FestBuzz, an application developed in Edinburgh by Jennie Lees, which puts together “crowd-sourced” reviews of events at the Edinburgh Festival, helping users check what other people like or dislike by scanning “the word on the tweet” instead of depending on “expert” reviews in conventional media outlets.

When it comes to  privacy issues in general, Oberlander takes the view that Europe has the chance to get it right when it comes to information security by providing individual protection without stifling innovation. “Smart technologies promise a lot, but there’s always a price,” he says. “You have to give up information about yourself. We have to help people make sensible decisions about these bargains, and we’re just not there yet.”

As well as being interested in public engagement with science, Oberlander (who comes from a family of architects) also has a passion for the visual arts and loves to show his visitors the School of Informatics’ collection of works by the Edinburgh artist Eduardo Paolozzi, who tried to visualise the art of the machine – the complex and abstract connections between different concepts and data – in a series of screenprints inspired by Britain's “father of computing” Alan Turing. He is also director of Inspace, a project to “make research visible, by exploring the cultural significance of informatics and new media practice,” run by the School of Informatics in partnership with New Media Scotland. One recent project –’s “life.turns” – gathers images from photographers based in countries all around the world, and Oberlander stresses that such mass participation highlights the fact that groups of individuals with the right tools can achieve things that organisations will never be able to do. 

This visual theme is also reflected in Oberlander’s interest in graphical communication and multi-modal interfaces, including a project in Glasgow to develop a “no-touch” computer which responds to hand gestures rather than keyboard or mouse. For example, you may wish to invite someone to share a resource like a web page. The question is what gestures are we comfortable with – and which gestures may be misinterpreted or even cause offence in different cultures? 

Oberlander believes that the ideal information system should be about “conversation not navigation,” so systems learn and adapt to our personal needs. It’s not so much about technology as people. “And we can’t get the technology right,” he declares, “without understanding ourselves better.” 

Such questions will continue to challenge researchers and Oberlander believes that the ultimate breakthrough will come when the computer is invisible, so smart and unobtrusive that we don’t even notice it’s there – until it goes wrong. It is ironic, he continues, that the more technology changes, the more we like things to stay the same, at least on the surface.  As innovations are developed, they are used to make the world look much the same as it has always looked.  “

The future will look a lot more like the present than some of us like to think,” he says. “But that’s a matter of styling – there will be plenty of changes under the bonnet.”


"In search of the invisible computer". Science Scotland (Issue Ten)
Printed from on 03/07/20 10:12:43 PM

Science Scotland is a science & technology publication brought to you by The Royal Society of Edinburgh (