Language, freedom to think and all that jazz:Computer Books

Mary Boland reports on a research lab funded by electronics giant Sony that is pursuing the secrets of communication - among other things

Why do we love one kind of music and loathe another? What do we hear in a composition that moves us to tears while a friend looks on, nonplussed? Deep questions like these, and others about the evolution of language, are being asked at the basic research laboratory opened recently in Paris by Japan's electronics giant, Sony.

In one project, images of the musical instruments used in a composition are viewed on-screen in 3D virtual reality. Using a mouse, or eventually perhaps with the wave of a hand, users will zoom in on an instrument, increasing its volume and clarity while other instruments fade slightly. Or you could wander around the "stage setting", with each move having the corresponding impact on the sound.

"Basically, what we're doing is trying to understand better the musical experience from the viewpoint of the listener," says Luc Steels, director of Sony Computer Science Laboratory (CSL), Paris. "The idea is to bring out the richness in the music, which is only accessible if you become sensitive to it."

This project might end up on Sony personal TV or computer screens. But the researchers say their work is not targeted towards creating new products. Sony's goal, says Steels, an expert on artificial intelligence (AI), "is to let scientists do their thing, which comes out of curiosity about fundamental problems; about methodology of exploration. It is not guided or influenced by short-term goals. The moment you have a short-term goal, you cut off the scientific enterprise."

Sony spends between 6 and 10 per cent of annual sales on research and development. For 1996, this amounted to $2.3 billion. The laboratory was opened last October in Paris's Latin Quarter. The location was chosen for two reasons: "Partly because in Europe we have a strong position in research and science," says Steels, who is maintaining his academic post at the Flemish-speaking Free University of Brussels; and also because "the constant concern in Europe to improve the quality of life, and to put a human at the centre of all considerations". Steels has been working on the origins of language, and has made it one of the points of departure for the laboratory's activities. In his view, the computer must improve the quality of life, and should evolve with people and society, taking the culture of users into account. The research at Sony's Paris lab is concerned with finding out what people think, how we think, and, in relation to Steels' own project on language, how we speak and how language is created and evolves. There are four full-time researchers at work in the laboratory, including Angus McIntyre, a graduate from Edinburgh University, and a Japanese researcher, Chisato Numaoka from Keio University. Sony intends the laboratory - its first for fundamental research in Europe - to work closely with European universities. It has taken on a PhD student from Jussieu in Paris, and is looking towards forming alliances with universities in Britain, including Edinburgh and Sussex.

Back in the music section Francois Pachet, a graduate from Paris VI, is navigating another kind of musical space - not so much a physical space as a harmonic one. Instead of moving around musical instruments, he is working with the harmonic structure of a series of chords from a piece of jazz, which is represented on the screen as a series of bars. By moving around the structure, he can play with the harmonic detail, and navigate to various levels of depth within it.

Other musical research includes experiments with dynamic mixing. Instead of buying a pre-mixed compact disc, you would have various independent musical sources to mix together as you listen. "What we're trying to look into is the history of people's music experience," says Steels. "What is the evolution that takes place when people like a certain thing but then become bored? What is it that brings people to evolve their musical taste? Maybe there's a way to help people on this path."

Sony CSL Paris plans developments in Community Place, a 3D environment already in operation on the Internet. This would allow musicians anywhere to play together in real time.

The project with apparently the least immediate commercial potential is Steels' language research. In his view, the future of technology will depend largely on systems' adaptability and their potential to work independently. He is raising certain basic questions linked to a centuries-old debate: How does language originate? How does grammar evolve? How do lexicons develop? He is monitoring how groups of distributed agents can spontaneously and autonomously develop their own language by taking part in a series of language games. The goal is to create a system that independently adapts itself to change.

"The major problem in all computer technology today, particularly in AI technology, is that the systems are frozen," says Steels. As systems are based on analyses of existing languages, lexicons, and grammars, they remain constant, despite continuous linguistic evolution. "The major reason why language systems and also expert systems like knowledge engineering technology fail at some point is because it's all frozen intelligence, and intelligence is something that almost by definition evolves. That's why this project on the origins of language is really putting the question in a brutal way: What are the mechanisms for evolution and what are the mechanisms for adaptation?"

Over a demonstration of the language games, Angus McIntyre explains how spatially distributed agents begin the process of communication, and then build up their shared vocabulary. A graph indicates how well and how fast they achieve a common language. "But simply being able to communicate doesn't mean you have an intelligible language," he explains. "They might understand certain words but may not choose the same word for something else, so it's not a shared vocabulary."

He measures to what extent new languages develop when groups of agents are scattered. Predictably, when new agents are added which do not know the existing agents' languages, the levels of communication drop significantly. However, they eventually rise again, this time to even higher points than before the new agents arrived. "What we're looking at," says McIntyre, "is how, as the population of the community increases, language and understanding actually develop."