« Back to article

Print this article

Now, a language test for computers!

What would it mean for a computer to actually understand the meaning of a sentence written in ordinary English, or French, or Urdu, or Mandarin?

One test might be whether the computer could analyse and follow a set of instructions for an unfamiliar task.

Researchers at the Massachusetts Institute of Technology's Computer Science and Artificial Intelligence Lab have begun designing machine-learning systems that perform exactly the same function "with surprisingly good results", according to an MIT release.

Regina Barzilay, associate professor of computer science and electrical engineering, her graduate student S R K Branavan, and David Silver of the University College London showcased their approach to computer-learning by playing the 'Civilization' game at this year's annual meeting of the Association for Computational Linguistics.

Click NEXT to read more...

Now, a language test for computers!

Civilization is a computer game in which the player guides the development of a city into an empire across centuries of human history.

When researchers augmented a machine-learning system so that it could use a player's manual to guide the development of a game-playing strategy, its victory rate jumped from 46 per cent to 79 per cent.

Complex computer games such as Civilization include algorithms that allow players to play against a computer.

The games' programmers have to develop strategies for the computer to follow and write the code that executes these strategies. "Games are used as a test bed for artificial-intelligence techniques, simply because of their complexity," says Branavan, the first author on both ACL papers.

Click NEXT to read more...

Now, a language test for computers!

"Every action that you take in the game doesn't have a predetermined outcome, because the game or the opponent can randomly react to what you do. So, you need a technique that can handle very complex scenarios that react in potentially random ways."

Barzilay and Branavan's system begins with virtually no knowledge of the task it's intended to perform, or the language in which the instructions are written.

It has a list of actions it can take, like right-clicks or left-clicks, or moving the cursor.

It also has access to information displayed on the screen and ways of gauging its success. But it doesn't know what actions correspond to what words in the instruction set.

But as it takes various actions, different words appear on the screen and it can look for instances of those words in the instruction set.

It can also search the surrounding text for associated words and develop hypotheses about what actions those words correspond to.