First Place at Competition of AI Computer Gamers

CITEC team wins finals of Microsoft multi-agent reinforcement learning challenge

Since 2017, the tech company Microsoft has put on an international competition for Minecraft, a popular computer game. The goal: getting artificial intelligence (AI) computer agents to learn to work with each other. The virtual agents are controlled by algorithms. This year, 133 computer scientists put their algorithms to the test at the tournament, including CITEC researcher Dr. Andrew Melnik and his two colleagues, computer science students Lennart Bramlage and Hendric Voß from Bielefeld University. Their agent scored the most points in the final round of the MARLÖ competition.

Dr. Andrew Melnik (front), from the Cluster of Excellence CITEC, with Henric Voß (left) and Lennart Bramlage (right), both computer science students: The team took first place in the “Marlö” tournament with their virtual agent. Photo: CITEC/Universität Bielefeld A research group from Microsoft set up this competition using the Minecraft computer game to encourage researchers and students from the field of computer science to explore and apply recent advances in AI. The competition serves to promote the creation of AI agents that are capable of identifying and solving problems on their own. "Reinforcement Learning" is a popular machine learning method that is used for this purpose. Additionally, this method is currently being used in autonomous driving and robotics, as well as in financial markets.

In the MARLÖ competition, virtual agents have to tackle a number of playful, cooperative tasks, like catching a pet together or building a structure made of blocks as quickly and precisely as possible. These AI agents learn to cooperate with other randomly assigned AI agents to score the most points in doing small tasks in the virtual world. The competition is organised in the form of a tournament, and Dr. Andrew Melnik and his CITEC team took first place in the finale. Second place went to the team from New York University, while third place went to Nanchang University.

“This competition is about general artificial intelligence that is capable of dealing with various situations on its own,” says CITEC researcher Dr. Andrew Melnik, who works in the Neuroinformatics research group headed by Professor Dr. Helge Ritter. “In order to be successful, our agent had to perform several tasks at the same time, such as running after a pet while keeping an eye on where the other players were.”

“What was especially challenging was the changing appearance of objects in the virtual world between games,” says Melnik. This include, for instance, the weather and lighting conditions. “Whether it’s rainy or sunny affects the appearance of the target object and makes recognition more difficult.”

To program their gaming agents, the CITEC team used an approach called “MimicStates.” “This is a form of imitation learning,” explains Melnik. “Recordings of game sequences from human players are used to demonstrate to the agent the actions needed to successfully complete the task.”

As the winner of the competition, the CITEC team was awarded two travel grants. With one of the grants, the three team members will be able to attend a scientific conference of their choice, where they will present their work related to the competition. The second travel grant is to be used to attend Applied Machine Learning Days 2020 in Lausanne, Switzerland – one of the biggest conferences in this area of research. In addition to the travel grants, the team will also receive funding to use the cloud-computing platform Microsoft Azure.

The MARLÖ competition is hosted by Microsoft, Queen Mary University of London, and CrowdAI. The initiative for the project came from Microsoft’s research and development division. For Microsoft researcher Dr. Katja Hoffman, the computer world of Minecraft offers the ideal environment in which to train AI agents to learn to act rationally and cooperatively. The name MARLÖ stands for “Multi-Agent Reinforcement Learning in Malmö” –Malmö is the technical platform in Minecraft created by Microsoft where the competition takes place.

Minecraft is a computer game in which the user designs their own world using building-blocks – similar to Legos, but virtual. Minecraft is set in a 3-D world with virtually any number of players, bringing a level of complexity to the game that approaches that of the real world.

Further information:

Dr. Andrew Melnik, Universität Bielefeld
Cluster of Excellence Cognitive Interaction Technology (CITEC)
Telephone: +49 521 106-12120