Python Package

Historically the ALE had to be built from scratch if you wanted to use the Python interface. This posed numerous issues especially when many end users might not have control over their system (e.g., running experiments on a compute cluster). As of v0.7 we now publish the Python package ale-py. Currently, we prebuild wheels for the following platforms:

If you are on one of these supported platforms you can install the ALE as easily as pip install ale-py. After installation you can go ahead and import the ALE interface as such:

from ale_py import ALEInterface

ale = ALEInterface()

ROM Management

Dealing with ROMs has historically been a pain point in the ALE. First off, we never gave strict guidance on which ROMs are supported. Not only is this an issue from a compatibility standpoint but subtle differences between games exist depending on a variety of factors. For example, the classic game Breakout was distributed in North America using the NTSC colour encoding. In other parts of the world, the same game may have been distributed with different encodings, e.g., PAL in much of Asia or SECAM for Russia. Although these region-specific ROMs play similarly they can have very different visual outputs.

If we want to get to a place in the community where we can feel confident comparing results on these types of benchmarks we have to take a tougher stance on standardizing these small details. This is why we now require ROMs using the new ROM management tools to match a known supported ROM. We publish a list of MD5 hashes for ROMs we support at Arcade-Learning-Environment#md5.txt. Furthermore, you can check if a ROM is supported by calling ALEInterface.isSupportedROM with a valid path.

Adding ROMs

The ALE will now support two methods for ROM management:

1. The command-line tool ale-import-roms.

You can import supported ROMs by simply using the command line tool ale-import-roms. This is installed with the new ale-py package. Assume you have ROMs in the directory roms/. You could import all the supported ROMs as follows:

$ ale-import-roms roms/

UPPORTED]       space_invaders    roms/space_invaders.bin
UPPORTED]             breakout          roms/breakout.bin


OT SUPPORTED]                             roms/combat.bin

Imported 2 / 3 ROMs

2. Installing a Python package which hooks into the ale-py.roms entry-point.

I’ll gloss over the details here but authors of Python packages can choose to register ROMs with the ALE. This is especially helpful for organizations who want to be able to easily install ROMs from an internal PyPI registry. For more information there’s an example package at Arcade-Learning-Environment/examples/python-rom-package.

Once a ROM has been registered with the ALE you’ll be able to import the ROM filename dynamically from the ale-py.roms package:

from ale_py import ALEInterface
from ale_py.roms import SpaceInvaders

ale = ALEInterface()
ale.loadROM(SpaceInvaders)

SDL2 Support

Previously if you wanted to visualize your agent you had one of two choices:

1. Compile the ALE on your local machine with SDL support

2. Collect the image observations and either a) save the images or b) display them using something like Pillow, matplotlib, etc.

This has its obvious drawbacks, what we’d really like is the ability to immediately be able to display the screen with audio support, native frame-rate (ignoring frame skips), proper resolution scaling, HiDPI support, etc. We now get this for free out of the box. Simply install ale-py and enable display_screen and sound and you’re ready to visualize your agents. This is made possible by distributing SDL2 with ale-py and only loading the shared library when the user enables screen or audio support.

OpenAI Gym

Since Gym was released it has been using a fork of the ALE, i.e., OpenAI/atari-py. This fork has lagged behind the official release and it’s finally time to consolidate things. This allows us to remain in control over the benchmark and provide direct support for Gym.

The transition should be seamless, installing gym[atari] will now install ale-py instead of atari-py. Furthermore, ale-py will look for supported ROMs that may have been imported with atari-py although this behaviour will eventually be deprecated.

The legacy game IDs, environment suffixes -NoFrameskip, -Deterministic, and versioning -v0, -v4 remain unchanged. We do suggest that users transition to the -v5 versioning which is contained in the ALE namespace. v5 environments follow the best practices outlined in Machado et al. 2018^[1]. Comparing each of these versions we now have:

With the new -v5 versioning we don’t support any ID suffixes such as -NoFrameskip or -Deterministic, instead you should configure the environment through keyword arguments as such:

import gym

env = gym.make('ALE/Breakout-v5',
    obs_type='rgb',                   # ram | rgb | grayscale
    frameskip=4,                      # frame skip
    mode=None,                        # game mode, see Machado et al. 2018
    difficulty=None,                  # game difficulty, see Machado et al. 2018
    repeat_action_probability=0.25,   # Sticky action probability
    full_action_space=False,          # Use all actions
    render_mode=None                  # None | human | rgb_array
)

The one keyword argument of note is render_mode. To realize the benefits of natively rendering with SDL we must supply the render mode when constructing the environment. We highly discourage using env.render().

Additional Features

We’ll briefly outline some other notable changes in this release. For a complete list see the updated changelog: Arcade-Learning-Environment/CHANGELOG.md.

Newly Supported Games

Note: If you’re using any of these games with Gym you need to use the v5 versioning. Version 0 and 4 are only for legacy games supported by atari-py.

Thanks to Thomas Köppe for upstreaming these contributions from DeepMind.

Determinism

Up until now, the ALE hasn’t been completely deterministic. There exist games that implement their own PRNG seeded by various methods which depend on the console’s state. We now properly control for this source of stochasticity and can now say with confidence that the emulator is 100% deterministic.

It’s worth noting that this source of randomness was observed in 20% of games we support to varying degrees. See Arcade-Learning-Environment#412 for more information. We’d like to thank Yuhang Song and Mark J. Nelson for their help in investigating this issue.

Modern CMake Support

We’ve completely rewritten our build system to support this release. Out is the old Makefile build system of the past and in with modern CMake. With CMake as a first-class citizen, this should alleviate some difficulties in implementing C++ agents. Along with these changes, we now support vcpkg natively and recommend installing your dependencies with vcpkg.

Acknowledgments

This release took the better part of two years and wouldn’t have been possible without all of our contributors. In particular, I’d like to thank Marlos C. Machado for imparting his ALE knowledge to me. Marc G. Bellemare and Michael Bowling for answering many of my ALE-related questions over the years. Chris Hesse @ OpenAI helped jump-start the Gym related changes. A special thanks to Thomas Köppe @ DeepMind for upstreaming their contributions. Lastly, I’d like to acknowledge J. K. Terry, Benjamin Black, and Tristan Deleu for their contributions and help with Gym.

References

1. Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents. Marlos C. Machado, Marc G. Bellemare, Erik Talvitie, Joel Veness, Matthew J. Hausknecht, and Michael Bowling. 2018. Journal of Artificial Intelligence Research 61 (2018), 523–562.

Endnotes

The visualizations depicted at the top of this page are from a trained Rainbow agent from the Dopamine project. These trajectories are rendered natively by the ALE in your browser. Refresh the page to view a new game.