Version 0.7 of the Arcade Learning Environment (ALE) brings lots of exciting improvements to the popular reinforcement learning benchmark. This release focuses on consolidating the ALE into a cohesive package to reduce fragmentation across the community. To this end, the ALE now distributes native Python wheels, replaces the legacy Atari wrapper in OpenAI Gym, and includes additional features like out-of-the-box SDL support for visualizing your agents.
Historically the ALE had to be built from scratch if you wanted to use the Python interface.
This posed numerous issues especially when many end users might not have control over their system
(e.g., running experiments on a compute cluster). As of v0.7 we now publish the Python package
Currently, we prebuild wheels for the following platforms:
If you are on one of these supported platforms you can install the ALE as easily as
pip install ale-py.
After installation you can go ahead and import the ALE interface as such:
from ale_py import ALEInterfaceale = ALEInterface()
Dealing with ROMs has historically been a pain point in the ALE. First off, we never gave strict guidance on which ROMs are supported. Not only is this an issue from a compatibility standpoint but subtle differences between games exist depending on a variety of factors. For example, the classic game Breakout was distributed in North America using the NTSC colour encoding. In other parts of the world, the same game may have been distributed with different encodings, e.g., PAL in much of Asia or SECAM for Russia. Although these region-specific ROMs play similarly they can have very different visual outputs.
If we want to get to a place in the community where we can feel confident comparing results on these types of benchmarks we have to take a tougher stance on standardizing these small details.
This is why we now require ROMs using the new ROM management tools to match a known supported ROM.
We publish a list of MD5 hashes for ROMs we support at Arcade-Learning-Environment#md5.txt.
Furthermore, you can check if a ROM is supported by calling
ALEInterface.isSupportedROM with a valid path.
The ALE will now support two methods for ROM management:
1. The command-line tool
You can import supported ROMs by simply using the command line tool
ale-import-roms. This is installed with the new
ale-py package. Assume you have ROMs in the directory
roms/. You could import all the supported ROMs as follows:
$ ale-import-roms roms/[SUPPORTED] space_invaders roms/space_invaders.bin[SUPPORTED] breakout roms/breakout.bin[NOT SUPPORTED] roms/combat.binImported 2 / 3 ROMs
2. Installing a Python package which hooks into the
I’ll gloss over the details here but authors of Python packages can choose to register ROMs with the ALE. This is especially helpful for organizations who want to be able to easily install ROMs from an internal PyPI registry. For more information there’s an example package at Arcade-Learning-Environment/examples/python-rom-package.
Once a ROM has been registered with the ALE you’ll be able to import the ROM filename dynamically from the
from ale_py import ALEInterfacefrom ale_py.roms import SpaceInvadersale = ALEInterface()ale.loadROM(SpaceInvaders)
Previously if you wanted to visualize your agent you had one of two choices:
Compile the ALE on your local machine with SDL support
Collect the image observations and either a) save the images or b) display them using something like Pillow, matplotlib, etc.
This has its obvious drawbacks, what we’d really like is the ability to immediately be able to display the screen
with audio support, native frame-rate (ignoring frame skips), proper resolution scaling, HiDPI support, etc.
We now get this for free out of the box. Simply install
ale-py and enable
sound and you’re
ready to visualize your agents. This is made possible by distributing SDL2 with
ale-py and only loading the shared library
when the user enables screen or audio support.
Since Gym was released it has been using a fork of the ALE, i.e., OpenAI/atari-py. This fork has lagged behind the official release and it’s finally time to consolidate things. This allows us to remain in control over the benchmark and provide direct support for Gym.
The transition should be seamless, installing
gym[atari] will now install
ale-py instead of
ale-py will look for supported ROMs that may have been imported with
atari-py although this behaviour
will eventually be deprecated.
The legacy game IDs, environment suffixes
-Deterministic, and versioning
-v4 remain unchanged.
We do suggest that users transition to the
-v5 versioning which is contained in the
v5 environments follow the best practices outlined in Machado et al. 2018. Comparing each of these versions we now have:
|Version||Frame Skip||Sticky Actions Prob.||Action Set||Max Frames|
With the new
-v5 versioning we don’t support any ID suffixes such as
-Deterministic, instead you should
configure the environment through keyword arguments as such:
import gymenv = gym.make('ALE/Breakout-v5',obs_type='rgb', # ram | rgb | grayscaleframeskip=5, # frame skipmode=0, # game mode, see Machado et al. 2018difficulty=0, # game difficulty, see Machado et al. 2018repeat_action_probability=0.25, # Sticky action probabilityfull_action_space=True, # Use all actionsrender_mode=None # None | human | rgb_array)
The one keyword argument of note is
render_mode. To realize the benefits of natively rendering with SDL
we must supply the render mode when constructing the environment. We highly discourage using
We’ll briefly outline some other notable changes in this release. For a complete list see the updated changelog: Arcade-Learning-Environment/CHANGELOG.md.
Newly Supported Games
The ALE now supports the following games:
- Basic Math
- Dark Chambers
- Flag Capture
- Haunted House
- Human Cannonball
- Mario Bros
- Miniature Golf
- Tic Tac Toe 3D
- Video Checkers
- Video Chess
- Video Cube
- Word Zapper
Note: If you’re using any of these games with Gym you need to use the
v5 versioning. Version 0 and 4 are only for
legacy games supported by
The ALE now supports additional modes/difficulties in the following games:
- Lost Luggage
- Tron Dead Discs
- Mr. Do
- King Kong
Thanks to Thomas Köppe for upstreaming these contributions from DeepMind.
Up until now, the ALE hasn’t been completely deterministic. There exist games that implement their own PRNG seeded by various methods which depend on the console’s state. We now properly control for this source of stochasticity and can now say with confidence that the emulator is 100% deterministic.
It’s worth noting that this source of randomness was observed in 20% of games we support to varying degrees. See Arcade-Learning-Environment#412 for more information. We’d like to thank Yuhang Song and Mark J. Nelson for their help in investigating this issue.
Modern CMake Support
We’ve completely rewritten our build system to support this release. Out is the old Makefile build system of the past and in with modern CMake. With CMake as a first-class citizen, this should alleviate some difficulties in implementing C++ agents. Along with these changes, we now support vcpkg natively and recommend installing your dependencies with vcpkg.
This release took the better part of two years and wouldn’t have been possible without all of our contributors. In particular, I’d like to thank Marlos C. Machado for imparting his ALE knowledge to me. Marc G. Bellemare and Michael Bowling for answering many of my ALE-related questions over the years. Chris Hesse @ OpenAI helped jump-start the Gym related changes. A special thanks to Thomas Köppe @ DeepMind for upstreaming their contributions. Lastly, I’d like to acknowledge J. K. Terry, Benjamin Black, and Tristan Deleu for their contributions and help with Gym.
- Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents.
Marlos C. Machado, Marc G. Bellemare, Erik Talvitie, Joel Veness, Matthew J. Hausknecht, and Michael Bowling. 2018. Journal of Artificial Intelligence Research 61 (2018), 523–562.
The visualizations depicted at the top of this page are from a trained Rainbow agent from the Dopamine project. These trajectories are rendered natively by the ALE in your browser. Refresh the page to view a new game.