Environment Handlers

Minecraft is an extremely complex environment which provides players with visual, auditory, and informational observation of many complex data types. Furthermore, players interact with Minecraft using more than just embodied actions: players can craft, build, destroy, smelt, enchant, manage their inventory, and even communicate with other players via a text chat.

To provide a unified interface with which agents can obtain and perform similar observations and actions as players, we have provided first-class for support for this multi-modality in the environment: the observation and action spaces of environments are gym.spaces.Dict spaces. These observation and action dictionaries are comprised of individual fields we call handlers.


In the documentation of every environment we provide a listing of the exact gym.space of the observations returned by and actions expected by the environment’s step function. We are slowly building documentation for these handlers, and you can click those highlighted with blue for more information!


Visual Observations - pov, third-persion

pov : Box(width, height, nchannels)

An RGB image observation of the agent’s first-person perspective.



third-person : Box(width, height, nchannels)

An RGB image observation of the agent’s third-person perspective.


This observation is not yet supported by any environment.




Camera Control - camera

camera : Box(2) [delta_pitch, delta_yaw]

This action changes the orientation of the agent’s head by the corresponding number of degrees. When the pov observation is available, the camera changes its orientation pitch by the first component and its yaw by the second component. Both delta_pitch and delta_yaw are limited to [-180, 180] inclusive





Tool Control - camera

equip : Enum('none', 'air', 'wooden_axe', 'wooden_pickaxe', 'stone_axe', 'stone_pickaxe', 'iron_axe', 'iron_pickaxe'))

This action equips the first instance of the specified item from the agents inventory to the main hand if the specified item is present, otherwise does nothing. none is never a valid item and functions as a no-op action. air matches any empty slot in an agent’s inventory and functions as an un-equip action.






Agents may un-equip items by performing the equip air action.