Studying to Remedy Voxel Constructing Embodied Duties from Pixels and Pure Language Directions
Authors: Alexey Skrynnik, Zoya Volovikova, Marc-Alexandre Côté, Anton Voronov, Artem Zholus, Negar Arabzadeh, Shrestha Mohanty, Milagro Teruel, Ahmed Awadallah, Aleksandr Panov, Mikhail Burtsev, Julia Kiseleva
Summary: The adoption of pre-trained language fashions to generate motion plans for embodied brokers is a promising analysis technique. Nevertheless, execution of directions in actual or simulated environments requires verification of the feasibility of actions in addition to their relevance to the completion of a purpose. We suggest a brand new technique that mixes a language mannequin and reinforcement studying for the duty of constructing objects in a Minecraft-like setting in keeping with the pure language directions. Our technique first generates a set of persistently achievable sub-goals from the directions after which completes related sub-tasks with a pre-trained RL coverage. The proposed technique fashioned the RL baseline on the IGLU 2022 competitors.