“Scaffolding” is a powerful principle taken from psychology for organizing and optimizing the progress of a learner. The goal of my project is to combine this powerful structuring of a learning process with recent advances in deep and hierarchical reinforcement learning in order to make learning of an artificial agent more efficient and feasible when interaction data is much more limited than considered in current studies.