Download or read book Adaptive Perception for Efficient Spatio-temporal Language Grounding in Dynamic Environments written by Siddharth Patki and published by . This book was released on 2023 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: "As robots are becoming increasingly prevalent in shared spaces such as homes and offices, the need for efficient and effective human-robot collaboration has become imperative. One of the key aspects of human-robot collaboration is the ability of robots to understand and interpret human instructions. However, the challenges associated with understanding natural language instructions in cluttered and dynamic environments remain significant. Recent approaches to grounded language understanding reason only in the context of an instantaneous state of the world. Though this allows for interpreting a variety of utterances in the current context of the world, these models fail to interpret utterances which require the knowledge of past dynamics of the world, thereby hindering effective human-robot collaboration in dynamic environments. Extending the contemporary models to reason about utterances which require knowledge of the past dynamics of the world introduces non-trivial challenges related to to both world state estimation and symbol grounding. Specifically in relation to world modeling, constructing a comprehensive model of the dynamic world that tracks the states of all objects in the robot's workspace is computationally intractable and difficult to scale with increasing clutter in the environments. On the other hand, a poorly detailed model of the environment limits the diversity of the utterances that can be interpreted and executed. A fundamental research question then is, how to efficiently reason over this rich information in a manner that enables robots to efficiently execute a variety of instructions in highly cluttered and dynamic worlds. In this thesis I present an arc of research which investigates how the information in language can be utilized to construct task-specific representations of the world, enabling faster and more accurate symbol grounding in cluttered and dynamic environments. Specifically, first this thesis presents a learned model of language and perception called Language Guided Adaptive Perception (LG-AP) that allows language to steer the interpretation of raw observations to create world models that are minimal but sufficient for the grounding robot instructions in static environments. Second, this thesis presents a novel approach called Language Guided Temporally Adaptive Perception (LG-TAP), that facilitates the construction of temporally compact models of dynamic worlds through closed-loop grounding and perception. This document includes is a discussion of the synergies that exist among these contributions and how adapting perception by exploiting the information in language can improve runtime efficiency and accuracy of robot instruction following in static and dynamic environments with high degree of clutter."--Pages xii-xiii.