Hello I'm Ayush! A researcher passionate to equip embodied agents with physical commonsense reasoning capabilities. Currently, I am working on creative tool use and language based abstractions in household task planning. Through my previous works I've worked on commonsense reasoning and scene understanding in embodied agents.
Most recently I worked on commonsense based object selection while being advised by Dr. Dianbo Liu and Dr. Anirudh Goyal. Prior to that, I worked on developing commonsense based object navigation techniques during my time as a research assistant at Robotics Research Center, IIIT Hyderabad. I've obtained my bachelors in B.E Electronics & Instrumentation Engineering from BITS Pilani, Pilani in 2022.
During my undergraduate days, I was extremely passionate about insect behavior and collective intelligence. My bachelors thesis was based on developing Honey Bee Vision inspired Obstacle Avoidance Algorithms under the guidance of Dr. Sridhar Ravi
Education
-
B.E. in Electronics & Instrumentation, 2022
BITS Pilani, Pilani
News
-
Apr 2024 CommonSense Object Affordance got accepted in TMLR! Camera ready coming soon...
Nov 2023 My project proposal got selected for OpenAI Researcher Access Program
May 2023 Presented CLIPGraphs at PT4R Workshop at ICRA 2023
May 2023 CLIPGraphs got accepted in RO-MAN 2023
Jan 2023 Presented Sequence Agnostic MultiON in RnD Showcase at IIIT-H
Jan 2023 Sequence Agnostic MultiON got accepted in ICRA 2023
May 2022 Graduated and Began as an RA at IIIT-H
Jan 2022 My project proposal got selected for BITS-AUGSD Undergraduate Project Funding.
Publications
Physical Reasoning and Object Planning for Household Embodied Agents
TMLR 2024|arxiv|dataset
Demystifying the decision making process behind choosing an object for task completion, we develop a 3 step architecture and curate datasets to power future research in this domain. Further, we evaluate various LLM baselines and report the findings.
Sequence Agnostic MultiON
ICRA 2023 |arxiv|video|blog
You are already in a kitchen, and tasked to find a fridge. Would you search for it in the current area or other places in house? We train a RL policy based off semantic relationship between static objects to generate efficient long term goals to enable quick retreival of a list of objects.
CLIPGraphs: Multimodal Graph Networks to Infer Object-Room Affinities
RO-MAN 2023 |arxiv|code|page
Leveraging upon the knowledge that we humans have highly developed Object-Utility and Room-Utility relationships; we generate human commonsense aligned latent embeddings useful for varius Embodied AI tasks. We do this by developing a Graph Neural Network by processing Human Preference datasets and Foundation Model Features.