Synthetic Object Preference Adaptation Data

This dataset involves a 2D or 3D agent moving from a start to goal pose while interacting with nearby objects. These objects can influence position of the agent via attraction or repulsion forces as well as influence orientation via attraction to object's orientation. This dataset can be used to pre-train general policy behavior, which can be later fine-tuned quickly for a person's specific preferences. Example use-cases include:

• self-driving cars maintaining distance from other cars
• robot pick-and-place tasks with intermediate subtasks (ie: scanning factory items before dropping them off)

Overall, pre-training initial policy behavior to be fine-tuned later is a powerful paradigm and is arguably essential for robots to handle changing environments and user preferences. This is compared to the paradigm of training on massive amounts of data and remaining fixed at test time, hoping that generalization alone will help the agent handle new scenarios.