Publications

2024

1. arXiv

   Unsupervised Dynamics Prediction with Object-Centric Kinematics

   Yeon-Ji Song, Suhyung Choi, Jaein Kim, Jin-Hwa Kim*, Byoung-Tak Zhang*

   Under review

   Abs PDF Code

   Human perception involves discerning complex multi-object scenes into time-static object appearance (\ie, size, shape, color) and time-varying object motion (\ie, location, velocity, acceleration). This innate ability to unconsciously understand the environment is the motivation behind the success of dynamics modeling. Object-centric representations have emerged as a promising tool for dynamics prediction, yet they primarily focus on the objects' appearance, often overlooking other crucial attributes. In this paper, we propose Object-Centric Kinematics (OCK), a framework for dynamics prediction leveraging object-centric representations. Our model utilizes a novel component named object kinematics, which comprises low-level structured states of objects' position, velocity, and acceleration. The object kinematics are obtained via either implicit or explicit approaches, enabling comprehensive spatiotemporal object reasoning, and integrated through various transformer mechanisms, facilitating effective object-centric dynamics modeling. Our model demonstrates superior performance when handling objects and backgrounds in complex scenes characterized by a wide range of object attributes and dynamic movements. Moreover, our model demonstrates generalization capabilities across diverse synthetic environments, highlighting its potential for broad applicability in vision-related tasks.

2. arXiv

   Continuous SO(3) Equivariant Convolution for 3D Point Cloud Analysis

   Jaein Kim, Heebin Yoo, Dong-Sig Han, Yeon-Ji Song, Byoung-Tak Zhang

   Under review

   Abs PDF

   abs

3. arXiv

   Unsupervised Visual Dynamics Learning with Object Kinematics

   Yeon-Ji Song, Byoung-Tak Zhang

   Under review

   abs

2023

1. NeurIPS

   Learning Object Appearance and Motion Dynamics with Object-Centric Representations

   Yeon-Ji Song, Hyunseo Kim, Suhyung Choi, Jin-Hwa Kim*, Byoung-Tak Zhang*

   NeurIPS Workshop on Causal Representation Learning, 2023

   Abs PDF Website

   Human perception involves discerning objects based on attributes such as size, color, and texture, and making predictions about their movements using features such as weight and speed. This innate ability operates without the need for conscious learning, allowing individuals to perform actions like catching or avoiding objects when they are unaware. Accordingly, the fundamental key to achieving higher-level cognition lies in the capability to break down intricate multi-object scenes into meaningful appearances. Object-centric representations have emerged as a promising tool for scene decomposition by providing useful abstractions. In this paper, we propose a novel approach to unsupervised video prediction leveraging object-centric representations. Our methodology introduces a two-component model consisting of a slot encoder for object-centric disentanglement and a feature extraction module for masked patches. These components are integrated through a cross-attention mechanism, allowing for comprehensive spatio-temporal reasoning. Our model exhibits better performance when dealing with intricate scenes characterized by a wide range of object attributes and dynamic movements. Moreover, our approach demonstrates scalability across diverse synthetic environments, thereby showcasing its potential for widespread utilization in vision-related tasks.

2. CLeaR

   On Discovery of Local Independence over Continuous Variables via Neural Contextual Decomposition

   Inwoo Hwang, Yunhyeok Kwak, Yeon-Ji Song, Byoung-Tak Zhang*, Sanghack Lee*

   The 2nd Conference on Causal Learning and Reasoning (CLeaR), 2023

   Abs PDF

   Conditional independence provides a way to understand causal relationships among the variables of interest. An underlying system may exhibit more fine-grained causal relationships especially between a variable and its parents, which will be called the local independence relationships. One of the most widely studied local relationships is Context-Specific Independence (CSI), which holds in a specific assignment of conditioned variables. However, its applicability is often limited since it does not allow continuous variables: data conditioned to the specific value of a continuous variable contains few instances, if not none, making it infeasible to test independence. In this work, we define and characterize the local independence relationship that holds in a specific set of joint assignments of parental variables, which we call context-set specific independence (CSSI). We then provide a canonical representation of CSSI and prove its fundamental properties. Based on our theoretical findings, we cast the problem of discovering multiple CSSI relationships in a system as finding a partition of the joint outcome space. Finally, we propose a novel method, coined neural contextual decomposition (NCD), which learns such partition by imposing each set to induce CSSI via modeling a conditional distribution. We empirically demonstrate that the proposed method successfully discovers the ground truth local independence relationships in both synthetic dataset and complex system reflecting the real-world physical dynamics.

3. KSC

   Unsupervised Video Prediction with Object-Centric Representations

   Yeon-Ji Song, Byoung-Tak Zhang

   Proc. Korea Software Congress, 2023

   abs

2022

1. KSC

   Self-Supervised Reinforcement Learning with Object-Centric Representations

   Yeon-Ji Song, Inwoo Hwang, Byoung-Tak Zhang

   Proc. Korea Software Congress, 2022

   abs

2. KCC

   Self-Supervised Multi-Object Reinforcement Learning via Mutual Information

   Yeon-Ji Song, Chang-Hoon Jeong, Byoung-Tak Zhang

   Proc. Korea Computer Congress, 2022

   abs

2021

1. KCC

   Deep RL-based Optimal Path Planning and Obstacle Avoidance for Mobile Robots

   Yeon-Ji Song, Youngjae Yoo, Chung-Yeon Lee, Byoung-Tak Zhang

   Proc. Korea Computer Congress, 2021

   abs