Publications

2024

1. arXiv

   DBMovi-GS: Motion-aware Dynamic View Synthesis from Blurry Monocular Video via Sparse-Controlled Gaussian Splatting

   Under review

   Abs PDF

   abs

2. arXiv

   Unsupervised Dynamic Video Prediction with Object-Centric Kinematics

   Yeon-Ji Song, Suhyung Choi, Jaein Kim, Jin-Hwa Kim*, Byoung-Tak Zhang*

   Under review

   Abs PDF Code

   Human perception involves discerning complex multi-object scenes into time-static object appearance (i.e., size, shape, color) and time-varying object motion (i.e., position, velocity, acceleration). For machines to interact in the real world in a manner akin to human intelligence, they must comprehend the physical properties of objects, which is the motivation for successful video prediction. Recently, transformers with object-centric representations have emerged as promising for video prediction, yet they primarily focus on objects’ appear- ance, often overlooking other crucial attributes. In this paper, we propose Object-Centric Kinematics (OCK), a framework for dynamic video prediction leveraging object-centric rep- resentations. We introduce a novel component named Object Kinematics, which comprises low-level structured states of objects, thereby capturing objects’ motion as an additional attribute on top of objects’ appearance. The Object Kinemat- ics are obtained via diverse approaches, enabling compre- hensive spatiotemporal object reasoning, and are integrated through various transformer mechanisms, facilitating effec- tive dynamic long-term video prediction and generation. Our model demonstrates superior performance in handling objects and backgrounds within complex scenes characterized by di- verse object attributes and motions. Furthermore, our model exhibits long-term generalization capabilities, highlighting its potential for broad applicability in vision-related tasks.

3. ECCV

   Continuous SO(3) Equivariant Convolution for 3D Point Cloud Analysis

   Jaein Kim, Heebin Yoo, Dong-Sig Han, Yeon-Ji Song, Byoung-Tak Zhang*

   The 18th European Conference on Computer Vision (ECCV), 2024

   Abs PDF

   The inherent richness of geometric information in point cloud underscores the necessity of leveraging group equivariance, as preserving the topological structure of the point cloud up to the feature space provides an intuitive inductive bias for solving problems in 3D space. Since manifesting the symmetry by means of model architecture has an advantage over the dependence on the augmentation, it has been a crucial research topic in the point cloud field. However, existing methods have limitations in the non-continuity of groups or the complex architecture causing computational inefficiency. In this paper, we propose CSEConv: a novel point convolution layer equivariant under continuous SO(3) actions. Its structure is founded on the framework of group theory, realizing the convolution module defined on a sphere. Implementing its filters to be explicit, continuous, and rigorously equivariant functions defined upon the double coset space is the distinctive factor which makes our method more scalable than previous approaches. From the classification experiments on synthetic and real-world point cloud datasets, our method achieves the best accuracy, to the best of our knowledge, amidst point-based models equivariant against continuous rotation group.

2023

1. NeurIPS Workshop

   Learning Object Appearance and Motion Dynamics with Object-Centric Representations

   Yeon-Ji Song, Hyunseo Kim, Suhyung Choi, Jin-Hwa Kim*, Byoung-Tak Zhang*

   NeurIPS Workshop on Causal Representation Learning, 2023

   Abs PDF Website

   Human perception involves discerning objects based on attributes such as size, color, and texture, and making predictions about their movements using features such as weight and speed. This innate ability operates without the need for conscious learning, allowing individuals to perform actions like catching or avoiding objects when they are unaware. Accordingly, the fundamental key to achieving higher-level cognition lies in the capability to break down intricate multi-object scenes into meaningful appearances. Object-centric representations have emerged as a promising tool for scene decomposition by providing useful abstractions. In this paper, we propose a novel approach to unsupervised video prediction leveraging object-centric representations. Our methodology introduces a two-component model consisting of a slot encoder for object-centric disentanglement and a feature extraction module for masked patches. These components are integrated through a cross-attention mechanism, allowing for comprehensive spatio-temporal reasoning. Our model exhibits better performance when dealing with intricate scenes characterized by a wide range of object attributes and dynamic movements. Moreover, our approach demonstrates scalability across diverse synthetic environments, thereby showcasing its potential for widespread utilization in vision-related tasks.

2. CLeaR

   On Discovery of Local Independence over Continuous Variables via Neural Contextual Decomposition

   Inwoo Hwang, Yunhyeok Kwak, Yeon-Ji Song, Byoung-Tak Zhang*, Sanghack Lee*

   The 2nd Conference on Causal Learning and Reasoning (CLeaR), 2023

   Abs PDF

   Conditional independence provides a way to understand causal relationships among the variables of interest. An underlying system may exhibit more fine-grained causal relationships especially between a variable and its parents, which will be called the local independence relationships. One of the most widely studied local relationships is Context-Specific Independence (CSI), which holds in a specific assignment of conditioned variables. However, its applicability is often limited since it does not allow continuous variables: data conditioned to the specific value of a continuous variable contains few instances, if not none, making it infeasible to test independence. In this work, we define and characterize the local independence relationship that holds in a specific set of joint assignments of parental variables, which we call context-set specific independence (CSSI). We then provide a canonical representation of CSSI and prove its fundamental properties. Based on our theoretical findings, we cast the problem of discovering multiple CSSI relationships in a system as finding a partition of the joint outcome space. Finally, we propose a novel method, coined neural contextual decomposition (NCD), which learns such partition by imposing each set to induce CSSI via modeling a conditional distribution. We empirically demonstrate that the proposed method successfully discovers the ground truth local independence relationships in both synthetic dataset and complex system reflecting the real-world physical dynamics.

Domestic

1. arXiv

   Dynamic View Synthesis from Blurry Videos via 3D Gaussian Splatting

   Yeon-Ji Song, Byoung-Tak Zhang*

   Under review

   abs

2. KCC

   Unsupervised Visual Dynamics Learning with Multi-Object Kinematics

   Yeon-Ji Song, Byoung-Tak Zhang*

   Proc. Korea Computer Congress, 2024

   ✨ Best Presentation Award

   abs

3. KSC Oral

   Unsupervised Video Prediction with Object-Centric Representations

   Yeon-Ji Song, Byoung-Tak Zhang*

   Proc. Korea Software Congress, 2023

   abs

4. KSC Oral

   Self-Supervised Reinforcement Learning with Object-Centric Representations

   Yeon-Ji Song, Inwoo Hwang, Byoung-Tak Zhang*

   Proc. Korea Software Congress, 2022

   abs

5. KCC

   Self-Supervised Multi-Object Reinforcement Learning via Mutual Information

   Yeon-Ji Song, Chang-Hoon Jeong, Byoung-Tak Zhang*

   Proc. Korea Computer Congress, 2022

   abs

6. KCC

   Deep RL-based Optimal Path Planning and Obstacle Avoidance for Mobile Robots

   Yeon-Ji Song, Youngjae Yoo, Chung-Yeon Lee, Byoung-Tak Zhang*

   Proc. Korea Computer Congress, 2021

   abs