Ziyao (Adonis) Zeng 曾子尧

Since the age of 13, deeply touched by Foundation by Isaac Asimov, my dream has been to create an AI who can think like humans (just like the dream of Prof. Jürgen Schmidhuber). When humans perceive the surrounding environment, we see (vision), hear (audio), feel (tactile) simultaneously to reason (language) and understand (neural signal), then interact (action) with the world. Therefore, I conduct research on Multimodal Representation Learning for Agentic AI, which spans both embodied agents (robotics) and digital agents (LLM/VLM-based agents). Despite differing in their input modalities and action spaces, they are unified by a common goal: the need to perceive, reason, and act intelligently within their environments.

My key research questions are (1) how to obtain good multimodal representations for agents and robots, and (2) how to use those representations to make agents and robots better perceive, reason, understand, and interact with the world. Specifically, my line of work in "Language for 3D Vision" (DepthCLIP, PointCLIPv2, WorDepth, RSA, Iris) explores how vision-language models can perceive and understand the world like humans do. Besides perceiving and understanding the world, I am exploring predicting the world and acting within it, including world models (DTWM, OpenLongTail, CrashTwin), LLM Agents (RuleSmith, SciAgentArena, CCPO), and 3D visual coding (3DCodeVerse). In the long term, since I want to build an AI who thinks like humans, I also care about AI consciousness and welfare (spiritual bliss attractor).