Sirui's Homepage

Affiliations University of California, San Diego


9500 Gilman Dr

La Jolla, CA 92093, USA

I am a CSE master student at University of California San Diego (UCSD), doing research in the field of Computer Graphics (CG) and Human-computer Interaction (HCI). I am interested in the general problem of better enabling creative professionals to use AI tools for their design tasks, both from CG and HCI perspectives. Some specific topics I am currently working on include:

  • Computer Graphics
    1. Physically-based real/ fake detector
    2. Near field light estimation
  • Human-computer Interaction
    1. Leveraging contextual information to improve the performance of AI models in creative tasks
    2. Developing new Gen AI interface to facilitate the convergence in the collaborative design setting

Here are some topics I am excited to explore in the near future:

  • Sketch-based/ layer-based Generative AI Control
  • Generative AI for non-photorealistic rendering in movies
  • AI-based creative tool for 3D modeling & architectural design
  • Physically-based generative/ prediction model for 3D scene understanding
  • Design collaboration in VR environments

selected publications

  1. physion++.gif
    Physion++: Evaluating Physical Scene Understanding with Objects Consisting of Different Physical Attributes in Humans and Machines
    Hsiao-Yu Tung , Mingyu Ding , Zhenfang Chen , Sirui Tao, Vedang Lad , Daniel Bear , Chuang Gan , Josh Tenenbaum , Daniel Yamins , Judith Fan , and  others
    In Proceedings of the Annual Meeting of the Cognitive Science Society , 2023
  2. physion.gif
    Physion: Evaluating Physical Prediction from Vision in Humans and Machines
    Daniel Bear , Elias Wang , Damian Mrowca , Felix Binder , Hsiao-Yu Tung , Pramod RT , Cameron Holdaway , Sirui Tao, Kevin Smith , Fan-Yun Sun , Fei-Fei Li , Nancy Kanwisher , Josh Tenenbaum , Dan Yamins , and Judith Fan
    In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks , 2021