Jacob Zhiyuan Fang

Email: zy.fang [at] asu [dot] edu

I am a 5th-year Ph.D. candidate in the Department of Computer Science at Arizona State University (ASU) and a member of APG Lab led by Dr. Yezhou Yang. I also work closely with Prof. Chitta Baral . My research interests mainly lie in Vision and Language, particularly in representation learning and image/video captioning. Meanwhile, I'm also interested in working on efficient VL, aiming to build lightweight, data-efficient and real-world applicable VL models.

Before joining ASU, I obtained my BEng degree from Department of Computer Science and Engineering at Southern University of Science and Technology. I've also spent great times collaborating with researchers in Microsoft, UC, Irvine, Chinese Academy of Sciences as research intern.



Mirosoft, Azure Florence - Vision and Language, Redmond, USA

       Research Intern,  Summer 2021.

       Mentors: Jianfeng Wang, Zhe Gan, Xiaowei Hu, Zicheng Liu, Lijuan Wang

Mirosoft, AI Cloud & Platform, Redmond, USA

       Research Intern,  Summer 2020.

       Mentors: Jianfeng Wang, Lei Zhang, Lijuan Wang, Zicheng Liu

MM Lab, Chinese Academy of Science, Shenzhen, China

       Research Intern,  2017

       Mentors: Yu Qiao, Zhifeng Li

IGB, University of California, Irvine, USA

       Research Intern,  Summer 2016

       Mentors: Juan Wang, Pierre Baldi

Southern University of Science and Technology, Shenzhen, China

       Bachelor of Engineering,  2013-2017

       Academic Mentors: Qi Hao, Kun Chen


I am fortunate to have worked with some great researchers and I sincerely appreciate all the helps from them:
Shu Kong (CMU) Lijuan Wang (Microsoft) Xiaowei Hu (Microsoft) Zicheng Liu(Microsoft) Jianfeng Wang (Microsoft) Zhe Gan (Microsoft)
Tianshu Yu (CUHK) Yu Qiao (MMLAB) Chengxi Ye (Amazon) Xiao Zhang (CUHK) Tejas Gokhale (ASU) Pratyay Banerjee (ASU)
Zhifeng Li (Tencent) Charless Fowlkes (UCI) Shailaja Sampat (ASU) Pierre Baldi (UCI) Lei Zhang (IDEA) Lingqi Zhang (Upenn)
Yandong Wen (CMU) Chitta Baral (ASU) Juan Wang (IIT) Kun Chen (SUSTech) Huiliang Shao (ASU) Zhe Wang (MSRA)

Also shout out to my Ph.D. Committee Supervisors:
Prof. Yezhou Yang (ASU) Prof. Chitta Baral (ASU) Prof. Huan Liu (ASU) Prof. Zicheng Liu (Microsoft)

Selected Publications & Preprints

Google Scholar for full publication list
<⋆>: equal contribution.
Zhiyuan Fang, Jianfeng Wang, Xiaowei Hu, Lijuan Wang, Yezhou Yang, Zicheng Liu
Compressing Visual-linguistic Model via Knowledge Distillation
International Conference on Computer Vision (ICCV), Virtual, Aug, 2021
Zhiyuan Fang, Jianfeng Wang, Lijuan Wang, Lei Zhang, Yezhou Yang, Zicheng Liu
SEED: Self-supervised Distillation For Visual Representation
International Conference on Representation Learning (ICLR), Virtual, May, 2021
Zhe Wang⋆, Zhiyuan Fang⋆, Jun Wang, Yezhou Yang
ViTAA: Visual-Textual Attributes Alignment in Person Search by Natural Language
European Conference on Computer Vision (ECCV), Virtual, July, 2020
Zhiyuan Fang⋆,Tejas Gokhale⋆, Pratyay Banerjee, Chitta Baral, Yezhou Yang
Video2Commonsense: Generating Commonsense Descriptions to Enrich Video Captioning
Conference on Empirical Methods in Natural Language Processing, (EMNLP, long paper), Nov 2020
[PDF][Website & Data][Code]
Zhiyuan Fang, Shu Kong, Charless Fowlkes, Yezhou Yang
Modularized Textual Grounding for Counterfactual Resilience
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, June 2019.
[Website][PDF][Poster][Code] [Slide]
Zhiyuan Fang, Shu Kong, Tianshu Yu, Yezhou Yang
Weakly Supervised Attention Learning for Textual Phrases Grounding
CVPR 2018 Language And Vision Workshop
Xiao Zhang, Zhiyuan Fang, Yandong Wen, Zhifeng Li, Yu Qiao
Range Loss for Deep Face Recognition with Long tail
International Conference on Computer Vision (ICCV), Venice, Italy, 2017.
Juan Wang, Zhiyuan Fang, Ning Lang, Huishu Yuan, Min-Ying Su, and Pierre Baldi
A Multi-Resolution Approach for Spinal Metastasis Detection using Deep Siamese Neural Networks
Computers in Biology and Medicine, 2017 (Honor Paper, 5%)

Professional activities

  • Conference reviewer & Programme Committee:
    CVPR-19/20/21;   Neurips-21;   ICLR-22;   ICCV-21;   AAAI-21;   WACV-20/21;   BMVC-19/20/21;   ACCV-20;   ICRA-21;   ...
  • Journal reviewer:
    Transactions on Image Processing (TIP)
  • Teaching associative:
      FIN208: Data Mining, 2017 Spring, SusTech
      CSE205: Object Oriented Programming, 2017 Fall, 2018 Spring, ASU
      CSE230: Computer Organization and Assembly Programming, 2018 Spring, ASU
      CSE310: Data Structures and Algorithms, 2017 Fall, 2018 Fall, 2021 Fall, ASU
      CSE510: Artificial Intelligence, 2019 Spring, ASU & Coursera Online Master Degrees


Designed via Bootstrap. Credits to my friend Jian Kang.