Jacob Zhiyuan Fang
I may not maintain this website in future. Check new website.
I obtained my Ph.D. degree from the Department of Computer Science at Arizona State University (ASU) as a member of APG Lab led by Dr. Yezhou Yang. Before joining ASU, I obtained my BEng degree from Department of Computer Science and Engineering at Southern University of Science and Technology. I've also spent great times collaborating with researchers in Microsoft, UC, Irvine, Chinese Academy of Sciences as research intern.
Education
- Ph.D. (Computer Science), Arizona State University. 2017.8 - 2022.5
- BEng (Computer Science and Engineering), Southern University of Science and Technology. 2013.9 - 2017.6
Experiences
|
Amazon Alexa-AI, Sunnyvale, USA Applied Scientist, June 2022. |
|
|
Mirosoft, Azure Florence - Vision and Language, Redmond, USA Research Intern, Summer 2021. Mentors: Jianfeng Wang, Zhe Gan, Xiaowei Hu, Zicheng Liu, Lijuan Wang |
|
|
Mirosoft, AI Cloud & Platform, Redmond, USA Research Intern, Summer 2020. Mentors: Jianfeng Wang, Lei Zhang, Lijuan Wang, Zicheng Liu |
|
|
MM Lab, Chinese Academy of Science, Shenzhen, China Research Intern, 2017 Mentors:
Yu Qiao,
Zhifeng Li |
|
|
IGB, University of California, Irvine, USA Research Intern, Summer 2016 |
|
|
Southern University of Science and Technology, Shenzhen, China Bachelor of Engineering, 2013-2017 |
|
Selected Publications & Preprints
 |
Huiliang Shao, Zhiyuan Fang, Yezhou Yang, CAVAN: Commonsense Knowledge Anchored Video Captioning International Conference on Pattern Recognition, Montreal, Aug, 2022 [PDF] |
 |
Arnav Chakravarthy, Zhiyuan Fang, Yezhou Yang, Tragedy Plus Time: Capturing Unintended Human Activities from Weakly-labeled Videos Robustness in Sequential Data Workshop (ROSE) @ CVPR 2022, Oral Presentation, Louisiana, June, 2022 [PDF][Code][Website] |
 |
Zhiyuan Fang Building Vision and Language Models with Implicit Supervision and Increased Efficiency Doctorate Dissertation, April, 2022 [PDF] |
 |
Zhiyuan Fang, Jianfeng Wang, Xiaowei Hu, Lin Liang, Zhe Gan, Lijuan Wang, Yezhou Yang, Zicheng Liu Injecting Semantic Concepts into End-to-End Image Captioning Conference on Computer Vision and Pattern Recognition, (CVPR), New Orleans-Louisiana, June, 2022 [PDF][Code][Slide][Poster] |
 |
Zhiyuan Fang, Jianfeng Wang, Xiaowei Hu, Lijuan Wang, Yezhou Yang, Zicheng Liu Compressing Visual-linguistic Model via Knowledge Distillation International Conference on Computer Vision (ICCV), Virtual, Aug, 2021 [PDF] |
 |
Zhiyuan Fang, Jianfeng Wang, Lijuan Wang, Lei Zhang, Yezhou Yang, Zicheng Liu SEED: Self-supervised Distillation For Visual Representation International Conference on Representation Learning (ICLR), Virtual, May, 2021 [PDF][Presentation][Code] |
 |
Zhiyuan Fang, Shu Kong, Zhe Wang, Charless Fowlkes, Yezhou Yang Weakly-Supervised Temporal-Language Association with Referring Attention arXiv preprint arXiv:2006.11747 [ Rejected 3 times by: ICCV19, CVPR20, WACV20  ][PDF] [Video Demo][Website] |
 |
Zhe Wang⋆, Zhiyuan Fang⋆, Jun Wang, Yezhou Yang ViTAA: Visual-Textual Attributes Alignment in Person Search by Natural Language European Conference on Computer Vision (ECCV), Virtual, July, 2020 [PDF][Code] |
 |
Zhiyuan Fang⋆,Tejas Gokhale⋆, Pratyay Banerjee, Chitta Baral, Yezhou Yang Video2Commonsense: Generating Commonsense Descriptions to Enrich Video Captioning Conference on Empirical Methods in Natural Language Processing, EMNLP, Long&Oral Presentation, Nov 2020 [PDF][Website & Data][Code] |
 |
Tejas Gokhale, Shailaja Sampat, Zhiyuan Fang, Yezhou Yang, Chitta Baral Cooking With Blocks : A Recipe for Visual Reasoning on Image-Pairs CVPR 2019 Vision Meets Cognition Workshop [Website] |
 |
Zhiyuan Fang, Shu Kong, Charless Fowlkes, Yezhou Yang Modularized Textual Grounding for Counterfactual Resilience IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, June 2019. [Website][PDF][Poster][Code] [Slide] |
 |
Zhiyuan Fang, Shu Kong, Tianshu Yu, Yezhou Yang Weakly Supervised Attention Learning for Textual Phrases Grounding CVPR 2018 Language And Vision Workshop [PDF][Poster][Slides] |
 |
Xiao Zhang, Zhiyuan Fang, Yandong Wen, Zhifeng Li, Yu Qiao Range Loss for Deep Face Recognition with Long tail International Conference on Computer Vision (ICCV), Venice, Italy, 2017. [PDF][Poster][Code] |
 |
Juan Wang, Zhiyuan Fang, Ning Lang, Huishu Yuan, Min-Ying Su, and Pierre Baldi A Multi-Resolution Approach for Spinal Metastasis Detection using Deep Siamese Neural Networks Computers in Biology and Medicine, 2017 (Honor Paper, 5%) [PDF][Code] |
Collaborators
I am fortunate to have worked with some great researchers and I sincerely appreciate all the helps from them:| Shu Kong (CMU) | Lijuan Wang (Microsoft) | Xiaowei Hu (Microsoft) | Zicheng Liu(Microsoft) | Jianfeng Wang (Microsoft) | Zhe Gan (Microsoft) |
| Tianshu Yu (CUHK) | Yu Qiao (MMLAB) | Chengxi Ye (Amazon) | Xiao Zhang (CUHK) | Tejas Gokhale (ASU) | Pratyay Banerjee (ASU) |
| Zhifeng Li (Tencent) | Charless Fowlkes (UCI) | Shailaja Sampat (ASU) | Pierre Baldi (UCI) | Lei Zhang (IDEA) | Lingqi Zhang (Upenn) |
| Yandong Wen (CMU) | Chitta Baral (ASU) | Juan Wang (IIT) | Kun Chen (SUSTech) | Huiliang Shao (ASU) | Zhe Wang (MSRA) | ... |
Also shout out to my Ph.D. Committee Supervisors:
| Prof. Yezhou Yang (ASU) | Prof. Chitta Baral (ASU) | Prof. Huan Liu (ASU) | Prof. Zicheng Liu (Microsoft) |
Professional activities
- Conference reviewer & Programme Committee:
CVPR-19/20/21; Neurips-21; ICLR-22; ICCV-21; AAAI-21; WACV-20/21; BMVC-19/20/21; ACCV-20; ICRA-21; ... - Journal reviewer:
Transactions on Image Processing (TIP)
Pattern Recognition, Elsevier
IEEE Transactions on Circuits and Systems for Video Technology
- Teaching associative:
FIN208: Data Mining, 2017 Spring, SusTech
CSE205: Object Oriented Programming, 2017 Fall, 2018 Spring, ASU
CSE230: Computer Organization and Assembly Programming, 2018 Spring, ASU
CSE310: Data Structures and Algorithms, 2017 Fall, 2018 Fall, 2021 Fall, ASU
CSE510: Artificial Intelligence, 2019 Spring, ASU & Coursera Online Master Degrees
- Academic Seminar Organizer:
Frontier topics in Vision and/or Language, Spring 2021 - Conference Workshop Organizer:
O-DRUM: Open-Domain Retrieval Under Multi-Modal Setting, June 2022
Awards
- CVPR-2022 Traval Award, CVPR, 2022
- SCAI Doctoral Fellowship, ASU, 2022
- Outstanding Reviewer, BMVC, 2019
- Engineering Grad Fellowship, Ira A. Fulton School of Engineering. 2018-2019, 2021-2022
- Prominent Artificial Intelligence Journal Paper Award Computers in Biology and Medicine, 2018