Currently, I am a research assistant professor in the School of Computing at KAIST. I work with Prof. Juho Kim in KIXLAB and Prof. Insik Shin in CPS Lab.
My research interests are focused on eliciting and leveraging diverse annotations from people to convert raw data to meaningful information, which can harness artificial intelligence systems and provide valuable insight to end users.
Keywords: #Human Computation; #Crowdsourcing; #Human-Computer Interaction; #Artificial Intelligence
Conference and Journal Papers
- Zhefan Ye, Jean Y. Song, Zhiqiang Sui, Stephen Hart, Jorge Vilchis, Arbor, Walter S. Lasecki, and Odest C. Jenkins. Human-in-the-loop Pose Estimation via Shared Autonomy. In Proceedings of the ACM International Conference on Intelligent User Interfaces (IUI 2021). Best Paper Honorable Mention
- Stephan J. Lemmer, Jean Y. Song, and Jason J. Corso. Crowdsourcing More Effective Initializations for Single-target Trackers Through Automatic Re-querying. In Proceedings of the ACM/SIGCHI Conference on Human Factors in Computing Systems (CHI 2021).
- Yoonjoo Lee, John Joon Young Chung, Jean Y. Song, Minsuk Chang, and Juho Kim. Personalizing Ambience and Illusionary Presence: How People Use "Study with Me" Videos to Create Effective Studying Environments. In Proceedings of the ACM/SIGCHI Conference on Human Factors in Computing Systems (CHI 2021).
- Jean Y. Song, John Joon Young Chung, David F. Fouhey, and Walter S. Lasecki. C-Reference: Improving 2D to 3D Object Pose Estimation Accuracy via Crowdsourced Joint Object Estimation. In Proceedings of the ACM International Conference on Computer Supported Cooperative Work and Social Computing (CSCW 2020).
- Divya Ramesh, Anthony Z. Liu, Andres J. Echeverria, Jean Y. Song, Nicholas R. Waytowich, and Walter S. Lasecki. Yesterday’s Reward is Today’s Punishment: Contrast Effects in Human Feedback to Reinforcement Learning Agents. In Proceedings of the International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2020). Pragnesh Jay Modi Best Student Paper
- Yan Chen, Maulishree Pandey, Jean Y. Song, Walter S. Lasecki, and Steve Oney. Improving Crowd-Supported GUI Testing with Structural Guidance. In Proceedings of the ACM/SIGCHI Conference on Human Factors in Computing Systems (CHI 2020).
- John Joon Young Chung, Jean Y. Song, Sindhu Kutty, Sungsoo (Ray) Hong, Juho Kim, and Walter S. Lasecki. Efficient Elicitation Approaches to Estimate Collective Crowd Answers. In Proceedings of the ACM International Conference on Computer Supported Cooperative Work and Social Computing (CSCW 2019). Austin, TX. Best Paper Honorable Mention
- Jean Y. Song, Raymond Fok, Juho Kim, and Walter S. Lasecki. FourEyes: Leveraging Tool Diversity as a Means to Improve Aggregate Accuracy in Crowdsourcing. In ACM Transactions on Interactive Intelligent Systems, Volume 19, Issue 1, Article No. 3 (TiiS 2019).
- Jean Y. Song, Stephan J. Lemmer, Michael Xieyang Liu, Shiyan Yan, Juho Kim, Jason J. Corso, and Walter S. Lasecki. Popup: Reconstructing 3D Video Using Particle Filtering to Aggregate Crowd Responses. In Proceedings of the ACM International Conference on Intelligent User Interfaces (IUI 2019). Los Angeles, CA.
- Jean Y. Song, Raymond Fok, Alan Lundgard, Fan Yang, Juho Kim, and Walter S. Lasecki. Two Tools are Better Than One: Tool Diversity as a Means of Improving Aggregate Crowd Performance. In Proceedings of the ACM International Conference on Intelligent User Interfaces (IUI 2018). Tokyo, Japan. Best Student Paper Honorable Mention
Posters, Demos, and Workshop Papers
- Andrew M. Vernier, Jean Y. Song, Edward Sun, Allison Kench, and Walter S. Lasecki. Towards Universal Evaluation of Image Annotation Interfaces. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST 2019). New Orleans, LA.
- Jean Y. Song, Raymond Fok, Fan Yang, Kyle Wang, Alan Lundgard, and Walter S. Lasecki. Tool Diversity as a Means of Improving Aggregate Crowd Performance on Image Segmentation Tasks. In HCOMP Workshop on Human Computation for Image and Video Analysis (HCOMP GroupSight 2017). Quebec City, Quebec. 2017.
- Sai R. Gouravajhala, Jean Y. Song, Jinyeong Yim, Raymond Fok, Yanda Huang, Fan Yang, Kyle Wang, Yilei An, and Walter S. Lasecki. Towards Hybrid Intelligence for Robotics. In Collective Intelligence Conference (CI 2017). New York, NY.
- Jean Y. Song and Charles R. Meyer. 2D-3D Image Registration using Thin-Plate Spline and Volume Rendering. SPIE Medical Imaging 2015. Orlando, FL.
- Jean Y. Song, Jeffrey A. Fessler, and Charles R. Meyer. Adaptive Filtering on Conditional Mutual Information for Intermodal Non-Rigid Image Registration. IEEE Nuclear Science Symposium and Medical Imaging Conference (NSS/MIC 2014). Seattle, WA.
- Jean Y. Song. Eliciting and Leveraging Input Diversity in Crowd-Powered Intelligent Systems. University of Michigan Ph.D. Thesis. 2019.
Research in robotics and autonomous vehicles suffers from a lack of realistic training data and environments in which to test new approaches. Rare and unusual events such as traffic accidents occur several orders of magnitude less frequently than is needed to collect large enough training and testing sets, presenting a fundamental bottleneck in the research and deployment of such systems. Thus, we propose to use a crowdsourced human-in-the-loop approach to guide computer vision algorithms to extract measurement information from large video corpora, allowing us to create simulations of scene dynamics for training and testing.
Dialog videos contain rich contextual, emotional, and intentional cues of the characters and their surroundings. In this project, we aim to build a crowdsourcing platform that collects these information from a large dialog video dataset. The collection and aggregation process can be challenging because the temporal dimension of the dataset has to be considered, and the labels can be highly subjective. We combat these challenges by exploring crowdsourcing techniques to design workflows and answer aggregation methods that efficiently collects multi-dimensional labels and overcome the subjective nature of the collected annotations.
In designing crowdsourcing tasks, we want to achieve as high accuracy as possible from the given resources. In this work, we introduced an approach of leveraging tool diversity as a means of improving aggregate crowd performance. We define tool diversity as a property of a system (or a task), that enables to use different tools for a same task. In semantic image segmentation tasks, we show that our approach improves the aggregate accuracy significantly, compared to using a single best tool alone.