About
Keywords: #Human Computation; #Crowdsourcing; #Human-Computer Interaction; #Artificial Intelligence
Publications
Conference and Journal Papers
- Dongyoon Han*, Junsuk Choe*, Seonghyeok Chun, John Chung, Minsuk Chang, Sangdoo Yun, Jean Y. Song, Seong Joon Oh. "Neglected Free Lunch - Learning Image Classifiers Using Annotation Byproducts " In Proceedings of the International Conference on Computer Vision (ICCV 2023). (* Equal contribution) (To appear)
- Jean Y. Song*, Sangwook Lee*, Jisoo Lee, Mina Kim, and Juho Kim. "ModSandbox: Facilitating Online Community Moderation Through Error Prediction and Improvement of Automated Rules. " In Proceedings of the ACM/SIGCHI Conference on Human Factors in Computing Systems (CHI 2023). (* Equal contribution)
- Seoyun Son, Junyoung Choi, Sunjae Lee, Jean Y. Song, and Insik Shin. "It is Okay to be Distracted: How Real-time Transcriptions Facilitate Online Meeting with Distraction. " In Proceedings of the ACM/SIGCHI Conference on Human Factors in Computing Systems (CHI 2023).
- Sunjae Lee, Hoyoung Kim, Sijung Kim, Sangwook Lee, Hyosu Kim, Jean Y. Song, Steven Y. Ko, Sangeun Oh, and Insik Shin. "A-Mash: Providing Single-app Illusion for Multi-app Use through User-centric UI Mashup. " In Proceedings of the International Conference On Mobile Computing And Networking (MobiCom 2022).
- Yoonjoo Lee, John Joon Young Chung, Taesoo Kim, Jean Y. Song, and Juho Kim. "Promptiverse: Scalable Generation of Scaffolding Prompts through Human-AI Knowledge Graph Annotation. " In Proceedings of the ACM/SIGCHI Conference on Human Factors in Computing Systems (CHI 2022).
- Sunjae Lee, Hayeon Lee, Hoyoung Kim, Sangmin Lee, Jeong Woon Choi, Yuseung Lee, Seono Lee, Ahyeon Kim, Jean Y. Song, Sangeun Oh, Steven Y. Ko, and Insik Shin. "FLUID-XP: Flexible User Interface Distribution forCross-Platform Experience. " In Proceedings of the International Conference On Mobile Computing And Networking (MobiCom 2021).
- Zhefan Ye, Jean Y. Song, Zhiqiang Sui, Stephen Hart, Jorge Vilchis, Arbor, Walter S. Lasecki, and Odest C. Jenkins. "Human-in-the-loop Pose Estimation via Shared Autonomy. " In Proceedings of the ACM International Conference on Intelligent User Interfaces (IUI 2021). Best Paper Honorable Mention
- Stephan J. Lemmer, Jean Y. Song, and Jason J. Corso. "Crowdsourcing More Effective Initializations for Single-target Trackers Through Automatic Re-querying. " In Proceedings of the ACM/SIGCHI Conference on Human Factors in Computing Systems (CHI 2021).
- Yoonjoo Lee, John Joon Young Chung, Jean Y. Song, Minsuk Chang, and Juho Kim. "Personalizing Ambience and Illusionary Presence: How People Use "Study with Me" Videos to Create Effective Studying Environments. " In Proceedings of the ACM/SIGCHI Conference on Human Factors in Computing Systems (CHI 2021).
- Jean Y. Song, John Joon Young Chung, David F. Fouhey, and Walter S. Lasecki. "C-Reference: Improving 2D to 3D Object Pose Estimation Accuracy via Crowdsourced Joint Object Estimation. " In Proceedings of the ACM International Conference on Computer Supported Cooperative Work and Social Computing (CSCW 2020).
- Divya Ramesh, Anthony Z. Liu, Andres J. Echeverria, Jean Y. Song, Nicholas R. Waytowich, and Walter S. Lasecki. "Yesterday’s Reward is Today’s Punishment: Contrast Effects in Human Feedback to Reinforcement Learning Agents. " In Proceedings of the International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2020). Pragnesh Jay Modi Best Student Paper
- Yan Chen, Maulishree Pandey, Jean Y. Song, Walter S. Lasecki, and Steve Oney. "Improving Crowd-Supported GUI Testing with Structural Guidance. " In Proceedings of the ACM/SIGCHI Conference on Human Factors in Computing Systems (CHI 2020).
- John Joon Young Chung, Jean Y. Song, Sindhu Kutty, Sungsoo (Ray) Hong, Juho Kim, and Walter S. Lasecki. "Efficient Elicitation Approaches to Estimate Collective Crowd Answers. " In Proceedings of the ACM International Conference on Computer Supported Cooperative Work and Social Computing (CSCW 2019). Austin, TX. Best Paper Honorable Mention
- Jean Y. Song, Raymond Fok, Juho Kim, and Walter S. Lasecki. "FourEyes: Leveraging Tool Diversity as a Means to Improve Aggregate Accuracy in Crowdsourcing. " In ACM Transactions on Interactive Intelligent Systems, Volume 19, Issue 1, Article No. 3 (TiiS 2019).
- Jean Y. Song, Stephan J. Lemmer, Michael Xieyang Liu, Shiyan Yan, Juho Kim, Jason J. Corso, and Walter S. Lasecki. "Popup: Reconstructing 3D Video Using Particle Filtering to Aggregate Crowd Responses. " In Proceedings of the ACM International Conference on Intelligent User Interfaces (IUI 2019). Los Angeles, CA.
- Jean Y. Song, Raymond Fok, Alan Lundgard, Fan Yang, Juho Kim, and Walter S. Lasecki. "Two Tools are Better Than One: Tool Diversity as a Means of Improving Aggregate Crowd Performance. " In Proceedings of the ACM International Conference on Intelligent User Interfaces (IUI 2018). Tokyo, Japan. Best Student Paper Honorable Mention
Posters, Demos, and Workshop Papers
- Hyungyu Shin, Nabila Sindi, Yoonjoo Lee, Jaeryoung Ka, Jean Y. Song, and Juho Kim. "XDesign: Integrating Interface Design into Explainable AI Education. " In Proceedings of the ACM Technical Symposium on Computer Science Education (SIGCSE TS 2022).
- Andrew M. Vernier, Jean Y. Song, Edward Sun, Allison Kench, and Walter S. Lasecki. "Towards Universal Evaluation of Image Annotation Interfaces. " In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST 2019). New Orleans, LA.
- Jean Y. Song, Raymond Fok, Fan Yang, Kyle Wang, Alan Lundgard, and Walter S. Lasecki. "Tool Diversity as a Means of Improving Aggregate Crowd Performance on Image Segmentation Tasks. " In HCOMP Workshop on Human Computation for Image and Video Analysis (HCOMP GroupSight 2017). Quebec City, Quebec. 2017.
- Sai R. Gouravajhala, Jean Y. Song, Jinyeong Yim, Raymond Fok, Yanda Huang, Fan Yang, Kyle Wang, Yilei An, and Walter S. Lasecki. "Towards Hybrid Intelligence for Robotics. " In Collective Intelligence Conference (CI 2017). New York, NY.
- Jean Y. Song and Charles R. Meyer. " 2D-3D Image Registration using Thin-Plate Spline and Volume Rendering. " SPIE Medical Imaging 2015. Orlando, FL.
- Jean Y. Song, Jeffrey A. Fessler, and Charles R. Meyer. " Adaptive Filtering on Conditional Mutual Information for Intermodal Non-Rigid Image Registration. " IEEE Nuclear Science Symposium and Medical Imaging Conference (NSS/MIC 2014). Seattle, WA.
Thesis
- Jean Y. Song. "Eliciting and Leveraging Input Diversity in Crowd-Powered Intelligent Systems. " University of Michigan Ph.D. Thesis. 2019.
Projects
Robust 4D Simulation of Rare Events enabled by Human-Augmented Computer Vision
Research in robotics and autonomous vehicles suffers from a lack of realistic training data and environments
in which to test new approaches. Rare and unusual events such as traffic accidents occur several orders of
magnitude less frequently than is needed to collect large enough training and testing sets, presenting a
fundamental bottleneck in the research and deployment of such systems. Thus, we propose to use a crowdsourced
human-in-the-loop approach to guide computer vision algorithms to extract measurement information from large
video corpora, allowing us to create simulations of scene dynamics for training and testing.
Crowdsourcing Emotion, Intention, and Context Annotations from Dialog Videos
Dialog videos contain rich contextual, emotional, and intentional cues of the characters and their surroundings.
In this project, we aim to build a crowdsourcing platform that collects these information from a large dialog video
dataset. The collection and aggregation process can be challenging because the temporal dimension of the dataset
has to be considered, and the labels can be highly subjective. We combat these challenges
by exploring crowdsourcing techniques to design workflows and answer aggregation methods that efficiently collects
multi-dimensional labels and overcome the subjective nature of the collected annotations.
Improving Aggregate Crowd Performance on Crowd-Assisted Image Segmentation
In designing crowdsourcing tasks, we want to achieve as high accuracy as possible from the given
resources. In this work, we introduced an approach of leveraging tool diversity as a means of improving aggregate
crowd performance. We define tool diversity as a property of a system (or a task), that enables to use
different tools for a same task. In semantic image segmentation tasks, we show that our approach
improves the aggregate accuracy significantly, compared to using a single best tool alone.