Jie Gao
Jie (Sophia) Gao

Jie (Sophia) Gao

Malone Postdoc Fellow
Johns Hopkins University

Malone Center for Engineering in Healthcare Data Science and AI Institute Center for Language and Speech Processing
CV Google Scholar Twitter LinkedIn

Human-Centered Interpretation of Agentic AI

Building a science of interpretive work across humans, autonomous agents, and their collaborations.

I am currently a Malone Postdoc Fellow at Johns Hopkins University, where I am mentored by Mark Dredze, Ziang Xiao, and Chien-Ming Huang. My research helps people make sense of large datasets and develop intuition from the insights they reveal. I use a range of social science methodologies to support this process. I also bring human perspectives into AI systems to make them more understandable, interpretable, and intuitive. My work is interdisciplinary, spanning AI, HCI, software engineering, and the social sciences. I publish my work in venues such as NeurIPS, EMNLP, CHI, TOCHI, and ICPC. I founded and currently lead the first Interpreting Agent Behavior (IAB) Workshop@NeurIPS 2026. I was also a co-organizer of the LLMs as Research Tools workshop @CHI 2024. Previously, I was a Postdoctoral Associate at MIT's M3S program in Singapore, advised by Thomas Malone. I received my Ph.D. in Information Systems Technology and Design from Singapore University of Technology and Design (SUTD), advised by Simon Perrault. During my Ph.D., I was a visiting student at the University of Notre Dame and the National University of Singapore. See more in my CV.

News

Recent updates
More news (37)
  • Organizing the IAB Workshop @NeurIPS 2026, on bringing interpretation methods from HCI and the social sciences to build interpretability for autonomous agents.
  • Gave a talk, “Toward Seeing What Agents Do: Interpretive Work in the Age of Agentic AI,” at Monash University
  • Gave a talk, “Toward Seeing What Agents Do: Interpretive Work in the Age of Agentic AI,” at Fudan University (News)
  • Our preprint How to Interpret Agent Behavior is out! We introduce Act*ONOMY, a hierarchical taxonomy for analyzing autonomous agent behavior at runtime.
  • Our CodeMap paper received an award at ICPC 2026 (a CORE A conference in code comprehension)! ACM SIGSOFT Distinguished Paper Award
  • Happy to give a talk at Advanced HCI at JHU (Slides)
  • Gave a Claude Code How-To Session (Slides) for 40+ JHU researchers (Master's students, PhD students, postdocs, and faculty): covering practical usage and best practices of Claude Code and OpenClaw, provided live demos.
  • Our paper on supporting developers in understanding new codebases has been accepted to the 34th IEEE/ACM International Conference on Program Comprehension (ICPC 2026)! Check out the preprint: Understanding Codebase like a Professional! Human-AI Collaboration for Code Comprehension | CodeMap Website
  • I attended EMNLP2025 in Suzhou, China. We presented our position paper, From Noise to Nuance: Enriching Subjective Data Annotation through Qualitative Analysis on generalizing qualitative data analysis to subjective data annotation domain.
  • I am serving as an Associate Chair for CHI2026 under subcommittee Understanding People — Statistical and Quantitative Methods.
  • I attended VL/HCC 2025 conference in North Carolina. Met new friends and had a great time!
  • I attended CHI 2025 conference in Japan, and met a lot of old and new friends!
  • We introduce mindcoder.ai, a web application designed to support automated qualitative analysis. The user only needs to follow three steps to obtain qualitative analysis results-1) upload data, 2) start coding, and 3) get analysis report! See how to use-video.
  • Started my postdoc fellowship at Johns Hopkins University!
  • I will join the Malone postdoctoral fellow program at Johns Hopkins University next March! I will be working with Mark Dredze, Ziang Xiao and Chien-Ming Huang to explore the potential of LLM in qualitative analysis methodology usage in healthcare.
  • I am serving as an Associate Chair for CHI2025 under subcommittee "Understanding People — Statistical and Quantitative Methods" this year!
  • Attended CHI2024 and met old and new friends! I co-organized a workshop "LLMs as Research Tools".
  • Our Late-breaking Work "A Taxonomy for Human-LLM Interaction Modes-An Initial Exploration" has been accepted to CHI 2024.
  • We have launched CollabCoder Website. Construction and refinement is ongoing!
  • One first-authored paper "CollabCoder" and one coauthored paper "Help Me Reflect" have been accepted to CHI2024! (Preprints will have minor refinements in final versions.)
  • I successfully defended my PhD thesis titled "Tool Support for Human-AI Collaboration in Qualitative Data Analysis".
  • We will organize a workshop "LLMs as Research Tools" at CHI2024, from 11-16 May 2024 in Honolulu, Hawaii, USA.
  • Starting from early 2024, I am excited to become a part of the Singapore-MIT Alliance for Research and Technology as a Postdoctoral Researcher, where I will be mentored by Prof. Thomas W. Malone. I'm thrilled about this opportunity!
  • I attended EMNLP2023 at Singapore. Happy to learn many new things and friends!
  • Presented CollabCoder project in the Immersive Reality Lab at NUS. Thanks for the invitation from Prof Khoo Eng Tat!
  • Attended CSCW2023 and presented CollabCoder demo virtually! (due to visa and funding issues)
  • One coauthored paper "GlassMessaging" has been accepted to Ubicomp2023!
  • One coauthored paper has been accepted to ICSME2023!
  • One coauthored paper "VISAR" has been accepted to UIST2023!
  • One first-authored paper "CoAIcoder" has been accepted to TOCHI!
  • Our "CollabCoder" platform has been selected for a demo work at CSCW2023!
  • Excited to start as a visiting Ph.D. student at Notre Dame, working with Prof Toby's team!
  • Work as an associate chair for the Late-Breaking Work Committee of CHI2023.
  • Thrilled to begin as a visiting Ph.D. student at NUS-HCI lab, looking forward to collaborating with Prof Shen and the lab team!
  • Glad to attend CHI2022 in person in New Orleans, LA, USA. Thanks for the funding support from Simon.
  • Worked as an associate chair for the Late-Breaking Work Committee of CHI2022.

Research

Building a science of interpretive work across humans, autonomous agents, and their collaborations.

Agentic AI Multi-Agent Systems Human-AI Collaboration AI-assisted Qualitative Analysis AI-assisted Code Comprehension

Qualitative Analysis

Helping humans understand unstructured text

How can AI support, rather than replace, the interpretive work of reading unstructured text? I design and evaluate systems that help people code qualitative data, align interpretations across multiple coders, and develop theories from raw text, while keeping human judgment central.

Read relevant papers (7)

Code Comprehension

Helping humans understand code

Real-world codebases are messy, and newcomers often struggle to build an accurate mental model. I build tools that help developers understand unfamiliar code, review AI-generated changes critically, and collaborate with AI assistants without giving up their own judgment.

Read relevant papers (2)

Agent Behavior Analysis

Helping humans understand agents

As AI systems take on increasingly autonomous roles, the bottleneck shifts from "can they do it" to "can humans understand or control what they do." I study how humans can understand and control agents when they run autonomously.

Read relevant papers (3)
Interpreting Agent Behavior · NeurIPS 2026 Workshop

Beyond these three lines, I also work on other topics such as human-LLM interaction modes, AI-assisted writing, online deliberation, and messaging on smart glasses.

Selected publications

Google Scholar
  1. 2026

    How to Interpret Agent Behavior

    Jie Gao, Kaiser Sun, Jen-tse Huang, Katherine Van Koevering, Sijie Ji, Heyuan Huang, Weiyan Shi, Zhuoran Lu, Ziang Xiao, Daniel Khashabi, Mark Dredze

    Conference on Neural Information Processing Systems (NeurIPS 2026) [Paper]
  2. 2026

    Using Grounded Theory for Agent Behavior Analysis at Scale

    Zhuoran Lu, Yangyang Yu, Zhuoyan Li, Yibo Meng, Nan Jiang, Chengxi Zang, Jie Gao, Ziang Xiao

    Conference on Empirical Methods in Natural Language Processing (EMNLP 2026) · Findings [Paper]
  3. 2026

    Agent-as-Peer-Debriefer: A Multi-Agent Framework with Perspective-Based Refinement for Qualitative Analysis

    Zhimin Lin, Kun Cheng, Zhiyao Shu, Junhua Fang, Juntao Li, Fan Bai, Jie Gao#

    # Corresponding author

    Conference on Empirical Methods in Natural Language Processing (EMNLP 2026) · Main Conference · Acceptance rate <15.4% (2,719/17,669 papers) [Paper]
  4. 2026

    Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents under Knowledge Conflict

    Kaiser Sun, Bernal Jiménez Gutiérrez, Hongjun Liu, Jingyu Zhang, Jie Gao, Mark Dredze, Daniel Khashabi

    Conference on Empirical Methods in Natural Language Processing (EMNLP 2026) · Findings
  5. 2026

    Understanding Codebase like a Professional! Human–AI Collaboration for Code Comprehension

    Jie Gao, Yue Xue, Xiaofei Xie, Junming Cao, SoeMin THANT, Erika Lee, Bowen Xu

    IEEE/ACM International Conference on Program Comprehension (ICPC 2026) [Paper] [Website]
    ACM SIGSOFT Distinguished Paper Award
  6. 2024

    CollabCoder: A Lower-barrier, Rigorous Workflow for Inductive Collaborative Qualitative Analysis with Large Language Models

    Jie Gao, Yuchen Guo, Gionnieve Lim, Tianqin Zhang, Zheng Zhang, Toby Jia-Jun Li, Simon Tangi Perrault

    ACM Conference on Human Factors in Computing Systems (CHI 2024) [Paper] [Website] [Code]
  7. 2023

    CoAIcoder: Examining the effectiveness of AI-assisted human-to-human collaboration in qualitative analysis

    Jie Gao, Kenny Tsu Wei Choo, Junming Cao, Roy Ka-Wei Lee, Simon Perrault

    ACM Transactions on Computer-Human Interaction (TOCHI 2023) [Paper]
More publications
  1. 2025

    From Noise to Nuance: Enriching Subjective Data Annotation through Qualitative Analysis

    Ruyuan Wan, Haonan Wang, Ting-Hao ‘Kenneth’ Huang, Jie Gao#

    # Corresponding author

    Fourth Workshop on Bridging Human-Computer Interaction and Natural Language Processing (HCI+NLP), at the Conference on Empirical Methods in Natural Language Processing (EMNLP 2025) [Paper]
  2. 2024
  3. 2024

    Help Me Reflect: Leveraging Self-Reflection Interface Nudges to Enhance Deliberativeness on Online Deliberation Platforms

    Shun Yi Yeo, Gionnieve Lim, Jie Gao, Weiyu Zhang, Simon Tangi Perrault

    ACM Conference on Human Factors in Computing Systems (CHI 2024) [Paper]
  4. 2024

    A Taxonomy for Human-LLM Interaction Modes: An Initial Exploration

    Jie Gao*, Simret Araya Gebreegziabher*, Kenny Tsu Wei Choo, Toby Jia-Jun Li, Simon Tangi Perrault, Thomas W Malone

    ACM Conference on Human Factors in Computing Systems (CHI 2024) · Extended Abstracts [Paper]
  5. 2024

    LLMs as Research Tools: Applications and Evaluations in HCI Data Work

    Marianne Aubin Le Quéré, Hope Schroeder, Casey Randazzo, Jie Gao, Ziv Epstein, Simon Tangi Perrault, David Mimno, Louise Barkhuus, Hanlin Li

    ACM Conference on Human Factors in Computing Systems (CHI 2024) · Workshop Proposal [Paper]
  6. 2023

    VISAR: A Human-AI Argumentative Writing Assistant with Visual Programming and Rapid Draft Prototyping

    Zheng Zhang, Jie Gao, Ranjodh Singh Dhaliwal, Toby Jia-Jun Li

    ACM Symposium on User Interface Software and Technology (UIST 2023) [Paper]
  7. 2023

    GlassMessaging: Towards Ubiquitous Messaging Using OHMDs

    Nuwan Janaka, Jie Gao, Lin Zhu, Shengdong Zhao, Lan Lyu, Peisen Xu, Maximilian Nabokow, Silang Wang, Yanch Ong

    Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT 2023) [Paper]
  8. 2023

    Characterizing the complexity and its impact on testing in ml-enabled systems: A case sutdy on rasa

    Junming Cao, Bihuan Chen, Longjie Hu, Jie Gao, Kaifeng Huang, Xuezhi Song, Xin Peng

    IEEE International Conference on Software Maintenance and Evolution (ICSME 2023) [Paper]
  9. 2022

    Differences of Challenges of Working from Home (WFH) between Weibo and Twitter Users during COVID-19

    Jie Gao, Xiayin Ying, Junming Cao, Yifan Yang, Pin Sym Foong, Simon Perrault

    ACM Conference on Human Factors in Computing Systems (CHI 2022) · Extended Abstracts [Paper]

* Equal contribution

Research software

CodeMap

A human-AI collaborative tool for code comprehension. Helps developers understand complex codebases through interactive visualization and AI-guided exploration.

[Paper]

MindCoder

An AI-powered platform for flexible qualitative data analysis. Supports open coding, sub-theme grouping, and theme generation with human-AI collaboration.

[Paper]

CollabCoder

A GPT-powered workflow for collaborative qualitative analysis. Enables multiple researchers to code data together with AI assistance, improving inter-rater reliability and efficiency.

[Paper]

Talks

Invited Talks, Guest Lectures & Tutorials

Honors & Services

Fellowships & Awards

Workshop Organizer

Associate Chair / Area Chair

Invited Reviewer

Travel

Earlier travel