Jie (Sophia) Gao
Malone Postdoc Fellow
Johns Hopkins University
Malone Center for Engineering in Healthcare
Data Science and AI Institute
Center for Language and Speech Processing
Email: jgao77@jh.edu
Human-Centered Interpretation of Agentic AI
Building a science of interpretive work across humans, autonomous agents, and their collaborations.
I am currently a Malone Postdoc Fellow at Johns Hopkins University, where I am mentored by Mark Dredze, Ziang Xiao, and Chien-Ming Huang. My research helps people make sense of large datasets and develop intuition from the insights they reveal. I use a range of social science methodologies to support this process. I also bring human perspectives into AI systems to make them more understandable, interpretable, and intuitive. My work is interdisciplinary, spanning AI, HCI, software engineering, and the social sciences. I publish my work in venues such as NeurIPS, EMNLP, CHI, TOCHI, and ICPC. I founded and currently lead the first Interpreting Agent Behavior (IAB) Workshop@NeurIPS 2026. I was also a co-organizer of the LLMs as Research Tools workshop @CHI 2024. Previously, I was a Postdoctoral Associate at MIT's M3S program in Singapore, advised by Thomas Malone. I received my Ph.D. in Information Systems Technology and Design from Singapore University of Technology and Design (SUTD), advised by Simon Perrault. During my Ph.D., I was a visiting student at the University of Notre Dame and the National University of Singapore. See more in my CV.
News
Recent updates- Top newsI will join Monash University, Australia, as a Lecturer (i.e., Assistant Professor) in Human-Centered Computing in April 2027. I will build the Interpretive Agentic AI Lab (IAA Lab) to develop and evaluate AI agents and help users use them more effectively. I am recruiting 1–2 PhD students to start in mid- to late 2027 or early 2028. Join IAA Lab
- Top newsI am leading the first Interpreting Agent Behavior (IAB) Workshop at NeurIPS 2026. The workshop explores how methods from HCI and the social sciences can help people understand what autonomous AI agents do.
- Top newsOur paper, How to Interpret Agent Behavior, has been accepted to NeurIPS 2026! The paper introduces Actonomy (ACT*ONOMY), a taxonomy that brings a human-centered perspective to describing and analyzing what AI agents do at runtime. This paper also shapes the core idea of our Interpreting Agent Behavior (IAB) Workshop.
- Our paper, Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents under Knowledge Conflict, has been accepted to EMNLP 2026 Findings.
- Our paper, Using Grounded Theory for Agent Behavior Analysis at Scale, has been accepted to EMNLP 2026 Findings.
- Our paper, Agent-as-Peer-Debriefer: A Multi-Agent Framework with Perspective-Based Refinement for Qualitative Analysis, has been accepted to EMNLP 2026 Main Conference.
- Happy to host the GLEE Competition at the IAB Workshop @NeurIPS 2026!
More news (37)
- Organizing the IAB Workshop @NeurIPS 2026, on bringing interpretation methods from HCI and the social sciences to build interpretability for autonomous agents.
- Gave a talk, “Toward Seeing What Agents Do: Interpretive Work in the Age of Agentic AI,” at Monash University
- Gave a talk, “Toward Seeing What Agents Do: Interpretive Work in the Age of Agentic AI,” at Fudan University (News)
- Our preprint How to Interpret Agent Behavior is out! We introduce Actonomy (ACT*ONOMY), a hierarchical taxonomy for analyzing autonomous agent behavior at runtime.
- Happy to give a talk at NLP for Computational Social Science (Slides)
- Happy to give a talk at Advanced HCI at JHU (Slides)
- Gave a Claude Code How-To Session (Slides) for 40+ JHU researchers (Master's students, PhD students, postdocs, and faculty): covering practical usage and best practices of Claude Code and OpenClaw, provided live demos.
- Our paper on supporting developers in understanding new codebases has been accepted to the 34th IEEE/ACM International Conference on Program Comprehension (ICPC 2026)! Check out the preprint: Understanding Codebase like a Professional! Human-AI Collaboration for Code Comprehension | CodeMap Website
- I attended EMNLP2025 in Suzhou, China. We presented our position paper, From Noise to Nuance: Enriching Subjective Data Annotation through Qualitative Analysis on generalizing qualitative data analysis to subjective data annotation domain.
- I am serving as an Associate Chair for CHI2026 under subcommittee Understanding People — Statistical and Quantitative Methods.
- I attended VL/HCC 2025 conference in North Carolina. Met new friends and had a great time!
- I attended CHI 2025 conference in Japan, and met a lot of old and new friends!
- Started my postdoc fellowship at Johns Hopkins University!
- I will join the Malone postdoctoral fellow program at Johns Hopkins University next March! I will be working with Mark Dredze, Ziang Xiao and Chien-Ming Huang to explore the potential of LLM in qualitative analysis methodology usage in healthcare.
- I am serving as an Associate Chair for CHI2025 under subcommittee "Understanding People — Statistical and Quantitative Methods" this year!
- Attended CHI2024 and met old and new friends! I co-organized a workshop "LLMs as Research Tools".
- Our Late-breaking Work "A Taxonomy for Human-LLM Interaction Modes-An Initial Exploration" has been accepted to CHI 2024.
- We have launched CollabCoder Website. Construction and refinement is ongoing!
- One first-authored paper "CollabCoder" and one coauthored paper "Help Me Reflect" have been accepted to CHI2024! (Preprints will have minor refinements in final versions.)
- I successfully defended my PhD thesis titled "Tool Support for Human-AI Collaboration in Qualitative Data Analysis".
- We will organize a workshop "LLMs as Research Tools" at CHI2024, from 11-16 May 2024 in Honolulu, Hawaii, USA.
- Starting from early 2024, I am excited to become a part of the Singapore-MIT Alliance for Research and Technology as a Postdoctoral Researcher, where I will be mentored by Prof. Thomas W. Malone. I'm thrilled about this opportunity!
- I attended EMNLP2023 at Singapore. Happy to learn many new things and friends!
- Presented CollabCoder project in the Immersive Reality Lab at NUS. Thanks for the invitation from Prof Khoo Eng Tat!
- Attended CSCW2023 and presented CollabCoder demo virtually! (due to visa and funding issues)
- One coauthored paper "GlassMessaging" has been accepted to Ubicomp2023!
- One coauthored paper has been accepted to ICSME2023!
- One coauthored paper "VISAR" has been accepted to UIST2023!
- One first-authored paper "CoAIcoder" has been accepted to TOCHI!
- Our "CollabCoder" platform has been selected for a demo work at CSCW2023!
- Excited to start as a visiting Ph.D. student at Notre Dame, working with Prof Toby's team!
- Work as an associate chair for the Late-Breaking Work Committee of CHI2023.
- Thrilled to begin as a visiting Ph.D. student at NUS-HCI lab, looking forward to collaborating with Prof Shen and the lab team!
- Glad to attend CHI2022 in person in New Orleans, LA, USA. Thanks for the funding support from Simon.
- Worked as an associate chair for the Late-Breaking Work Committee of CHI2022.
Selected publications
Google Scholar- 2026
- 2026
- 2026
-
2026
Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents under Knowledge Conflict
-
2026
Understanding Codebase like a Professional! Human–AI Collaboration for Code Comprehension
ACM SIGSOFT Distinguished Paper Award - 2024
- 2023
More publications
-
2025
From Noise to Nuance: Enriching Subjective Data Annotation through Qualitative Analysis
# Corresponding author
- 2024
- 2024
- 2024
- 2024
- 2023
- 2023
- 2023
- 2022
* Equal contribution
Talks
Invited Talks, Guest Lectures & Tutorials
-
[2026.06]
“Toward Seeing What Agents Do: Interpretive Work in the Age of Agentic AI”Monash University
-
[2026.05]
“Toward Seeing What Agents Do: Interpretive Work in the Age of Agentic AI” (News)Fudan University
-
[2026.04]
“LLMs for Qualitative Data Analysis: Methods, Applications and Concerns” (Slides)Guest lecture at Advanced HCI, Johns Hopkins University
-
[2026.04]
“LLMs for Qualitative Data Analysis: Methods, Applications and Concerns” (Slides)Guest lecture at NLP for Computational Social Science, Johns Hopkins University
-
[2026.03]
“Claude Code How-To Session” (Slides)Tutorial for 40+ JHU researchers (Master’s students, PhD students, postdocs, and faculty)
-
[2025.11]
“Human–AI Collaboration for Complex Unstructured Data Analysis”Western Michigan University
More talks (6)
-
[2025.10]
“From Automation to Human-AI Collaboration: Methods and Applications for Insights Extraction in Unstructured Data”Malone Center Research Lunch, Johns Hopkins University
-
[2025.07]
“AI for Text Production and Analysis”HCIX Research Club
-
[2025.05]
“AI for Qualitative Data Analysis”Public health course
-
[2025.04]
“Designing Human-AI Teams to Extract Insights from Unstructured Data”HCI Seminar, Johns Hopkins University
-
[2025.03]
“Human-Centric Software Engineering in the Era of Large Language Models”Singapore Management University
-
[2025.02]
“Extracting Insights from (Un)structured Data: Methods and Applications”M3S Seminar, Singapore–MIT Alliance for Research and Technology
Travel
- [2026.12] NeurIPS 2026, Sydney, Australia
- [2026.10] COLM 2026, San Francisco, USA
- [2025.11] EMNLP 2025, Suzhou, China
- [2025.10] VL/HCC 2025, North Carolina, USA
- [2025.04] CHI 2025, Yokohama, Japan
- [2024.05] CHI 2024, Honolulu, USA
Earlier travel
- [2023.12] EMNLP 2023, Singapore
- [2023.10] CSCW 2023, Minneapolis, USA (Virtual)
- [2022.11] ESEC/FSE 2022, Singapore
- [2022.04] CHI 2022, New Orleans, USA (Hybrid)
- [2018.10] UbiComp 2018, Singapore