CollabCoder Logo

CollabCoder

A Lower-barrier, Rigorous Workflow for Inductive Collaborative Qualitative Analysis with Large Language Models
Jie Gao1,2, Yuchen Guo1, Gionnieve Lim1, Tianqin Zhang1, Zheng Zhang1, Toby Jia-Jun Li3, Simon Tangi Perrault1
1Singapore University of Technology and Design    2Singapore-MIT Alliance for Research and Technology    3University of Notre Dame

Overview

CollabCoder is a lower-barrier, rigorous workflow for inductive collaborative qualitative analysis. By integrating Large Language Models into the coding process, CollabCoder helps research teams achieve higher quality and more efficient qualitative analysis.

The system supports the full qualitative analysis pipeline — from independent open coding with AI-assisted suggestions, through iterative discussion and conflict resolution between coders, to final codebook development with LLM-generated code groupings. CollabCoder lowers the barrier to rigorous qualitative research while maintaining methodological standards.

Three-Phase Workflow

Phase 1

Independent Open Coding

On-demand code suggestions from LLMs help coders produce initial codes with higher confidence.

Phase 2

Iterative Discussion

Conflict mediation within the coding team produces a list of agreed-upon code decisions.

Phase 3

Codebook Development

LLM-generated suggestions help form code groups based on the decided codes.

System Demo

CollabCoder provides an integrated workspace for collaborative qualitative analysis powered by LLMs.

collabcoder.app
Rigorous qualitative analysis, powered by AI
✏️
AI Code Suggestions
GPT-powered code recommendations with confidence levels and keyword support.
🤝
Conflict Mediation
Side-by-side comparison with IRR metrics and AI-suggested resolutions.
📊
Codebook Builder
Drag-and-drop code grouping with AI-generated themes and categories.
🚧 Live demo — to be updated

Phase 1: Editing Interface

Coders can input customized codes for raw data, choose from GPT recommendations or top relevant codes, add keyword support, assign certainty levels (1-5), and review their individual codebook.

Editing Interface for Phase 1
The editing interface supports independent open coding with AI-assisted suggestions.

Phase 2: Comparison Interface

Two coders' codes are displayed side-by-side. The system calculates similarity between code pairs and Inter-Rater Reliability (IRR), helping teams identify agreements and resolve disagreements through discussion or GPT-suggested decisions.

Comparison Interface for Phase 2
The comparison interface facilitates consensus-building between coders.

Phase 3: Code Group Interface

Code decisions are compiled into unique codes. Users can group them manually or use AI-generated themes, drag codes into groups, and save the final codebook.

Code Group Interface for Phase 3
The code group interface enables efficient codebook development.

BibTeX

@inproceedings{10.1145/3613904.3642002, author = {Gao, Jie and Guo, Yuchen and Lim, Gionnieve and Zhang, Tianqin and Zhang, Zheng and Li, Toby Jia-Jun and Perrault, Simon Tangi}, title = {CollabCoder: A Lower-barrier, Rigorous Workflow for Inductive Collaborative Qualitative Analysis with Large Language Models}, year = {2024}, isbn = {9798400703300}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3613904.3642002}, doi = {10.1145/3613904.3642002}, booktitle = {Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems}, articleno = {11}, numpages = {29}, keywords = {Collaborative Qualitative Analysis, Grounded Theory, Inductive Qualitative Coding, Large Language Models}, location = {Honolulu, HI, USA}, series = {CHI '24} }