OATH-Frames: Characterizing Online Attitudes towards Homelessness with LLM Assistants

We introduce a novel framework to understand, synthesize and characterize large-scale public attitudes towards societal issues through a collaboration between social work experts and LLMs. Specifically, we introduce a framing typology: OATH-Frames, (Online Attitudes Towards Homelessness): nine hierarchical frames capturing public attitudes towards homelessness as expressed on Twitter. We provide three kinds of annotations for posts from Twitter: expert-only, LLM-assisted expert and predicted annotations from a multilabel classification model. Our LLM-assisted expert annotation for socially sensitive tasks follows a new scalable framework which incorporates expert insights and chain-of-thought (CoT) explanations for designing better LLM prompts. Our 2.4M OATH-Frames-annotated posts enable a large-scale analysis across states and time periods, revealing changing trends in attitudes with key sociopolitical events. OATH-Frames surface harmful language towards people experiencing homelessness (PEH), which we show are often mislabeled by popular sentiment and toxicity classifiers, highlighting the value of our typology.