Generative AI Meets Open-Ended Survey Responses: Participant Use of AI and Homogenization

Phys.org  November 25, 2024 Generative AI tools present new challenges for data quality in online surveys and experiments. A team of researchers in the US (New York University, Stanford University, Cornell University) examined participants’ use of large language models to answer open-ended survey questions and described empirical tendencies in human vs LLM-generated text responses. From social science research participants, 34%, reported using LLMs to help them answer open-ended survey questions. Researchers found that LLM responses are more homogeneous and positive. According to the researchers homogenization patterns may mask important underlying social variation in attitudes and beliefs among human subjects, raising […]