(Cari) Beth Head, a Sociology Ph.D. candidate, teamed up with collaborators to author a paper, “Large language model applications for evaluation: Opportunities and ethical implications,” which was published in a November 2023 issue of New Directions for Evaluation.
Large Language Models (LLMs) are a type of artificial intelligence (AI) that provides exciting opportunities to synthesize large amounts of data, from various sources, and generate text for evaluation practitioners. The authors discuss these opportunities while also highlighting technological challenges, ethical dimensions (e.g., perpetuating biases), and advancing recommendations for the evaluation community to work collaboratively to overcome these challenges.
The paper is open source and can be found online at https://onlinelibrary.wiley.com/doi/pdf/10.1002/ev.20556