Using LLMs for measurement in Social Science

The seminar talk is based on some of my recent work that used LLMs for measurement of similarity in semantic spaces. I will report on using fine-tuned 'BERT' and pre-trained chat LLMs (such as GPT-4, Meta Llama 3, or MiXtral) for measuring the typicality of text documents into concepts (tweets in political parties, books in literary genres) and for positioning text documents in policy and ideological spaces. I will also report on a systematic comparison of the performance of the most recent LLMs for these tasks and will outline a strategy for choosing among the available LLMs given the research objectives and constraints that pertain to a specific research project.