Can LLMs Recognize the Difference Between Persuasion and Manipulation? Rutgers Researchers Introduce SafePersuasion to Help

Rutgers researchers have developed a new classification framework and a human-labeled collection of online comments, to provide a foundation for studying human persuasion and evaluating whether large language models (LLMs) can recognize manipulative language and tactics.

This framework distinguishes between rational persuasion and manipulation while identifying the specific techniques used in each. By creating a shared taxonomy and dataset, the framework makes these communication strategies easier for researchers to analyze and compare.

“We wanted to understand where persuasion crosses the line into manipulation and whether large language models can recognize that difference,” said lead author SC&I Ph.D. student Haein Kong. “Our findings highlight the need to carefully evaluate AI systems before relying on them to make these judgments. Persuasion is part of everyday online communication, but there is an important distinction between helping someone make an informed decision and exploiting their psychological or cognitive vulnerabilities.”

Despite the distinction between persuasion and manipulation, Kong said, “everyday manipulative communication remains relatively understudied, with limited shared resources for examining how it differs from rational persuasion. We developed this resource to address this gap. Ultimately, we hope this work supports online discussions in which people can consider different perspectives and make informed decisions without being manipulated.”

Beyond serving as a foundation for research, the dataset and framework could have practical applications in media literacy education and the development of tools that identify manipulative communication online. For policymakers, the framework provides a taxonomy and real-world examples that can inform discussions about potentially harmful persuasion techniques.

The paper, “SafePersuasion: A Dataset, Taxonomy, and Baselines for Analysis of Rational Persuasion and Manipulation,” by Kong; SC&I Postdoctoral Fellow A. M. Muntasir Rahman; Assistant Professor Ruixiang Tang of the Rutgers Computer Science Department; and SC&I Professor of Library and Information Science Vivek K. Singh was published in Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL 2025). 

This matters for researchers and developers because recognizing that a message is manipulative and identifying how that manipulation works are distinct challenges.

Haein Kong 

Given today’s digital landscape, the authors said it has become increasingly important for researchers to be able to evaluate persuasive language to determine if it could be harmful. The authors said while persuasive language has long been a tactic used to influence beliefs, attitudes, and behaviors, “today, across social media and online platforms, this type of content is pervasive, appearing in political campaigns, marketing, fundraising appeals, and more. These strategies span a broad spectrum, from rational and ethical appeals to highly manipulative tactics, some of which pose significant risks to individuals and society.” 

In recent years, the authors noted, researchers have increasingly used LLMs to assess persuasive language and generate persuasive content. “The lack of studies and resources on safety in persuasion can result in risky scenarios, such as applications built based on a potentially unsafe persuasion dataset or users being exposed to manipulative persuasion. Although our work focuses on human persuasion, we build on this growing recognition that understanding the boundary between rational persuasion and manipulative influence is a prerequisite for both responsible NLP modeling and future AI safety research.” 

To conduct the study, the authors evaluated the baseline performance of three large language models in detecting manipulation and its subtypes. 

Describing their additional findings, Kong said overall, models found it more difficult to identify specific persuasive techniques than to distinguish manipulation from rational persuasion. 

Although this study focused on human-written communication, the framework and dataset could provide a starting point for investigating AI-generated manipulation and informing the development of safeguards against manipulative AI behavior.

Haein Kong 

“This matters for researchers and developers,” Kong said, “because recognizing that a message is manipulative and identifying how that manipulation works are distinct challenges.”

Beyond these findings, Kong added that this work connects concepts from persuasion research with concrete examples that can support future AI safety research. 

“Although this study focused on human-written communication, the framework and dataset could provide a starting point for investigating AI-generated manipulation and informing the development of safeguards against manipulative AI behavior,” Kong said. 

As AI systems become increasingly involved in creating and evaluating online content, the researchers hope their work will help advance efforts to distinguish ethical persuasion from manipulation and support safer, more trustworthy digital communication.

Learn more about the Ph.D. Program at the Rutgers School of Communication and Information.