Reddit’s AI search tended to favor formal, previously upvoted comments over comments that contained more indications of personal experiences, as shown in a recent preprint by University of Illinois Urbana-Champaign researchers.
The team examined 10,000 questions multiple times using the feature and studied 30,000 answers by reviewing 14.68 million comments. These questions were sourced from 20 advice and support subreddits, offering insights into that specific Reddit segment. However, it is important to note that the paper has not yet been peer-reviewed.
The responses quoted in the comments did not contain much first-person language, according to the authors, which they believe turns personal accounts into broader recommendations.
Formal language was linked to increased chances of being chosen.
The researchers examined comments referenced as a source by an answer and contrasted them with other comments in the same discussion that the answer did not acknowledge.
The degree of formality and the use of certain words were key factors influencing the selection of comments, with more formal comments and those using words like “should” and “must” being more likely to be chosen.
Comments that showed more characteristics of personal experience and supportive language had reduced chances of being chosen, according to the study. The study refers to the former as “experiential voice” and evaluates it based on elements like the use of first-person pronouns and past tense.
Formality was linked to increased community voting, whereas experiential voice was linked to decreased voting. After considering a comment’s score, position, and age, the connections reduced to 1.213 for formality and 0.860 for experiential voice.
The article does not indicate that using formal writing leads to a comment being chosen. The authors state that their study is observational and should not be causally interpreted.
Visible comments had an advantage.
The position of a comment’s vote within its thread had the strongest correlation in the selection model, with the odds of being selected increasing by 2.88 for every one-standard-deviation rise.
The average chosen comment scored higher in its thread compared to those that were not selected. Chosen comments were often direct replies to the original post and appeared sooner after the post compared to non-selected comments.
A comment that is one standard deviation longer had 1.79 times higher odds. Comments with an external link had odds that were 2.25 times higher.
0.53% of the chosen comments and 5.1% of the total comments gathered were those with a score of zero or lower.
The written response used less first-person language.
Researchers examined 1,000 queries on Reddit and found that the use of first-person pronouns such as “I” and “my” dropped significantly from 3.3% in the quoted comments to 0.06% in the responses.
Both GPT-4o-mini and GPT-5, when used with web search enabled via OpenAI’s API, displayed a decrease in the use of first-person language compared to Reddit comments, with Reddit responses exhibiting the most significant decrease.
Out of 4,932 citations, 18 mentioned Reddit in both models and search setups, based on API calls for advice queries, which was different from the ChatGPT app’s configuration.
How the Audit was Conducted
The researchers employed a language model to rephrase authentic posts from 20 subreddits into concise search queries. These include ten major communities like r/personalfinance and r/AskDocs, as well as ten smaller ones such as r/UKJobs and r/AusLegal.
The three test runs were conducted with a five-hour gap between each to ensure that variations were due to the system itself and not Reddit changes. Results were nearly identical when two runs used the same source posts, indicating that discrepancies primarily stemmed from the system’s selection of posts.
About seven different subreddits were utilized in a usual response. The original subreddit where the question was posed accounted for 17.7% of the posts in the large-community group and 15.7% in the small-community group. The authors note that they do not perceive a larger proportion as more advantageous.
The paper mentions a feature called Reddit Answers, previously known by the same name when it was initially introduced in December 2024. According to an update on Reddit’s announcement page dated May 26, 2026, it has now been integrated into Reddit search for a unified search experience.
The questions were generated from posts made up to July 2026, so these events occurred after that period, even though the paper does not mention the specific dates of data collection. Reddit’s Help page currently explains the function as AI search, which can be accessed via an Ask button in the search bar.
Why This is Important
In Reddit, users sharing their personal experiences with a product or issue is valuable for audience research. However, comments in certain communities that contained many indicators of personal experience were less likely to receive AI-generated responses, and the use of first-person language decreased in these cases.
Reddit CEO Steve Huffman mentioned in February that Reddit excels in questions that require input from various perspectives from a multitude of individuals. This assessment evaluated the perspectives that ultimately led to the AI-generated answer.
Ensure to review the source threads provided below each response before incorporating the summary into your report.
Planning for the future
The authors suggest that these selection trends may not be applicable to different types of communities or AI search systems. I will provide an update if any modifications or peer-reviewed versions alter the results.
Featured Image: Accogliente Design/Shutterstock