AI-powered bots increase social media post engagement but do not boost overall user activity

A recent study shows that AI-powered social media bots can increase user engagement on posts, but they fall short of encouraging users to post more overall.

The study, “Does Social Bot Help Socialize? Evidence from a Microblogging Platform,” focused on user engagement with CommentRobot, a large language model–powered bot launched on Weibo, China’s leading microblogging platform.

The work is published in the journal Information Systems Research.

At the core of the research project, the social bot automatically generated comments on users’ posts in public threads on the platform. The researchers found that when human posts receive bot comments, their peers are more likely to engage with those posts, but human authors of focal posts (hereafter posters) were not any more likely to increase their social media activity.

The study was conducted by Yang Gao of the University of Illinois Urbana-Champaign, Maggie Mengqing Zhang of the University of Virginia, and Mikhail Lysyakov of the University of Rochester.

Key findings were that when people receive bot-generated comments, their posts receive 23% more comments, and 11% more likes.

“Our research studied the bots at several complex levels, from bot comment quality to which users were targeted and how human peers responded to the public interactions between the bot and the poster,” said Gao.

Gao said that the quality of the bot comments matters. Social bot comments that were considered relevant and included certain social cues were more likely to generate engagement. The researchers detected a pattern where social bots often prioritized less active users, but that it was active users who more significantly benefited from receiving bot comments.

“It’s often assumed that people are more likely to engage with other people and not bots, but what we found is that when the bots are able to integrate relevant social cues into their comments, this stimulates a response from people,” said Zhang. “This in turn increases engagement.”

“What may be most interesting about this dynamic,” said Lysyakov, “is that the subsequent engagement is often not directly with the bot’s comments, but rather with other human users who also decided to engage in discussion.”

While all of this heightens user activity and engagement around a single social media post, the study authors found that overall, this did not increase the likelihood that they would become more active on the platform as posters.

The researchers analyzed over 106,000 posts by 64,000 users on Weibo in January 2024, focusing on first-time interactions with CommentRobot. They used econometric models, instrumental variable analysis, robustness checks and an online randomized experiment with 348 active Weibo users to confirm their findings.

“All of this suggests that while AI-powered social bots can help increase visibility and engagement around posts, platforms should refine their deployment strategies,” said Gao. “Poorly targeted or low-quality comments may limit their effectiveness, and platforms cannot assume bots will increase overall user activity.”