Imagine you’re watching a movie, in which a character puts a chocolate bar in a box, closes the box and leaves the room. Another person, also in the room, moves the bar from a box to a desk drawer. You, as an observer, know that the treat is now in the drawer, and you also know that when the first person returns, they will look for the treat in the box because they don’t know it has been moved.

You know that because as a human, you have the cognitive capacity to infer and reason about the minds of other people—in this case, the person’s lack of awareness regarding where the chocolate is. In scientific terms, this ability is described as Theory of Mind (ToM). This “mind-reading” ability allows us to predict and explain the behavior of others by considering their mental states.

We develop this capacity at about the age of four, and our brains are really good at it.

“For a human brain, it’s a very easy task,” says Zhaozhuo Xu, Assistant Professor of Computer Science at the School of Engineering—it barely takes seconds to process.

“And while doing so, our brains involve only a small subset of neurons, so it’s very energy efficient,” explains Denghui Zhang, Assistant Professor in Information Systems and Analytics at the School of Business.

How LLMs differ from human reasoning

Large language models or LLMs, which the researchers study, work differently. Although they were inspired by some concepts from neuroscience and cognitive science, they aren’t exact mimics of the human brain. LLMs were built on artificial neural networks that loosely resemble the organization of biological neurons, but the models learn from patterns in massive amounts of text and operate using mathematical functions.

That gives LLMs a definitive advantage over humans in processing loads of information rapidly. But when it comes to efficiency, particularly with simple things, LLMs lose to humans. Regardless of the complexity of the task, they must activate most of their neural network to produce the answer. So whether you’re asking an LLM to tell you what time it is or summarize “Moby Dick,” a whale of a novel, the LLM will engage its entire network, which is resource-consuming and inefficient.

“When we, humans, evaluate a new task, we activate a very small part of our brain, but LLMs must activate pretty much all of their network to figure out something new even if it’s fairly basic,” says Zhang. “LLMs must do all the computations and then select the one thing you need. So you do a lot of redundant computations, because you compute a lot of things you don’t need. It’s very inefficient.”

New research into LLMs’ social reasoning

Working together, Zhang and Xu formed a multidisciplinary collaboration to better understand how LLMs operate and how their efficiency in social reasoning can be improved.

They found that LLMs use a small, specialized set of internal connections to handle social reasoning. They also found that LLMs’ social reasoning abilities depend strongly on how the model represents word positions, especially through a method called rotary positional encoding (RoPE). These special connections influence how the model pays attention to different words and ideas, effectively guiding where its “focus” goes during reasoning about people’s thoughts.

“In simple terms, our results suggest that LLMs use built-in patterns for tracking positions and relationships between words to form internal “beliefs” and make social inferences,” Zhang says. The two collaborators outlined their findings in the study titled “How large language models encode theory-of-mind: a study on sparse parameter patterns,” published in npj Artificial Intelligence.

Looking ahead to more efficient AI

Now that researchers better understand how LLMs form their “beliefs,” they think it may be possible to make the models more efficient.

“We all know that AI is energy-expensive, so if we want to make it scalable, we have to change how it operates,” says Xu. “Our human brain is very energy efficient, so we hope this research brings us back to thinking about how we can make LLMs to work more like the human brain, so that they activate only a subset of parameters in charge of a specific task. That’s an important argument we want to convey.”

Thirty-eight years later, The Running Man is back on our screens, playing to a world that seems to have caught up with the original’s idiocy. This new one features a considerably less bulky, but no less watchable star in Glen Powell, playing runner Ben Richards. Fired from various jobs for insubordination, and tending to a sick toddler, he’s press-ganged into joining America’s favorite kill-or-be-killed game show, after a producer identifies him as “quantifiably the angriest man to ever audition.”

The show’s premise has been tweaked a bit, too. Instead of navigating a series of video-game-like levels for the length of a TV broadcast, Richards must now survive in the real world for 30 days, surveilled by hovering network TV camera droids, pursued by armed-to-the-teeth “hunters,” private police goons, and a general public who spot and film runners using a proprietary app on their smartphones. The longer he lasts, and the more pursuers he can kill, the more money he makes. He’s cheered (and booed) by a massive audience of brain-dead oafs called Running Fans, glued to their screens 24/7. Like Schwarzenegger’s Richard before him, Powell makes the transition from onscreen villain to beloved folk hero, mugging for the cameras as his antics drive the ratings.

If it sounds familiar, it’s because this new version of The Running Man, which is cowritten and directed by Edgar Wright (Hot Fuzz, Scott Pilgrim vs. the World), draws as much from the original film and Stephen King’s source novel as it does from present-day reality. A modern-day America overseen by a game show president, where ICE squads team up with Dr. Phil McGraw to turn deportation raids into reality television, would seem ripe for a Running Man remake. But that’s the problem. Satire relies on caricature. And the new version is barely exaggerative. Does the very idea of a lethal game show seem that far off, in a world where the success of Netflix’s South Korean thriller series Squid Game (itself a variation on the The Running Man format) spawned an actual, licensed Squid Game-style competitive reality TV show? Or when a grinning zillennial YouTuber named “MrBeast” baits contestants with ten grand to sit in a bathtub full of snakes? A few weeks ago I watched live as rookie New York Giants’ running back Cam Skattebo’s ankle twisted 45-degrees, as if cranked by some invisible wrench, while a bar-full of rival fans cheered.

Autonomous vehicles (AVs) have been tested as taxis for decades in San Francisco, Pittsburgh and around the world, and trucking companies have enormous incentives to adopt them.

But AV companies rarely share the crash- and safety-related data that is crucial to improving the safety of their vehicles—mostly because they have little incentive to do so.

Is AV safety data an auto company’s intellectual asset or a public good? It can be both—with a little tweaking, according to a team of Cornell researchers.

A new data-sharing roadmap

The team has created a roadmap outlining the barriers and opportunities to encourage AV companies to share the data to make AVs safer, from untangling public versus private data knowledge, to regulations to creating incentive programs.

“The core of AV market competition involves who has that crash data, because once you have that data, it’s much easier for you to train your AI to not make that error. The hope is to first make this data transparent and then use it for the public good, and not just profit,” said Hauke Sandhaus, M.S. ’24, a doctoral candidate at Cornell Tech and co-author of “My Precious Crash Data,” presented Oct. 16 at the ACM on Human-Computer Interaction.

His co-authors are Qian Yang, assistant professor at the Cornell Ann S. Bowers College of Computing and Information Science; Wendy Ju, associate professor of information science and design tech at Cornell Tech, the Cornell Ann S. Bowers College of Computing and Information Science and the Jacobs Technion-Cornell Institute; and Angel Hsing-Chi Hwang, a former postdoctoral associate at Cornell and now assistant professor of communication at the University of Southern California, Annenberg.

Barriers to sharing AV safety data

The team interviewed 12 AV company employees who work on safety in AV design and deployment, to understand how they currently manage and share safety data, the data sharing challenges and concerns they face, and their ideal data-sharing practices.

The interviews revealed the AV companies have a surprising diversity of approaches, Sandhaus said. “Everyone really has some niche, homegrown data set, and there’s really not a lot of shared knowledge between these companies,” he said. “I expected they would be much more commonality.”

The research team discovered two key barriers to sharing data—both underscoring a lack of incentives. First, crash and safety data includes information about the machine-learning models and infrastructure that the company uses to improve safety.

“Data sharing, even within a company, is political and fraught,” the team wrote in the paper. Second, the interviewees believed AV safety knowledge is private and brings their company a competitive edge.

“This perspective leads them to view safety knowledge embedded in data as a contested space rather than public knowledge for social good,” the team wrote.

And U.S. and European regulations are not helping. They require only information such as the month when the crash occurred, the manufacturer and whether there were injuries. That doesn’t capture the underlying unexpected factors that often cause accidents, such as a person suddenly running onto the street, drivers violating traffic rules, extreme weather conditions or lost cargo blocking the road.

Potential solutions for safer autonomous vehicles

To encourage more data-sharing, it’s crucial to untangle safety knowledge from proprietary data, the researchers said. For example, AV companies could share information about the accident, but not raw video footage that would reveal the company’s technical infrastructure.

Companies could also come up with “exam questions” that AVs would have to pass in order to take the road. “If you have pedestrians coming from one side and vehicles from the other side, then you can use that as a test case that other AVs also have to pass,” Sandhaus said.

Academic institutions could act as data intermediaries with which AV companies could leverage strategic collaborations. Independent research institutions and other civic organizations have set precedents working with industry partners’ public knowledge. “There are arrangements, collaboration, patterns for higher ed to contribute to this without necessarily making the entire data set public,” Qian said.

The team also proposes standardizing AV safety assessment via more effective government regulations. For example, a federal policymaking agency could create a virtual city as a testing ground, with busy traffic intersections and pedestrian-heavy roads that every AV algorithm would have to be able to navigate, she said.

Federal regulators could encourage car companies to contribute scenarios to the testing environment. “The AV companies might say, ‘I want to put my test cases there, because my car probably has passed those tests.’ That can be a mechanism for encouraging safer vehicle development,” Yang said. “Proposing policy changes always feels a little bit distant, but I do think there are near-future policy solutions in this space.”