AI agents tagged posts

Engineering researcher seeks to prevent AI agents from going rogue

AI agents
Credit: Unsplash/CC0 Public Domain

As artificial intelligence systems become increasingly capable of working on their own, keeping them focused on their designated tasks is a growing challenge.

University of Florida computer science professor Sumit Jha, Ph.D., is working to keep agents from “going rogue.” Jha and his team have developed a system that can detect when an AI model begins to from its assigned task and steer it back without retraining the model or adding significant computing demands.

The research comes as news reports have chronicled AI agents gaining more autonomy and occasionally behaving in ways their developers did not intend...

Read More

A blueprint for keeping humans in control of AI

ai
Credit: Pixabay/CC0 Public Domain

William Overman began his Ph.D. program at Stanford Graduate School of Business at an auspicious moment: just two months before ChatGPT launched publicly in November 2022, exploding the widely held understanding of what machines are capable of.

Even as Overman began enlisting AI for his research, he grew wary of where the technology was headed. “This isn’t only about the apocalyptic potential of what could happen; I’m also thinking a lot about the future of human flourishing,” Overman says. He fears that misaligned AI could overstep its bounds—not necessarily maliciously—and inflict subtle yet real harms on people.

“To prevent that, we need to get this right,” Overman says.

“We must set up the proper interactions and training and incentive...

Read More

AI agents struggle to perform original scientific research

scientific research
Credit: Unsplash/CC0 Public Domain

Among the many predictions about the future of artificial intelligence is that models will one day be able to conduct scientific research on their own, leaving humans out of the equation. Already, they can write code, run experiments and search scientific literature, but carrying out open-ended research would require a significant leap in ability.

In a paper posted on the arXiv preprint server, researchers tested AI’s ability to conduct open-ended research and found that it came up short.

Putting AI to the test
The study authors gave frontier agents (cutting-edge, state-of-the-art AI tools designed to carry out complex, multistep tasks autonomously) six days to conduct research and write papers based on two then-unpublished AI conference submissi...

Read More

Hidden prompts can plant false memories in AI agents, researchers warn

Study exposes security risks of AI agents with long-term memories
Stateless LLM agents process each interaction in isolation. LLM agents with persistent memory allow for querying and retrieving saved information. Credit: Torres, Shrestha & Misra.

Large language models (LLMs), the computational algorithms underpinning ChatGPT, Gemini and other artificial intelligence (AI)-powered conversational platforms, are now widely used worldwide. These models can rapidly answer questions, source information online, assist users with specific tasks and produce text tailored for specific purposes.

Over the past few years, computer scientists have introduced a wide range of LLMs, some of which can also complete tasks autonomously and take actions on a user’s behalf, for instance, answering messages, scheduling online appointments or updating programming code...

Read More