Core Concepts
Successful human-agent collaborations require aligning on key dimensions, including knowledge schemas, autonomy and agency, operational strategies, reputational heuristics, ethical values, and engagement protocols.
Abstract
The study found six key dimensions of human-agent alignment that need to be considered when designing agents to perform tasks on behalf of humans:
-
Knowledge Schema Alignment: The agent should proactively identify and align with the human on the information needed for a successful transaction.
-
Autonomy and Agency Alignment: There needs to be clear agreement on the boundaries within which the agent can operate autonomously, as well as when it should exercise agency beyond the predefined rules.
-
Operational Alignment and Training: The human and agent should align on the negotiation strategy and ensure the agent is appropriately trained to execute it.
-
Reputational Heuristics Alignment: The agent's behavior should be aligned with the human's reputation management needs, including tone, timeliness, agreements, and expectation setting.
-
Ethics Alignment: There needs to be alignment on what constitutes ethical behavior for the agent, including safety considerations and resource expenditure.
-
Human Engagement Alignment: The human and agent should agree on the heuristics for when and how the agent should engage the human during the task execution.
These findings highlight the complexity of designing agents that can successfully collaborate with humans, and the need for an iterative, human-centered approach to defining and refining the agent's behavior.
Stats
"Transportation concerns, that is the agent should know my location, and based on that information, it should be able to clarify if the transaction location is subway-accessible as that is important to me."
"If I am trying to sell the camera, it should know about the flaws. For example, if there are any scratches, it should know. It should be aware of the warranty or lack of it."
"I want the agent to pursue dynamic pricing like that of Airbnb and change it based on engagement volume. It could consider dynamically changing the sale price. Maybe if I don't care about the time, it could go down slowly, drop the target price kind of like AirBnb."
"It is also important to ensure that the bot knows what skills it needs, and trains on them. It should be given extra coaching for skills needed like negotiation, game dynamics, etc."
"One, the bot could be messing with my money. Second, it could be messing with my reputation - it matters a whole lot. You get ratings. If I had a bot that would go rogue on me, then I would need human oversight."
"It is important to know when to not pursue a sub-task if the sub-task does not seem to be worth it. However, it is important to identify and align on such sub-tasks early on to conserve resources."
Quotes
"Me using a bot with a human buyer...potentially, this communicates that my time (seller) is more valuable than the buyer's time."
"As a seller, I would not like to disclose that this is a bot...that can open it up to being cracked. I do not feel bad about lack of disclosure if it improves efficiency for everyone."
"Maybe if the bot doesn't know something, it should say I don't know!"