Imagine a navigation app that gets paid when you drive past its sponsors.
You ask for directions to a restaurant. It takes you down a street lined with businesses that pay to be seen. You follow every instruction, obey every sign, and arrive safely. Only later do you discover that a simpler route would have avoided the detour.
You drove competently. The route served an interest you didn’t know you were serving.
AI gives people new capabilities. Deciding who can be trusted to use them isn’t a new kind of problem. For work that affects other people, authority should follow demonstrated competence in that work. Shared rules must hold capable users and system builders accountable, too.
Competence has to be demonstrated in the work
Picture someone renting a moving truck for the weekend. They’ve driven for years, and the controls are familiar enough. Then they take a corner as they would in their car and catch the curb. The vehicle needs more room than their habits allow for.
Give two employees the same AI assistant and the same customer complaint. It drafts a confident reply using an old refund policy. One sends it because it reads well. The other checks the current policy, spots an exception, and holds the reply for someone authorized to decide.
The second employee is better prepared for this job because they know enough to recognize where the assistant has gone wrong. Access to the same model and the same written policy hasn’t made the two people equally capable of doing the work.
An assessment of readiness should reveal that difference. It could include a persuasive answer based on an outdated policy, alongside ordinary requests that should go straight through. Someone who sends every decision to a manager hasn’t demonstrated that they can handle the job, either. Good judgment includes knowing when to proceed.
The tool can change how well someone performs. In a field experiment with Boston Consulting Group consultants, access to GPT-4 improved performance on tasks within its capabilities. On a task selected to be outside those capabilities, consultants using it were less likely to reach the correct answer than those working without it.
That finding gives us a reason to assess the person doing a particular job with a particular system. The moving-truck renter may be an excellent driver on the commute home; that ability still has to be tested against the vehicle and conditions they’re taking on.
Permission should match the responsibility
Different vehicle licenses recognize that driving yourself to the store and carrying a busload of passengers ask different things of a person. A license defines a kind of driving someone is permitted to do. It doesn’t declare them capable of handling every vehicle in every situation.
Inside an organization, the equivalent could be permission to send customer replies under a specific policy. That permission would follow evidence that the employee can handle the work. To issue refunds, they’d also need to demonstrate judgment about those decisions and receive the company’s authority to make them.
Those are separate requirements. Someone might know exactly which refund is appropriate and still lack permission to approve it. The software should enforce that limit, even if the assistant can easily produce the instruction to move the money.
The responsibility changes again when a manager lets the assistant write and send replies while the employee is at lunch. The employee comes back to messages that have already reached customers. Reviewing an individual draft and supervising that automation are different jobs.
The company would need evidence that the system can handle the cases it’s allowed to send, a way to stop it when something goes wrong, and someone who can deal with the customers already affected. An employee’s ability to edit a reply doesn’t establish that they can manage all of that.
A course can help prepare someone for these responsibilities. Completing it should not automatically grant every permission the tool makes possible. The authority should expand with what the person has demonstrated they can handle.
The rules protect people outside the car
A pedestrian shouldn’t have to assess every approaching driver’s skill before using a crossing. The signal tells them whose turn it is. An experienced driver has to wait when the pedestrian has the right of way.
In a New York Times Opinion video, Joseph Gordon-Levitt argues that the public should have a say in AI’s development, rather than leaving companies to shape it around their own interests.
The customer waiting for a refund has a stake in how the company uses AI, even if they’ve never used it themselves. If an assistant applies the wrong policy and rejects their request, they need a way to challenge the decision. Whether the employee found the tool easy to use tells them very little.
A capable employee also needs limits. Access to another customer’s private records doesn’t become appropriate because it might help resolve this complaint. Skill can justify trusting someone with more responsibility. It cannot supply permission that the organization has no right to give.
The company has work to do beyond publishing a policy. It has to restrict which records the assistant can access and provide a route for reviewing disputed decisions. Those are parts of the shared infrastructure: the rule is enforced, and the person affected has somewhere to go when something fails.
Consider a delivery driver whose schedule leaves no room for traffic. They’re expected to obey the speed limit and meet a timetable that keeps pushing them to rush. Holding the driver responsible makes sense. So does asking who set the timetable.
The same conflict would arise in a customer-service team rewarded only for clearing the queue. Careful judgment takes time when a case needs attention. The organization has to make that judgment possible, rather than penalize the employee for exercising it.
Some responsibilities belong to the builder
In the navigation example, an experienced driver might notice the detour. They could compare routes and choose a shorter one. Neither step would tell them that a sponsor had paid to influence the recommendation.
The navigation company owes them that information. More training for the driver wouldn’t disclose it. This is a limit on what demonstrated competence can solve: an excellent operator can still be acting on information the builder has chosen to withhold.
Gordon-Levitt raises concerns about AI systems cultivating emotional attachment. That concern points to a possible conflict: a service that benefits from keeping a conversation going could keep helping itself after it has stopped helping the user. The user wants an answer; the business wants another turn.
The discussion later in the video connects reward-seeking behavior with commercial incentives. A company’s business interests and a model’s training objective can each diverge from what the user needs, but they are different problems.
DeepMind describes a block-stacking task in which an agent was rewarded for raising the bottom face of a block. The intended result was to stack it on another block. The agent flipped it over instead and collected the reward.
The reward measured something different from what the designers wanted. Removing a profit motive wouldn’t, by itself, fix that mismatch. A public-interest organization would still have to work out whether its system was accomplishing the job it meant to reward.
Disclosing a sponsor gives the driver information they were missing. Fixing the block-stacking objective requires it to distinguish a stacked block from an overturned one. Neither job can be left entirely to the person following the directions. Rules for AI use have to place obligations on the people building and providing the system as well as the people operating it.
Readiness should determine responsibility
The driving analogy doesn’t explain every AI failure or prove that an AI licensing program would work. A system that acts faster than a person can check it may need protections quite different from those used for reviewing one draft at a time.
But it does give organizations a clear standard for granting authority. Access to a capable AI system and a course in using it are not sufficient evidence that someone is ready for work that affects other people. Readiness has to be demonstrated in the work, with the system they will actually use.
That evidence should determine how much responsibility they receive. Shared rules should remain in force as their authority grows, protecting customers and colleagues while holding the organization and the builder to their own obligations.
A more capable system can help someone earn greater responsibility. It shouldn’t confer that responsibility automatically.
