There's No Replicator Yet

An AI agent did nearly all of a family member's data entry, missed one or two items, and almost got written off. All-or-nothing is a replicator's bar, and nothing I use with AI clears it yet.

A family member was copying from a sheet in one browser tab into forms in another. I showed her how to install an AI agent and its browser extension, and we asked it to do the data entry.

She was amazed and creeped out watching the computer work on its own. Then it missed one or two items, and she almost dismissed the whole thing.

I teased her: “Well, if it can’t do 100% of it, then you’re acting like it should not do any of it at all.” By my estimate, it had done 99% of the work.

All-or-nothing is the right bar for a Star Trek replicator: ask for dinner and it appears exactly as you meant it. Nothing I use with AI works like that yet, and the machines change by the day.

Until the replicator exists, someone has to know how to cook. Here, cooking means judging what the machine missed and deciding whether the rest was worth keeping.


Most People Around Me Need It Ready-Made

Short of a replicator, there are four ways to get dinner, from no cooking to all of it.

What you doThe AI version
Microwave dinnerHeat it and eat what comes outSomeone else sets it up; you press go and take what comes back
Factor (ready-made meals delivered)Pick from a menu of meals cooked to a planYou pick a ready-made AI tool built for one kind of job, like a meeting note-taker, and use it as delivered
HelloFresh (meal kits)Cook from the recipe card with the ingredients in the boxYou get the tools installed and the steps written down, then run them and fix what comes out wrong
HomemadeChoose the ingredients, taste, adjustYou decide what to hand over, check the result, and change your approach when the model changes

Most people around me need the first row. My family member is still on it.

Installing the agent didn’t move her. She judged the result by the replicator’s bar, all or nothing. Moving toward the last row means judging whether the rest is worth keeping.

I’ve done this kind of setup many times over at a volleyball club where I do photo and media work. In October I used AI agents to build three 18-second tryout videos and sent all three to one of the club’s directors. Four minutes later the director picked two and asked for different photos in one. By that afternoon, both were scheduled to post.

The club’s October family newsletter started from the other end. I sent two staff members an AI-built draft and asked them to choose an athlete to spotlight and send a few details and a photo. One offered a junior high player to spotlight and asked me to add a coaches’ meet-and-greet. The finished email went to 1,217 contacts on September 30.

In the ladder’s terms, the director ordered from a menu and sent one dish back with a note. The newsletter staff brought ingredients. Both times, the cooking was mine.

I can cook with these tools, up to a point. Even the agent I build with is a packaged product I didn’t make, which I wrote about in Who Is Still Driving?


Swapping the Model Changes What Comes Out

New models ship constantly, and swapping one in can change the result. A cook tests the swap before trusting it. Someone using the tool ready-made doesn’t see the swap.

I use AI agents to write code. At the end of a working session, I run a check on the session’s transcript. Among other things, it looks for passwords and keys that ended up in the transcript by mistake.

On October 7, a smaller, cheaper model came out. I wanted to know whether it could run that check, so I ran five past transcripts through both models.

In one transcript, a key had been printed into the output of a command. The larger model flagged it and said to replace it. The cheaper model missed it on both of its runs.

One case says little about the model in general. It was enough to keep the check on the larger model.

Then I added a plain scan for anything shaped like a key, so catching a leak no longer depends on which model is paying attention.

Deciding which model to trust with which job is part of cooking. Someone using a ready-made tool doesn’t get to make that call.


Sometimes 100% Is the Right Bar

Her instinct isn’t always wrong. If those forms had moved money, one or two missed items could have cost more than everything the agent saved.

My 99% is also an estimate. Here the missed items were obvious at a glance, and she didn’t have to check every entry to find them. That made the trade easy.

When misses are harder to spot, finding them is work too, and that checking belongs in the comparison.

Deciding when 99% is good enough is the cook’s job.


Less Cooking, Once the Tool Can Say What It Missed

In Who Is Still Driving?, I wrote that a product needing an expert to catch its mistakes should say so. I listed three things I want to know before I rely on an agent:

  • What mistakes will I need to catch?
  • What can I do if something goes wrong?
  • How will I know the job is finished?

Those questions also mark how much cooking a tool still leaves you. When a tool can answer them for itself, and tell my family member which items it skipped, her only job will be deciding whether the misses matter.

Until then, a result that comes back mostly done needs two checks. One is the time it saved, minus the time it took to find and fix what it missed. The other is what a miss nobody found would cost.

There's No Replicator Yet
Share:

More in AI & Automation