What custom AI agents do on my projects, and what stays with me
When a new client asks how I work, I tell them early that I use custom AI agents. Some people relax when they hear it. Others get nervous. Both reactions make sense, so here is what it actually means on a project, and where the line sits.
The short version
I’ve built websites since 2016, more than 150 of them. The way the work gets done has changed since then. Now I write the spec, custom agents I’ve set up do much of the building and testing, and I review every change before it ships.
The agents don’t talk to clients, and they have no say in what gets built or when it’s finished.
What the agents do
I run four roles, each set up for my projects, my conventions and my checklists.
The builder agent writes code from the spec. The test agent writes and runs tests. The review agent reads each change and flags problems. The ops agent handles deployments and keeps an eye on things once they’re running.
Where they help most is volume. The MEFMA event app has 63 automated unit tests. Fluffy’s café workspace has 49 test files. My own product, Unmu, has 1,886 automated tests passing. Writing that many tests alone, by hand, used to be the first thing a small project cut. With agents it’s routine. That doesn’t make the tests right by default. I still read what each one checks, because a test that checks the wrong thing is worse than no test: it tells you everything is fine.
Deciding where AI doesn’t belong
The most useful decisions on a project are often about where not to use AI at all.
Dr. Ehab Hamarneh runs an Arabic coaching academy, Be You Training, and I built the support desk behind it. Tickets get sorted into categories like Payment, Courses and Password. That sorting is deliberately keyword-based: it’s free, instant and predictable, and a support agent can see exactly why a ticket landed where it did. If a person changes a category by hand, the system doesn’t overwrite their choice.
The chat widget on the site works the same way. It searches the academy’s own knowledge base first. If an article matches, it answers from that article, with no AI call at all. Only when nothing matches does it either try an AI answer (if that’s switched on) or hand the customer to a person, which opens a real ticket. Every AI call it does make is logged with its token usage, so the cost is visible.
A model could have done all of that. It would have been slower, cost money on every message, and been harder to explain to the support team. Choosing the plain option is my job. Agents are good at doing what they’re asked, and they rarely ask whether the task needs AI in the first place.
Deciding what’s safe
The same desk receives newsletters and sales pitches alongside real customer email. Those junk messages are closed and tagged, never deleted. Deleted mail can’t be recovered, and a real customer misjudged as junk would vanish without anyone noticing. Closed and tagged, they stay searchable and one click from being reopened.
Arabic needed its own care. Customers spell the same course name several ways: with or without a shadda, with different forms of alef, with Arabic or Western digits. The tagging normalises all of that before matching, so one rule catches every spelling. It only ever adds tags. An agent’s own tags are never removed.
None of these choices is clever. They come from thinking about what could go wrong for this particular business, and that thinking is the part I don’t hand off.
Refusing to fake it
On the MEFMA event app, guests were meant to get a QR code in their table email. For a while, the real per-guest QR from the booking system wasn’t verified yet. The easy move was a placeholder QR that looked right. The checklist said the opposite, in plain words: do not render a fake QR. The option stayed switched off until real codes were confirmed.
The same goes for test results. When I load-tested MEFMA’s awards game with 1,000 simulated players, the first attempt, run from a desktop, failed: connections timed out after 203 live streams opened. That failed run is kept in the report as evidence, next to the run that passed. The report also says what the passing test doesn’t prove, such as venue Wi-Fi or real phones. An agent can write that report. Deciding that the limits paragraph stays in is mine.
What you deal with
You deal with me. We talk on WhatsApp in plain language. I write a short spec you can read and change before anything is built. You see a demo every week. After launch, I support what I shipped.
If something breaks, you don’t get “the AI did it”. You get me, and I answer for it, because I reviewed it and shipped it.
What this means if you’re hiring someone
Most builders use AI tools now, whether they mention it or not. The useful questions are about the work around the tools. Ask who reads the code before it goes live, and how it’s tested. Then ask who you call when it stops working at 9pm on a Thursday.
For my projects the answers are short: I read it, the tests are in the project, and you call me.
If you have questions about how this works on a real project, you can ask me on WhatsApp.
Related work: See the systems this applies to
