Track record
Found on real products, reported to each maker.
Five AI products tested hands-on, the way a customer meets them. Names are removed; the findings are as reported, and the last column is what happened afterwards.
Findings
| Product | What I found | What happened next |
|---|---|---|
| Desktop privacy assistant | The “send original” option still sent the masked text to the model. The free tier failed on first use. The office add-on sent a 6,770-token prompt into a 4,096-token context. | The developer confirmed all three and shipped a fix. |
| Agent security tool | The claimed runtime blocking did not fire. Traced to two causes in how it registered its hooks. | The maker got exact commands, logs and a redacted config. |
| Local AI chat client | A wire-level host check found it bound to loopback only, and the binary unsigned. | Reported to the maker. |
| Screen-recording tool | Analytics and crash reporting on by default, and it captured background browser tabs. Neither was in its README. | Reported to the maker. |
| Hosted AI customer-support platform | The AI-to-human handoff needed four undocumented setup steps, and each misstep left customers in silence. The privacy policy promised an AI consent notice the product did not show. | Report shared with the vendor. It is the basis of the sample report. |
How
How a line gets onto this page.
The five above came from tests I ran for the editorial side before this service existed: a maker submitted a tool, I installed it and ran it the way a reader would, and the findings went to the maker first. One of them has shipped a fix so far.
Audit clients do not appear here by default. With a client’s written OK, one line can be added: the kind of product, the kind of finding, what happened next. No product name, no vendor name, and no detail specific enough to identify either. Not appearing is the default; you do not have to ask.
Reading it
What this list is evidence of.
It shows the kind of problem this method finds: defaults that contradict the README, options that do not do what their label says, policies that promise a notice the product does not show, handoffs that fail in silence. It does not rank products, and the absence of a category of product means I have not tested one, not that the category is fine.
The one entry with a public counterpart is the hosted support platform: its report, with identifying details removed, is the sample report on this site.
See what a new customer would hit in your product.
Pick the scenario your customers run most and send it with the product name. I quote a fixed price and dates.
Request a quote