How we test AI tools
The five-step method behind every score on ByRole AI.
Every review on ByRole AI follows the same five steps. If a tool has not been through them, it does not get a score.
1. Start from the job
Before we look at any tool, we list the tasks that fill a working week in that role. For an HR team that might be screening applications, answering policy questions and writing job descriptions. Tools are then judged on those tasks, not on their feature list.
2. Use it on realistic work
We sign up the way any customer would, connect sample data that mirrors the role (a CRM export, a set of leases, a payroll file with real names removed) and run the same tasks through every competing tool.
3. Measure what matters
- Time saved on the core tasks, compared with doing them by hand
- Accuracy, and how often we had to fix the output
- Setup effort for a non-technical person
- Real cost on the plan a small team would actually need
- Data handling, including where your data goes and whether it trains the model
4. Score it out of 10
| Score | Verdict | What it means |
|---|---|---|
| 8.5 to 10 | Excellent | We would pay for it ourselves |
| 7 to 8.4 | Good | Worth it for the right team, with clear limits |
| 5 to 6.9 | Mixed | Useful in places, but try alternatives first |
| Under 5 | Skip it | Not worth the money or the setup time today |
5. Keep it current
AI tools change every month. We re-check prices and features at least once a quarter and re-test whenever a major update lands. The date at the top of each review shows when it was last updated.
What we never do
- Accept payment for a review, a score or a place in a list
- Let a vendor read or change a review before it is published
- Score a tool we have not used ourselves