A vendor reference is a customer who was chosen because they are happy, told in advance what the call is about, and is going to tell you the truth. All three of those are true at once, and the third is what makes the first two so effective. Nobody has to lie for a reference call to be uninformative.
We supply references too, and we select them the same way everyone does. So treat what follows as a description of how to get past our preparation as much as anyone else’s.
The question that does not work
"What would you do differently?" It is the most-recommended question in every buying guide and it reliably produces a rehearsed answer, because it is the most-recommended question in every buying guide. The reference has answered it before. Usually the answer is "we would have committed more internal resource earlier", which is both true and free of information.
Four that do
- "Walk me through your last month-end, in order, with the times." Not whether the close got faster — the actual sequence. What runs, what waits, who is waiting for it. A close that genuinely works can be narrated without effort. One that does not produces a description with gaps, and the gaps are where the spreadsheets live.
- "What is still in a spreadsheet?" There is always something. The useful part is whether they can tell you why, and whether the why is a deliberate decision or a workaround that calcified. This question is hard to prepare for because the honest answer feels like an admission and is not one.
- "Who at the vendor did you talk to when something was actually broken, and how long did that take?" The name matters more than the duration. A reference who names an individual is describing a support relationship. One who says "we raised a ticket" is describing a process, which is fine, but it is a different product experience than the one being sold to you.
- "What did you have to change about how you work?" Every implementation forces process change somewhere. A reference who reports none either had unusually aligned processes or is describing a system that was customised into a copy of the old one — and the second is worth knowing about, because it is 14% of the stalled implementations in our sample.
Ask for the reference you were not offered
The stronger move is orthogonal to the questions: ask for a customer in your industry, at your size, who went live in the last eighteen months, and see what happens.
Sometimes there is not one, which is an answer. Sometimes there is one and they were not offered, which is a more interesting answer. The response to the request is more informative than most of the call that follows it.
The reference list is a selected sample. The question worth asking is what it was selected from.
What this looks like pointed at us
Applied to erp.io, that request exposes real things: we have no vertical specialists for industries we have not worked in, our references skew toward companies that were replacing small-business accounting rather than a mid-market ERP, and we cannot produce a local partner in your city because we do not have one. Those constraints are on the team page because they are the ones that lose us work honestly.
The rest of the evaluation sequence — getting a long list down to three, and building a total cost of ownership that is not fiction — is in the selection guide and the TCO guide. How we score products, and why we rank ourselves fourth on our own directory, is in the editorial policy.