A vendor reference is a customer who was chosen because they are happy, told in advance what the call is about, and is going to tell you the truth. All three of those are true at once, and the third is what makes the first two so effective. Nobody has to lie for a reference call to be uninformative.
We supply references too, and we select them the same way everyone does. So treat what follows as a description of how to get past our preparation as much as anyone else’s. Most ERP vendor evaluation processes never get this far, because the standard reference-check questions are the wrong instrument.
The question that does not work
"What would you do differently?" It is the most-recommended question in every buying guide and it reliably produces a rehearsed answer, because it is the most-recommended question in every buying guide. The reference has answered it before. Usually the answer is "we would have committed more internal resource earlier", which is both true and free of information. It is the single weakest step in most ERP selection process design, precisely because everyone already knows to ask it.
Four that do
- "Walk me through your last month-end, in order, with the times." Not whether the close got faster — the actual sequence. What runs, what waits, who is waiting for it. A close that genuinely works can be narrated without effort. One that does not produces a description with gaps, and the gaps are where the spreadsheets live.
- "What is still in a spreadsheet?" There is always something. The useful part is whether they can tell you why, and whether the why is a deliberate decision or a workaround that calcified. This question is hard to prepare for because the honest answer feels like an admission and is not one.
- "Who at the vendor did you talk to when something was actually broken, and how long did that take?" The name matters more than the duration. A reference who names an individual is describing a support relationship. One who says "we raised a ticket" is describing a process, which is fine, but it is a different product experience than the one being sold to you.
- "What did you have to change about how you work?" Every implementation forces process change somewhere. A reference who reports none either had unusually aligned processes or is describing a system that was customised into a copy of the old one — and the second is worth knowing about, because it is 14% of the stalled implementations in our sample.
Ask for the reference you were not offered
The stronger move is orthogonal to the questions: ask for a customer in your industry, at your size, who went live in the last eighteen months, and see what happens. This one request tells you more about real-world vendor reference checks than the entire scripted call that follows it.
Sometimes there is not one, which is an answer. Sometimes there is one and they were not offered, which is a more interesting answer. The response to the request is more informative than most of the call that follows it.
The reference list is a selected sample. The question worth asking is what it was selected from.
What this looks like pointed at us
Applied to erp.io, that request exposes real things: we have no vertical specialists for industries we have not worked in, our references skew toward companies that were replacing small-business accounting rather than a mid-market ERP, and we cannot produce a local partner in your city because we do not have one. Those constraints are on the team page because they are the ones that lose us work honestly.
The rest of the evaluation sequence — getting a long list down to three, and building a total cost of ownership that is not fiction — is in the selection guide and the TCO guide. How we score products, and why we rank ourselves fourth on our own directory, is in the editorial policy. Treat all of it as a software evaluation checklist, not a sales conversation — the format only works if you run it the same way for every vendor.
The four questions above are aimed at surfacing exactly the failure modes we see most often: data that turns out to be worse than assumed, decisions that stall because nobody owns them, and once the system is live, a close that never gets faster or an agent that has been allowed to do more than it should. A good reference will have opinions about all four.
A good reference call will also have opinions on problems that show up later than go-live: how many custom fields their implementation added without a review process, whether their consolidated close was designed before the first entity migrated or discovered afterward, how their integrations resolve a field two systems both tried to change, and whether their chart of accounts was redesigned or just copied over.