Most RFPs for AI-native ERP ask the wrong questions in the wrong proportions. The template gets forwarded from the last selection, a column is added for "AI capability," vendors return 200 pages of yeses, and the scoring matrix picks the one that said yes most convincingly. The result is a shortlist decided by prose quality.
Vendor theatre thrives in that gap. Marketing-grade answers survive because the scorecard rewards coverage breadth rather than evidence depth, and because reviewers rarely agree in writing what counts as proof. The fix is mechanical: tighter weightings, explicit pass or fail gates, and a required proof artefact attached to every scored answer. Done well, the scoring sheet eliminates two or three vendors before anyone books a demo.
What follows is how to build that sheet for an agentic ERP evaluation, which requirements deserve weight, and which answers should end the conversation.
Start With Gates, Not Scores
Weighted scoring is a comparison tool. It is not a filter. A vendor that fails a genuine must-have should be removed from the matrix entirely, not awarded partial credit for a near miss. Industry guidance on RFP mechanics is consistent on this: mandatory criteria sit outside the weighted score, and a late submission, missing certification, or unanswered mandatory question can disqualify a proposal before scoring begins.
For an AI ERP, five gates belong at the top of the document, each a yes-or-no with a required artefact:
- Ledger immutability. The AI never writes directly to the general ledger. Required artefact: an architecture diagram showing the control plane between the agent and the posting API, plus the name of the deterministic rule layer.
- Daily reconciliation proof. Subledger-to-GL and bank-to-GL variances are produced and signed off daily, not at month end. Required artefact: a sample reconciliation report with timestamp and reviewer identity.
- Role-scoped autonomy. Each agent runs with least-privilege permissions and a documented approval threshold. Required artefact: the policy file or configuration excerpt.
- Audit log completeness. Every agent action, including read-only retrievals, is logged with inputs, policy applied, and outcome. Required artefact: an exported log for a sample three-way match.
- Human override and rollback. Any posted entry can be reversed by a named human role, with lineage preserved.
A vendor that answers "roadmap" to any of these fails the gate. The SEC's 2024 enforcement actions against two advisers for AI-washing claims, with penalties of $225,000 and $175,000, are a useful reminder that unverified AI assertions carry regulatory weight of their own. Treat vendor answers as representations, and demand the artefact that would survive a deposition.
Weight Functional Fit Higher Than You Think
The second mistake is spreading weight too evenly. A ten-category rubric with 10% each produces a scorecard that cannot discriminate between vendors. Mid-market ERP selection practice puts functional fit at 35–45% of the total evaluation score, with the remainder split across technology, implementation, commercial terms and vendor viability. For an agentic ERP, that functional block has to include the agent workflows themselves, not just the underlying modules.
A workable weighting for the written RFP:
- Functional fit, including agent workflow coverage: 40%
- AI controls, auditability and governance: 20%
- Technical and integration fit: 15%
- Implementation, migration and data load: 10%
- Total cost of ownership over five years: 10%
- Vendor viability and references: 5%
Two moves make this defensible. First, score functional fit on a 0 to 5 scale where native out-of-the-box capability earns the top mark and heavy customisation earns one, so a vendor cannot inflate the score with "configurable" answers. Second, publish the weights to vendors before they respond. Weighting adjustments after responses arrive turn the scorecard into a rationalization engine rather than a scoring model.
Require a Proof Artefact for Every AI Answer
Hallucination is not a theoretical risk inside a finance system. Independent benchmarks show hallucination rates between 15% and 52% across commercial models, and production telemetry on multi-step agent workflows reports 20–40% error rates on tool-call chains. These are the workflows an AP agent runs. An evaluation that scores agent capability on the vendor's self-description, with no artefact, is scoring a brochure.
For every scored AI question, require one of four proof types:
- A log excerpt from a comparable customer tenant, redacted but intact in structure, showing the extract-code-check-post chain for the workflow in question.
- A policy or control configuration showing how the agent is prevented from acting outside its scope.
- A test harness output showing agent accuracy on a published benchmark the buyer can reproduce.
- A named reference running that workflow in production for at least six months.
Answers without artefacts receive zero on that line, not partial credit. This single rule removes most of the padding from a 180-page response and surfaces the vendors who have actually run the workflow in production. For a deeper view of what the ledger permits and what it structurally refuses, the erp.io platform page documents the control-plane split that the gate above tests for, and the piece on what an agent cannot do inside a ledger covers the architectural reasoning.
Score the Dangerous Combinations, Not the Average
A composite score hides the thing that will go wrong. A vendor can earn an 82% total while scoring a 2 on exactly the workflow that handles the buyer's largest cash movement. The remedy is to score risk at the intersection of workflow and answer type, and to flag the dangerous cells before averaging.
The matrix below shows how to think about that intersection. "Roadmap" answers on high-value workflows are the cells that most often survive scoring and most often cause the post-go-live incident. Arthur AI's survey finding that 34% of enterprises reported a customer-facing LLM incident in the past year, with average remediation above $50,000 in regulated industries, is the price of ignoring these cells.
Give Each Phase a Deadline the Vendor Can See
Timelines leak. An RFP that opens on week one and never names its scoring deadline will drag past the fiscal-year window the project was approved under. Published dates discipline both sides. The sequence below is the one most mid-market agentic ERP evaluations can defend, assuming three to four shortlisted vendors and two finalists for scripted demos.
Two of these dates matter more than the rest. The clarification deadline forces vendors to surface inconsistencies between their functional and technical teams in writing, where they can be scored. The scripted demo week is where the buyer-supplied script against actual business scenarios replaces the vendor's rehearsed storyline, and where a refusal to follow the script is treated as a red flag.
Answers That Should End the Evaluation
Some answers are not low scores. They are exits. These are the responses that should remove a vendor from the shortlist regardless of the composite total, because each one indicates the vendor either does not understand the control model or is willing to misrepresent it.
None of these are unusual. Each has appeared in real agentic ERP responses in the past twelve months, and each has survived scoring in evaluations that failed. Pre-committing to disqualification criteria in the scoring document, signed by the evaluation panel before responses arrive, is what keeps a strong total score from overriding a bad answer on a critical control. The pre-signature matters: the panel should also agree the contract terms worth insisting on before the first demo, so the commercial conversation does not later re-weight the technical one.
What the Scorecard Buys You
Panorama's 2025 research puts the overall ERP failure rate at 68%, with budget overruns averaging 189% across industries. The RFP is not the only cause of that number, but it is where most of the misreads start. A scorecard that gates on controls, weights functional fit heavily, demands an artefact for every AI claim, scores risk at the intersection rather than the average, and names its disqualifying answers in advance will remove the vendors that would otherwise have produced the overrun.
The scorecard is the deliverable the audit committee will ask for later. Build it so the final choice is defensible in a sentence: this vendor passed every gate, scored highest on weighted fit, and produced artefacts for every AI control. For evaluation teams that want a second read on their current rubric, a free assessment will stress-test the gates and the weightings against the specific workflows in scope, and a scripted demo is the next step once the shortlist is down to two.