System selection when every vendor demos well
Demonstrations are performed by people who know the software, using data chosen to suit it. Judging a system on one is like judging a car by watching someone else drive it around a car park.
Three vendors present. All three look capable, all three answer yes to most questions, and the shortlist gets decided by which presenter was most impressive or which price was lowest.
Six months into the implementation the differences become obvious, and they are never the differences that were discussed.
Why demos mislead without anyone lying
A demonstration is a rehearsed performance on clean data by an expert. Nothing about it is dishonest and nothing about it resembles your Tuesday.
The scenarios chosen are the ones the software handles elegantly. The data has no duplicate customers, no product coded twice, no partial delivery from 2019 still open. The presenter knows the shortcut for every screen. And crucially, “can it do X” is almost always answerable as yes, because with enough configuration or development most systems can do most things. The question that matters is what it costs to make it do X, and whether that survives an upgrade.
What to do instead
Write down your five hardest cases first. Not features. The specific things your business does that give software trouble. The contract production run billed to another label. The order that ships from three locations. The customer with consignment stock. The job that gets re-priced after completion.
Then make each vendor demonstrate those, with your data, in front of the people who actually perform the process. This single change does more than everything else combined, because it moves the demo from their ground onto yours.
Ask how, not whether. “Yes it does that” and “yes, with a custom module we would build” are radically different answers beginning with the same word. Every yes should be followed by: is that configuration, extension or development, and who maintains it.
Talk to a reference at your scale, in your sector, without the vendor present. Ask what surprised them, what the implementation cost against the original estimate, and what they would do differently. Vendors supply references who are happy; the useful information is in what they were unhappy about anyway.
Price the whole thing. Licences, implementation, data migration, integration, training, the internal time of your own people, and the ongoing cost of whatever customisation you just agreed to. The licence figure is often the smallest number in that list and the only one on the proposal.
Test the exit. Can you get your data out, in a usable form, without paying for the privilege? Ask during the sales process, while you have leverage, and get the answer in writing.
The internal question that decides more than the software
Who is going to own this?
Not the project. The system, afterwards. The person who understands why it is configured the way it is, who reviews the exceptions, and who says no when somebody wants a field added to support one report.
Implementations rarely fail because the wrong product was chosen. They fail because nobody was given that role, and a good product with nobody owning it drifts into the same state as the systems it replaced.
And a declaration
We implement Odoo, so we are not a neutral party in every selection.
The way we handle that is to price discovery and system selection separately from implementation, and to declare the conflict whenever a recommendation lands on something we deliver. It also means we can reach the conclusion that you should keep and extend what you have, and still be paid for the work of finding out.
If a selection process is being run by whoever would implement the winner, at minimum know that going in.