Why the bad runs are in here
Every sourcing vendor shows you a good campaign. So did we, for about a year: one run, a European pharmaceutical ingredient, presented on the homepage as evidence that the thing works.
That run was real. It was also the only one, it was in a category almost nobody in our target market buys, and it had been chosen after the fact. Which makes it a testimonial, not a measurement.
So we commissioned 8 campaigns against the categories our buyers actually work in, wrote the briefs before we knew what would come back, and published all of them. Between them they read 4656 company pages and left 472 suppliers on the shortlists. Every one of them came in under the pass mark the engine itself set before searching. That gap is the most useful part of this article, and the reason it is worth reading.
The pass mark almost nothing reaches
Before it searches, the engine estimates how many suppliers exist for a brief and sets itself a pass mark at roughly seventy percent of that estimate, capped at 250. It then reports whether it got there.
It did not get there. Not on the broad briefs, where the estimate ran into the thousands and the cap made the target unreachable by construction. Not on the narrow domestic ones either, where the target scaled down to something plausible and the runs still landed under it.
This is worth stating plainly because it is the opposite of a sales claim: the product's own scoring says it underdelivered on every campaign in this article. What that actually means is more interesting than either the number or the failure. The pass mark measures coverage of an estimated market. A buyer does not need coverage of a market. A buyer needs enough qualified names to run a real comparison, and then the search stops being the problem.
A missing certificate never drops a good maker. It lowers a score, and the score is a number you can argue with.
Run by run
Biggest file first. Each line is what the campaign record says, followed by what we make of it.
Folding carton packaging, searched worldwide. 170 on the shortlist in 6.2 minutes, 134 of them with an email address. The biggest file of the 8 and the fastest, which is not a coincidence. Packaging converters publish product pages, list their minimums, and sit in dense clusters the search can walk. China led the country count, Germany came second within a couple of places, and the file mixes names a buyer knows with names a buyer would never have found. That mix is the point: Stora Enso and Graphic Packaging are the ones you would have Googled anyway, and the German and Italian converters underneath them are what twenty six languages buys you.
Catering for a 400 guest launch, Chicago. 77 on the shortlist in 11.7 minutes, 50 of them with an email address. The narrowest brief in the programme, tied to one city and one guest count, and the best service result. The names that came back are the Chicago catering scene rather than a national directory scraped for the word Chicago, and only one company in the whole run was rejected for being in the wrong place, which is what a city-level search is supposed to look like. Two of the entries are the same operator under two trading names, which is the honest weakness here: the file needs a human pass for duplicates before anybody sends an enquiry.
Scaffolding contractors, United States. 67 on the shortlist in 10.2 minutes, 39 of them with an email address. The second largest file, and a good result with one hole in it. The brief asked for certified crews and insurance, and the engine found the contractors but almost none of the certificates. Scaffolding firms in the United States treat certification and insurance as things you are told during the bid, not things they publish, so the two clauses that would actually disqualify a contractor are the two the web could not answer. The shortlist is a real list of scaffolding companies. It is not a list of pre-qualified ones, and the difference matters.
Medical disposables, FDA registered, China excluded. 58 on the shortlist in 11.6 minutes, 28 of them with an email address. This run read more company pages than any other and threw away the overwhelming majority of them. That ratio is the brief doing its job: "FDA registered" and "sterile" are claims a page either makes or does not, so the screen is unusually decisive. Excluding China visibly changed the shape of the file, putting the United States on top with Italy and Sweden behind it, which is roughly where a compliance-led medical buyer would expect to end up. Fewer than half arrived with an email address, and that is the number to plan around, not the shortlist size.
Deep groove ball bearings 6204, producers only, China excluded. 39 on the shortlist in 12.3 minutes, 21 of them with an email address. The hardest brief of the set, and the one that shows the difference between a search and a screen. A part number like 6204 is sold by thousands of distributors and made by a few dozen firms. The brief said producers only, and the file came back with India in front, then the United States and Germany, and a Latvian and a Turkish maker scoring at the top alongside a British one. It took the longest of any run and returned a third of what the packaging brief did, because there is genuinely less to find.
Industrial spare parts, distributors holding United States stock. 31 on the shortlist in 16 minutes, 13 of them with an email address. The slowest run in the programme, and the one that shows what an over-broad brief does. "Industrial spare parts" is not a category, it is a department, and the search collapsed it into the sub-category with the densest web presence. Read the top of the file: seals, gaskets, sealing devices, almost nothing else. Every one is a legitimate answer to the brief as written and the set as a whole answers a narrower question than the one asked. Ask for a department and you get whichever corner of it publishes best.
Cold chain pharmaceutical transport, United States. 23 on the shortlist in 11.2 minutes, 14 of them with an email address. A service brief, so it was searched inside one country, and the first usable name appeared in well under a minute. That speed is what a narrow domestic brief buys. The interesting number here is not the shortlist, it is the rejections: the engine found and discarded a substantial block of firms that were the right business in the wrong country. That is the tax on a domestic search, and it is the reason a service brief should never be run worldwide.
Commercial kitchen equipment dealers, United States. 7 on the shortlist in 4 minutes, 1 of them with an email address. The clear failure of the set, and the most instructive one. The brief asked for dealers with installation and service coverage. What came back was manufacturers: Turbo Air, Hobart, Alto-Shaam, Middleby. Every one of them scores high against "commercial kitchen equipment" and none of them is a dealer. American foodservice dealers are regional, they sell on the phone, and their websites carry a territory map rather than a product catalogue, which is close to invisible to a screen built to read product pages. Almost none of the shortlist arrived with an email address. This brief needed rewriting, not re-running, and it is the clearest example of where a web-reading search hits its limit.
Three things the data says that the marketing does not
One. A shortlist is not a contact list. Across all 8 runs, 300 of 472 shortlisted suppliers arrived with an email address, which is 64 percent. The rest are real companies with real websites and a contact form. Somebody still has to fill in that form. Any tool that quotes you a shortlist size without quoting the contactable share is quoting the easier number.
Two. Speed tracks the shape of the category, not the size of the result. The packaging run returned the largest file in the shortest time. The bearings run returned a fraction of that and took twice as long. Dense categories with product pages are fast and generous; categories where the real makers are few and quiet are slow and thin. The clock tells you about the market you are buying in, not about the software.
Three. Domestic service briefs fail differently from global product briefs. A worldwide product search fails by returning too much adjacent material, which the screen then discards in bulk. A domestic service search fails by returning the wrong kind of company entirely, as the kitchen equipment run did when it handed back manufacturers instead of dealers. The first failure is cheap and self-correcting. The second needs the brief rewritten by somebody who knows the trade.
Where this leaves you
If your categories look like the packaging and bearings runs, product families with makers who publish what they make, you can run this yourself and have a shortlist the same afternoon. That is the first way of working with us and it needs nothing installed.
If your categories look like the kitchen equipment run, where the right supplier is defined by things a website does not say and your team holds the rules that decide, a search you run yourself will keep returning the wrong kind of company. That is the second way of working with us: the agents get pointed at your own supplier database first and taught your qualification rules, and what you end up holding is a pipeline your team runs, not a login.
We would rather tell you which of those you are before you pay for the wrong one.
472 suppliers across 8 briefs, 1053 rejections still on the record with the reason each one was given.