The Role of Product Reviews in Procurement

To provide a bit more of a teaser for the upcoming "Buying Security Products" eBook, let me rant a bit about product reviews. First of all, I hate product reviews. OK, I said it. With few exceptions, they do not reflect real life situations, and can be easily manipulated. What do you mean manipulated? I'll plead the 5th on that one, but suffice it to say I've lost very few reviews in my time because I understood the process. Of course, there are exceptions to that (and I'm sure my friends in the trade press will be pretty annoyed with me), but I call it like I see it.

So, what got me going? First was just seeing a couple of reviews (not just the stand-alone review of NFR that I blasted in this post), and I was struggling to get a feel for how that information is going to help a customer make a buying decision. In one of the reviews, 4 out of the 8 products got some type of "recommended" status. What the hell is that about? They may as well have said, all these products are the same - so it doesn't matter. Or they should have admitted, my methodology had me comparing apples to oranges, so if you are looking for apples - do this and oranges - do that. But they didn't, so end users are left to their own devices to figure out what matters.

Now, I may as well remind everyone what I used to do for a living. As VP of Marketing for any emerging technology company, product reviews are either your best friend or a friggin' nightmare. I've had numerous examples of both during my marketing tenure. It was always amazing to me that two different reviews would come out within a few weeks of each other and get totally different results and draw totally different conclusions. Had the product changed? No, of course not. It's basically the methodology that varies and that can have a dramatic impact on the "winner" of a review.

Let's look at reviews within the context of the procurement process (which of course will be detailed ad nauseum in the eBook). Product reviews can be useful in determining the long list during the time you are learning how to solve your problem (Step 3 - EDUCATE), prior to actually start talking to folks that will try to sell you stuff (Step 4 - ENGAGE).

Now, I get that no one has infinite time. So you can't talk to everyone. When I was in the anti-spam business, there were legitimately 50 companies that could stop spam in some way, shape or form. You can't possibly talk to all of them, so how do you pare it down?

Basically, a review can help do that. Most reviews will have a grouping of folks near the top. And if a space is hot, there will be a number of reviews that have been published. The reality is the cream does rise to the top. So if a set of vendors is always at the top of the list, then they are probably someone that should make the long list. Conversely, if a vendor is just not in the ballgame consistently, then just blow them out right there and then.

DO NOT MAKE A DECISION BASED ON ONE REVIEW. I'm not sure I can say it any clearer than that. 

But, does being a "Best Buy" or "Recommended" mean a damn thing? Not in my opinion. Determining the winner of these things is so subjective that sometimes it's impossible to understand why the winner actually won. This is pretty frustrating as you marketing folks out there know. "I don't know why the other guys won" does not make the CEO very happy.

And the vendors are masterful at using each "win" to make it seem that everyone else sucks. Don't believe them. Reviews are just another data point to help you focus your efforts on products that have the highest likelihood of solving your problem. But, they SHOULD NOT be used to determine a short list (2-3 vendors). You could be disqualifying the right product early on in the process because their marketing folks don't understand the process.

Like everything else having to do with procurement, the more data you gather - the higher the likelihood that you'll make the right decision.