Last weekend at the wedding I attended, I spoke for some time with Matt from Consumer Reports. He works in the auto division, and he filled me in on how Consumer Reports does its research.
When CR evaluates products, employees approach the companies as typical consumers. For example, rather than the companies sending CR souped-up “review models,” like they do for other consumer review outfits, CR employees will go to a dealership “under cover,” haggle with the salesman, and purchase a car. This way, they’re getting the same deal and the same car everyone else gets.
This is how the non-profit organization operates for all of its products. From their mission statement:
To maintain our independence and impartiality, CU accepts no outside advertising, no free test samples, and has no agenda other than the interests of consumers… We buy all the products we use as test samples. We receive no special treatment. We accept no free samples. If a manufacturer sends us a free product, we return it.
Tomorrow I’ll post a quick blurb from their latest issue about what their undercover consumers discovered about “data brokers” like ChoicePoint—companies that sell your personal information without allowing you complete access.
Subscribe
The way Consumer Reports is structured is what has allowed them to keep their integrity over the years, inlike say, the “Better Business Bureau”. Companies DO pay the BBB. On the flip side, you can report companies you have had bad dealings with to http://www.ripoffreport.com . I hope you don’t mind me sharing the link (not my site- great resource![]())
The CR process reminds me of a good restaurant review. The dining staff and management have no clue that their food and service are under review.
The outcome: an ethical snapshot of a product or service.
I followed CR’s test taste on coffees in which cheaper beans (Eight ‘O Clock and even the Dunkin Donuts brew) scored higher than many of the high-end (expensive) brands.
How are they funded? According to the mission statement of Consumer Union, the parent organization, “CU supports itself through the sale of our information products and services, individual contributions, and a few noncommercial grants.” they may be free of “Corporate” taint, but they are very closely aligned with certain political and activist agendas. I believe they are deeply entwined with the Association of Trial Lawyers of America . Before you congratulate them on their integrity and impartiality take a look at their loyalties. They’re not as “independent” as you suggest.
Teddy,
1. You believe or you know? Can you find a source for reference?
2. If they are “deeply entwined” with the Trial Lawyers, how would that have an effect on their consumer reviews?
Everyone needs to get funded, and it’s true that everyone has a bias and an agenda. In my opinion, Consumer Reports’ methods are sound and result in more accurate reviews than those published by outfits that placate their advertisers.
Flexo,
You ask if I can find a source for my reference – and my answer is no; I’m not inclined to do that much research over a casual comment post
on a weblog. If, in your mind, that means the possibility should not be considered – so be it.
As for the Trial Lawyers, they are a deeply political group as you can see from their website if you care to visit. They advocate for union shops only and what some would consider alarmist product-safety standards. Such affiliations could result in favoring one manufacturer over another (union vs. non-union shop), or slanting studies to produce data designed to support law suits.
My point was simply that their model of performing reviews does not necessarily give them integrity and ethics. As you yourself
wrote “…it’s true that everyone has a bias and an agenda�.
I still suggest that rather than taking their purity at face value, a wise consumer would consider that there may be other means of bias. After weighing that possibility, go with the source you consider most trustworthy.
If you take a look at what I had written, I didn’t call their intentions pure. Of course it would be wise to consider all points of view available.
“Such affiliations could result in favoring one manufacturer over another (union vs. non-union shop), or slanting studies to produce data designed to support law suits.”
CR seems to favor non-union shops when it comes to autos, so this bias may be difficult to prove.
Here’s an interesting article from CNET about their procedures for technology reviews.
My comment about pure intentions was directed at the first two commenters who referred to “keep[ing] their integrity over the years” and “an ethical snapshot of a product or service”, not to your original post.
In retrospect, I suspect that Consumer Reports doea a good job on balance. They were not the ideal example for me to use.
The broader point I would like to make is that a non-profit or not-for-profit organization can have hidden motives just as readily as a capitalist corporation. I believe that too many people are lulled into trust by these designations. Everyone is getting their funding from somewhere, and charitable foundations can be just as corrupting as industry lobbies when they have an ideology to promote.
Thanks for allowing me to join in the conversation, and keep up the good work – I like your blog.
Thanks, Teddy—glad you’re coming back.
I worked for a non-profit, and I may again someday, so I have first hand familiarity to funding issues. Our choice in equipment was dictated by the manufacturer that provided the incredibly low prices for the equipment and the underwriting for the large events. The town in which we were headquartered allowed us to occupy a floor of the municipal building without rent—or at least with increasingly late rent payments—as long as we performed certain functions for the town.
Any kind of funding often comes with something in return—the conflict is when what is given in return is not completely honest information.