In my experience, making reviews easier to give causes the review quality and usefulness to go down. This happened when Netflix went from 5 stars to a simple like/dislike. I’m not sure why Amazon didn’t just block non-verified-purchaser reviews, increasing spammer costs significantly.
I suspect that Amazon reviews are going to be even less useful now. Especially given things like this:
> Amazon does not provide many specifics about how a product’s overall star rating is calculated, other than stating that it is not a simple average but instead uses “machine-learned models” that take into account factors such as how recent the rating or review is and whether it was a verified purchase or not.
I'm pretty sure that a ton of the review fraud on amazon comes through verified purchases. You refund/pay people to buy the product on amazon and leave a review in the best case, in the worst case you operate accounts buying your own stuff, and then flow the inventory back and basically pay an amazon tax for leaving good reviews.
IMO this is pretty solvable by looking at an account's purchase history too, but I don't think it's just as simple as blocking non-verified-purchase reviews.
And while you're dreaming that purchase history solves that, the scam world has long moved on.
Currently, companies pay people to buy items. They can keep those items, they just need to leave a good review. There are intermediaries who handle lots of sellers, so people buy a mixed bag of random garbage in exchange for the occasional review.
Yes, you can probably test for statistical anomalies, but I'm willing to bet that's quickly countered too - just have people buy occasional legit items so their profile is "statistically normal".
As far as I can tell, Amazon tries to fight that by keeping their ML model secret so scammers don't learn too quickly, but essentially, they're currently finding out what the Internet learned about SEOs manipulating search results.
Yes, I've seen articles where people who do this say there are a bunch of Facebook groups for exactly this purpose. Manufacturers/sellers ask people to buy their goods on the group and write good reviews. Once they get the proof of the purchase/review, they Paypal (or whatever) money to the "reviewer." I remember an article I read where some woman's house was overflowing with junk she didn't want because she was writing so many of these fake reviews. I would think Amazon could apply their awesome machine learning to figure out which accounts are pumping out these suspect reviews and perhaps de-prioritizing and maybe even rate limiting and delaying publishing of their reviews. I'd think that would do a lot to alleviate the problem.
> What is higher than the expected number of 5-star reviews?
That's left as an exercise for the reader ;)
But you could simplify this to new products, which if ungamed, would probably follow a predictable curve of discovery, as opposed to a games "instant 5 star" rating.
That is a very simple algorithm! It’s also something the sort of people who do this will overcome relatively quickly. Automated or systemic attacks and mitigation is a war of escalation.
I suspect solving it is hard to impossible. You can heuristic anything as suspicious at a glance but moderation at scale fails. You personally can decide "this review is bullshit" and block without issue but start flagging in a false positive or even true positive by a manipulator looking to start shit could cause considerable backlash as a futile attempt at pleasing everyone is made. Transparent and consistent rules invite gaming and opaque ones invite accusations of malfeasance and the uncertainty promotes bad feelings and bad behavior.
Worse still is that even taking a stance to not take a stance because you know it cannot be done will bring backlash as there are many who demand you take their stance, even allowing easy distribution of self chosen block and filter lists are not enough. There are many who demand the appliance experience.
There's also low scale review fraud like my sister bought a pet cam and gave it a 3 star review, and the seller started to offer escalating offers for her to remove or increase the rating from full refund, another camera, refund + free camera, and refund + money for the inconvenience.
It does bug me that scummy vendors know my email address and physical address. What's next? They send goons to my house demanding a good review or my dog gets it?
The same applies to links and Google. Buying/renting backlinks costs money, and still pretty much everybody that wants to rank in competitive fields does it.
Perhaps they could limit refunded products to 3 star or below? Really there isn't a good reason to positively review something and take it back, if it was your fault eg if it didn't fit, then you could just not review it.
> I’m not sure why Amazon didn’t just block non-verified-purchaser reviews, increasing spammer costs significantly.
I don't think it'd do much good. There are already large groups that subsidize verified purchases to get 5 star reviews. I read an article about it, confirmed some investigation on Facebook of on my own.
Basically someone runs a Facebook group where sellers advertise free product, promising Paypal reimbursement of the purchase price in exchange for reviews. There are at least (or were, I haven't checked recently) hundreds of Facebook groups across many languages with thousands of members each doing these activities. The reviewers are randos who like free stuff, and I don't think anyone could detect them if they only casually participate in the review scams.
I rather look at the content of the reviews than their ratings, there's been times where someone left what looks like an honest review and it's 4 / 5 but they only write positive things. I don't know what the solution is, but I definitely think more thorough reviews would help.
If you return a bad product, try and get a replacement to see if it was just bad luck, but also please do write a detailed review.
If you get a good product and it barely has reviews please review it.
someone left what looks like an honest review and it's 4 / 5 but they only write positive things
That may well be me. I don't think that "meets requirements" should merit a 5. When I do reviews, I view a bare "meets requirements" as 3 or maybe 4, depending on the type of product. I want to leave some headroom to be able to point out products that really do excel.
Another ambiguity is whether the rating is on an absolute scale, or normalized for value (it's not a perfect product, but it's super-cheap).
I think that on Netflix's case they are not going for reviews. They want to know what you like, to suggest something you'll probably like too. Doesn't matter if it's 4 or 5 stars.
> I’m not sure why Amazon didn’t just block non-verified-purchaser reviews, increasing spammer costs significantly.
I'm not sure this would help much. I've been asked by vendors to leave a review and if it's 5 stars, I will get something in return. I think for some people this would be a "well, nobody is getting hurt, right?" decision and they'd just do it.
I suspect that Amazon reviews are going to be even less useful now. Especially given things like this:
> Amazon does not provide many specifics about how a product’s overall star rating is calculated, other than stating that it is not a simple average but instead uses “machine-learned models” that take into account factors such as how recent the rating or review is and whether it was a verified purchase or not.