Rating or over rating.

6:24 p.m. on April 29, 2016 (EDT)
TOP 25 REVIEWER REVIEW CORPS
3,569 reviewer rep
1,335 forum posts

Not sure if it's just me, but my impression is that the most common rating for a review is 5 stars. I feel this could be due to a variety of reasons:

  • More of an urge to review gear you are really excited about.
  • Subconcious or maybe conscious need to justify a purchase.
  • Lack of in depth and honest analysis of the benefits and flaws of a piece of gear.
  • An abundance of really good gear.
  • Not enough time spent with the gear to fully evaluate its true potential, like durability.

Admittedly it's difficult to hold back a review...you really want to spread the word or have a chance at a really cool freebie being offered that month. I have several pieces of gear from last year I am having a hard time holding back until another trip or two so understand, but consider the usefulness of the review to others as well. I tend to ignore reviews by folks who only rate things highly. I might be missing out on a good review but tend to trust reviewers with more variable ratings as well as in depth reviews...anyone else do the same?

8:10 p.m. on April 29, 2016 (EDT)
REVIEW CORPS
4,534 reviewer rep
6,027 forum posts

Phil,

I think that most of the gear we get via the Gear Review Corps is pretty good. Getting items like the QuietStoveCap that I got one time and gave a 3.5 star (readers gave me 8 votes for the review) are pretty rare. Gear that I review having bought is subject to examination and evaluation before I waste my time. If I can readily tell that the gear is junk, it doesn't get reviewed.

Of all my reviews, I had one other 3 star, a Kelty Windfoil tent, that was nominally 3 person, but required 4 of us struggling hard to get pitched (our expedition was supplied 5 of the tents). That was in Antarctica, where getting the tents pitched in storm is life-critical (we didn't have a storm while pitching them). People should know if there is that big a flaw. I had a 4-star for a Big Agnes inflatable insulated mattress. The mattress itself was mostly very good, but the first sample punctured itself on the plastic support blocks inside. It was a bit over-rated for the claimed use on snow, but really nice on bare ground (i.e. nosnow or ice under me). The defective one was replaced by Big Agnes fairly quickly.

To me, 5-star means fully meets all my expectations. 4-star means pretty good, but not perfect. 3-star means might be ok for some conditions, but falls short of what I expect. 2-star means the item falls pretty short of my expectations, but might work if you aren't too critical. 0-stars, I just won't look at, though if Alicia or Seth sent it to me, it's gonna get some really negative remarks with pictures of its total failure.

I have an item that Alicia sent me to review that someone else on the GRC turned down that looks like it might not make 4 full stars, though it isn't really all that bad, plus the company normally makes very good gear. 

In other words, anything I review has been subject to pre-screening before I spend my time on it. I sometimes also throw in a comparison to a similar product and say something like "Junk company's product is awful by comparison to this product, which barely rates 4 stars". Why waste my time on a product that shows it is trash in the first day of use, when I can be testing something that might require 6 months of fun, er, I mean, thorough wringing out on long fun outings?

And besides, I refuse to test anything whose failure is life-critical if a first examination shows the probability of failure is high.

8:19 p.m. on April 29, 2016 (EDT)
TOP 25 REVIEWER REVIEW CORPS
3,569 reviewer rep
1,335 forum posts

I probably should have clarified a bit...I use thoroughness of review first and yours certainly meet that criteria so any 5 star rating from someone like you I take at face value. I was speaking less to the review corps and long time reviewers and more to what seems to me an occasional run of new reviews with 5 stars that don't seem to fully evaluate the gear but rather just promote it. 

I was not meaning to disparage anyone in particular Or try to artificially get folks to reduce their ratings. More to throw out food for thought to reviewers with the tendency to rate high and analyze less. 

8:29 p.m. on April 29, 2016 (EDT)
TOP 25 REVIEWER
1,204 reviewer rep
393 forum posts

That's a good question, and I wonder if it means that the abundance of product information available on the web is keeping people from buying as much subpar gear? 

Something else that this topic got me thinking about is whether people weight how & where a reviewer uses a product. For example, I rated MSR Lightning Ascent snowshoes 5 stars, but I don't use them in extreme terrain. Would a 3- or 4-star review from someone who DOES use them in extreme terrain carry less, equal, or more weight than mine?

9:13 p.m. on April 29, 2016 (EDT)
107 reviewer rep
490 forum posts

I prefer to hear about what works for you and why. the pros and cons. the star rating lets me know how much. Then if I recognize the reviewer I may read further. Some folks just want to write a novel and not all of us need to know that this Product is .76555 millimeters to long or 7/10 of a second slower at xyz deg. unless like Bill says its life threatening or mission critical. I do believe we have the intelligence to figure out the rest. PROS=whats best about it. CONS= fundamental flaws. Reviewer=Phil, Bill, ect all. people who have reasonable experience and have consistently shown good judgement.  

9:23 p.m. on April 29, 2016 (EDT)
1,977 reviewer rep
362 forum posts

Phil Smith said:

I wonder if it means that the abundance of product information available on the web is keeping people from buying as much subpar gear? 

That certainly is the case with me. I do a lot of homework and thinking and sometimes spreadsheet analysis before I pull the trigger on a major piece of gear, so by the time I've decided on it I'm pretty confident I will be at least reasonably happy with it. If I buy a piece of major gear that I feel deserves less than a 4-star rating then I screwed up.

9:27 p.m. on April 29, 2016 (EDT)
271 reviewer rep
1,919 forum posts

The 5 stars I see are general the hit once review people...Exactly what you listed..The 3 and 4 's I read twice..To see where, when and how long they used it..Hate to say it at times I feel the Review Corps evaluations are to short in duration..But I understand this is a compromise for the company as well.So I am not complaining just saying...

12:49 a.m. on April 30, 2016 (EDT)
158 reviewer rep
391 forum posts

For me most of the gear I've had for years and wouldn't hesitate recommending it. I hold on to my 5 star gear and lesser gear gets returned,given away,ect. I have reviewed knives on here that had no reviews but are not for the outdoors so I did a quick review and made sure that people knew they were not outdoors knives but more for self defense..spyderco matriarch and ulize. Pants,boots,and packs,ect I'll swear by. 

I don't think some people like to admit that they bought subpar anything,its almost a ego blow. I know I went through tons of crap gear before I knew what to look for and I'm sure I'll get something that doesn't make the cut again. I think it is important to be honest in a review what maybe a plus for you maybe a negative for someone else.My reviews haven't been that great and I'll be working on that. I guess everyone has different standards and opinions, but I wait till I've had and used and abused something before it gets 5 stars.

I do think I fall into the category your referring to Phil. My reviews do need work and more thought so I'm man enough to say that I'm guilty.

5:23 a.m. on April 30, 2016 (EDT)
TOP 25 REVIEWER
1,204 reviewer rep
393 forum posts

tracker clayton 2 said:

I don't think some people like to admit that they bought subpar anything,its almost a ego blow. 

 That's true, and it may be that's another reason why there are fewer 1 & 2 star reviews than statistics say there should be.

6:21 a.m. on April 30, 2016 (EDT)
158 reviewer rep
391 forum posts

sorry I ment I do fall into the category your referring to FlipNC, but it also works for Phil too.

8:06 p.m. on May 4, 2016 (EDT)
REVIEW CORPS
1,979 reviewer rep
475 forum posts

I think there are a number of factors at play, in no particular order

#1 there really is a huge abundance of good gear these days.

#2 generally there is access to at least some information about bad gear online, so a bit of research generally helps you avoid glaringly terrible gear

#3 people do need to validate their decisions/purchases so there is a bit of that

#4 somewhat related to #3, there tends to be a "honeymoon" period with new gear when people get a bit overexcited, this is why I dont like reviewing until I have really given things a try

#5 sometimes people get frustrated with bad gear and just dont want to deal/bother with it, and this may extend to them not caring enough to go online and review

11:21 p.m. on May 4, 2016 (EDT)
GUIDE/OUTFITTER
297 reviewer rep
116 forum posts

My 2 cents I have never given 5 stars because there is no perfect tool for all jobs. 

With that said with the world of information these days that if you do your research you should get what you expect. 

Pretenders don't last long in this new world of information overload

2:10 p.m. on May 5, 2016 (EDT)
REVIEW CORPS
1,747 reviewer rep
743 forum posts

Most of the 5 star "problem is likely due to the fact that those who write reviews are going to be motivated to do so...my guess is that just as there is a over-representation of 5 stars there is likely to be an over-representation of 1 stars...because there are not a lot of folks out there motivated to write a review about a piece of unexceptional but adequate piece of gear...and there are a lot of folks who are upset or overly impressed with their gear willing to write a review.

5:53 p.m. on May 5, 2016 (EDT)
TRAILSPACE STAFF
346 reviewer rep
995 forum posts

There are a lot of good theories in this thread, and I suspect that there's an element of truth to most of them.

This apparent bias toward higher ratings is pretty common across review sites, not just Trailspace.

For the record, our current distribution is:

8% 1-star
6% 2-star
10% 3-star
33% 4-star
44% 5-star

Our average rating is 4.07 stars and the median rating is 4.5 stars. This is basically unchanged from 5 years ago.

8:59 p.m. on May 5, 2016 (EDT)
TRAILSPACE STAFF
346 reviewer rep
995 forum posts

Review Corps follow a different pattern: ratings are clustered a bit more in the middle of the range, with the vast majority at 4 stars:

1% 1-star
4% 2-star
20% 3-star
62% 4-star
14% 5-star

The average Review Corps rating is 4.05 and the median Review Corps rating is 4 stars.

Interestingly, the combined portion of 4 and 5-star reviews is nearly identical between Review Corps (76%) and all reviews (77%).

Since Review Corps reviewers don't have anything invested in the products they're testing, this lends some credence to the "most stuff is pretty good" theories.

That said, there is some selection bias in the gear that we test through Review Corps. Of the hundreds (thousands?) of products we see every year at Outdoor Retailer, we only select a relative handful to review. There's some apparent crap (but also some good stuff) that never even makes it to our reviewers. We also haven't tested much, if any, generic/big-box/house-brand gear.

9:19 p.m. on May 5, 2016 (EDT)
11,722 reviewer rep
1,399 forum posts

I think there is also a tendency for people to move to the extreme on internet opinions. I'm not sure why, but you certainly see some intense "black or white" positions taken on many things, including inconsequential things. You see extremes of topics ranging from politics to carrying guns in the backcountry to Kim Kardashian selfies--you rarely see middle-of-the-road opinions anymore.  So I can see that spilling over into reviews. 

Beyond the star rating, I want to see if a thorough review was written. I want to know how it was tested, how long it was tested, and how it stands up overtime. 

Of course the worst reviews on Amazon say stuff like, "5-stars: I bought this for my grandson for Christmas, and I know he is going to love using this." 

10:00 p.m. on May 5, 2016 (EDT)
REVIEW CORPS
1,979 reviewer rep
475 forum posts

I think one of the reasons review corps items get a higher rating could be that we are substantially more thorough than average, and we are judging the item against set criteria and manufacturer claims. If those claims are met, you cant dock them points for random bias you might have. 

Along those lines, I thoroughly disagree with the early poster's remark about not giving 5 stars because nothing is perfect. We arent judging on some subjective notion of perfection, we are judging on whether or not it meets manufacturer claims and is adequate for the task at hand. If it is, then 5 stars is warranted in some cases. 

10:47 p.m. on May 5, 2016 (EDT)
158 reviewer rep
391 forum posts

I think there are certain items that some can review rather quickly. Maybe it's tents,packs,stoves,ect because they have had alot or have a hobby that involves it, me for example I have alway been into knives and tomahawks and yep I've been burned. Now I read up on makers, companies, steels, anything that has to do with them. I've spent tons of money on knives I've also sold alot of them. I can handle a knife or tomahawk and tell if it's junk but I can't with a tent or other items I gotta test them. My pack I love same with my footwear and fishing rods ect,but I can't say they are the best. If it's not a knife or tomahawk and I recommend it I've had it atleast a year or longer.  I had said earlier that most people don't want to admit they bought a subpar item as its almost an ego blow. Now if you review the item and give it a 5 star rating and people buy it and they all get a piece of crap your credibility is shot so no1 will pay attention when you do get good gear. I've done some bad reviews when I first got here now I know what people expect so when I do another review it will be much different. So before you throw out 5 stars think about your credibility and is it really a 5 star item?

It's also hard to review all the knives I got and the 20 some tomahawks I have. It's easier to post pictures and have someone or numerous people ask about a item and then do a review. Want to know about a Winkler hawk I got 2 I'll gladly tell you about both and review both,want to know about a Helm forge or grind item I got 12 items,ect. It's not that I don't wanna review them all its more that it would take forever and in the end I don't think most care about tomahawks or knives here except a select few.

8:06 a.m. on May 6, 2016 (EDT)
107 reviewer rep
490 forum posts

Been giving a lot of thought to this and the way it looks is that the star rating doesn't mean it is perfect or not, but how enthusiastic the reviewer feels about the product at that time. I've never viewed their rating as an I should take this as truth and go out and buy it but as something worth looking into. 

9:38 a.m. on May 6, 2016 (EDT)
TOP 25 REVIEWER REVIEW CORPS
3,569 reviewer rep
1,335 forum posts

Those statistics are interesting Dave. It says, to me, that the review corps probably tests gear more thoroughly and/or are at least slightly more selective in handing out 5s. No big surprise there, as the group has been vetted and provides quality reviews consistently.

I agree if the gear has been selected carefully through thorough research, then 4s and 5s are the most likely result. My lower ratings have usually been for bargain items that I like to try out when I see a good deal, but,as the saying goes you get what you pay for many times.

I must admit that in looking back not only have my reviews improved but my ratings shifted. Soon after joining here, I changed a couple of my older gear reviews upon purchasing newer and better equipment and realizing that I was rating too high initially. Always learning on this site...

9:44 a.m. on May 6, 2016 (EDT)
158 reviewer rep
391 forum posts

I actually went back to some of my knife reviews all spydercos and my Winkler tomahawk review and explained why they got 5 stars last night.

11:17 a.m. on May 6, 2016 (EDT)
REVIEW CORPS
4,534 reviewer rep
6,027 forum posts

An aspect of review ratings that hasn't been mentioned here is the ratings of the reviews themselves. As one who has received "Killer Review" ratings on all but one review since the system was initiated in 2011, it has been interesting to try to figure out why one review gets 15 upvotes and another only 2.  My conclusion thus far is that it has a lot to do with how much interest there is in the particular type of gear.

I would think that food would be a very high interest item. Yet, very few of the food reviews get high "helpful" ratings. Then again, food items get lower star ratings in many cases - a message to the backpacking food manufacturers? OTOH, there is one item I notice has a very large number of reviews with having mostly 5-star ratings from the reviewer, almost all by a single reviewer, with mostly "0" helpful ratings.

Tents, clothing, sleeping bags, and footwear seem to generate a lot of interest, although the reviews themselves seem to get a wide range of "helpful" ratings. Not too surprising though is that the lower "helpful" ratings are of the "bought it, opened the package, it is GREAT" type. In other words, no indication of actual usage and no discussion of what is so great or any indication that there might be cons.

As time has gone along, I will note that I have picked up many good ideas on how to test, what is relevant to the readers, and usefulness of photos and videos to the readers.

1:12 p.m. on May 6, 2016 (EDT)
TOP 25 REVIEWER REVIEW CORPS
6,757 reviewer rep
1,647 forum posts

I like the thoughts on this issue and I think its very important not to be biased toward loving gear that is truly, only mediocre. This thread is a good reminder to me to be thorough and objective.

Anyone ever remember Alicia offering any Ozark Trail, Stansport (those brands may occasionally make something decent) or other mass-market trash for us to review?  Me neither.  IMO, that is the greatest factor in the higher ratings; TS just seeks or accepts products from better manufacturers.

2:43 p.m. on May 6, 2016 (EDT)
REVIEW CORPS
4,534 reviewer rep
6,027 forum posts

Jeff's comment reminds me - some years ago Trailspace used to get lots of posts reading "The poles on my Ozark Trail tent broke/don't fit/are missing several poles/left me with several extra poles", plus comments about the zippers not working, tent leaking, etc. After many responses to let people know that TS is not a manufacturer, dealer, or repair service, particularly of Big Box Store "outdoor" gear, the policy of just plain not answering other than pointing out that if there is a problem with a product, you need to contact the store or manufacturer. Eventually people got the hint. I don't recall seeing a plea for help in a couple years now.

OTOH, there still seem to be a small number of posts on gear that are clearly outside the range of interest of the original intentions of Trailspace and the interests and needs of the vast majority of visitors and members of Trailspace, although those could be posted in "Off Topic".

5:39 p.m. on May 6, 2016 (EDT)
TOP 25 REVIEWER
1,204 reviewer rep
393 forum posts

Bill S said:

An aspect of review ratings that hasn't been mentioned here is the ratings of the reviews themselves. As one who has received "Killer Review" ratings on all but one review since the system was initiated in 2011, it has been interesting to try to figure out why one review gets 15 upvotes and another only 2.  My conclusion thus far is that it has a lot to do with how much interest there is in the particular type of gear.

 

I think part of it may be that some people think an up vote means you agree with the review, rather than you found the review helpful to your decision-making process. You could find a 1- or 2-star review helpful if something in it made you change your mind about spending money on that piece of gear ("Wide" wasn't, sizing is off compared to a standard, item leaked/tore/broke too easily, etc.) An up vote in this case, preferably with a short explanation in a comment, may help draw more attention to the review. 

Another possibility is the Internet made some of us lazy. When I'm doing a general search for "hiking boots" the fact that there are hundreds to go through makes me not want to leave the first page. Same goes for google searches - "ugh! I'm not clicking 'next' TWICE! I want this to be EASY!" As I learn more, I know certain brands fit better in general so I can safely ignore 80% of them. As I told an instructor giving a class on something new to me at work when he asked if we had any questions, "I don't know enough about this thing to know which questions to ask!"

Edit: Is it appropriate to give an up vote if you're not in the market for that particular piece of gear, such as if you already own it? Or is a comment about your good experience with it more appropriate? (I'd say negative comments seem like they'd be better expressed in a review of your own. That's a good idea for positive comments as well.)

11:45 p.m. on May 6, 2016 (EDT)
REVIEW CORPS
1,979 reviewer rep
475 forum posts

I frequently give up-votes to well done reviews, regardless of my initial interest in the item (or lack thereof), and regardless of the rating. 

A good, thorough review deserves credit. I get to learn about a random piece of gear I hadnt even thought about, and I get someones take on how fit for purpose it is. If I find it well written and informative, even if it wasnt information I was needing, I think it deserves acknowledgement.

12:55 a.m. on May 7, 2016 (EDT)
158 reviewer rep
391 forum posts

I think people should ask questions about gear. You may have done a great review but someone may have that random question you didn't address. 

Just because i know it will be seen and does in a way have to do with reviews....Extra stitching shouldn't be cut or burned off footwear or clothes it's usually left to accommodate stretching and wear over time, but people don't know this and say the stitching came undone after 1 outing or 1 wash.

8:04 a.m. on May 9, 2016 (EDT)
11,722 reviewer rep
1,399 forum posts

TJ1984 said:

I frequently give up-votes to well done reviews, regardless of my initial interest in the item (or lack thereof), and regardless of the rating. 

A good, thorough review deserves credit. I get to learn about a random piece of gear I hadnt even thought about, and I get someones take on how fit for purpose it is. If I find it well written and informative, even if it wasnt information I was needing, I think it deserves acknowledgement.

 Yes, exactly. I endeavor to provide a positive comment on any review where it's clear the reviewer sought to give details. And I vote up reviews where folks take the time to post relevant photos and fuller details.

Aside from the testing, it takes me several hours to put together a review. I want folks to feel it was worth the effort and time, even if it's not a product I'm interested in. 

We get too many crappy reviews not to acknowledge the good ones!

10:16 a.m. on May 9, 2016 (EDT)
TOP 10 REVIEWER REVIEW CORPS
7,931 reviewer rep
1,430 forum posts

I'm with you two and vote for good reviews even if it is gear I'm not interested in buying at the moment. Recognizing good reviews rewards the effort the writer took to hit all the marks and provide a clear picture of what an item was all about, good or bad. It also promotes the good reviews so they show up before the less informative ones.

12:14 p.m. on May 9, 2016 (EDT)
271 reviewer rep
1,919 forum posts

So the comment section at the bottom of the review is for questions and for that a boys? Or is it also to point out where not in the cons a well known con to the gear that's being reviewed that wasn't in  there?

5:47 p.m. on May 9, 2016 (EDT)
TOP 10 REVIEWER REVIEW CORPS
7,931 reviewer rep
1,430 forum posts

denis daly said:

So the comment section at the bottom of the review is for questions and for that a boys? Or is it also to point out where not in the cons a well known con to the gear that's being reviewed that wasn't in  there?

 I'd see no problem with asking the reviewer if he ran into that "well known con" or linking to your own review of the same product where you detailed the "con". Don't think I'd get into an argument about it in the comments though, that's what the forums are for :)

9:24 p.m. on May 9, 2016 (EDT)
REVIEW CORPS
4,534 reviewer rep
6,027 forum posts

A couple of times, I have read a review where the reviewer made a significant error due to lack of knowledge (Thankfully, these were non-Gear Review Corps people). In one of them, the reviewer came back with a "I am the world's greatest expert". I added an extra comment with a pointer to several places on the web where he could get the correct and complete information. In another, where the error was actually pretty minor, but due to a claim by the manufacturer, the reviewer responded with a "I didn't know that, thanks for pointing it out, I will make a correction".

Something not mentioned so far - most manufacturers are glad to get constructive criticism.

10:38 p.m. on May 9, 2016 (EDT)
158 reviewer rep
391 forum posts

Bill S .....I agree a call or email to a manufacturer will usually always get you good info and constructive criticism.

6:51 a.m. on May 10, 2016 (EDT)
271 reviewer rep
1,919 forum posts

LS said: 

 I'd see no problem with asking the reviewer if he ran into that "well known con" or linking to your own review of the same product where you detailed the "con". Don't think I'd get into an argument about it in the comments though, that's what the forums are for :)   

I try not to even argue on the forum because it doesn't work..LOL people are going to do their thing because of ego...Sounds bad but pretty much true...

8:50 a.m. on May 10, 2016 (EDT)
TOP 10 REVIEWER REVIEW CORPS
7,931 reviewer rep
1,430 forum posts

Oh I was just making a joke Denis. We have some respectful differences of opinion here sometimes, but very little actual arguing. Sometimes I think we're a bit too respectful when it comes to people suggesting dangerous or bad ideas, but as you say, you can't teach them better anyway so why bother?

9:29 a.m. on May 10, 2016 (EDT)
TRAILSPACE STAFF
1,684 reviewer rep
4,127 forum posts

I agree that an upvote is meant to recognize that a review is either personally helpful ("Great info and pictures of the vestibule that helped me choose my tent!") and/or to recognize the quality efforts of a reviewer, even if you're not interested in that product personally.

Usually, you do not need to be personally interested in a specific piece of gear to recognize if a review is well done and helpful or not. So, if you think someone did a great job on the review, whether the review is positive or negative, feel free to give them an upvote.

Plus, when you vote certain reviews up, it gives them a higher visibility and priority on that product page. So, other readers who come along to read the reviews find the best info up front.

As for comments, here are some of the things I think they're most useful for:

  • Constructive comments, which may be recognizing and acknowledging a helpful review and info ("thanks for the great review and pictures of the tent setup")
  • Questions about the product, which may be of personal interest ("I like to winter camp. How did your stove do in the cold?") or of a broader nature ("I've read in several reviews that this tent has really poor ventilation. Did you experience that?")

If you had a different experience and/or disagree with the reviewer, generally the best thing to do is to just go write your own review and let it stand on its own. More reviews (and pictures) are almost always more helpful than fewer reviews.

You can civilly bring up differences of opinion in a comment and acknowledge your own contradictory experience, as long as it's in a civil way ("Thanks for the review. Unfortunately, my boots failed on their second hike. How are yours faring?") as long as it's not an attack, but a valid question or constructive comment.

If you've got more to say about the product from firsthand experience, then writing a review is the most helpful action.

Thanks to everyone who shares such helpful reviews and info, and to everyone who recognizes those reviews and reviewers.

9:34 a.m. on May 10, 2016 (EDT)
254 reviewer rep
185 forum posts

I think it is some of those factors, but also a cultural grade inflation.  5 has become "no problems" when it should be "much better than average." Otherwise how can we tell the good, works as it should, gear from the awesome, changed my life, gear?  

I bought something on ebay recently and the seller said if you don't leave 5 stars, then don't leave anything at all because anything less than 5 stars is negative.   I would prefer to leave 5 stars for "above and beyond."  When did "above average" or "pretty good" become negative?

11:10 a.m. on May 10, 2016 (EDT)
REVIEW CORPS
4,534 reviewer rep
6,027 forum posts

tracker clayton 2 said:

Bill S .....I agree a call or email to a manufacturer will usually always get you good info and constructive criticism.

 Tracker, since you are not on the Trailspace Gear Review Corps, you are not aware that those of us who are TSGRC are not allowed to personally contact the manufacturers directly. If it is necessary to have something explained, a replacement, and so on, Seth and Alicia are the go-between between reviewer and supplier. The major reason is to minimize questions of bias in the review process. Another is that the occasional rare manufacturer can get pretty nasty (I have had that experience myself, though Seth was the contact point -- still, the manufacturer sent me PMs and got pretty aggressive with Seth).

There have been cases where the product failed, was the wrong size, didn't get shipped to the correct address, etc, so Seth arranged a replacement.

When the review is completed, some companies allow the reviewer to keep the reviewed item. Some require the item to be returned because the company wants to see the amount of wear, or when the item is a prototype.

As you might infer, all of us in the TSGRC are very grateful to Seth and Alicia for handling logistics, potential disputes and other problems.

The manufacturers and distributors do read the reviews, and they do take the pros and cons into consideration for future improvements in their products.

For private reviewers or any customers, direct contact with the company is appropriate. However, I will note that sometimes someone has posted a really nasty and aggressive tirade about a product in the forums. But the old adage about "you can catch more flies with honey than with vinegar" applies. If you just out and out attack the "stupid, idiotic" (plus other non-quotable verbiage), you won't get much help. Truth to be told, the "attack" kinds of reviews (the ones I have seen came from non-TSGRC people) are almost always "driver error".

2:13 p.m. on May 10, 2016 (EDT)
TOP 10 REVIEWER REVIEW CORPS
3,158 reviewer rep
2,248 forum posts

DrPhun said:

I think it is some of those factors, but also a cultural grade inflation.  5 has become "no problems" when it should be "much better than average." Otherwise how can we tell the good, works as it should, gear from the awesome, changed my life, gear?  

I bought something on ebay recently and the seller said if you don't leave 5 stars, then don't leave anything at all because anything less than 5 stars is negative.   I would prefer to leave 5 stars for "above and beyond."  When did "above average" or "pretty good" become negative?

 DrPhun,

This makes me think about something I've observed professionally (I work in IT) over the last decade or so; if someone simply does their job they really stand out.

I get praise at times and think "well, ok, I like getting recognition and positive feedback for sure, but I'm just doing my job and nothing extra".  It's somewhat of a sad commentary but it seems so few people do what they are paid to do these days.

9:01 p.m. on May 10, 2016 (EDT)
158 reviewer rep
391 forum posts

Bill S.....Your right I did not know those in TSGRC could not directly contact a company atleast you have Seth and Alicia. At times I wish I had someone else to deal with a company or maker!  I deal with alot of knife and tomahawk companies and some no matter how nice I am are on the attack from the get go, where others will spend hours talking with me about their products. I flat out won't buy from a company or individual who is let's say less than nice no matter how much I may like their products. 

I really can't imagine going and testing and dealing with all the different gear yall get and not being able to jump on the phone myself...lol. I also can't imagine changing my gear up alot I tend to stick with gear I have till its falling apart. I commend yall for the work you do to bring us a review and leave a trusted piece of gear at home.

8:50 a.m. on May 11, 2016 (EDT)
254 reviewer rep
185 forum posts

I see that too.  I find that I now stand out at work just by showing up when I am supposed to i.e. meetings, scheduling time off in advance, answering the phone, etc (and I am in a professional job.)

5:17 p.m. on May 11, 2016 (EDT)
TRAILSPACE STAFF
1,684 reviewer rep
4,127 forum posts

Bill S said:

As you might infer, all of us in the TSGRC are very grateful to Seth and Alicia for handling logistics, potential disputes and other problems.

One clarification for everyone: while Seth has contributed in many important ways to Trailspace over the years, including in the past as our coordinator for Review Corps, since last year he has worked exclusively on the advertising side at Trailspace as our Media Sales Manager.

Since we keep editorial (which includes Review Corps) completely separate from advertising, Seth is no longer in contact with brand reps in any editorial capacity or about Review Corps reviews.

I hope that made sense.

2:20 p.m. on May 14, 2016 (EDT)
TOP 25 REVIEWER REVIEW CORPS
3,569 reviewer rep
1,335 forum posts

I started this thread but got too busy to keep up.  Great input from everyone on the ratings and different viewpoints.  I always try to upvote a really good review as well, even if it is not a piece of gear I would buy or be interested in. 

Patman - I hear you on the work expectations.  Always surprised when I feel I am just doing what I am paid to do but others say it is not the norm...hope that trend changes in the future although I will admit I am not the best at the work-life balance thing...tend to swing wildly from one to the other!

10:44 p.m. on May 14, 2016 (EDT)
TOP 25 REVIEWER
1,204 reviewer rep
393 forum posts

I think 5 stars is a fair rating for something that does everything it's advertised to do. A pair of boots that claim to be waterproof can't go above & beyond by keeping your feet drier than dry, after all.

Subjective criteria, such as the ease of pitching or striking a tent or how warm a sleeping bag is, are going to be different for everyone, but in the case of the sleeping bag a sleeper who's warm at the rated temp should give 5 stars for that criterion, at least.

As far as what a piece of gear would have to do to earn 1 or even 2 stars, I can't say, because I can't remember buying anything that crappy. Or, perhaps the conditions in which I used it didn't let the flaws show themselves. 

4:59 p.m. on May 16, 2016 (EDT)
609 reviewer rep
28 forum posts

I unfortunately was not able to read every answer so I could be repeating some. But personally when it comes to my star rating... i base that solely upon the question.. "does it do what it says it will do". If it does so in ever aspect I give it five stars. Even if it does not work for me personally, i feel that if it is as described and does everything it says that it will do it deserves five stars. As i do not feel that my personal preferences should play into the integrity of a piece of gear. I normally will discuss the cons that I have with it personally, but that does not mean it did not work as the manufacturer said it would. Happy Trails 

2:13 p.m. on August 12, 2016 (EDT)
INCENTIVIZED REVIEWER
2,539 reviewer rep
17 forum posts

I find this discussion fascinating, as a gear reviewer, I always give the star rating on how I feel about the item being reviewed, since to me, no matter what the manufacturer says an item will do it's up to the person using it to decide whether it fills the requirements he or she has. You have all given me things to think about, though.

6:05 a.m. on August 14, 2016 (EDT)
8 reviewer rep
23 forum posts

I just did a quick and dirty review of my Snugpak blanket and it did all is said and more so it got 5 stars. Someone else had reviewed it before me and I used their review when it came to buying this item. I love the good reviews they help me spend my money wisely.

8:24 p.m. on August 16, 2016 (EDT)
TOP 25 REVIEWER REVIEW CORPS
3,569 reviewer rep
1,335 forum posts

Finally getting back to some overdue reviews after a busy summer and revisited this thread.  I find now when I look through other reviews and these responses, that we have a variety of approaches all with their merits.  I look at a reviewer's overall rating scale to judge the rating on a certain product.  Just because someone gives a 5 and another a 4 doesn't mean their opinion is much different on the piece of gear...just their rating scale.  Either way works for me as long as you have enough reviews to make the trend apparent.

Happy reviewing everyone!

May 23, 2019
Quick Reply

Please sign in to reply

 
More Topics
This forum: Older: Similar item review question Newer: Review of Discontinued Gear
All forums: Older: Backpacker Magazine Renewal Newer: what is your favorite duffel bag?