META - Gear reviews

7:57 a.m. on March 2, 2011 (EST)
120 reviewer rep
137 forum posts

Just by scanning the most recent gear reviews on the front page each day, it is obvious that a whole lot of 5-stars get handed out.  I was just curious as to how people rated their gear – for instance, if you do rate your stuff, how much use does it get first?  I am probably guilty of one over-zealous gear review on this site, and perhaps I was a little unfair in another review.  

I know that it's not wise to base your opinions solely off of gear that hasn't personally been tested, and that it is even more foolish to base an opinion off of one data point (and a subjective one, at that).  I just find that a lot of times, by the time I am comfortable rating a piece of gear, it is no longer being manufactured to those specifications (i.e., next year's model, etc.).  For instance, I would love to review my Lowa Scarab II's, but they aren't made any more, so what's the point?

I am really excited about the new "Trailspace approved" gear test reviews.

8:27 a.m. on March 2, 2011 (EST)
TRAILSPACE STAFF
234 reviewer rep
951 forum posts

The average rating on Trailspace is 4.0 stars, and the median rating is 4.5 stars.

(I heard recently that the average rating on review sites is typically about 4.3 -- apparently Trailspace users give slightly less glowing reviews than do people elsewhere.)

I think the root of the phenomenon is that people are most motivated to write about gear they love (or the rare piece that's failed spectacularly). Maybe your Scarab IIs (go for it!) are a case in point?

 

11:23 a.m. on March 2, 2011 (EST)
171 reviewer rep
223 forum posts

Interesting point. I think one of the reason the gear get so good review is the fact that we buy gear on wich we've researched on a lot. In that case buying gear that works for us is simplified and we dont buy stufff that doesn't work. I've worked in the outdoor retail industry as a shop manager and saw a lot of what is done, talked to a lot of reps, compagny endesigner and so forth. It has given me a better view of what I like and need in most case. For example, I love Marmot gear and most othe stuff I use works great, but I bought it after trying out a lot of stuff that didn't before. Now I know and rarely buy stuff that wont work for me, hence the good rates.

11:46 a.m. on March 2, 2011 (EST)
120 reviewer rep
137 forum posts

Agreed on both points.  I guess what motivated this post was that I am going to make an effort to review all of my gear, and not just the stuff that falls on either end of the spectrum.  It can be frustrating when looking for new gear, and everything you research is a 4.5 or 5 star item.  What is even more frustrating is when you don't have an REI anywhere close to you (or other similar retailer) where you can give your potential purchase a test run before committing.

12:44 p.m. on March 2, 2011 (EST)
TRAILSPACE STAFF
715 reviewer rep
3,166 forum posts

Based on having read every one of the 21,000 reviews on the site, at least once, I agree with most of the points above.

At least initially, people tend to review a piece of gear they love and are excited about or a piece of gear that failed them in some way and they hate.

OK, not everyone is quite that extreme, but reviewers tend to first review gear at one end of the spectrum or the other: "good" gear or "bad" gear, with a leaning toward reviewing the good stuff.

I think it's a natural inclination to want to highlight the stuff you have positive assocations with.

After those initial reviews, some more dedicated members and reviewers get more broad and systematic and start to review all (or at least more) of their gear, including stuff that falls in the middle.

Sometimes we overlook the steady gear that just keeps doing its job uneventfully without notice, but that gear deserves a review too. It also adds depth to the content.

And there's no harm in reviewing a piece of discontinued gear. It can still be interesting for some readers and gives a profile of the brand's quality.

I say, if you've used the gear well then review it, whether it's new or old, or a 1, 3, or 5 star product.

11:25 p.m. on March 2, 2011 (EST)
102 reviewer rep
2,295 forum posts

Also since we own the gear we review, we are disinclined to test it in a manner that may damage it.  For example how many "accidentally" drop their stoves or spill gooey stroganoff on them just to see if it can muster the abuse?  Or how many of us intentionally pitch tents in progressively stronger winds until they eventually shred or the poles snap?  How many actually use something other than their finger to gage the force of said wind?  Thus most stellar ratings are actually unqualified, as the equipment was never subjected to situations that truly established their limitations. 

I will be testing the quality of several brands of whisky on my next jaunt.  I will employ an extensive and exhaustive test regimen, and promise to test these products to their limit and total demise.  Now if I can convince Trailspace to set up a beverage category in their gear review section...

Cheers,   Ed 

4:22 a.m. on March 3, 2011 (EST)
52 reviewer rep
312 forum posts

Ed, we are currently testing some Ledaig single malt and initial results are encouraging.

I agree with the OP: by the time I think I know how good some gear really is, they no longer make it (though having so much stuff doesn't help). I therefore try and do early reviews of things which, to my mind, throw up a red flag. I believe the ability to edit older reviews is in the pipeline?

You Americans should have a kind of gear exchange. For instance, when a new bit of gear comes out, it could be posted to various users at the appropriate time and be put through the wringer within a few months. The postage would be shared and possibly the initial price but it would be worth it if you intended purchasing it. This would be like the useful multi-reviews that Trailspace has been doing, but for everything else.

8:30 a.m. on March 3, 2011 (EST)
102 reviewer rep
2,295 forum posts

May I suggest, if not already considered:

Testing it at the summit of Mt Blanc, and any huts you pass through en route.  Testing its performance on a scenic overview of Lake District.  (Has to function well at both high and low altitude).  Compare affects of enjoying fireside at home as well as fireside at camp and on the beach.  Test with chocolate.  And if there is still enough left over, test comparing affect at different event - Birthdays, holidays, and back yard BBQ are some that come to mind.  Heck if you need another bottle to complete the entire circut make all necessary efforts, provided extending the test is warranted.

Be sure to use proper safety equipment and restraint (wouldn't want to cloud your judgment and render a flawed assessment or hurt somebody).  We await your report. :)

Ed

10:10 a.m. on March 3, 2011 (EST)
120 reviewer rep
137 forum posts

and promise to test these products to their limit and total demise.

Cheers,   Ed 

Wait.  Testing until the beverage's demise or your own?  Both?  :)

12:05 p.m. on March 3, 2011 (EST)
344 reviewer rep
1,124 forum posts

We do need a reassessment of some of our gear. I went out for 3 days this week, in a steady light rain. A tent of mine failed. Eventhough it had gone though driving rain and heavy winds. It seems that the seams got soaked and started to leak this time. I was very suprised, and my down bag got wet. I had to bug out.

4:12 p.m. on March 3, 2011 (EST)
102 reviewer rep
2,295 forum posts

and promise to test these products to their limit and total demise.

Cheers,   Ed 

Wait.  Testing until the beverage's demise or your own?  Both?  :)

 I love my drink but have no fondness for being drunk let alone hung over.  Thus drink till it's gone, even it that takes months.  But if you are in a rush to recycle that glass bottle, do as you wish, just keep away from the steeps, sharps, machines, and vehicles.  I should probably include camp fires too; a couple of my buddies who do like to get their drunk on have fallen into the flames - one did it twice.  Amazingly and as usual, no harm occurred, not even a scorched eye brow.

Ed

4:51 p.m. on March 3, 2011 (EST)
29 reviewer rep
29 forum posts

In my humble opinion, I think there are 2 primarry factors that lead to high ratings:

1. Sites using public reviews are inherently flawed as the majority of the equipment reviewed comes from the type of folks who are drawn to reviews and to the process. Those of us on this site are inherently concerned with getting the best products we can and bring this circumspect approach to our purchases, most likely comparing different brands and features on the majority of the gear we purcahse. Because of this, as a group, we are less likely to purchase and use (and ultimately review) crap. This self selection adds a bias towards quality products.

2. Most gear is more than capable of providing what the majority of us require of it . Case in point: I love my new Snow Peak Giga, but in truth, it has yet to out-perform an old Gaz stove I had 20 years ago and that never failed me on trips up Whitney, mornings where it was left outside in below freezing temps and started right up, etc. The flaw here is that my personal use is so far within the tolerances and capabilities of my gear that my reviews can't help but be positive. There are obviously folks here to push gear much further toward the limits, but I've also read plenty of "I-just-opened-the-box" reviews, so I figure I'm not alone in my lack of torture testing.

7:02 p.m. on March 3, 2011 (EST)
TRAILSPACE STAFF
715 reviewer rep
3,166 forum posts

I agree with the OP: by the time I think I know how good some gear really is, they no longer make it (though having so much stuff doesn't help). I therefore try and do early reviews of things which, to my mind, throw up a red flag. I believe the ability to edit older reviews is in the pipeline?

Yes, it is. In the meantime, I highly encourage members to write a second review to update matters. Don't feel like you need to wait till there's a slick way to update and edit your reviews to write a second update. Better to share the info than not.

 

CapitolaDan, I think you raise some good points above.

I'd add that someone like yourself who has taken the time to think about their gear and reviews in such a way is exactly the type who should write more reviews!

OK, I know you've already done several, for which we thank you. I just wanted to point out that having realistic expectations is essential to writing useful reviews.

1:51 p.m. on March 4, 2011 (EST)
29 reviewer rep
29 forum posts

Quick clarification to avoid having my earlier comment misconstrued: I believe that the reviews here have VERY high value in allowing myself and others to gain from others experience. I personally find them enjoyable to read and have benefited from hearing how folks with uses similar to my own. I've also used them in my own buying recent buying decisions and am very happy with each purchase.

My mention of a "flaw" pertains only to using an average of all equipment ratings here to indicate the state of all backpacking/camping gear, etc..  It's an academic point, but I make my living in marketing/research, so forgive my diversion into the weeds.

3:43 p.m. on March 4, 2011 (EST)
72 reviewer rep
311 forum posts

This is a great post.I too have some very old and no longer made gear that still works very well.These include tents,stoves,sleeping bags,clothing etc.No need to replace if they still work and only weigh a few grams more.On the other hand some of the new fabrics,designs etc are much better so I have replaced some gear and have been very happy with my purchase's I have made.Bottom line to me is that no gear can be fully tested by most untill many years and trips have passed.Both good and very bad weather conditions are what allow us to make the reviews honest and informative.For my self I will try to update the reviews I have made on this web site inorder to give more in depth info to those shopping for new Items.Thanks to all of you for makeing this a great forum!

 

2:53 p.m. on March 5, 2011 (EST)
REVIEW CORPS
1,407 reviewer rep
442 forum posts

I agree Skimanjohn...great post!

Reviewing discontinued gear is vital to new model purchase decisions for me, as highlights to good design and execution in a certain gear category can help me to formulate need vs want decisions and evaluate certain design features in the new product. Alicia's comment that she's read every review (wish I had the time to do the same) re-inforces a wise piece of advice from my favorite teacher in high school: "READ, READ VORACIOUSLY!"

The other value to discontinued gear reviews, no matter the age of the product, is that it can provide you with data for consideration in purchasing from the fine universe of used gear. Quality gear is designed to exceed the outdoor experiences of most users. For me, I almost always purchase my gear at a heavy discount. It is typically last year's model (or sometimes 2-3 years past) and is new old-stock. I read every review I can find before purchasing, and this community's extensive library, combined with the wisdom of its members, really helps me make the right decision. I try to pay it forward by reviewing every outdoor related gear/apparel I own.

In honor or Dr Claire Tremain (my Humanities Teacher in H.S.) and Trailspace, I say "REVIEW, REVIEW VORACIOUSLY!"

12:00 p.m. on March 6, 2011 (EST)
75 reviewer rep
306 forum posts

As for me personally, I give a lot of 4 and up star ratings to the gear I have tested mostly due to the face that I do a lot of research on a type of item before I go and buy any specific brand.  I read as many reviews about an item as I can and filter out the non-helpful ones.  Therefore, I know I am getting a pretty good quality product.  Then for my review, I just write up how I have used something and what are the pros, cons and anything else that I have experienced with it.

I like the idea of reviewing items no longer made...they are still out there (ie eBay, Craigslist, and our very own Trailspace classifieds...)

12:21 a.m. on March 8, 2011 (EST)
REVIEW CORPS
1,347 reviewer rep
1,320 forum posts

Hmmm, I see my average review score is 3.875.  I think I'd write more reviews, except I like to use it enough to really feel qualified to write about it, before doing so ... and by then it tends to be "old news", so I don't bother.  For example, last summer I got some new gear, but have only used it a few times so far, and only in good weather ...  and all of it worked well, so my scores so far would be very high.  But if I used the same gear in heavy rain or cold weather, it might be a different story (or might not)...

Then again, commercial reviewers (e.g. writers for commercial publications) don't use the gear for years before writing about it.

Maybe I should just start writing about my "initial impressions", hmmm...

Anyway, I'm with what others have said ... I research my purchases to death before laying down my money, so I tend to have a pretty good idea before making the purchase that I'll like the item.  It doesn't always turn out that way, but tends to be biased in that direction...

 

 

 

7:13 a.m. on March 8, 2011 (EST)
120 reviewer rep
137 forum posts

bheiser,

That was exactly my point.  A failure in a piece of gear only requires it to fail once, in which case awarding 1 or 2 stars to a product is warranted.  If your pack straps break in the first 20 miles, you probably won't replace the straps only to "give it a second chance" prior to review.  However, in order for a product to be 4.5-5 star worthy, it needs a fair amount of abuse, imo.  I get out whenever I can, but I don't bushwhack across Alaska (at least not yet), so I feel like it will take me longer to review gear that deserves higher marks.  And it is this gear that deserves to be rated soonest!

11:41 a.m. on March 8, 2011 (EST)
171 reviewer rep
223 forum posts

I think that gear can be seperated in two categories for a review. Soft good and hard goods.

Let me put it this way. For me technical clothing/bags/tents/sleeping bags is purchased in the intentions of abusing it in adventure conditions, for example when I climb. The gear I buy almost always fails/dies due to the abuse I put it through. Hole will appear because of embers, stiches will brake because of abrasions etc etc... In that regard it would be foolish of me to expect it to last more than what 2 to 3 years?? I wear out at least three pairs of ice climbing gloves per seasons And from what I hear it aint that much!

Then there's hard goods. Like skis, axes, ropes, biners, MSR XGKs (that dont want to die by the way...), knifes, and anything else that not made of whatever kind of tissue. Those things, for me are purchased in the hope of never having to replace them unless for it's something better, and then some. If I can I'll keep em a life time all the better. (execpt for climbing gear, you know...). That little stove is on it's 12 year and still kicking.

For me how soft goods performs is way more important than there longevity. Of course going on a walk in the park wont tell you effective a climbing jacket is, but three days of brittle ice on a high wall will. What is failure is dependant on everybdoy perveived vision of it. When we review, stating our vision might be good?

We have to be realistic on what we ask of our products.  And remember that performance and longevity dont always have to go together in an unseperable way. Of course we'd want to, it just cant always happen. It's like asking for the most ultra light mountain bike made out of 100% carbon fiber. There is reasons why that doesn't exist.

2:57 p.m. on March 9, 2011 (EST)
102 reviewer rep
2,295 forum posts

..We have to be realistic on what we ask of our products.  And remember that performance and longevity dont always have to go together in an unseperable way. Of course we'd want to, it just cant always happen...

So Louis, you bring up a point that got me thinking. 

I wonder if Johnny Black is more suited for the slopes than Johnny Red?  Is blended Scotch better suited for the summit, while bourbon outperforms back at camp?  How long can I expect a pint to last under severe conditions?  Is it best to bring along a three legged stool, a four legged chair, or hammock when using above said equipment?  How 'bout a safety helmet?  What is a good brand of auxillary air sickness bags to bring when using spirits at elevation?


monkey.jpg
Self portrait with new monkeycam.  Yep, it makes me look just like one.
In-depth review to follow.
Don't ask and please don't tell!

Ed

11:33 p.m. on March 13, 2011 (EDT)
38 reviewer rep
395 forum posts

Three legged stool  - 3.5 stars   lite, but I find them a bit tippy

Four legged chair  - 2.0 stars   comfy, but way to heavy

Sitting on the ground  - 5.0  stars   no extra weight in pack, no where to fall too, and very comfy when the bourbon has acheived a 6.0 star rating

10:45 p.m. on March 14, 2011 (EDT)
102 reviewer rep
2,295 forum posts

Three legged stool  - 3.5 stars   lite, but I find them a bit tippy

Four legged chair  - 2.0 stars   comfy, but way to heavy

Sitting on the ground  - 5.0  stars   no extra weight in pack, no where to fall too, and very comfy when the bourbon has acheived a 6.0 star rating

Everything seems comfortable when bourbon reaches 6.0, including sleeping in hammocks dangling from cliffs on Big Wall climbs.  Personally I don't drink enough to reach that comfort zone, but will suffer the additional weight necessary for a good night's sleep. :)
Sleeping-on-hanging-stance.jpg

Field testing the Cloudnine Bivouac platform.  Comes with optional rain fly and masseuse.   Get both options – you’ll need them, especially after hauling this hanging platform up ten pitches.  

Ed

December 22, 2014
Quick Reply

Please sign in to reply

 
More Topics
This forum: Older: Mountain Trail in Spain. Let's hike! Newer: Adventure camping protection
All forums: Older: Preceived value Newer: Happy 10th Birthday, Trailspace!