Testing Sleeping Bags

OK, I have a virtually non-existent backpacking gear company. Amongst the existing backpacking gear that I was dissatisfied with, was sleeping bags. So I talk to the wife, and we do all this stuff and start sewing quilts and filling my head with all the world's knowledge about sleeping bags and whatnot and I realized that I don't have a way to test my gear.

I mean, I can get someone to get inside for the night, and I can get thermometers with the 'low point' setting and so forth, but I really, really want to know stuff. Detailed stuff.

The EN 13537 is out of the question, as I don't have that kind of money. Further, I just don't believe in the efficacy of the test. I know, I'm a dude - who at this point doesn't even have a spare bedroom to sew in, so what do I know. But, it gnaws at me, pesters me. I hate you, EN 13537.

Resentful, I about trying to learn how everyone else is testing their sleeping bags and quilts and so forth. I don't get anywhere. So, I start trying to understand how a person keeps warm, what makes a person warm, how humans regulate their temperatures and whatnot. I think that I learn that we are wet beings, loosing heat through many means that you and I already know about. But, a few things do stick with me. Namely that people are sloppy. Our skin

I learn that I have to measure humidity and temperature on a sleeping person inside of the sleeping bag. But, I also have to measure temperature and humidity just outside of the sleeping bag, to see the difference. From these readings, if they are tracked over time, I should be able to track a bag's performance, right?

That is what I am trying to determine. I have purchased a batch of humidity and temperature logging sensors and have been testing them. I set them to take  a reading every five minutes. Here is an example of a short test:

I don't know if this gets bigger, but you can click on the picture at the website.

I tried to link to it, but I'm not supposed to.

Here is my thinking in a nutshell:

1) before crashing for the night while backpacking, place one sensor inside of your sleeping bag and one outside. Hit the time stamp button on your sensor to make a time stamp in it's recording, indicating sleepy time.

2) go to sleep. If you wake up to change clothing, are too cold are too warm, to take or pee or whatever, press the 'time stamp' button.

3) Wake up, press the time stamp button again, write down what you wore and other related nonsense on a notepad.

4) get this info, jam it into Excel and poo out a graph like the one above.

I think that if I can do this enough times, I will begin to be able to establish operational parameters of sleeping gear. Maybe 100 tests to begin with?

If I gather other data at the same time, I should be able to shed light on shelter qualities, maybe even the effects of sleeping pads, vapor barriers and whatnot.

Gathering age, gender and body type information may be able to discern what insulation levels various groups are happiest with.

After that, I'd like to build an online database to show and share all the collected statistical (?) information.

Here is where I am at: I've run a dozen or so tests in quilts. But, then I realized that I needed to contain the sensors in pockets in the quilts or bag for more reliable data. So, the data looks promising as a model, but not great.

I've also been figuring out how to process the data in Excel and how to spit out usable graphs. But, not very well.

I also wrote a provisional patent and my little brother is building a database program to manage the data. I put up a plan on my website, hoping to recruit people to test their existing gear. It's under "Ardeth Sleep System Protocol".

I didn't want to do any of this, I just want a way to test my gear, but I'm very happy if anyone can test their own gear. Then, we can all compare our stuff and say stuff like, "well, the data clearly indicates that PeePeeLoft's CLO is reduced by 9% after 50 nights in the backcountry." Or, "couples' arousal rates clearly jumped by 23% when wearing vapor barrier socks."

I don't want to spam, but I don't know how to ask a bunch of people all at the same time. So, there it is. Please tell me what you think.

It sounds like a reasonably sound way to test. It would obviously take testing to find out if your testing was a valid test! But it sounds like your on the right track. Other than testing like you mention, the other way is in real world tests.

Make up a quilt/bag whatever, and do some preliminary testing on it yourself, and then send it out to someone to test in the field for you under real world conditions. Having a seperate 3rd party conduct a test for you helps to reduce the bias that could theoretically be used by yourself when posting specs on your bags/quilts. Because that person would post a review that was out of your control as to what was in the content. For example, it could be posted on trailspace.

From what i understand, below 70F you need about 1 inch of loft for every 15 degrees. (this is down, not sure about synthetic) , so for example my 0F quilts from another cottage industry maker has 5inches of loft. I find it to be very accurate, and is probably more realistically a -5 or -10F rating. But at 0F i am more than comfortable with only a baselayer.

Best of luck, i am sure its challenging to really get the business off the ground making sleeping bags etc.

I am actually trying to open this up to anyone who wants to test their own gear. Do you think that overly ambitious?

I just don't see your average joe shelling out money to test gear he already owns, if thats what your implying. That market would surely die before it ever got started IMO.

There isn't a charge for this. I actually expect that only people that are really into backpacking or other, similar pursuits would be interested in doing this. A very small group. But, if I need a system like this to test my gear, then other cottage-sized businesses would see it's usefulness. I'm thinking that consumers would want this information, once enough is gathered, to make informed decisions.

Who will pay? Maybe nobody needs to pay. Maybe other manufacturers are interested enough to spend $1000 for the hardware and support to start their own program. At this point, I've spent about I haven't spent that much to getting this up and running. I've not nailed it down quite yet, but as I can get the data processed, I am starting to see it's utility for me as a builder. Do you think people would do it for free? Would you? Would you do it in order to test free gear?

I am not 100% clear on what it is your asking really. Would people do what for free? Would they allow you to send them sensors to use in their bags to test them?(that just doesn't make sense to me, especially from a business stand point)

I do test things from time to time. If part of a test was to use said item with some special sensor as part of the test, then i would have no issue doing so. But it would not influence a review any, it would just be information to be sent back to the manufacturer.

IMO you need to do some objective testing with this system to make sure its even worth your while to even pursue this. It's one of those things that may sound great, and look great on paper, but in actuallity may not be. For example: How do you know what the temperature should be at inside the bag? What are you comparing it to? things like that. Just because you have a sensor in a bag and its giving you some numbers, that doesn't neccesarily mean anything. The EN testing uses a heated manican that is maintained at a certain temperature, that is how they can compare the numbers. You would need to monitor the temp outside the bag, the temp inside the bag, and your skin and core temperatures to really get any objective information i would think.

If you want to send me a bag you think is rated to -5 or colder, I would be happy to take it to Mt Everest with me on my base camp trek in March and test it. I wouldn't charge you a dime!

Sorry giftogab, I'm wrapped up with trying to push out a batch of 20°F bags, and they ain't coming out too pretty. But, I'd love to have you do a test run on your gear, if you're still interested.

I'm sorry, Rambler, I'm not a very good writer. I really have got to push my way through writing just about anything. While my post above and my website info both attempt to clarify how we're doing the tests, I guess I'm not writing it very well. But, I appreciate the opportunity to elaborate!

Questions:

"Would people do what for free? Would they allow you to send them sensors to use in their bags to test them?"

Yes, I am wondering that. Would you, based upon the information on my website?

Assuming that they protocol pans out, from a business standpoint, I get, 1) inexpensive data on the performance of the gear I produce, 2) data on competitor's gear, 3) publicity from a system that my company has brought into being, 4) the ability for a potential customer to interface with something that my company has produced, for free.

Further, from a personal standpoint, I hope that I may be bringing something of value to the backpacking community. I've witness many conversations about this insulation being better than that, about the effects of time on sleeping bags and others. I think we'll actually be able to monitor and measure these things! I sorta think that's cool.

"How do you know what the temperature should be at inside the bag? What are you comparing it to?"

I don't want to tell anyone what the temperature should be inside the bag. I figure the tester knows that better than anyone and we should just let them tell us. For instance, if a tester is in a bag, with only silkweight baselayers on, is comfortable down to an external temperature down to 20°F, but at 15°F needs to put on socks, we can begin to define the bag's lower limits! When tester wakes up to put on socks, they will push the time stamp button on the inside sensor. With that time stamp, we'll be able to see the temperature inside the bag and outside - and the temperatures leading up to that time stamp.

"You would need to monitor the temp outside the bag, the temp inside the bag, and your skin and core temperatures to really get any objective information i would think."

I think mostly just the temps inside and outside the bag. Also, humidity levels and other information such as clothing worn, et. al. We also hope to gather a good deal of peripheral information. I don't pretend to have everything figured out, but this conversation is helping me to think all this stuff out, and I appreciate it! I wish I could convey this information a little better.

Ok, that makes a little more sense. It sounds potentially like a very innovative design. I would be up to giving it a whack i think, more out of curiosity than anything else really.

Hey Jason, I just checked out the website and think it's a neat idea. At the very least it might help your company gain some publicity. My only question, and where I kinda get lost on the idea, is that you would end up getting 20 different makes/models of sleeping bags from 20 different people, how does that help you further your business? Or are you trying to see if this new style of testing is a business in itself? If I were you I'd get some of your bags out to people in different locales to get some constructive criticism back. Just my two cents. Please keep us up to date, there are many people here, who I believe, would love to support a smaller homegrown business. 

TheRambler: Curosity is just fine. I'd love to have you take out the sensors and give me a critique.

Jake W: Trust me; I want to be making sleeping bag and my other gear, not doing this testing program. But, here is why I am doing this now:

1) I don't have any other way of testing my bags and getting objective data. Subjective information is easily obtained.

2) If it's successful, it'll lend credibility to my company and publicity.

3) Testing my gear is great, but they need to be put in context. That context will be supplied by curious individuals like yourself that is will use this protocol on their own gear.

4) It's not just about business, it's also about backpacking. I want to know how to better travel fast and light, and part of this means having information about the insulating values of my sleep system. How do all of the components of my sleep system help keep me warm?

5) Call me crazy, but I sorta see the EN 13537 standard as a test method for the exclusive use of the powerful, wealthy brands. For $3000 a pop, what cottage company has access? Not me. What backpacker can go out and submit a bag that they just bought to check the manufacturer's claims? A $200 REI bag now costs you $3200 and you still have questionable data. With this, you can buy that $200 bag and test it for free. You will have all the subjective information from using it, but you will also have some objective data.

To quote a most horrible movie, Point Break, "You know nothing. In fact, you know less than nothing. If you knew that you knew nothing, then that would be something, but you don't."

Pretty much, I at least, know that I know nothing, so that must be something, right?

Try it out. It's free. Let me see if I can produce something of value to you and to the community. All I ask is that you take them out for a trip that is more than a couple of days. If you're with someone else, even better. I still have to pay for shipping and processing the data manually takes some time. I don't have much money, in fact, I've been unemployed for about a year and there is no work on the horizon. So, this is my time to make it happen.

If your serious, and the sensors dont take up much space, and you already have a rough process of how i should use them, i would be willing to do this for you. I am leaving on a 7 day trip this friday feb 3rd around noon. Not sure if you'd want to fedex 2 day them or something. But if you wanted to, you could.

Temps for this trip at the moment based on the current forecast look to be highs in the 30-40F range and the lows look to be 15-25F, maybe a few degrees lower at elevation.

My pack is fairly full, but if they are somewhat small i am sure i could squeeze them in. Especially if i don't need a bunch of extra batteries etc for them.

You can send me a PM if you want to pursuen further.

NOTE: I am a hammock camper, and use a underquilt and topquilt, and not a traditional sleeping bag.

What kind of sensors are you using?  Brand, make, model, etc.  What are you using to capture the data?  Have you ever seen these before? http://www.maxim-ic.com/products/ibutton/ibuttons/thermochron.cfm 

TheRambler: I will PM you.

ocalacomputerguy: Those are very nice. I've seen some of the smaller sensors like these, but I pretty much purchased the least expensive sensors that fit my minimum requirements. If this protocol takes off, and I can invest more, I'd like to get better sensors.

Currently, we have been using the LogTag HAXO-8 dataloggers http://www.logtagrecorders.com/products/haxo.htm. They seem to hit the mark with the sensing range, verified weight of 1.2 oz. and reasonable size. They can be reasonably cleaned with rubbing alcohol and have a simple interface. Unlike some other data loggers, these have a button to leave a time stamp, which I have come to believe is required for this type of testing. We've taken them out on numerous 2 and 3 day trips, along with 7, 9, and 14 day trips. But, it was only recently that I noticed that we need to be able to attach pockets to the sleep envelope (bag, quilt or blanket) to standardize sensor placement inside of the sleeping envelope.

The sensors have a USB interface via cradle and LogTag has software for the to access the data. I set the software to automatically save to a csv format and I then import into Excel for manual processing.

testing needs to be in a controlled environment,tightly regulated and documented(a freezer)or you can dismiss.the whole problem with setting measurement parameters is control and verification of variances.tolerances need to be specified.good luck,maybe you'll have more success than the european standard ENBLAHBLAH.P.S.if its not adopted universally its useless.

I am not sure the objectivity you seek using sensors is all it is cracked up to be.

Instruments may form a consensus among themselves, regarding the performance of a product, but in the case of something like a sleeping bag I think such testing has limited value, definitely not value sufficient to expend a lot of effort or money in this endeavor.  First off, even if you can come up with the “perfect” rating system, the typical consumer rarely purchases more than a couple of sleeping bags in their entire life, thus does not have a good basis from which to handicap his personal subjective experiences in sleeping bags against some standardized “objective” evaluation.  The fact we all have different tolerances to cold is further complicated in that we all have different ways these differences express themselves.  Some folks have cold feet; for others it is their head.  How can I tell if a bag addresses my preferences?  Furthermore even our personal criteria varies a fair amount, depending on daily psychological, biological and dietary variances.  Lastly we use our bags differntly, placing them on differnt pads, and sometimes placing other things in the bag besides our person.  If you actually came up with a system that addersses all these criteria, chances are it will confuse more than elucidate.  

I understand your concerns, and share in your frustration over lack of better methods to successfully match buyer to bag.  I have purchased a dozen bags over the decades, yet still find bag shopping a challenge.  In the end I cave into a rather simplistic methodology for my purchases.  I use some basic knowledge regarding the merits of various material and designs used to produce bags, a touch-and-feel session with a bag of interest, evaluating its loft and workmanship, and lastly the subjective feedback from others.  I do know where my preferences vary from the typical consumer, but have no idea how my preferences stack up to a sensor.

Ed

unk, Testing does not need a controlled environment, it only needs a monitored environment.
whomeworry, On the website, I show how these variables can easily be accounted for. All the things that you talk about is precisely why I am doing what I am doing.
The current favorite test method uses a heated manikin in a laboratory. What I am offering is countless humans doing real field tests, with sensors attaching temperature data to the user's experiences.
So, the difference is that EN 13537 has a manikin telling you how good a bag is, whereas ASSP provides user's opinions coupled with real-world data.

Which would be more reliable? Manikin A117D1 or 10 users with a similar age, body type, gender, backpacking experience, and gear? The tester's data will be able to show at what temperatures the bag got to before the users became uncomfortable.

With enough data points and a careful surveying, we can have access to lots of very interesting data, such as the effects that sleeping pads, baselayers, tents, body physiology and age have upon an individual's sleeping experience.

Many people talk about the effects of use on a sleeping bag. But, nobody really knows. This protocol will be able to shed light on questions just like this for the first time.

I really like where you are going with this. Real world scenarios real world applications. Cant tell u how many times ive been frosen due to the poor temp rating on a bag. Heck sell the bags with the sensors in them and offer a discount or reward or what have you, for data or reviews. I would definetly by a bag or any gear from a start up business with those kinds of inscentives.

Which would be more reliable? Manikin A117D1 or 10 users with a similar age, body type, gender, backpacking experience, and gear? The tester's data will be able to show at what temperatures the bag got to before the users became uncomfortable.

too many variables with human testing.. For instance ground temperature needs to be accounted for (not the same as air temperature)

So here the mat used under the bag presents another variable.

If you go to bed cold you will be colder inside the same bag than when you get in warm.

 If you have had a good dinner a couple of hours before you get in the bag you will sleep better than if you have just eaten or had a lousy dinner (calory deficient dinner)

 Going to sleep with a clean body and clean clothes (first layer)  will be warmer than with dirty/damp clothes and a sweatty body.

 Some people sleep naked, most have some clothes on. What clothes? Altitude  may affect how you sleep.

 An Alaskan native will feel warmer than a San Francisco one at exactly the same temp and with the same gear.

If you are tired you may sleep colder than when rested.

Sounds and smells can affect our sleep too.

To put it another way.. If you were making beer you could not get an accurate alcohol content by getting people to drink it and then see how many it takes for them to get drunk. And yes , I am very well aware that many here will disagree..

Franco

Now for my 2 cents.

I think that the mannequin method is an excellent starting place and would be excellent for mass produced bags.  If the test is performed honestly and consistently it will give people an excellent reference.  People will discover if they need to add a few degrees or subtract a few degrees to be comfortable. 

The human method would probably be better as long as you get enough people to get a decent bell curve.  It will take into account things like different bag shapes and features.  It will also be more realistic in that people will toss and turn which is something that the mannequin method can't do.

Anything is better than the current rating system in America.

"Anything is better than the current rating system in America."

 just curious. How are bags tested in America ?

Franco

To save you the troble looking it up , this is from Marmot :

How does EN testing differ from other methods and is it better? When Marmot first started testing bags, especially when we used Gore-Tex® fabric, our designers would go into meat lockers and sleep over night. Once they were able to sleep through the night they looked at the amount of fill to determine the temp range. Nowadays it is a bit more scientific. The testing method used in the US is very similar to the EN test. They use a copper mannequin, but the clothing, pad, and math formula are different. Again we feel the EN test is the most reliable and accurate measure of the performance of a sleeping bag currently available.

Has the feedback from the EN testing improved the designs of Marmot bags? Yes it has. The vast majority of our bags have tested to our current temp ratings. In some cases, the tests showed our bags were actually warmer than the rating we gave it. In another instance, there was a spot that showed room for improvement which led to more efficient baffle construction. With our baseline testing started in 2004, we can now measure improvement over time through new fabrics, insulation, and construction techniques.

http://marmot.com/product/content/en-tested

(I just Googled that...)

From Franco: "If you have had a good dinner a couple of hours before you get in the bag you will sleep better than if you have just eaten or had a lousy dinner (calory deficient dinner)"

Franco makes very good points about human testing. This one in particular I can attest to with an anecdote-winter camping in Yosemite, I wake up because I am cold, I eat half a Clif Bar and go back to sleep and am fine the rest of the night. Half a Clif Bar, that was the difference between my being cold and comfortable. It had nothing to do with my bag or its rating, the variable was me and my fuel consumption.

For those of you dissing the EN test, what it does is create not just an absolute metric, which is really a range of comfort, but a method to compare one bag against another, which in and of itself is valuable.

Part of the problem with the EN results is how manufacturers use them in advertising. If you buy a bag based on the EN rating and the one used by the manufacturer is the survival rating, you will be cold at that temperature, I pretty much guarantee it because that is what that rating says-cold, but survivable, not comfortable, that rating being quite a bit higher.

So, you need to check about any rating-how it was done, what it means and that is not always clear from ads. Cheap bags rated at 20F are often nowhere near that, which is why buying a bag should involve more than just taking the adverts and specs at face value.

(editing to catch up on reading)

Thanks Dingo, I love that idea. And, I intend to have people use the sensors on their own gear to start building a database. Perhaps you'd be interested in doing a test on your next trip?

Franco, I think there is merit your beer test idea, I will integrate it into the test protocol starting tonight and work in the part about some people being naked somehow. I'm pretty sure that'll resolve your other issues. Seriously, I think I've mostly addressed most of the issues you've raised on the website or on previous posts. You work for TarpTent? Have you guys ever tried to test sleeping bags? How did you do it?

There is a lot more to the EN 13537 than what marketers tell us. I'm not an industry insider, but I've been doing much research on the subject as I am hoping the articles at http://www.ardethgear.com/ardeth-sleep-system-protocol.html and the posts above attempt to illustrate. I can't use the current standard, especially as a budding sleep system manufacturer.

It is my hope that I'll be able to have hundreds, then thousands of data points, spanning a wide range of users, a wide variety of gear, and a diverse range of sleeping bags.

Just curious Franco did you find any US standard temperature ratings for sleeping bags??

Here let me google that for you.

To save you the trouble reading all of the page that comes up, this is pertinent part.

Wikipedia:

In the U.S., a sleeping bag's rating typically indicates the lowest temperature at which it will keep the average sleeper warm. For example, with a 0° bag, you should be able to sleep comfortably in 0° temperature.

There is no standard measurement rating in the U.S., so a 20° bag from one company may not provide the same warmth as a 20° from another company. And there are other variables to consider, such as what you plan to wear when you sleep, what type of sleeping pad you use, and how well you hold the heat in your bag.

As you can see there is no system. Some companies use the EN method, but not all and they usually don't put the ratings on there US packaging.  Even if all companies tested using some sort of decent test if it's not the same test it's useless when comparing bags across different brands.  I suspect some companies don't even test their bags.

Since Tarptent was mentioned... TT only makes tents so no interest in sleeping bags whatsoever except for personal use since Henry is a keen hiker and I do also hike, when I can. My point was to highlight that  human testing has many variables and in fact, contrary to the assumption by some here ,a mannequin testing is better (more accurate)  than  a collection of anecdotal evidence .  That was confirmed in that Marmot link. If you read that page you will find that Marmot spelled out some of the same issues I have raised. However I am not suggesting that the EN testing should be mandatory due to the cost attached to it.

To clarify , I am not criticising Jason's efforts, rather pointing out that the results of his survey will have no more merit than the other "American" or otherwise non EN findings that some object to here.

On a side note, I am still puzzled by the fact that so many do not understand why a manufacturer cannot tell them accurately if they will be warm or not inside a particular bag at a particular temperature... Going back to the beer example, it really is the same thing. Beer manufacturers can't tell you how many you can drink before you are drunk. They can tell you the alcohol content (call that loft /amount of insulation) but cannot tell you how much of that you can ingest before you feel drunk .

 Franco

Franco, thanks for your opinion.
I know what TT does, but I was wondering, since you have TT as your website, if you knew if TT had ever built any sleeping bags and tried to test them. I figured you'd have the inside scoop.

I'm not offering to provide anecdotal evidence, that would just be a review. Have you read this thread and the website articles?

Jason Yes what I posted is just  my opinion so take it for what it is. Yes I have read your article on your web page. I suggest you post the same proposal in some other forums, you may get a different response.

 Franco

Please sign in to reply