9:25 p.m. on November 30, 2011 (EST)
Like any other outdoor gear, it depends on what you are going to do with it. First, I seriously doubt that you want a GPS. That means Global Positioning System. A GPS primarily consists of a constellation of satellites, a ground control system, and a community of users who operate user equipment, including among other things, GPSRs (which are GPS receivers). I suspect you really want a GPSR, since a GPS would be priced far out of anyone's price range other than a government. There are 2 currently operational global positioning systems - the US Navstar Global Positioning System and the Russian GLONASS. There are 4 more under development - the European Galileo, plus Japanese, Chinese, and Indian systems.
Further, you probably want a consumer GPSR, and since you are inquiring on Trailspace, I suspect you want a hiking-oriented GPSR, and not a marine, aviation, or automotive GPSR. Then again, you might want a geocaching-oriented GPSR. I will also assume you are not interested right now in geotagging your photographs. I will stick with hiking-oriented variety for now.
A number of the "features" are a matter of personal taste. Of the GPSRs currently on the market, my top recommendation is the Delorme PN-60. It has all the features you want and more. If you get the PN-60w version, you can add either a SPOT Communicator or the new inReach (currently being delivered, according to the ads). With these, you can send messages to your "team", and with the inReach you have 2-way text messaging anywhere on Earth. Some people really really want touch screens, not only on their phones and pads, but also on their GPSRs. Magellan and Garmin make touchscreen versions, plus most automotive GPSRs these days have touch screens. While I like my Magellan 640 for many things (especially geocaching) and make a lot of use of my TomTom automotive GPSR in new locations, getting around road construction, and in foreign countries, I find the touchscreen to be problematic in hiking/backpacking/ski-touring use, especially when I have to wear gloves. Touchscreens basically do not work well with gloves on, plus they always get greasy fingermarks and become less responsive and require frequent cleaning to be fully legible. For backcountry use, buttons just work better than touchscreens.
You say you want a compass. The compass in any electronic widget, including all GPSRs is a flux-gate compass. This means it requires fairly frequent recalibration (including every time you change the batteries). Flux-gate compasses shorten the battery life dramatically, from the 15-20 hour range for GPSRs without a compass or keeping the compass turned off down to 10 or less hours. Regardless of whether you have a compass in your GPSR, you really really really must have a "real" compass with you in the backcountry. As discussed many times here on Trailspace, an inexpensive baseplate compass ($10 or so) from Brunton, Suunto, or Silva (Johnson Worldwide Associates) will do everything you need in the backcountry, plus it won't die when the batteries run down or are cold.
You say you want an altimeter. All but the very cheapest current handheld GPSRs display the altitude. This is automatically part of the 4-dimensional positioning determination of a GPS receiver. The GPS-derived altitude is more accurate overall than a barometric altimeter, since it is not affected by changes in air pressure. But since you mention a barometer, I suspect you think you want a barometric altimeter in your GPSR. The mid-range and up GPSRs currently on the market all have barometric altimeters. When the barometric altimeter is turned on, it cuts battery life almost as much as the flux-gate compass, so much so that the manufacturers are recommending (ins some cases, stating is as "require") using only Ultimate Lithium batteries in their GPSRs with both compasses and barometric altimeters. Garmin, unfortunately, does not allow you to turn off the barometric altimeter, and in some of their models displays the GPS-derived altitude only when calibrating the barometric altimeter (although they do have an autocalibrate mode). There are separate barometric altimeters, both handheld and included in "wrist-top computers" such as heart-rate training monitors by Suunto and Polar. These have very long-lived batteries (2-5 years). But again, electronic altimeters, whether in a GPSR or elsewhere, are useless when the battery runs down.
As for the barometer itself, since a barometric altimeter does depend on air pressure changes to show the altitude, and the barometer depends on air pressure changes to be usable for weather tracking, you can use the barometer in its weather mode only when at a fixed location, or by recording the air pressure at locations of known altitude. Which means you should take an introductory meteorology course.
One feature appearing in more and more GPSRs, as well as your phone and pad is a camera. My Magellan 640 has a fairly nice camera, but by no means as nice as my little shirt pocket camera in terms of the photos it produces, and of course far poorer than my DSLR with its interchangeable lenses.
Ok, so after all that, what do I recommend? You can go to EMS, REI, Fry's. Best Buy, etc etc. and look at all sorts of GPSRs. You will find that Garmin dominates the market. The reason is not that they are any better or that the "user interface" is any easier to use (the easiest use interface is the one you first learn and get used to, besides which "the Garmin interface" changes between their button and touchscreen models and as new "features" are introduced). Rather, Garmin has had a very aggressive marketing campaign for years. Add to that Magellan's financial problems which has put them through a series of owners (currently a Chinese company), Lowrance's low profile in the outdoor market (they are big in marine and aircraft), and Delorme's newness to the market. Delorme has been big in the map market for decades, and the computerized map market for more than 10 years (their Street Atlas is the most complete and up to date computerized street map software - Rand McNally is a subsidiary, as is Thomas Bros.). Delorme has had a tiny module to use with your laptop in the car with their electronic map software for years, with the PN series (20, 40, and now 60 - I have all 3) being relative recent.
All of the big 3 have maps that will display on the screen of their handhelds, with capability of getting worldwide topographic maps. The problem is the tiny screens, of course - CARRY A PAPER MAP!!! You can get scanned USGS maps to go on most of the better models from each company. And you can get many foreign country street and topographic maps - extra cost, of course.
As mentioned above, the Delorme PN-60 is my current top choice (wait 6 months, and I will find something "newer, better, faster, ..."). My main reason is dependability and ease of interfacing with computerized maps. I have used the PN-60w in Antarctica, up to the summits of peaks in the Andes, wandering the deserts in Arizona, and other places in the year I have had it. I have used it with a SPOT to send tracks and messages back home to Barb and to my son in Wisconsin. At the OR Show last summer, I (and Alicia) got a chance to try the prototype of the inReach, which looks really good (particularly the free-form texting feature). I do like the Magellan touchscreen models a lot. I used the 640 in Antarctica, where I had a problem with taking my gloves off in the -30 to -40 weather, but have little problem with the touchscreen (except greasy fingerprints) here in California from Pacific shore to Donner Pass. Yeah, I do like my Garmin 60CSx, though the short battery life when the compass is on is annoying (I leave the compass off almost all the time). I haven't tried the newer 62CSx yet.