Although Sir Rowland Hill (accredited as the inventor of the postage stamp) intended the first stamps to be gummed, many early stamps were not.
Up until the late 1960's the type of adhesive used on stamps was "gum arabic". This is a natural adhesive made from the sap of an Acacia tree, and not, as is sometimes thought, from horses' hooves.
Unfortunately this gum suffered from some disadvantages. In tropical climates not only did it cause stamps to curl up, but sheets tended to stick together. It also stains the paper itself - making the watermark much more visible. The gum could also become glazed, brittle or even crack - and these cracks can eventually cause the stamp itself to tear.
Synthetic adhesives were then introduced, one of which is polyvinyl alcohol (or PVA). This type is almost invisible, so a yellowish colourant was added in order to make it possible to see whether or not the adhesive had been applied. The advantage of this synthetic type is that in warmer climates the stamps do not curl, and sheets are less likely to stick to one another.
Other types of adhesives used on stamps include dextrin (obtained by heating starch).
Due to the difficulty in identification, gum is rarely one of the criteria used in differentiating between common and rare stamps. It can make a diffence to the value of a stamp though. Some collectors of mint stamps want examples where the gum is in pristine condition - and that means the stamp can't be hinged. Of course traditionally collectors mounted stamps in their albums by using hinges, so early examples of mint un-hinged stamps are rare. And of course, this has meant that some unscrupulous individuals have taken to re-gumming early stamps i.e. removing the old damaged adhesive and applying a new coat.
Depending on how well this was done, it can be hard to identify. One way of seeing whether a stamp has been re-gummed is by examining the back with a magnifying glass. If the adhesive has pooled at the edges near the perforation, the likelihood is that this is no longer the original gum (because the original sheets of stamps would have been gummed before perforation took place).
Again using a magnifying glass, check the tips of the perforation. If they appear stiff, this may be because the gum was applied after the stamp was detached. Usually when a stamp is separated from its neighbours along the perforation, the torn edges appear slightly feathered.
Another way of identification is by comparison - if you know what the gum should look like, and that of the suspect stamp appears different, this could be an indication that the stamp has been doctored. If you intend buying an old stamp as "mint" i.e. never hinged, with full original gum, you should make sure it has been authenticated first.
Some stamps were mistakenly gummed on the printed side.
The most recent addition is self-adhesive gum. Whilst certainly easy to use by those posting a letter, it is not well-liked by collectors. The first self-adhesive stamps had insoluble gum. Soaking in water had no effect, so collectors were left with no option but to retain the envelope, or at least the part bearing the stamp.
In the 1980's some countries started using self-adhesive gums which were easier to soak off. This was not done to appease collectors, but simply because the glue was cheaper.
In order to collect self-adhesive stamps in mint condition, the stamps obviously need to remain on their backing paper - and the double thickness of this can cause unsightly bulging in the album, especially noticeable if there are a lot of these stamps!