Heat Flow

Hot and cold are very subjective labels - something can feel 'cold to the touch; when it is at the same temperature as something else which is 'warm to the touch' - metals typically feel colder than fabric even when both are at the same temperature, because metal conducts heat away quickly.
For this reason an objective measure of temperature is needed, and an objective definition of heat energy. Heat energy is the amount of energy a body has because of its temperature. A body at a temperature may contain a lot of energy or a little - the amount of heat energy in a body is equal to , so depending on the values of , where are the mass ans specific heat capacity respectively.
This is only a measure of ability to store heat. The ability to transfer heat depends only on temperature. A hotter body will always transfer heat to a colder body, so temperatures always tend to equalise. The human body has a temperature of 37 degrees, which is hotter than most of the objects that surround us. When we touch a piece of metal, the temperature is probably below 37 degrees, so we transfer heat to it, and very efficiently, because it is a metal, so it feels cool. When we touch a piece of fabric, even though it has a temperature below 37 degrees, we do not transfer a great deal of heat to it, because it is an insulator, and it feels (relatively) warm to the touch.