Of all the equipment in the recording studio, there is no doubt which produces the most noise – the analog tape recorder, multitrack or stereo. Noise is produced by random variations in the medium through which sound, or a sound signal travels or is stored.
Even real acoustic sound travelling in air has a certain noise content produced by the random vibrations of air molecules. This random vibration is sometimes known as 'Brownian motion', and increases with temperature. There is no Brownian motion at absolute zero – but neither is much recording done there!
A microphone with a very small diaphragm will pick up acoustic noise, as relatively few molecules carrying the sound wave are in contact with the diaphragm. The random motions are therefore significant in amplitude compared to the sound wave.
Electrical noise, known as 'Johnson noise' is caused by the random motion of electrons. A so-called 'low-impedance' circuit uses a lot of current, therefore a lot of electrons, and the random motions cancel out almost entirely. A high impedance circuit uses only a small current, therefore few electrons, and once again their random motions will be significant when compared with the signal.
Despite the above, noise due to randomness in the medium is rarely a problem in acoustics or electronics. It exists, but can be controlled.
Noise in analog tape recording is caused by the random orientations of the poles of the magnetic 'domains' on the tape. Think of a domain as a little bubble of magnetism that is as small as it possibly can be. It is always magnetized as much as it can be and never becomes demagnetised.
The process of erasing a tape randomizes the orientations of the poles of the magnetic domains. But since the domains have significant size, their randomness does not entirely cancel out and is heard as noise when the blank tape is played.
The recording process re-orients the poles of the domains. The stronger the signal, the more rigidly the domains are lined up. But in quiet sections of the signal, the randomness remains, hence noise remains.
In a good analog tape recorder, the noise level is around 66 decibels lower than the maximum signal level. This means it is around 1/2000th the level of the signal. During loud sections of the signal, the noise is inaudible, but during quiet sections, it can clearly be heard.
It is debatable whether noise is a desirable component of analog recording, but it is certainly a feature.
But noise isn't really the ogre it is made out to be. If levels are set correctly to maximize the use of the available dynamic range up to the point where the distortion is around 1% (maybe even 3% on percussive sounds), then there is no reason why it should be troublesome in the final mix, although some 'noise management' will be necessary on the part of the mix engineer.