Do you ever look at your compressor's gain reduction meter? Actually, it should be called a level reduction meter but 'gain reduction' is a term that has been handed down by history, so I will continue to use it.
Obviously, you should judge compression by ear, but it is helpful to have a visual indication of what's going on. If you want to hear more compression, then you need to see more segments on the gain reduction meter lighting up (or the needle going down more deeply), and faster movements.
But here's something you should look out for - while the instrument is playing (or singer singing), does the meter always show some degree of gain reduction? Even in the quietest (but still playing) sections?
If so, then you are indeed applying too much compression, through setting the threshold too low.
You will hear the effect of this in the transitions from not-playing to playing. Just after the gaps in playing in other words.
When the instrument comes in each time, the compressor suddenly has to leap into deep compression and this leap, though momentary, will probably sound ugly.
But if you set the threshold so that the gain reduction meter reads only a decibel or two in the quietest sections, the compressor never has to compress more than it needs to. The transition from not-playing to playing will be much smoother.
Of course, if you use a compressor for warmth, such as a variable-mu compressor or digitally-mimicked version, then deep compression might give you the sound you are after and you might accept a little roughness in the transitions.
Other than that however, it is a good guideline to set the threshold so that at the very quietest part of the recording, the amount of gain reduction drops almost to zero.
Great home recording starts with a great home recording studio. It doesn't need to be expensive if you know how to select the right equipment for your needs.