Noise is your enemy.
The simplest concept in compression is the idea that you average four nearby pixels together - if they're really similar? You can replace them with one piece of data (rather than the original four.)
Noisy footage happens at the source for two reasons: poorly lit and a very compressed starting point (I'm looking at you DSLR h.264 files!). The worse the starting point, the more important it is to shoot right - else you have to deal with noise, meaning it'll be harder to compress your footage.