Stamp Collecting or Physics

I read an interesting quote about science recently, by Ernest Rutherford (the kiwi physicist who demonstrated that matter is mostly empty space):

"All science is either physics or stamp collecting."-Ernest Rutherford

Now while this might seem cruel and insulting to the many who associate whith other sciences, I think I can see what he's getting at.
All science can in theory be derrived from physics, but doing so is impracticle and oftem impossible given our limited knowledge about physics and our lack of enormous computational power. Imagine trying to predict, for example, the results of a psychological experiment knowing the exact state of every particle within the system the experiment takes place in.
This is of course impossible given the probabalistic nature of quantum physics, but even if it were possible it would be so difficult to know the nature of all the particles and then calculate their actions that it would certainly be beyond current human ability.
While this is true, we still go ahead and conduct psychological experiments, record them and manipulate them as independent entities from the physics on which they are ultimately based.
This is the kind of 'stamp collecting' I think Rutherford was talking about.
All sciences have the same relationship with physics, with the exception of perhaps chemistry which is so close to physics that it is usually a direct application rather than independently concluded.
Some would argue that physics itsself is only an aplication of mathematics, but I draw the line at physics seeing as it is the most pure science that directly relates to the world. While mathematics is the application of reason on reason, it could not be used to (in theory) arrive at anything about the experiencable world independently.