Some prohibitions (especially Rabbinic prohibitions) attach to the action of an individual. Others attach to the object. In the case of Avodah Zarah the object is so contaminated that even Rabbinic Prohibition can attach to the object.
Source Sheet
