What your contractor does on a Friday night is his business. What your contractor does on the job site is yours. Specifically, you can't tolerate drug, alcohol or unreasonable prescription drug use on your site. Whether your contractor gets his drug of choice from the bottle, a street-corner dealer or from a doctor, if it impairs his ability to do the job well and safely, it might be time to find someone who can do the job right. And the same goes for your contractor's employees. If there's substance abuse on your job site, it's the contractor's fault.
Most large contracting firms have a zero-tolerance policy for drug and alcohol use on the job. If you hired your contractor through one of these firms, you may be able to replace your contractor without altering your contract too much.