There's a reason you hired a contractor instead of overseeing the project yourself: competence. Your contractor should be perfectly able to do the work he said he would do (what's outlined on the contract). You don't need to put up with shoddy, haphazard or unsafe work.
That said, you can't and shouldn't expect everything in a build or remodel to go perfectly smoothly. Tools break, workers get sick, and the straight lines on your old house may not be so straight. Whatever the issues, there are likely to be some bumps and bruises along the way to your dream house or dream addition. It's not necessarily a complete lack of these bumps that indicates a competent contractor; it's whether your contractor is able to roll with the punches. There's always a way to make it work. If, time and time again, your contractor can't make it work, you may need to replace him.