I installed one and it leaks water out of the holes when it is turned on. I don't think this should be happening . Can this be stopped?

Hi Larry, are we talking about the vacuum breaker or the yard hydrant? If we are talking about the vacuum breaker, no, it should not be leaking. Please make sure the rubber washer is in place, just like a garden hose sprayer washer. If that is missing, then yes, it could be leaking.