lower the pokemon's health and/or inflict it with a status.
all the balls work, but i avoid great balls myself. not worth the money for their cost/effectiveness ratio. anything with a difficult catch rate, you'll want to use ultras. anything with an easy catch rate, you get three regular balls for the price of one great ball.
but they are not broken; great balls do catch pokemon, if the RNG is kind to you. which it never is, RNGs hate everyone.