We sing from the same hymn book, generally repeating the same hundred-or-so hymns and ignoring the same leftovers. It's disconcerting to the congregation when we're asked to sing one of the leftover hymns -- but we struggle through it, guided by the organist and the loud-singing conductor.
(It's not the arm-waving, by the way, so much as the loud singing that gives the conductor the power to lead us in song. While no one will try this experiment, I can guarantee you that if the conductor just stood there and sang, there would be no change whatever in the performance of the congregation.
(Then again, if there were no conductor at all, just an organist, we would probably do as well. But the combination of accompaniment, vocal leading and arm-waving makes us more secure.)
If we are uncomfortable with the less-familiar hymns in the book, imagine what would happen if we had music from a completely different musical tradition!
Well, we don't have to imagine: It happened, and recently.