The song sung by the priest that Lady Hazlemier dances to is called "The Polka" and was written as a monologue by George Grossmith, a leading comedian of the day and the father of the actor George Grossmith junior.
Charlie, hearing Louisa enter the room, says, "He holds her with his glittering eye - The charming guest stands still, and listens like a three-years' child: The master hath his will." This is a paraphrase of some lines from Part One of "The Rime of the Ancient Mariner" by Samuel Taylor Coleridge.