When Dr. Clove hands Felix a copy of his book, "How to Exorcise at Home," the top of the book's rear binding has vertical white scratches. These marks are consistent with the way that Oscar would later take the book and strike the air conditioner several times, revealing that the book had already been used for this in a rehearsal or earlier take. (Shows that were filmed before a live audience were actually often filmed twice, before two different studio audiences. The different performances could then be cut together, to give the creators options about how to edit the show.)
When Oscar discovers the pigeon behind the radiator and initially hands it to Miriam (Elinor Donahue), in the closeups it's clearly a real pigeon, whereas in long shots it's a stuffed or otherwise fake bird. This was likely because the show was filmed before a live audience -- a real bird being handled on stage might well have gotten loose or made noises that would cause problems with the soundtrack. The three-camera setup for a live recording also would not have allowed for such a tight closeup, so it almost certainly was filmed and inserted during post-production.