Ok, from everything I've ever found this story is partially true.
First off, she left the show in September, a year after it opened on Broadway. She was the first of the OBC to leave, and it was more than likely over money.
HOWEVER, the version I heard is that Bishop was ill that night and had to leave suddenly. In fact it's even cited (or at least a performance where Bishop left) in "The Longest Line" as being January 31, 1976.
According to Patti D'Beck (u/s Bebe):
I was on the night Kelly Bishop walked off the stage. It was my first time performing on the line as Bebe and I was horrified, thinking my performance was so terrible that Kelly left the stage! Zach tried to keep her on, but Kelly left. You could hear pages flipping in the orchestra, because we were obviously cutting "At the Ballet" and skipping to "Sing." Michael felt the audience got so in touch with if we were up to or through "At the Ballet," and an actor couldn't finish the show, the understudy would not continue for them. There was a whole diagram on paper of who does what lines, who does the vocals, who does the accident scene. These were scenarios that we were supposed to be prepared for. But of course, there was always this scramble of "What the hell am I supposed to do now?"
http://www.youtube.com/huskcharmer