Since one can't really "decline" a nomination, she remained a nominee, ultimately losing to Donna Murphy.
That's not entirely true. When William Daniels declined his nomination for 1776, his name was, in fact, removed from the ballot. Otherwise, he would have almost certainly won (his costar, Ron Holgate, took the award in his absence). I've always assumed that the committee refused to remove Andrews' name because it made for good publicity at the time -- what would happen if she won, etc.
As for her returning to Broadway, I doubt it's likely, and almost certainly not in a musical. After her O2 performances back in 2010, I doubt she'll ever sing in public again.