The whole point of the story is that Emile is not a man of color, Nellie is surprised that this man she has fallen in love with married a Polynesian woman and has Polynesian kids. An African-American actor as Emile would completely throw off the balance of the story, it doesn't matter whether he "looks" black or not (I'm still not sure what that means but that's a whole other story), the point is Mitchell considers himself black, audiences know he is, and so the story wouldn't make sense. That's why it's so great he got to do it in concert and record it because he'd be ENTIRELY miscast in an actual production even if he is technically perfect for the role.
"Some people can thrive and bloom living life in a living room, that's perfect for some people of one hundred and five. But I at least gotta try, when I think of all the sights that I gotta see, all the places I gotta play, all the things that I gotta be at"