I second SporkGoddess. One of the very few things that I always found strange about the book (and therefore all other incarnations of the story) is Valjean doesn't have a single romantic bone in his body. There is NO romantic love story for him at all in all 1450 odd pages of the book.
I don't think that he felt that way for Fantine at all. He looked at her as someone who he had wronged, who he would be able to help, just as the bishop helped him. His caring for her was not at all romantic. He gave her shelter and then promised to take care of Cosette (again, trying to right his part in getting her fired), nothing is mentioned of a feeling of even lust, let alone actual LOVE for her.
Maybe it was the religious moral ground that Valjean found, but even if Fantine was well and came onto him, my image of the scene would be of him pushing her away, looking on her almost pityingly - that this poor creature is changing her feelings of gratitude for the rescue into what she believes he would want. I can definitely see her, if she wasn't sick, falling for him, but nothing anywhere in the book would indicate to me that he would return that.
Hi, Shirley Temple Pudding.