Well, every protagonist has an antagonist. If we look at Valjean as the protagonist, Javert is his antagonist.
Now, if we think of a villain has just some evil person with no motivation besides being evil, than Javert isn't. Most people aren't evil or villainous for the purpose of being an evil, villainous person. So, Javert can have a perfectly pure motivation and still be the villain of the story.
But, maybe I'm using or looking at villain incorrectly.
Some people paint, some people sew, I meddle.