Why is Father Hooper wearing a veil in The Minister's Black Veil?

The truth
is, the reader never truly finds out why Father Hooper is wearing the veil. It appears quite
suddenly, and the members of his church are most definitely put off by its addition to his
clothing: 

"I can't really feel as if good Mr.
Hooper's face was behind that piece of crape," said the sexton.

"I
don't like it," muttered an old woman, as she hobbled into the meeting-house. "He has
changed himself into something awful, only by hiding his face."

"Our parson has gone mad!" cried Goodman Gray, following him across the
threshold.

Hawthorne sets up this same curiosity in the
reader. Most...