It was mainly due to the Victorian interpretations of several events and actions in the novel and subsequent comments from contemporary critics.
Firstly, Tess committed murder when she killed Alec. Although the reader of course sympathises with Tess after all Alec did to her, it was still murder, and therefore immoral, in the eyes of Victorian law and hence Tess was heavily criticised for this act... and given the death penalty. Contemporary critic R. H. Hutton described Tess' actions thus:
'though pure in instinct, she was not faithful to her pure instinct'
Furthermore as an aside, Hardy hints his views on Tess' crime and punishment (and thus Victorian society's views) in this line at the very end of the book:
"Justice" was done
The presence of the quotation marks suggests Hardy's contempt of contemporary views on a situation like Tess'. They are unforgiving and condemn her without consideration of the crimes that the 'victim' (Alec) committed. The stress here is that he is criticising the same people who then criticised the subtitle of Tess of the d'Urbervilles.
Secondly, Tess became an adulteress when she married Angel and then married Alec when Angel travels to Brazil, however she really thought he would never return, which makes it excusable in our eyes. But this adultery would have been shocking and inexcusable to the Victorians and would certainly have been classed as immoral.
Thirdly, it was interpreted that, after Alec assaulted her, Tess lost what was called her 'physical purity' and hence Hardy came under fire for his subtitle of 'A Pure Woman (...)' since this contradicted the contemporary commonly accepted ideas of purity. (this article explains it well if you want to go into it.) I will paraphrase the basic idea: The Victorian idea was that 'purity' was lost in such a situation regardless whether consent was given. Virtue was synonimized with this 'purity' and hence Tess would not have been considered pure by many contemporary readers.
In addition, there has been ambiguity over the scene in which Tess is violated by Alec (although it is usually decided that it was non-consensual - discussed in this question) there were obviously those who believed it to be a seduction and hence a fault and immorality on Tess' part. This was fueled by the commonly accepted belief at the time that (quoted from the thesis below):
Olwen Hufton claims that the common Victorian belief was that pregnancy (...) “was held to demonstrate [the woman’s] active consent”
So, many thought that Tess was additionally morally impure due to this event, due to the fact she did conceive Sorrow.
Furthermore, there are also aspects of misogyny, as while Angel committed a consensual immoral act and Tess forgave him for it, when she confessed that she was violated as a young girl, an involuntary act in her case, he 'cannot' forgive her (reference)
“Forgive me as you are forgiven! I forgive you, Angel.”
“You–yes, you do.”
“But you do not forgive me?”
“O Tess, forgiveness does not apply to the case! You were one person; now you are another. My God–how can forgiveness meet such a grotesque–prestidigitation as that!”
Even though Angel admits that she was "more sinned against than sinning" he cannot forgive her as society's expectations were very different for women. They were expected to be strictly moral but men's actions to the contrary were brushed under the carpet. In our time we cannot comprehend this injustice, but as early 20th century critic Norman Page noted:
'what is fairly obvious to the modern reader, however, would not have been obvious to all Victorian readers'
Tess of the d'Urbervilles was Hardy's attempt at criticising this double standard among other negative aspects of Victorian society, a criticism which again caused controversy among his contemporaries.
In regards to anti-Tess critics, Mowbray Morris published a heavily critical review soon after Tess was published, in the Quarterly Review, but I have been unable to find a copy to find exact explanations.
(Main reference for points 1, 2 and 3 from this thesis from Utrecht University, in addition to specific cited sources)