Neither. Coleridge is referring to his own reservations about some of Wordsworth's poems. The context makes this clear:
A friend whose talents I hold in the highest respect, but whose judgment and strong sound sense I have had almost continued occasion to revere, making the usual complaints to me concerning both the style and subjects of Mr. Wordsworth's minor poems; I admitted that there were some few of the tales and incidents, in which I could not myself find a sufficient cause for their having been recorded in metre. I mentioned Alice Fell as an instance; "Nay," replied my friend with more than usual quickness of manner, "I cannot agree with you there!—that, I own, does seem to me a remarkably pleasing poem." In the Lyrical Ballads, (for my experience does not enable me to extend the remark equally unqualified to the two subsequent volumes,) I have heard at different times, and from different individuals, every single poem extolled and reprobated, with the exception of those of loftier kind, which as was before observed, seem to have won universal praise. This fact of itself would have made me diffident in my censures, had not a still stronger ground been furnished by the strange contrast of the heat and long continuance of the opposition, with the nature of the faults stated as justifying it.
Lyrical Ballads marked a break with prevailing trends in 18th C. poetry by including several poems that forgo elevated subjects and lofty diction, focusing instead on humbler subjects and ordinary language. Coleridge is defending Wordsworth's attempt to "ascertain how far the language of conversation in the middle and lower classes of society is adapted to the purposes of poetic pleasure" ("Advertisement" to the 1798 edition). He says that in some cases, he thinks Wordsworth might have gone too far, picking a topic so unremarkable that it is not worth writing poetry about: "there were some few of the tales and incidents, in which I could not myself find a sufficient cause for their having been recorded in metre." However, he finds that even those poems which he and others criticize for this reason have their fierce defenders. He says that the fact that the same poem can be both "extolled" and "reprobated" would be enough to make him "diffident in [his] censures". I.e., since others whose judgment he esteems likes poems that he doesn't like, he thinks that he might perhaps be wrong to criticize those poems. He goes on to say that the sheer and continued vitriol directed toward Wordsworth's experiments also reinforces his sense that the poems do not deserve such hostility, and that the problem is therefore with those who excoriate Wordsworth.