It's a literary device, leveraging the reader's imagination to increase the horror.
Many of the most effective horror stories work by leveraging the reader's imagination rather than describing horrific things directly. Detailed descriptions (or images, in a film/TV context) of gore often don't work as well as more subtle creeping horror. The Blair Witch Project, one of the most famous horror films, uses this technique to great effect: no ghost, witch, or monster is ever seen; there's no CGI or on-screen deaths; the film is terrifying because of what it implies without showing outright.
(More or less the same technique can be used in psychological torture: rather than making specific threats about exactly what they intend to do to their victim, an effective torturer can just send for, say, cooking oil and salt, or a basket of figs and some mice, and the victim's mind will make up worse uses for these things than the torturer themselves ever could.1)
In literature, Lovecraft was also a great user of this technique. I only read a few of his stories before deciding that they were all too similar to each other and giving up on the rest, but many of his characters have horrific experiences which are never described explicitly, because it's more effective to describe them vaguely and let the reader imagine the rest. For example, this excerpt from near the end of the short Lovecraft story "The Rats in the Walls":
Having grasped to some slight degree the frightful revelations of this twilit area—an area so hideously foreshadowed by my recurrent dream—we turned to that apparently boundless depth of midnight cavern where no ray of light from the cliff could penetrate. We shall never know what sightless Stygian worlds yawn beyond the little distance we went, for it was decided that such secrets are not good for mankind. But there was plenty to engross us close at hand, for we had not gone far before the searchlights shewed that accursed infinity of pits in which the rats had feasted, and whose sudden lack of replenishment had driven the ravenous rodent army first to turn on the living herds of starving things, and then to burst forth from the priory in that historic orgy of devastation which the peasants will never forget.
God! those carrion black pits of sawed, picked bones and opened skulls! Those nightmare chasms choked with the pithecanthropoid, Celtic, Roman, and English bones of countless unhallowed centuries! Some of them were full, and none can say how deep they had once been. Others were still bottomless to our searchlights, and peopled by unnamable fancies. What, I thought, of the hapless rats that stumbled into such traps amidst the blackness of their quests in this grisly Tartarus?
Once my foot slipped near a horribly yawning brink, and I had a moment of ecstatic fear. I must have been musing a long time, for I could not see any of the party but the plump Capt. Norrys. Then there came a sound from that inky, boundless, farther distance that I thought I knew; and I saw my old black cat dart past me like a winged Egyptian god, straight into the illimitable gulf of the unknown. But I was not far behind, for there was no doubt after another second. It was the eldritch scurrying of those fiend-born rats, always questing for new horrors, and determined to lead me on even unto those grinning caverns of earth’s centre where Nyarlathotep, the mad faceless god, howls blindly to the piping of two amorphous idiot flute-players.
Their surroundings aren't described in much detail, but the horror story is extremely effective, partly because it leaves us unsure as to what they actually witnessed. Partly also because of the extended foreshadowing and leadup to the ultimate horror, and the spine-chilling ending that comes just after the passage quoted above, but the lack of detailed information is definitely a part of what makes it stick in the mind: long after finishing the story, we keep thinking about (maybe even having nightmares about) exactly what they saw, what was down there in those black and hideous depths.
In "Dagon", it's clear right from the start of the story that this incomplete-information device will be used. (Actually it's pretty likely even from before you start reading the story, if you know how Lovecraft writes, but be that as it may.) The narrator is setting us up for that creeping, lingering horror that comes from letting our own imaginations run amok on what we didn't fully grasp from the story.
Beyond a meta-reference to a Lovecraftian literary device, this also increases the horror in a more literal sense. The narrator is telling us that nobody who hasn't actually experienced it can really appreciate it. That accentuates the horror right from the start. We know we're going to read a horror story, and we're also being told that we can never be affected as much as the narrator was. Once again, this leaves it to us to imagine the true extent of the effect on the narrator, and to try to comprehend how twisted his mind is after what he's seen. Same idea again.
Finally, I challenge your claim that the story is going heavy on description and exact details. In typical Lovecraft fashion, it starts off detailed, but as the grotesqueness increases, the detail decreases, until we're left with the vaguest of vague descriptions of the final sight which literally drove the narrator insane. Here's an excerpt from the final paragraphs before he wakes up in a San Francisco hospital (emphasis mine):
It was the pictorial carving, however, that did most to hold me spellbound. Plainly visible across the intervening water on account of their enormous size, were an array of bas-reliefs whose subjects would have excited the envy of a Doré. I think that these things were supposed to depict men—at least, a certain sort of men; though the creatures were shewn disporting like fishes in the waters of some marine grotto, or paying homage at some monolithic shrine which appeared to be under the waves as well. Of their faces and forms I dare not speak in detail; for the mere remembrance makes me grow faint. Grotesque beyond the imagination of a Poe or a Bulwer, they were damnably human in general outline despite webbed hands and feet, shockingly wide and flabby lips, glassy, bulging eyes, and other features less pleasant to recall. Curiously enough, they seemed to have been chiselled badly out of proportion with their scenic background; for one of the creatures was shewn in the act of killing a whale represented as but little larger than himself. I remarked, as I say, their grotesqueness and strange size; but in a moment decided that they were merely the imaginary gods of some primitive fishing or seafaring tribe; some tribe whose last descendant had perished eras before the first ancestor of the Piltdown or Neanderthal Man was born. Awestruck at this unexpected glimpse into a past beyond the conception of the most daring anthropologist, I stood musing whilst the moon cast queer reflections on the silent channel before me.
Then suddenly I saw it. With only a slight churning to mark its rise to the surface, the thing slid into view above the dark waters. Vast, Polyphemus-like, and loathsome, it darted like a stupendous monster of nightmares to the monolith, about which it flung its gigantic scaly arms, the while it bowed its hideous head and gave vent to certain measured sounds. I think I went mad then.
Of my frantic ascent of the slope and cliff, and of my delirious journey back to the stranded boat, I remember little. I believe I sang a great deal, and laughed oddly when I was unable to sing. I have indistinct recollections of a great storm some time after I reached the boat; at any rate, I know that I heard peals of thunder and other tones which Nature utters only in her wildest moods.
That's not what I call a detailed description. The monster is hardly described at all in any specific way, except that it has scaly arms. We don't know its shape, its exact size, what sounds it made, really anything about what it looked like. Even the carved creatures described earlier aren't described properly, and the very clear implication is that to do so would cause us to never sleep soundly again. By leaving out all these details, of course, Lovecraft is disturbing our sleep far more than he would have by leaving them in. Our imaginations can create monsters far more grotesque and awful than a writer's pen could possibly achieve. That, in a nutshell, is the key to Lovecraftian writing, and to a lot of really successful and memorable horror.
1 Gold star to whoever gets this reference without Googling.
