I don't believe the Raven symbolizes death at all, but rather life, in grief of having to live after a loved one is dead. As Poe himself put it in his essay Philosophy of Composition:
The reader begins now to regard the Raven as emblematical — but it is not until the very last line of the very last stanza, that the intention of making him emblematical of Mournful and Never-ending Remembrance is permitted distinctly to be seen
The phrase "mournful and never-ending remembrance" is the title of a biography of Poe. It's a theme Poe returned to several times, such as in Annabel Lee.
Of course authorial intent is only one factor to consider in interpreting a work; see this discussion. It does, however, jibe with my own understanding of the poem even before I read Poe's explicit thoughts on it.
The Raven is thus even sadder than death itself. The speaker craves forgetfulness, and will never have it: his heart shall be "lifted -- Nevermore". He could be content, or at least cope, with loneliness. But the Raven is a constant reminder: he lives not just with the loss but the knowledge of the loss.