It was his crazy, guilty conscience.
As well as the quote you mention, in which the murderer makes sure that the old man is dead by holding his hand to his heart for "many minutes" without feeling a beat, there's also this in the next paragraph, from when he's hiding the old man's body:
First of all I dismembered the corpse. I cut off the head and the arms and the legs.
I don't think he could possibly still be alive after that!
Besides, the murderer is quite clearly insane. We are constantly reminded of this throughout the story by his wild protestations that, despite what everyone thinks, he is not mad. It was an insane decision to kill the old man in the first place - because of the appearance of his eye? what the heck? And this insanity comes out perhaps most strongly of all at the very end of the story, when he thinks he hears the old man's heart. Just look at the way he describes the noise and the reaction to it:
I gasped for breath - and yet the officers heard it not. I talked more quickly - more vehemently; but the noise steadily increased. I arose and argued about trifles, in a high key and with violent gesticulations, but the noise steadily increased. Why would they not be gone? I paced the floor to and fro with heavy strides, as if excited to fury by the observation of the men - but the noise steadily increased. Oh God! what could I do? I foamed - I raved - I swore! I swung the chair upon which I had been sitting, and grated it upon the boards, but the noise arose over all and continually increased. It grew louder - louder - louder! And still the men chatted pleasantly, and smiled. Was it possible they heard not? Almighty God! - no, no? They heard! - they suspected! - they knew! - they were making a mockery of my horror! - this I thought, and this I think. But any thing was better than this agony! Any thing was more tolerable than this derision! I could bear those hypocritical smiles no longer! I felt that I must scream or die! - and now - again! - hark! louder! louder! louder! louder! -
The way the narrator lapses into sentence fragments suggests that his reason is deserting him altogether. The way he reacts so strongly to the perceived noise while the policemen don't react at all suggests that it's only in his head and not a real, audible sound.
This is, in fact, the point of the story. The murderer is a classic
unreliable narrator - he constantly claims not to be mad, although he clearly is, and in the end he dooms himself by his own guilt-ridden imaginings. Poe explores exactly the same idea in
"The Imp of the Perverse", in which a young man murders his older relative and gets away scot-free and unsuspected until his "perverse" guilty conscience makes him, against his own will, confess the deed.
