Byron's "Fragment of a Novel" was based on Polidori's The Vampyre, which in turn was based on a story told verbally by Byron as part of the same competition which produced Frankenstein.
The story of the competition is all too familiar to enthusiasts of horror literature. In the summer of 1816, Byron, Percy Shelley, Mary Shelley, and Polidori were stranded together for a few days in the Villa Diodati on Lake Geneva, until the thunderstorms blew over. The game - a contest in telling ghost stories - was intended to be a match between Shelley and Byron, despite being open to all four ... but it was Mary Shelley and Polidori whose works, Frankenstein and The Vampyre, have been better known up to this very day. The Vampyre went on to inspire Edgar Allan Poe and Bram Stoker himself.
From this detailed essay (emphasis mine):
What was it then that inspired Polidori to create a short story which so many other writers right up to the present day – whether they realise it or not, taking their inspiration direct from Bram Stoker without being aware of his own influences – have admired and sought to emulate?
The short answer is Byron. 'The Vampyre', to give it its proper credit, was inspired by a tale told earlier in the evening by Byron, one which was very much later published as 'A Fragment', and which bears startling theoretical and literary similarities to Polidori's piece. The influence of Byron was such that when 'The Vampyre' was originally published in 'The New Monthly Magazine' in April 1819 it was attributed to him. The rift between the two men having widened in the interim few years however, Byron denied all authorship and a month later Polidori stepped out of the shadows and claimed it for his own whilst admitting the influence of his great friend in its conception. "I beg leave to state that your correspondent has been mistaken in attributing that tale in its present form to Lord Byron. The fact is that though the groundwork is certainly Lord Byron's, its development is mine."
Note that The Vampyre was published on 1 April 1819 while "Fragment of a Novel" was published together with Byron's Mazeppa on 28 June 1819.
It is speculated that Byron wrote "Fragment of a Vampire" as a sort of counterstroke to Polidori's publication of The Vampyre, trying to prove himself and distance himself from association with the latter work:
On its publication "The Vampyre" was widely attributed to Byron, much to his irritation, and as a form of self-defence maybe, he lost no time in quickly publishing his own "Fragment", in which the vampire elements to be found in Polidori’s work are noticeably missing - perhaps eradicated to make the story as removed from "The Vampyre" as possible.
