Two phones connected by wires - it's hard to say much more than that.
Perhaps unsurprisingly, Lovecraft turned this nightmare into one of his short stories, "The Statement of Randolph Carter". If you read this story (the full text is available at the link I provided), you can see that it's almost the same as the story he told in his letter to Derleth, but with some changes to the phrasing. For example, the following paragraph from the Derleth letter (dated 11 December 1919):
"I'm really sorry", he said in a mellow, pleasant voice; cultivated, and not very deep, "to have to ask you to stay above ground, but I couldn't answer for the consequences if you were to go down with me. Honestly, I doubt if anyone with a nervous system like yours could see it through. You can't imagine what I shall have to see and do - not even from what the book said and from what I have told you - and I don't think anyone without iron-clad nerves could ever go down and come out of that place alive and sane. At any rate, this is no place for anybody who can't pass an army physical examination. I discovered this thing, and I am responsible in a way for anyone who goes with me - so I would not for a thousand dollars let you take the risk. But I'll keep you informed of every move I make by the telephone - you see I've enough wire to reach to the centre of the earth and back!"
becomes the following, similar but slightly altered, passage in the short story (dated December 1919):
“I’m sorry to have to ask you to stay on the surface,” he said, “but it would be a crime to let anyone with your frail nerves go down there. You can’t imagine, even from what you have read and from what I’ve told you, the things I shall have to see and do. It’s fiendish work, Carter, and I doubt if any man without ironclad sensibilities could ever see it through and come up alive and sane. I don’t wish to offend you, and heaven knows I’d be glad enough to have you with me; but the responsibility is in a certain sense mine, and I couldn’t drag a bundle of nerves like you down to probable death or madness. I tell you, you can’t imagine what the thing is really like! But I promise to keep you informed over the telephone of every move—you see I’ve enough wire here to reach to the centre of the earth and back!”
The technology described in this short story has been analysed in a blog post from Lovecraftian Science. Unfortunately, it seems that very little can be said about the "portable telephone" except that it connects two telephones directly by wire, just a very long wire. Certainly it's a far cry from modern-day "mobile phones". Quoting from that blog post:
As Leslie S. Klinger has noted in The Annotated H.P. Lovecraft (2014) on “The Statement of Randolph Carter,” information on a phone system as described by Lovecraft – where two telephones are directly connected to each other by wire – is not widely available. Apparently, Frederick F. Strong filed a patent application in 1905 for a portable telephone unit where two phones were connected with wire (Klinger, 2014).
Lovecraft's dream and story from 1919 come from an era when telephone technology was rather new and developing very rapidly:
Additionally, the first coast to coast long-distance telephone call was in 1915 between Alexander Graham Bell in New York City and his formal assistant Thomas Augustus Watson in San Francisco. By 1904 more than three million phones in the United States were connected to manual switchboard exchanges. Given the quick-based, technological evolution of the telephone, it was not surprising that Warren and Carter had a portable telephone outfit.
So it's hard to say exactly how far telephone technology had progressed in exactly Lovecraft's circles in exactly December 1919. However, I did find the patent application by Frederick F. Strong for a "portable telephone apparatus" (note the exact same phrase "portable telephone" as in Lovecraft's letter and story). You can read the patent application in full (4-page PDF link) (filed in 1905, granted in 1908) or just look at the pretty pictures:
It's probably impossible to know if Lovecraft was aware of this or similar inventions, but that seems to be what people would be thinking of in those days if they talked about "portable telephones": two boxes connected by long wires. Wireless technology wasn't much of a thing back then.
