The Boston Grotto group of cave explorers has a great little gallery page of their visits to Maine caves, including some strange cave mushrooms, and a link to a page by Eric Hendrickson with a bunch of info about Maine caves.
Is it possible that Lovecraft's character Edward Derby visited a Maine cave that was home to the "pit of the shoggoths" in HPL's story, The Thing on the Doorstep? "Cyclopean ruins" are mentioned, so it may just be some dank, antiquated basement in the Maine woods, but I'd like to think otherwise.
Derby had been married more than three years on that August day when I got that telegram from Maine. I had not seen him for two months, but had heard he was away "on business." ... And now the town marshal of Chesuncook had wired of the draggled madman who stumbled out of the woods with delirious ravings and screamed to me for protection. It was Edward - and he had been just able to recall his own name and address.
Chesuncook is close to the wildest, deepest, and least explored forest belt in Maine, and it took a whole day of feverish jolting through fantastic and forbidding scenery to get there in a car. I found Derby in a cell at the town farm, vacillating between frenzy and apathy. He knew me at once, and began pouring out a meaningless, half-incoherent torrent of words in my direction.
"Dan, for God's sake! The pit of the shoggoths! Down the six thousand steps... the abomination of abominations... I never would let her take me, and then I found myself there - Ia! Shub-Niggurath! - The shape rose up from the altar, and there were five hundred that howled - The Hooded Thing bleated 'Kamog! Kamog!' - that was old Ephraim's secret name in the coven - I was there, where she promised she wouldn't take me - A minute before I was locked in the library, and then I was there where she had gone with my body - in the place of utter blasphemy, the unholy pit where the black realm begins and the watcher guards the gate - I saw a shoggoth - it changed shape - I can't stand it - I'll kill her if she ever sends me there again - I'll kill that entity - her, him, it - I'll kill it! I'll kill it with my own hands!"
Lovecraft isn't the only one to use Maine subterranean realms as a setting for fantastic madness.
On November 4, 1948, Dr. Clark Savage, Jr., encounters what seem to be denizens of Hell in a Maine cavern. Dr. Karl Lenningen, a noted psychiatrist who accompanies Doc on this adventure, suggests that the experience was prompted by a subterranean hallucinogenic gas, but this does not seem tenable (Kenneth Robeson (Lester Dent), Up From Earth's Center).(this nugget found on a page with an excellent collection of fictive cave adventures) I'll try to get my hands on the full version of this tale and do a writeup on it for you all.
While I have yet to explore any Maine caves myself, I did go to the Adirondacks a couple of summers ago and had a blast wandering around Howe Caverns and Secret Caverns. If you find yourself out there, do NOT neglect Secret Caverns -- it's just up the road from Howe Caverns and totally worth it. The one with the crazy bat-faced entrance is Secret Caverns, and the other is Tristan getting crazy from cave madness in Howe Caverns.