According to an article on Ask About Ireland, the Hellfire Club originally had a Neolithic tomb on the site. The property was eventually owned by William "The Speaker" Conolly who used stone from the tomb to build a hunting lodge in 1725 . After Speaker Conolly died in 1729, the building stood unoccupied until The Hell Fire Club purchased the site.
The Hell Fire Club was alleged to "have devoted their time to the practice of unbridled hedonism," according to a talk given by Dave Walsh. There are numerous stories surrounding the site including devil worship, drunken orgies, human and animal sacrifices, and of course ghosts that wonder the site to this day.
We didn't see any ghosts on our visit to the site, just some graffiti and an expanded view of Dublin.