While I don't think your theory is really likely, I have noticed this curious occurence in my site's statistics as well. However, while I did notice a couple of pr0n referrers, they were generally only 1-2 hits each and I could find no reference to me when viewing the page.
However, the types of "one hit" referrers I get aren't limited to pr0n or marketing links. Many of them are personal homepages, video game forums, and basically a myriad other things that I could understand someone visiting my site to be interested in.
I asked one of my other friends about this once, who runs his own server, and knows more about that kind of thing than I did. He explained it to me as a bug in some browsers that sometimes records the last page you visited as a "referrer" even though it really wasn't. So if someone was looking at that pr0n page, then clicked their bookmark to visit you, then it might accidentally show up as a referrer stat. I don't know how true that is, but it makes enough sense to me to at least seem like a plausible explanation. He mentioned Internet Explorer as being the guilty browser, but I'm sure there are others with the same problem (if this is indeed happening at all.)
If there is a better explanation, I'd like to know about it, as this is one of the things that has baffled me for quite awhile.
This post was probably not written by the real Jin Wicked. Please see user "butter pie" for Jin's actual posts.