Came across an interesting site today called SearchLore. There's an older but excellent article there on the subject of deep searching and deep indexing - it covers a wide variety of issues and has a great 3D model of the web's structure in the form of a sphere.

Particularly interesting is the discussion on the link structure of the web - from the article:

"Like matter and anti-matter, to the "outside linked" pages we spoke of above, correspond an inversed related part of the web: the "outside linkers" pages. Indeed all the pages located in this specific area of the web do "point" to the Nucleus but are not pointed back from it. Imagine as an example the personal links page of a scientist: lotta interesting links to the Nucleus yet no need to publicize its existence. A page with information you may need is there, somewhere, without any link whatsoever that could bring you to it. Indeed there are no links back from the Nucleus to these pages."

Last, there's even a strange form of a blog/updated news section that's got some good links. It's great to look outside the SEO space once in a while for web search information, and this site has to be one of the most unique I've come across.