A couple of people pointed out that one server doesn't equal one "source"
for the purpose of results lists. I think this is a sound bit of wisdom.
One suggestion was to use the security idea of realms as named document
groups. I'm wondering if this will effectively map, or perhaps there
should be a separate scheme.
Immediately popping to mind is the "robots.txt" file, whose purpose in life
is to instruct indexing agents and their ilk. We're intending to use
robots.txt as the configuration file even for Web indexers.
With the latter idea in mind, I'm cross-posting to the robots list.
Nick