> Jacob and Dennis ARE RIGHT. It is annoying to stumble around and get stuck
> on so many broken links. But it's very time consuming to maintain large
> webs, particularly when they are so volatile. We are all experiencing
> phenomenal growth in use of the web, and one of the beauties of the web is
> that it is SO EASY to get a few pages going. It's not surprising there are
> tons of broken links out there. Not only are many of the people browsing
> the web NOT webmasters, but many are quite new to computers as well! I say
> all this interest can only help the cause, and we should welcome them. WE
> provide a service and ought to be helping these people, not slapping them
> around for making mistakes!
And then Brian Behlendorf adds:
> Howbout a program which, given a local html file, recurses through
> links on that file to other files, and the base case on the recursion
> is a link to a file on another machine. When it reaches this, it
> sends out a HEAD command, and if the file exists it moves on, but if
> it gets sent back a REDIRECT header, it finds this new document, and
> reports back to the web maintainer on the local machine that his link
> to "x" is wrong and needs to be changed to "x'". If the script were
> robust enough, this could be set up to do interactively, or even
> automatically. Thus the conundrum of moving pages or whole sites
> could work.
This thread is just too nice a coincidence. I will be presenting a paper
at the WWW94 conference on exactly that type of tool, although with a much
more robust design and efficient operation. The program is called the
Multi-Owner Maintenance spider (MOMspider).
The paper (or at least most of it) can now be read at:
http://www.ics.uci.edu/WebSoft/MOMspider/WWW94/paper.html
I would very much appreciate any comments. The missing bits of the paper
(the one figure and appendices) will be added tomorrow, so there's no need
to tell me about that -- I'd do it now, but I can barely see straight after
working all night.
....Roy Fielding ICS Grad Student, University of California, Irvine USA
(fielding@ics.uci.edu)
<A HREF="http://www.ics.uci.edu/dir/grad/Software/fielding">About Roy</A>