That is a good idea, I will have to do that too.

There may be even a bigger issue as I try to get that site relisted.

The fact that ANYTHING will turn up a page from the robots.

After they finish looking at the tags and links they try to send a bad name to get a bad result.

like sitename.com/video/thiscantbeanameforadirectoryatall and the damn thing turns up a page. This causes havoc and they will turn the spider off if it starts to go forever to get a 404 error instead of a regular page (even if it says no videos for this term).

You can try it with things like //+++===0123e193 and you will get a page. In your error logs you will always see these strange things and file names, they are robots seeing if you give a proper 200 server error with a 404 page and this script does not.