I’ve been trying to find out what User Agent string the MOSS Crawler uses when crawling http-sites. After writing a web app that logs HTTPUSERAGENT and then crawling it to get ahold of the string. I figured there had to be a easier way. And there is. Check the registry on the index box and navigatie to:

HKEY\_LOCAL\_MACHINE\\SOFTWARE\\Microsoft\\Office Server\\12.0\\Search\\Global\\Gathering Manager

Check the string value for UserAgent: Mozilla/4.0 (compatible; MSIE 4.01; Windows NT; MS Search 5.0 Robot)