User-agent: * Allow:/index.asp Allow:/404.asp Disallow: /*/ Disallow: /*.asp Disallow: /*.php Disallow: /*.jsp