网站自动采集系统(北京婚纱摄影：网站自动采集系统的三条伪静态)

　　网站自动采集系统(北京婚纱摄影：网站自动采集系统的三条伪静态)

　　网站自动采集系统要求很简单，就是先设定自动提取什么指定关键词，系统自动提取与指定关键词相匹配的网站中指定页面的链接。比如说是，就设定自动采集北京及北京周边城市的婚纱摄影店信息，百度搜索“北京婚纱摄影”，这时就能搜索到我们要提取的关键词“北京婚纱摄影”了。要制作自动采集系统，采集站就必须要满足以下三条：1、需要伪静态,什么是伪静态，就是有什么网站内容就搜索什么内容，伪静态定义：当网站没有特定关键词（非要关键词就使用）的时候可以直接查询，当有特定关键词时就需要查询指定的关键词，伪静态的数据没有压缩！2、设置robots文件，robots文件很简单，就是进行网站禁止什么网站的蜘蛛抓取，让抓取不到这些网站的网站蜘蛛不抓取。

　　robots文件格式如下：robots.txt{"disallow":[true],"disallowsingleclick":[true],"disallowchecking":[true],"disallowwithkeywords":[true],"disallowedurl":[true],"disallow":[true],"disallow1domain":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallowedurl":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow1domain":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow":[true],"disallow。