蜘蛛是一种半自动的程序，就象现实当中的蜘蛛在它的Web（蜘蛛网）上旅行一样，蜘蛛程序也按照类似的方式在Web链接织成的网上旅行。蜘蛛程序之所以是半自动的，是因为它总是需要一个初始链接（出发点），但此后的运行情况就要由它自己决定了，蜘蛛程序会扫描起始页面包含的链接，然后访问这些链接指向的页面，再分析和追踪那些页面包含的链接。从理论上看，最终蜘蛛程序会访问到Internet上的每一个页面，因为Internet上几乎每一个页面总是被其他或多或少的页面引用。-Spider is a semi-automatic process, like the reality which the spider in its web (spider web) travel, the spider in a similar way in the Web link woven into online travel. Reason why a spider is a semi-automatic because it always requires an initial link (starting point), but after the operation should by its own decision, and the spider scans the start page contains links and then visit these links point to the page, and then analysis and tracking links to those pages contain. In theory, the ultimate spider will visit every page on the Internet, because almost every page on the Internet is always more or less page references.