文件名称:NWebCrawler
-
所属分类:
- 标签属性:
- 上传时间:2013-12-19
-
文件大小:377.63kb
-
已下载:1次
-
提 供 者:
-
相关连接:无下载说明:别用迅雷下载,失败请重下,重下不扣分!
介绍说明--下载内容来自于网络,使用问题请自行百度
网络爬虫,
* 可配置:线程数、线程等待时间,连接超时时间,可爬取文件类型和优先级、下载目录等。
* 状态栏显示统计信息:排入队列URL数,已下载文件数,已下载总字节数,CPU使用率和可用内存等。
* 有偏好的爬虫:可针对爬取的资源类型设置不同的优先级。
* 健壮性:十几项URL正规化策略以排除冗余下载、爬虫陷阱避免策略的使用等、多种策略以解析相对路径等。
* 较好的性能:基于正则表达式的页面解析、适度加锁、维持HTTP连接等。-Web crawler
* 可配置:线程数、线程等待时间,连接超时时间,可爬取文件类型和优先级、下载目录等。
* 状态栏显示统计信息:排入队列URL数,已下载文件数,已下载总字节数,CPU使用率和可用内存等。
* 有偏好的爬虫:可针对爬取的资源类型设置不同的优先级。
* 健壮性:十几项URL正规化策略以排除冗余下载、爬虫陷阱避免策略的使用等、多种策略以解析相对路径等。
* 较好的性能:基于正则表达式的页面解析、适度加锁、维持HTTP连接等。-Web crawler
(系统自动生成,下载前可以参看下载内容)
下载文件列表
NWebCrawler/
NWebCrawler/bin/
NWebCrawler/bin/download/
NWebCrawler/bin/dump.txt
NWebCrawler/bin/NWebCrawler.exe
NWebCrawler/bin/NWebCrawler.pdb
NWebCrawler/bin/NWebCrawler.vshost.exe
NWebCrawler/bin/NWebCrawler.vshost.exe.manifest
NWebCrawler/bin/NWebCrawlerLib.exe
NWebCrawler/bin/NWebCrawlerLib.pdb
NWebCrawler/bin/NWebCrawlerLib_4_13_2013.log
NWebCrawler/bin/NWebCrawlerLib_4_14_2013.log
NWebCrawler/C# 编写的网络爬虫.txt
NWebCrawler/data/
NWebCrawler/data/pdc_09.txt
NWebCrawler/data/sina_12_28.txt
NWebCrawler/src/
NWebCrawler/src/Backup/
NWebCrawler/src/Backup/NWebCrawler/
NWebCrawler/src/Backup/NWebCrawler.sln
NWebCrawler/src/Backup/NWebCrawlerLib/
NWebCrawler/src/Backup/NWebCrawlerLib/Common/
NWebCrawler/src/Backup/NWebCrawlerLib/Common/Logger.cs
NWebCrawler/src/Backup/NWebCrawlerLib/Common/PriorityQueue.cs
NWebCrawler/src/Backup/NWebCrawlerLib/CrawleHistroyEntry.cs
NWebCrawler/src/Backup/NWebCrawlerLib/CrawlerThread.cs
NWebCrawler/src/Backup/NWebCrawlerLib/Downloader.cs
NWebCrawler/src/Backup/NWebCrawlerLib/NWebCrawlerLib.csproj
NWebCrawler/src/Backup/NWebCrawlerLib/Parser.cs
NWebCrawler/src/Backup/NWebCrawlerLib/Program.cs
NWebCrawler/src/Backup/NWebCrawlerLib/Properties/
NWebCrawler/src/Backup/NWebCrawlerLib/Properties/AssemblyInfo.cs
NWebCrawler/src/Backup/NWebCrawlerLib/Settings.cs
NWebCrawler/src/Backup/NWebCrawlerLib/UrlFrontierQueueManager.cs
NWebCrawler/src/Backup/NWebCrawlerLib/Utility.cs
NWebCrawler/src/Backup/NWebCrawler/config.ini
NWebCrawler/src/Backup/NWebCrawler/MainForm.cs
NWebCrawler/src/Backup/NWebCrawler/MainForm.Designer.cs
NWebCrawler/src/Backup/NWebCrawler/MainForm.resx
NWebCrawler/src/Backup/NWebCrawler/NWebCrawler.csproj
NWebCrawler/src/Backup/NWebCrawler/Program.cs
NWebCrawler/src/Backup/NWebCrawler/Properties/
NWebCrawler/src/Backup/NWebCrawler/Properties/AssemblyInfo.cs
NWebCrawler/src/Backup/NWebCrawler/Properties/Resources.Designer.cs
NWebCrawler/src/Backup/NWebCrawler/Properties/Resources.resx
NWebCrawler/src/Backup/NWebCrawler/Properties/Settings.Designer.cs
NWebCrawler/src/Backup/NWebCrawler/Properties/Settings.settings
NWebCrawler/src/Backup/NWebCrawler/SettingsForm.cs
NWebCrawler/src/Backup/NWebCrawler/SettingsForm.Designer.cs
NWebCrawler/src/Backup/NWebCrawler/SettingsForm.resx
NWebCrawler/src/NWebCrawler/
NWebCrawler/src/NWebCrawler.sln
NWebCrawler/src/NWebCrawler.suo
NWebCrawler/src/NWebCrawlerLib/
NWebCrawler/src/NWebCrawlerLib/bin/
NWebCrawler/src/NWebCrawlerLib/bin/Release/
NWebCrawler/src/NWebCrawlerLib/Common/
NWebCrawler/src/NWebCrawlerLib/Common/Logger.cs
NWebCrawler/src/NWebCrawlerLib/Common/PriorityQueue.cs
NWebCrawler/src/NWebCrawlerLib/CrawleHistroyEntry.cs
NWebCrawler/src/NWebCrawlerLib/CrawlerThread.cs
NWebCrawler/src/NWebCrawlerLib/Downloader.cs
NWebCrawler/src/NWebCrawlerLib/NWebCrawlerLib.csproj
NWebCrawler/src/NWebCrawlerLib/NWebCrawlerLib.csproj.user
NWebCrawler/src/NWebCrawlerLib/obj/
NWebCrawler/src/NWebCrawlerLib/obj/Debug/
NWebCrawler/src/NWebCrawlerLib/obj/Debug/DesignTimeResolveAssemblyReferencesInput.cache
NWebCrawler/src/NWebCrawlerLib/obj/Debug/NWebCrawlerLib.csproj.FileListAbsolute.txt
NWebCrawler/src/NWebCrawlerLib/obj/Debug/NWebCrawlerLib.exe
NWebCrawler/src/NWebCrawlerLib/obj/Debug/NWebCrawlerLib.pdb
NWebCrawler/src/NWebCrawlerLib/obj/Debug/TempPE/
NWebCrawler/src/NWebCrawlerLib/Parser.cs
NWebCrawler/src/NWebCrawlerLib/Program.cs
NWebCrawler/src/NWebCrawlerLib/Properties/
NWebCrawler/src/NWebCrawlerLib/Properties/AssemblyInfo.cs
NWebCrawler/src/NWebCrawlerLib/Settings.cs
NWebCrawler/src/NWebCrawlerLib/UrlFrontierQueueManager.cs
NWebCrawler/src/NWebCrawlerLib/Utility.cs
NWebCrawler/src/NWebCrawler/bin/
NWebCrawler/src/NWebCrawler/bin/Release/
NWebCrawler/src/NWebCrawler/config.ini
NWebCrawler/src/NWebCrawler/MainForm.cs
NWebCrawler/src/NWebCrawler/MainForm.Designer.cs
NWebCrawler/src/NWebCrawler/MainForm.resx
NWebCrawler/src/NWebCrawler/NWebCrawler.csproj
NWebCrawler/src/NWebCrawler/NWebCrawler.csproj.user
NWebCrawler/src/NWebCrawler/obj/
NWebCrawler/src/NWebCrawler/obj/Debug/
NWebCrawler/src/NWebCrawler/obj/Debug/DesignTimeResolveAssemblyReferencesInput.cache
NWebCrawler/src/NWebCrawler/obj/Debug/GenerateResource-ResGen.read.1.tlog
NWebCrawler/src/NWebCrawler/obj/Debug/GenerateResource-ResGen.write.1.tlog
NWebCrawler/src/NWebCrawler/obj/Debug/NWebCrawler.csproj.FileListAbsolute.txt
NWebCrawler/src/NWebCrawler/obj/Debug/NWebCrawler.exe
NWebCrawler/src/NWebCrawler/obj/Debug/NWebCrawler.MainForm.resources
NWebCrawler/src/NWebCrawler/obj/Debug/NWebCrawler.pdb
NWebCrawler/src/NWebCrawler/obj/Debug/NWebCrawler.Properties.Resources.resources
NWebCrawler/src/NWebCrawler/obj/Debug/NWebCrawler.SettingsForm.resources
NWebCrawler/src/NWebCrawler/obj/Debug/ResolveAssemblyReference.cache
NWebCrawler/src/NWebCrawler/obj/Debug/TempPE/
NWebCrawler/src/NWebCrawler/obj/Debug/TempPE/Properties.Resources.Designer.cs.dll
NWebCrawler/src/NWebCrawler/Program.cs
NWebCrawler/src/NWebCrawler/Properties/
NWebCrawler/src/NWebCrawler/Properties/AssemblyInfo.cs
NWebCrawler/src/NWebCrawler/Properties/Resources.Designer.cs
NWebCrawler/src/NWebCrawler/Properties/Resources.resx
NWebCrawler/src/N
NWebCrawler/bin/
NWebCrawler/bin/download/
NWebCrawler/bin/dump.txt
NWebCrawler/bin/NWebCrawler.exe
NWebCrawler/bin/NWebCrawler.pdb
NWebCrawler/bin/NWebCrawler.vshost.exe
NWebCrawler/bin/NWebCrawler.vshost.exe.manifest
NWebCrawler/bin/NWebCrawlerLib.exe
NWebCrawler/bin/NWebCrawlerLib.pdb
NWebCrawler/bin/NWebCrawlerLib_4_13_2013.log
NWebCrawler/bin/NWebCrawlerLib_4_14_2013.log
NWebCrawler/C# 编写的网络爬虫.txt
NWebCrawler/data/
NWebCrawler/data/pdc_09.txt
NWebCrawler/data/sina_12_28.txt
NWebCrawler/src/
NWebCrawler/src/Backup/
NWebCrawler/src/Backup/NWebCrawler/
NWebCrawler/src/Backup/NWebCrawler.sln
NWebCrawler/src/Backup/NWebCrawlerLib/
NWebCrawler/src/Backup/NWebCrawlerLib/Common/
NWebCrawler/src/Backup/NWebCrawlerLib/Common/Logger.cs
NWebCrawler/src/Backup/NWebCrawlerLib/Common/PriorityQueue.cs
NWebCrawler/src/Backup/NWebCrawlerLib/CrawleHistroyEntry.cs
NWebCrawler/src/Backup/NWebCrawlerLib/CrawlerThread.cs
NWebCrawler/src/Backup/NWebCrawlerLib/Downloader.cs
NWebCrawler/src/Backup/NWebCrawlerLib/NWebCrawlerLib.csproj
NWebCrawler/src/Backup/NWebCrawlerLib/Parser.cs
NWebCrawler/src/Backup/NWebCrawlerLib/Program.cs
NWebCrawler/src/Backup/NWebCrawlerLib/Properties/
NWebCrawler/src/Backup/NWebCrawlerLib/Properties/AssemblyInfo.cs
NWebCrawler/src/Backup/NWebCrawlerLib/Settings.cs
NWebCrawler/src/Backup/NWebCrawlerLib/UrlFrontierQueueManager.cs
NWebCrawler/src/Backup/NWebCrawlerLib/Utility.cs
NWebCrawler/src/Backup/NWebCrawler/config.ini
NWebCrawler/src/Backup/NWebCrawler/MainForm.cs
NWebCrawler/src/Backup/NWebCrawler/MainForm.Designer.cs
NWebCrawler/src/Backup/NWebCrawler/MainForm.resx
NWebCrawler/src/Backup/NWebCrawler/NWebCrawler.csproj
NWebCrawler/src/Backup/NWebCrawler/Program.cs
NWebCrawler/src/Backup/NWebCrawler/Properties/
NWebCrawler/src/Backup/NWebCrawler/Properties/AssemblyInfo.cs
NWebCrawler/src/Backup/NWebCrawler/Properties/Resources.Designer.cs
NWebCrawler/src/Backup/NWebCrawler/Properties/Resources.resx
NWebCrawler/src/Backup/NWebCrawler/Properties/Settings.Designer.cs
NWebCrawler/src/Backup/NWebCrawler/Properties/Settings.settings
NWebCrawler/src/Backup/NWebCrawler/SettingsForm.cs
NWebCrawler/src/Backup/NWebCrawler/SettingsForm.Designer.cs
NWebCrawler/src/Backup/NWebCrawler/SettingsForm.resx
NWebCrawler/src/NWebCrawler/
NWebCrawler/src/NWebCrawler.sln
NWebCrawler/src/NWebCrawler.suo
NWebCrawler/src/NWebCrawlerLib/
NWebCrawler/src/NWebCrawlerLib/bin/
NWebCrawler/src/NWebCrawlerLib/bin/Release/
NWebCrawler/src/NWebCrawlerLib/Common/
NWebCrawler/src/NWebCrawlerLib/Common/Logger.cs
NWebCrawler/src/NWebCrawlerLib/Common/PriorityQueue.cs
NWebCrawler/src/NWebCrawlerLib/CrawleHistroyEntry.cs
NWebCrawler/src/NWebCrawlerLib/CrawlerThread.cs
NWebCrawler/src/NWebCrawlerLib/Downloader.cs
NWebCrawler/src/NWebCrawlerLib/NWebCrawlerLib.csproj
NWebCrawler/src/NWebCrawlerLib/NWebCrawlerLib.csproj.user
NWebCrawler/src/NWebCrawlerLib/obj/
NWebCrawler/src/NWebCrawlerLib/obj/Debug/
NWebCrawler/src/NWebCrawlerLib/obj/Debug/DesignTimeResolveAssemblyReferencesInput.cache
NWebCrawler/src/NWebCrawlerLib/obj/Debug/NWebCrawlerLib.csproj.FileListAbsolute.txt
NWebCrawler/src/NWebCrawlerLib/obj/Debug/NWebCrawlerLib.exe
NWebCrawler/src/NWebCrawlerLib/obj/Debug/NWebCrawlerLib.pdb
NWebCrawler/src/NWebCrawlerLib/obj/Debug/TempPE/
NWebCrawler/src/NWebCrawlerLib/Parser.cs
NWebCrawler/src/NWebCrawlerLib/Program.cs
NWebCrawler/src/NWebCrawlerLib/Properties/
NWebCrawler/src/NWebCrawlerLib/Properties/AssemblyInfo.cs
NWebCrawler/src/NWebCrawlerLib/Settings.cs
NWebCrawler/src/NWebCrawlerLib/UrlFrontierQueueManager.cs
NWebCrawler/src/NWebCrawlerLib/Utility.cs
NWebCrawler/src/NWebCrawler/bin/
NWebCrawler/src/NWebCrawler/bin/Release/
NWebCrawler/src/NWebCrawler/config.ini
NWebCrawler/src/NWebCrawler/MainForm.cs
NWebCrawler/src/NWebCrawler/MainForm.Designer.cs
NWebCrawler/src/NWebCrawler/MainForm.resx
NWebCrawler/src/NWebCrawler/NWebCrawler.csproj
NWebCrawler/src/NWebCrawler/NWebCrawler.csproj.user
NWebCrawler/src/NWebCrawler/obj/
NWebCrawler/src/NWebCrawler/obj/Debug/
NWebCrawler/src/NWebCrawler/obj/Debug/DesignTimeResolveAssemblyReferencesInput.cache
NWebCrawler/src/NWebCrawler/obj/Debug/GenerateResource-ResGen.read.1.tlog
NWebCrawler/src/NWebCrawler/obj/Debug/GenerateResource-ResGen.write.1.tlog
NWebCrawler/src/NWebCrawler/obj/Debug/NWebCrawler.csproj.FileListAbsolute.txt
NWebCrawler/src/NWebCrawler/obj/Debug/NWebCrawler.exe
NWebCrawler/src/NWebCrawler/obj/Debug/NWebCrawler.MainForm.resources
NWebCrawler/src/NWebCrawler/obj/Debug/NWebCrawler.pdb
NWebCrawler/src/NWebCrawler/obj/Debug/NWebCrawler.Properties.Resources.resources
NWebCrawler/src/NWebCrawler/obj/Debug/NWebCrawler.SettingsForm.resources
NWebCrawler/src/NWebCrawler/obj/Debug/ResolveAssemblyReference.cache
NWebCrawler/src/NWebCrawler/obj/Debug/TempPE/
NWebCrawler/src/NWebCrawler/obj/Debug/TempPE/Properties.Resources.Designer.cs.dll
NWebCrawler/src/NWebCrawler/Program.cs
NWebCrawler/src/NWebCrawler/Properties/
NWebCrawler/src/NWebCrawler/Properties/AssemblyInfo.cs
NWebCrawler/src/NWebCrawler/Properties/Resources.Designer.cs
NWebCrawler/src/NWebCrawler/Properties/Resources.resx
NWebCrawler/src/N
本网站为编程资源及源代码搜集、介绍的搜索网站,版权归原作者所有! 粤ICP备11031372号
1999-2046 搜珍网 All Rights Reserved.