هل هناك الخطاء في ملف robots.txt

اسامه العراقي كتب:

السلام عليكم

مادفعني لكتابة الموضوع هو وجود مواضيع لم يتم فهرستها

لاحظت ذلك من خلال حساب المنتدى في ادوات مشرفي الموقع

المثال

اجمالي ماتمت فهرسته 968

تم حضرها من قبل ملف الروبوت تكست 567

ارجو الاطلاع على الملف وتبيان اذا ماكان خطاء به

الكود:

User-agent: Mediapartners-Google
Disallow:

User-agent: Mediapartners-Google*
Disallow:

User-agent: *
Disallow: /abuse
Disallow: /admgt/
Disallow: /contact
Disallow: /donate
Disallow: /go/
Disallow: /login
Disallow: /modcp
Disallow: /post
Disallow: /privmsg
Disallow: /spa/
Disallow: /sta/

User-agent: 008
User-agent: Accoona
User-agent: aipbot
User-agent: aipbot*
User-agent: aipbot/1.0
User-agent: Alexa
User-agent: Alexa Bitlybot
User-agent: Alexibot
User-agent: AltaVista Intranet V2.0 AVS EVAL search@freeit.com
User-agent: AltaVista Intranet V2.0 Compaq Altavista Eval sveand@altavista.net
User-agent: AltaVista Intranet V2.0 evreka.com crawler@evreka.com
User-agent: AltaVista V2.0B crawler@evreka.com
User-agent: Anonymous
User-agent: ApocalXExplorerBot
User-agent: appie
User-agent: Aqua_Products
User-agent: Argus/1.1
User-agent: Artabus
User-agent: Ask Jeeves
User-agent: asterias
User-agent: atSpider
User-agent: attentio
User-agent: AV Fetch 1.0
User-agent: AVSearch-3.0(AltaVista/AVC)
User-agent: AWS Cloud Based
User-agent: b2w
User-agent: b2w/0.1
User-agent: BackDoorBot
User-agent: BackDoorBot/1.0
User-agent: Baiduspider
User-agent: becomebot
User-agent: BecomeBot
User-agent: BigBrother
User-agent: BIGLOTRON (BETA 2;GNU/Linux)
User-agent: BizInformation
User-agent: Black Hole
User-agent: Black.Hole
User-agent: BlackWidow
User-agent: BlowFish
User-agent: BlowFish/1.0
User-agent: BoardPulse
User-agent: boitho.com-dc
User-agent: Bookmark search tool
User-agent: bot/1.0
User-agent: BotALot
User-agent: Bot mailto:craftbot@yahoo.com
User-agent: BotRightHere
User-agent: BrandProtect
User-agent: BuiltBotTough
User-agent: Bullseye
User-agent: Bullseye/1.0
User-agent: BunnySlippers
User-agent: CazoodleBot
User-agent: Cegbfeieh
User-agent: cfetch
User-agent: cfetch/1.0
User-agent: CheeseBot
User-agent: CherryPicker
User-agent: CherryPicker /1.0
User-agent: CherryPickerElite/1.0
User-agent: CherryPickerSE/1.0
User-agent: ChinaClaw
User-agent: Collage
User-agent: cometrics-bot
User-agent: complex_network_group
User-agent: convera
User-agent: ConveraCrawler
User-agent: ConveraCrawler/0.2
User-agent: ConveraCrawler/0.9d
User-agent: Convera Internet Spider V6.x
User-agent: ConveraMultiMediaCrawler/0.1
User-agent: Copernic
User-agent: CopyRightCheck
User-agent: cosmos
User-agent: Crescent
User-agent: Crescent Internet ToolPak HTTP OLE Control v.1.0
User-agent: Crescent Internet ToolPak HTTPOLE Control v.1.0
User-agent: Curl
User-agent: Custo
User-agent: CydralSpider
User-agent: Deepnet Explorer
User-agent: default.ida
User-agent: DigExt
User-agent: DISCo
User-agent: discobot
User-agent: DISCoFinder
User-agent: DISCo Pump
User-agent: DISCo Pump 3.0
User-agent: DISCo Pump 3.1
User-agent: DISCo Pump 3.2
User-agent: DittoSpyder
User-agent: DOC
User-agent: dotbot
User-agent: DotBot
User-agent: DotBot/1.1
User-agent: Download Demon
User-agent: Download Demon/3.2.0.8
User-agent: Download Demon/3.5.0.11
User-agent: Download Ninja
User-agent: Download Wonder
User-agent: DSurf
User-agent: Dulance bot
User-agent: dumbot
User-agent: eCatch
User-agent: eCatch/3.0
User-agent: echo!
User-agent: EchO!/2.0
User-agent: EirGrabber
User-agent: EliteSys Entry
User-agent: EmailCollector
User-agent: Email Extractor
User-agent: EmailSiphon
User-agent: EmailSmartz
User-agent: EmailWolf
User-agent: Enterprise_Search
User-agent: Enterprise_Search/1.0
User-agent: EroCrawler
User-agent: es
User-agent: ESIRover
User-agent: e-SocietyRobot
User-agent: Exabot
User-agent: Exabot/2.0
User-agent: Exabot-Images
User-agent: Express WebPictures
User-agent: Express WebPictures (www.express-soft.com)
User-agent: ExtractorPro
User-agent: EyeNetIE
User-agent: FairAd Client
User-agent: Fairshare
User-agent: Fasterfox
User-agent: Fetch
User-agent: Flaming AttackBot
User-agent: Flamingo_SearchEngine
User-agent: FlashGet
User-agent: FlashGet WebWasher 3.2
User-agent: Foobot
User-agent: FreeFind
User-agent: FreeWebMonitoring SiteChecker/0.1
User-agent: FrontPage
User-agent: FrontPage [NC,OR]
User-agent: FurlBot
User-agent: Gaisbot
User-agent: Gaisbot/3.0
User-agent: GetBot
User-agent: GetRight
User-agent: GetRight/2.11
User-agent: GetRight/3.1
User-agent: GetRight/3.2
User-agent: GetRight/3.3
User-agent: GetRight/3.3.3
User-agent: GetRight/3.3.4
User-agent: GetRight/4.0.0
User-agent: GetRight/4.1.0
User-agent: GetRight/4.1.1
User-agent: GetRight/4.1.2
User-agent: GetRight/4.2
User-agent: GetRight/4.2b (Portuguxeas)
User-agent: GetRight/4.2c
User-agent: GetRight/4.3
User-agent: GetRight/4.5
User-agent: GetRight/4.5a
User-agent: GetRight/4.5b
User-agent: GetRight/4.5b1
User-agent: GetRight/4.5b2
User-agent: GetRight/4.5b3
User-agent: GetRight/4.5b6
User-agent: GetRight/4.5b7
User-agent: GetRight/4.5c
User-agent: GetRight/4.5d
User-agent: GetRight/4.5e
User-agent: GetRight/5.0beta1
User-agent: GetRight/5.0beta2
User-agent: GetUrl
User-agent: GetWeb!
User-agent: Gigabot
User-agent: Go-Ahead-Got-It
User-agent: Go!Zilla
User-agent: Go!Zilla 3.3 (www.gozilla.com)
User-agent: Go!Zilla 3.5 (www.gozilla.com)
User-agent: Go!Zilla (www.gozilla.com)
User-agent: GrabNet
User-agent: Grafula
User-agent: grub
User-agent: grub-client
User-agent: Hackertarget.com
User-agent: Harvest
User-agent: Harvest/1.5
User-agent: Hatena Antenna
User-agent: HavIndex
User-agent: heritrix
User-agent: hloader
User-agent: HMView
User-agent: httplib
User-agent: httrack
User-agent: HTTrack
User-agent: HTTrack 3.0
User-agent: HTTrack 3.0x
User-agent: HTTrack [NC,OR]
User-agent: humanlinks
User-agent: ichiro
User-agent: IconSurf
User-agent: Igentia
User-agent: Image Collector
User-agent: Image Stripper
User-agent: Image Sucker
User-agent: Indy Library
User-agent: Indy Library [NC,OR]
User-agent: InfoNaviRobot
User-agent: InfoSpiders
User-agent: InterGET
User-agent: Internet Explore
User-agent: Internet Ninja
User-agent: Internet Ninja 4.0
User-agent: Internet Ninja 5.0
User-agent: Internet Ninja 6.0
User-agent: InternetSupervision
User-agent: IRLbot
User-agent: Iron
User-agent: Iron33/1.0.2
User-agent: Jeeves
User-agent: JennyBot
User-agent: Jetbot
User-agent: Jetbot/1.0
User-agent: JetCar
User-agent: Jobo
User-agent: JOC Web Spider
User-agent: kalooga
User-agent: KDD Exploror
User-agent: Kenjin Spider
User-agent: Kenjin.Spider
User-agent: Keyword Density
User-agent: Keyword.Density
User-agent: Keyword Density/0.9
User-agent: larbin
User-agent: Larbin
User-agent: larbin_2.6.2 (kabura@sushi.com)
User-agent: larbin_2.6.2 kabura@sushi.com
User-agent: larbin_2.6.2 (larbin2.6.2@unspecified.mail)
User-agent: larbin_2.6.2 larbin2.6.2@unspecified.mail
User-agent: larbin_2.6.2 larbin@correa.org
User-agent: larbin_2.6.2 listonATccDOTgatechDOTedu
User-agent: larbin_2.6.2 (listonATccDOTgatechDOTedu)
User-agent: larbin_2.6.2 (vitalbox1@hotmail.com)
User-agent: larbin_2.6.2 vitalbox1@hotmail.com
User-agent: larbin (samualt9@bigfoot.com)
User-agent: larbin samualt9@bigfoot.com
User-agent: LBot
User-agent: LeechFTP
User-agent: LexiBot
User-agent: libWeb/clsHTTP
User-agent: libWeb/clsHTTPDisallow: /
User-agent: libwww
User-agent: LightningDownload
User-agent: Linguee
User-agent: LinkedIn
User-agent: LinkextractorPro
User-agent: Linknzbot
User-agent: Linknzbot*
User-agent: Linknzbot 2004
User-agent: LinkScan
User-agent: LinkScan/8.1a Unix
User-agent: LinkScan/8.1a.Unix
User-agent: LinkScan/8.1a Unix Disallow: /
User-agent: linksmanager
User-agent: LinksManager
User-agent: LinksManager.com_bot
User-agent: LinkWalker
User-agent: LjSEEK
User-agent: LNSpiderguy
User-agent: looksmart
User-agent: LWP
User-agent: LWP*
User-agent: lwp-trivial
User-agent: lwp-trivial/1.34
User-agent: magpie-crawler
User-agent: Mail Sweeper
User-agent: Marketwirebot
User-agent: Mass Downloader
User-agent: Mass Downloader/2.2
User-agent: Mata Hari
User-agent: Mata.Hari
User-agent: MetagerBot
User-agent: MetaURI
User-agent: Microsoft.URL
User-agent: Microsoft URL Control
User-agent: Microsoft URL Control*
User-agent: Microsoft.URL.Control
User-agent: Microsoft URL Control - 5.01.4511
User-agent: Microsoft URL Control - 6.00.8169
User-agent: Microsoft URL Control - 6.01.9782
User-agent: MIDown tool
User-agent: MIIxpc
User-agent: MIIxpc/4.2
User-agent: Missigua Locator
User-agent: Mister PiX
User-agent: Mister.PiX
User-agent: Mister Pix II 2.01
User-agent: Mister Pix II 2.02a
User-agent: Mister PiX version.dll
User-agent: MJ12bot
User-agent: MLBot
User-agent: moget
User-agent: moget/2.1
User-agent: mozilla
User-agent: Mozilla
User-agent: Mozilla/2.0 (compatible; Ask Jeeves)
User-agent: mozilla/3
User-agent: mozilla/4
User-agent: Mozilla/4.0 (compatible; BullsEye; Windows 95)
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 2000)
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 95)
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 98)
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows ME)
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows NT)
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows XP)
User-agent: mozilla/5
User-agent: MRSPUTNIK
User-agent: MSIECrawler
User-agent: MSRBOT
User-agent: MS Search 4.0 Robot
User-agent: MS Search 5.0 Robot
User-agent: munky
User-agent: naver
User-agent: Naverbot
User-agent: NaverBot
User-agent: NaverBot-1.0
User-agent: Navroad
User-agent: NearSite
User-agent: NetAnts
User-agent: NetAnts/1.10
User-agent: NetAnts/1.23
User-agent: NetAnts/1.24
User-agent: NetAnts/1.25
User-agent: NetAttache
User-agent: NetAttache Light 1.1
User-agent: Netcraft Web Server Survey
User-agent: NetMechanic
User-agent: NetSpider
User-agent: Net Vampire
User-agent: Net Vampire/3.0
User-agent: NetZIP
User-agent: NetZip-Downloader
User-agent: NetZip-Downloader/1.0.62 (Win32; Dec 7 1998)
User-agent: NetZip Downloader 1.0 Win32(Nov 12 1998)
User-agent: NetZippy+(http://www.innerprise.net/usp-spider.asp)
User-agent: NetZippy+(http:/www.innerprise.net/usp-spider.asp)
User-agent: NICErsPRO
User-agent: NimbleCrawler
User-agent: NPbot
User-agent: NPBot
User-agent: NPBot/3
User-agent: Nutch
User-agent: Nutch*
User-agent: NutchCVS/0.06-dev
User-agent: NutchCVS/0.7.1
User-agent: NutchOrg
User-agent: oBot
User-agent: Ocelli
User-agent: Octopus
User-agent: Offline Explorer
User-agent: Offline.Explorer
User-agent: Offline Explorer/1.2
User-agent: Offline Explorer/1.4
User-agent: Offline Explorer/1.6
User-agent: Offline Explorer/1.7
User-agent: Offline Explorer/1.9
User-agent: Offline Explorer/2.0
User-agent: Offline Explorer/2.1
User-agent: Offline Explorer/2.3
User-agent: Offline Explorer/2.4
User-agent: Offline Explorer/2.5
User-agent: Offline Navigator
User-agent: OmniExplorer_Bot
User-agent: oneriot
User-agent: Openbot
User-agent: Openfind
User-agent: Openfind data gathere
User-agent: Openfind data gatherer
User-agent: Oracle Ultra Search
User-agent: OutfoxBot/0.5
User-agent: PageGrabber
User-agent: Papa Foto
User-agent: pavuk
User-agent: PBWF
User-agent: pcBrowser
User-agent: penthesilea
User-agent: PerMan
User-agent: PGBot
User-agent: PhpDig
User-agent: Pingdom GIGRIB (http://www.pingdom.com)
User-agent: postrank
User-agent: ProPowerBot
User-agent: ProPowerBot/2.14
User-agent: ProWebWalker
User-agent: psbot
User-agent: psycheclone
User-agent: Psycheclone
User-agent: Python-urllib
User-agent: QuepasaCreep
User-agent: QueryN Metasearch
User-agent: QueryN.Metasearch
User-agent: radian6 comment reader
User-agent: radian6 Feedfetcher
User-agent: Radiation Retriever
User-agent: Radiation Retriever 1.1
User-agent: RB2B-bot
User-agent: RealDownload
User-agent: RealDownload/4.0.0.40
User-agent: RealDownload/4.0.0.41
User-agent: RealDownload/4.0.0.42
User-agent: ReGet
User-agent: RepoMonkey
User-agent: RepoMonkey Bait & Tackle/v1.01
User-agent: RepoMonkey Bait & Tackle
User-agent: RepoMonkey Bait & Tackle/v1.01
User-agent: research-spider
User-agent: RMA
User-agent: Robozilla
User-agent: Roverbot
User-agent: RufusBot
User-agent: sbider
User-agent: Scooter/1.0
User-agent: Scooter/1.0 scooter@pa.dec.com
User-agent: Scooter/1.1 (custom)
User-agent: Scooter/2.0 G.R.A.B. V1.1.0
User-agent: Scooter/2.0 G.R.A.B. X2.0
User-agent: Scooter2_Mercator_x-x.0
User-agent: Scooter-3.0.EU
User-agent: Scooter-3.0.FS
User-agent: Scooter-3.0.HD
User-agent: Scooter-3.0QI
User-agent: Scooter-3.0.VNS
User-agent: Scooter-3.2
User-agent: Scooter-3.2.BT
User-agent: Scooter-3.2.DIL
User-agent: Scooter-3.2.EX
User-agent: Scooter-3.2.JT
User-agent: Scooter-3.2.NIV
User-agent: Scooter-3.2.SF0
User-agent: Scooter-3.2.snippet
User-agent: Scooter/3.3
User-agent: Scooter-3.3dev
User-agent: Scooter/3.3.QA.pczukor
User-agent: Scooter/3.3_SF
User-agent: Scooter/3.3.vscooter
User-agent: Scooter-ARS-1.1
User-agent: Scooter-ARS-1.1-ih
User-agent: Scooter_bh0-3.0.3
User-agent: Scooter_trk3-3.0.3
User-agent: scooter-venus-3.0.vns
User-agent: Scooter-W3-1.0
User-agent: Scooter-W3.1.2
User-agent: Scrubby
User-agent: SearchDaimon.com-dc
User-agent: searchpreview
User-agent: seekbot
User-agent: Seekbot
User-agent: Seekbot/1.0
User-agent: SEOprofiler
User-agent: Shai'Hulud
User-agent: Shim-Crawler
User-agent: ShopWiki
User-agent: ShopWiki/1.0
User-agent: SightupBot
User-agent: SiteBot
User-agent: SiteSnagger
User-agent: Slurp China
User-agent: SlySearch
User-agent: SmartDownload
User-agent: SmartDownload/1.2.76 (Win32; Apr 1 1999)
User-agent: SmartDownload/1.2.77 (Win32; Aug 17 1999)
User-agent: SmartDownload/1.2.77 (Win32; Feb 1 2000)
User-agent: SmartDownload/1.2.77 (Win32; Jun 19 2001)
User-agent: Snapbot
User-agent: Snappy
User-agent: Softlayer Server
User-agent: Sogou web spider
User-agent: sootle
User-agent: sosospider
User-agent: SpankBot
User-agent: spanner
User-agent: Speedy
User-agent: SpiderBot
User-agent: Sqworm
User-agent: Sqworm/2.9.85-BETA (beta_release; 20011115-775; i686-pc-linux
User-agent: ssearcher100
User-agent: Stanford
User-agent: Stanford Comp Sci
User-agent: suggybot
User-agent: SuperBot
User-agent: SuperBot/2.6
User-agent: SuperBot/3.0 (Win32)
User-agent: SuperBot/3.1 (Win32)
User-agent: SuperHTTP
User-agent: SuperHTTP/1.0
User-agent: Surfbot
User-agent: SurveyBot
User-agent: suzuran
User-agent: Szukacz
User-agent: Szukacz/1.4
User-agent: tAkeOut
User-agent: Teleport
User-agent: TeleportPro
User-agent: Teleport Pro
User-agent: Teleport Pro/1.29
User-agent: Teleport Pro/1.29.1590
User-agent: Teleport Pro/1.29.1634
User-agent: Teleport Pro/1.29.1718
User-agent: Teleport Pro/1.29.1820
User-agent: Teleport Pro/1.29.1847
User-agent: Telesoft
User-agent: Templeton
User-agent: Teoma
User-agent: The Intraformant
User-agent: The.Intraformant
User-agent: TheNomad
User-agent: TightTwatBot
User-agent: Titan
User-agent: toCrawl
User-agent: toCrawl/UrlDispatcher
User-agent: True_Robot
User-agent: True_Robot/1.0
User-agent: turingos
User-agent: TurnitinBot
User-agent: TurnitinBot/1.5
User-agent: Tweetmeme
User-agent: TwengaBot
User-agent: Twiceler
User-agent: URL Control
User-agent: UrlDispatcher
User-agent: ://URLFAN
User-agent: URL_Spider_Pro
User-agent: URLy Warning
User-agent: URLy.Warning
User-agent: VCI
User-agent: VCI WebViewer VCI WebViewer Win32
User-agent: vobsub
User-agent: VoidEYE
User-agent: vscooter
User-agent: w3mir
User-agent: WatchDog/3.0
User-agent: WebAuto
User-agent: WebAuto/3.40 (Win98; I)
User-agent: WebBandit
User-agent: WebBandit/3.50
User-agent: WebCapture
User-agent: WebCapture 2.0
User-agent: WebCatcher
User-agent: webcopier
User-agent: WebCopier
User-agent: WebCopier v.2.2
User-agent: WebCopier v2.5
User-agent: WebCopier v2.6
User-agent: WebCopier v2.7a
User-agent: WebCopier v2.8
User-agent: WebCopier v3.0
User-agent: WebCopier v3.0.1
User-agent: WebCopier v3.2
User-agent: WebCopier v3.2a
User-agent: webcopy
User-agent: WebCopy
User-agent: webcrawl.net
User-agent: WebEmailExtrac
User-agent: WebEMailExtrac.*
User-agent: WebEnhancer
User-agent: WebFetch
User-agent: webfetch/2.1.0
User-agent: WebFetcher
User-agent: WebGo IS
User-agent: Web Image Collector
User-agent: Web.Image.Collector
User-agent: WebLeacher
User-agent: WebmasterWorld Extractor
User-agent: WebmasterWorldForumBot
User-agent: webmirror
User-agent: WebMirror
User-agent: WebReaper
User-agent: Web Reaper
User-agent: WebReaper [info@webreaper.net]
User-agent: WebReaper v9.1 - www.otway.com/webreaper
User-agent: WebReaper v9.7 - www.webreaper.net
User-agent: WebReaper v9.8 - www.webreaper.net
User-agent: WebReaper vWebReaper v7.3 - www,otway.com/webreaper
User-agent: WebReaper [webreaper@otway.com]
User-agent: WebSauger
User-agent: WebSauger 1.20b
User-agent: WebSauger 1.20j
User-agent: WebSauger 1.20k
User-agent: website extractor
User-agent: Website eXtractor
User-agent: Website eXtractor (http:/www.asona.org)
User-agent: Website Quester
User-agent: Website.Quester
User-agent: Website Quester - www.asona.org
User-agent: Website Quester - www.esalesbiz.com/extra/
User-agent: Webster Pro
User-agent: Webster.Pro
User-agent: WebStripper
User-agent: WebStripper/2.02
User-agent: WebStripper/2.03
User-agent: WebStripper/2.10
User-agent: WebStripper/2.12
User-agent: WebStripper/2.13
User-agent: WebStripper/2.15
User-agent: WebStripper/2.16
User-agent: WebStripper/2.19
User-agent: Web Sucker
User-agent: webvac
User-agent: WebVac
User-agent: WebVulnCrawl
User-agent: WebVulnScan
User-agent: WebWalk
User-agent: WebWasher
User-agent: WebWhacker
User-agent: WebZip
User-agent: WebZIP
User-agent: WebZIP/2.75 (http://www.spidersoft.com)
User-agent: WebZIP/2.75 (http:/www.spidersoft.com)
User-agent: WebZIP/3.65 (http://www.spidersoft.com)
User-agent: WebZIP/3.65 (http:/www.spidersoft.com)
User-agent: WebZIP/3.80 (http://www.spidersoft.com)
User-agent: WebZIP/3.80 (http:/www.spidersoft.com)
User-agent: WebZip/4.0
User-agent: WebZIP/4.0 (http://www.spidersoft.com)
User-agent: WebZIP/4.0 (http:/www.spidersoft.com)
User-agent: WebZIP/4.1 (http://www.spidersoft.com)
User-agent: WebZIP/4.1 (http:/www.spidersoft.com)
User-agent: WebZIP/4.21
User-agent: WebZIP/4.21 (http://www.spidersoft.com)
User-agent: WebZIP/4.21 (http:/www.spidersoft.com)
User-agent: WebZIP/5.0
User-agent: WebZIP/5.0 (http://www.spidersoft.com)
User-agent: WebZIP/5.0 (http:/www.spidersoft.com)
User-agent: WebZIP/5.0 PR1 (http://www.spidersoft.com)
User-agent: WebZIP/5.0 PR1 (http:/www.spidersoft.com)
User-agent: wget
User-agent: wGet
User-agent: Wget
User-agent: Wget/1.10.2
User-agent: Wget/1.5.2
User-agent: Wget/1.5.3
User-agent: Wget/1.6
User-agent: Wget/1.7
User-agent: Wget/1.8
User-agent: Wget/1.8.1
User-agent: Wget/1.8.1+cvs
User-agent: Wget/1.8.2
User-agent: Wget/1.9-beta
User-agent: whitevector crawler
User-agent: Whitevector+Crawler
User-agent: Widow
User-agent: WikioFeedBot
User-agent: wikiwix-bot-3.0
User-agent: Willow
User-agent: WinHTTrack
User-agent: Wise-Guys
User-agent: woozweb-monitoring
User-agent: woriobot
User-agent: WWW-Collector
User-agent: WWW-Collector-E
User-agent: WWWOFFLE
User-agent: Xaldon WebSpider
User-agent: Xaldon WebSpider 2.5.b3
User-agent: Xenu
User-agent: Xenu Link Sleuth
User-agent: Xenu's
User-agent: Xenu's Link Sleuth 1.1c
User-agent: xGet
User-agent: Yahoo-MMCrawler
User-agent: YahooSeeker/CafeKelsa
User-agent: Yeti
User-agent: YodaoBot
User-agent: YRSPider
User-agent: Zao
User-agent: Zealbot
User-agent: Zeus
User-agent: Zeus 11389 Webster Pro V2.9 Win32
User-agent: Zeus 11652 Webster Pro V2.9 Win32
User-agent: Zeus 18018 Webster Pro V2.9 Win32
User-agent: Zeus 26378 Webster Pro V2.9 Win32
User-agent: Zeus 30747 Webster Pro V2.9 Win32
User-agent: Zeus 32297 Webster Pro V2.9 Win32
User-agent: Zeus 39206 Webster Pro V2.9 Win32
User-agent: Zeus 41641 Webster Pro V2.9 Win32
User-agent: Zeus 44238 Webster Pro V2.9 Win32
User-agent: Zeus 51070 Webster Pro V2.9 Win32
User-agent: Zeus 51674 Webster Pro V2.9 Win32
User-agent: Zeus 51837 Webster Pro V2.9 Win32
User-agent: Zeus 63567 Webster Pro V2.9 Win32
User-agent: Zeus 6694 Webster Pro V2.9 Win32
User-agent: Zeus 71129 Webster Pro V2.9 Win32
User-agent: Zeus 82016 Webster Pro V2.9 Win32
User-agent: Zeus 82900 Webster Pro V2.9 Win32
User-agent: Zeus 84842 Webster Pro V2.9 Win32
User-agent: Zeus 90872 Webster Pro V2.9 Win32
User-agent: Zeus 94934 Webster Pro V2.9 Win32
User-agent: Zeus 95245 Webster Pro V2.9 Win32
User-agent: Zeus 95351 Webster Pro V2.9 Win32
User-agent: Zeus 97371 Webster Pro V2.9 Win32
User-agent: Zeus Link Scout
User-agent: ZyBorg
Disallow: /

Sitemap: http://asama.lamuntada.com/sitemap.xml

وشكرا لكم مقدما

الزعيم كتب:السلام عليكم
 
كل مرة تفتح مثل هذه المواضيع بالضبط و كل مرة اكرر نفس الكلام بالضبط. يا اخوة من الطبيعي ان تسجل المواقع اخطاء زحف بالعشراء و حتى بالآلاف و هي تظهر و تختفي و نحن نتأقلم معها من عندنا دون الحاجة الى مثل هذه المواضيع التي قد تخيف من ليس له علم بخبايا الامور.
 
تخطاء الزحف لن تعيق منتداك في شيء فهناك مواقع تسجل آلاف اخطاء الزحف و هي الاولى في الارشفة على كلماتها الدليلية و زوارها بعشرات الآلاف يومياً. فكلما زاد المحتوى و تصفحه زاد معه احتمال وقوع خطئ زحف و ما خطئ الزحف الا روبوت غوغل حصل على رابط و لما اراد ان يدخل اليه لم يستطع لا اقل و لا اكثر و قد يكون الامر حذف موضوع او غيره...
 
المرجو ان يكون الامر قد اتضح نهائياً