Understanding Search Engines their robots and spiders

Understanding Search Engine Robots and Spiders

Search engines work by cataloging information contained on web sites throughout the entire WWW world wide web. Search engines cannot catalogue in detail all the information on each and every site due to the massive amounts of data online. So search engines rely on software programs called robots or spiders that are launched onto the web. Robots follow URLs from one web page to another cataloging key information about each page this information is sent back to the main search database.

Search Logs

Each and every web server maintains a set of records called logs, which provides data about the type of visitor what link they came from to get to your page etc. These logs also show if a robot or spider has been to your site. To determine whether or not your site has been visited by a search engine's robot. You can either subscribe to a Stat counter company with detailed tools that show you a log of each visit and visitor to your site. A company called Web Trends offers software that will analyze these web server logs to see if a robot has visited or not. Robots are always searching websites. If a robot visits your site there are many methods that are available to ensure that the most amount of information about your site is exchanged with the robot. Which I will discuss in other articles

Block the RobotsYou can if you like also have these robots blocked. Why would anyone want to block traffic of friends or potential customers to read your blog etc. Sometimes it depends on a Company's decision or certain information that they feel should remain not publicized etc. You can block the robots by inserting blocking text. files within your html head tags also to be discussed in other articles.

Robot Submission
If a robot or spider has not investigated your site you will not be in the search database. A robot can be forced to visit your site by completing a registration form with the search engine.
Below are 3 major Search Engine submit pages. Google, Baidu and Bing
Url: http://www.bing.com/toolbox/submiturl
Url: www.google.com/addurl/?continue=/addurl

Url: www.baidu.com/search/url_submit.html

Free Submitters
I am of course old school and I suggest you find each online search engine submit page and manualy submit your site to each respective search engine but,
There are a few auto search engine submitting sites below that I know of. They will submit your site for you. Some are totally free others offer free submission and a paid package as well

Url: http://submissionurl.com/

Url: http://www.scrubtheweb.com/cgi-bin/addurl.cgi
WEB LAUNCH
Url: www.web-launch.com/submiturlfree.html
TRAFFIC ZAPP
Url: www.trafficzap.com/searchsubmit.php
Fast Submit Pro

Url:http://www.evrsoft.com/fastsubmit/myezsubmit.cgi

Or if you like you can automatically ping your site or blog with a service called ping-o-matic

Url:http://pingomatic.com/
Optimize itBut before you submit: your site please make sure it is thoroughly keyword (SEO) Search engined optimized.
If you need help with SEO there is a site listed below with a lot powerful free optimization tools and instructional videos.
Keyword Country
KEYWORD COUNTRY
Did you find this article helpful or useful? Please feel free to leave a comment



_____________________________
Secret: Niche Sites Sell
_________________________________________________________
This Online Article is by Dusty Crawford email: turnkeywebsite@ymail.com
A website owner and blogger. A mentoring program of sorts designed to teach the serious blogger or website owner to improve their quality of life through residual income from internet businesses. part time or full time

Peter Tramayne'sZero to 50 in 30 days!!

___________________________________________

Sponsor

Dropshiping Wholesalers