What are the names of few crawler for different search engines and also mention

  • 0 replies
  • 158 views
*

Offline mechanic

  • Engineer
  • *****
  • 3420
  • Gender: Male
  • Engineer
    • View Profile
    • Engineering Forum
What are the names of few crawler for different search engines and also mention its function?


Answer:


The term Crawling or web crawler is referred to an automated process through which different search engines filtrate web pages for proper indexing. Usually, the crawler used to go through different web pages because of the relevant keywords, content, and hyperlink. There are few crawlers for different search engines like:-

Google: – Google Bots crawler.

Yahoo: – Yahoo Slurp crawler.

Bing: – Msnbot crawler

Their main function is to go through from other linked pages on websites, usually, companies build their sitemaps for better accessibility and navigation.