Post by rakibsakibislam on Mar 13, 2024 22:04:43 GMT -6
Google launches a new crawler called GoogleOther, used by the Big G team to crawl the public web Google , in this period, is in continuous development, to guarantee its users an increasingly better experience on its search engine and therefore on the web pages that are "proposed" by Big G in SERP. To allow Googlebot to do a better crawling job, the company announces that it has launched another bot called GoogleOther, whose objective is to lend support to the other bot to scan the public web , and therefore the pages of websites found all over the world. inside the web.
The launch of this new web crawler was announced Find List by Google's Gary Illyes via LinkedIn and the news appeared on all specialized websites, including Search Engine Land . What do we know about GoogleOther? It is not clear whether he is linked to Google Bard or not, but what we know about this bot was described by the company spokesperson himself via social media and through the Google page (which added GoogleOther to its documentation on crawlers): “Generic crawler that can be used by various product teams to retrieve publicly accessible content from sites.
For example, it can be used for one-time crawls for internal research and development. The GoogleOther crawler always obeys the rules in the robots.txt file for the user agent token and global user agent (*) and uses the same IP ranges as Googlebot.” Why the new bot? This new bot will “eventually take some of the strain off Googlebot,” Illyes explained: “As we optimize how and what Googlebot crawls, one thing we wanted to ensure is that Googlebot's crawling processes are only used internally to create the index used by Search. So we've added a new crawler, GoogleOther, which will replace some of Googlebot's other jobs like R&D crawlers to free up some of the crawling capacity for Googlebot." Google will use it for internal purposes.
The launch of this new web crawler was announced Find List by Google's Gary Illyes via LinkedIn and the news appeared on all specialized websites, including Search Engine Land . What do we know about GoogleOther? It is not clear whether he is linked to Google Bard or not, but what we know about this bot was described by the company spokesperson himself via social media and through the Google page (which added GoogleOther to its documentation on crawlers): “Generic crawler that can be used by various product teams to retrieve publicly accessible content from sites.
For example, it can be used for one-time crawls for internal research and development. The GoogleOther crawler always obeys the rules in the robots.txt file for the user agent token and global user agent (*) and uses the same IP ranges as Googlebot.” Why the new bot? This new bot will “eventually take some of the strain off Googlebot,” Illyes explained: “As we optimize how and what Googlebot crawls, one thing we wanted to ensure is that Googlebot's crawling processes are only used internally to create the index used by Search. So we've added a new crawler, GoogleOther, which will replace some of Googlebot's other jobs like R&D crawlers to free up some of the crawling capacity for Googlebot." Google will use it for internal purposes.