Google now documents Search, Gemini, Shopping and other products through a shared crawling infrastructure. One key distinction for SEO and GEO is that Googlebot and Google-Extended do different jobs.
Googlebot
Googlebot crawls content for Google Search. Blocking important pages can limit Google’s ability to discover, understand and index them.
Google-Extended
Google describes Google-Extended as a control token for how published content may be used by Gemini apps and the Vertex AI API for Gemini. It is not the same as Googlebot and does not function as the standard Search crawler.
Can they be controlled separately?
Yes. A site can express preferences for Google-Extended without blocking Googlebot. This distinction matters because an overly broad robots.txt rule can have unintended SEO consequences.
GEO implications
Businesses should separate goals for Google Search visibility, generative Search visibility and Gemini availability. Crawl policy should follow those goals instead of treating every AI-related crawler as identical.
Sources
Google Crawling Infrastructure and Google AI optimization guide.