What effect will this line in robots.txt have on Google's crawling behavior? # Disallow: /
Choose an answer
Tap an option to check your answer.
Correct answer: It will not have any effect on crawling.
Why this is the answer
In robots.txt , a line that begins with # is treated as a comment, not as an active crawler directive. HubSpot’s documentation shows comment lines such as # Block all bots, followed separately by real crawl rules like Disallow entries. Because the slash rule here appears after #, Google would ignore it rather than apply it. Actual crawl blocking only happens when Disallow is written as an active directive under a matching user-agent. Developers HubSpot+1
Pass your exam — without the endless answer hunt
Get every verified question and explanation for this exam in one place, and save hours of prep. 1,000+ certifications · 20+ languages · free to start.
Pass your exam faster → No card needed