HTML · The document layer
What is the difference between canonical tags and robots directives?
Short answer
A canonical link names the preferred URL for duplicate content, while a robots meta tag tells crawlers whether to index or follow a page. robots.txt controls crawling, not indexing.
Canonical and robots solve different problems. Canonical is about consolidation: when several URLs show the same or similar content, it points to the one that should be treated as primary. Robots directives are about access and indexing.
<link rel="canonical" href="https://example.com/questions/html" />
<meta name="robots" content="noindex, follow" />The distinctions
- rel=canonical is a hint that consolidates ranking signals to one URL
- meta robots noindex asks search engines not to index the page
- meta robots nofollow asks them not to follow links on the page
- robots.txt controls which URLs a crawler may fetch, not whether they are indexed
- A page blocked by robots.txt cannot be read, so its noindex directive cannot be seen
Canonical tags should point to an absolute, stable URL. Every page in a duplicate set should usually point to the same canonical, and the canonical should return the preferred content directly.
Follow-ups you should expect
Interviewers use these to check whether the first answer was understanding or memorisation.
- 1Does robots.txt prevent indexing?
- 2Can a canonical tag point to a different domain?
- 3What does noindex, follow mean?
- 4What happens if a canonical URL is blocked?
Say it in the interview
A canonical tag tells search engines which URL is the preferred version when several pages have similar content. A robots meta tag controls whether a page should be indexed or whether its links should be followed. robots.txt is different again: it controls crawling, not indexing. I use canonical for consolidation and robots directives for indexing intent, and I avoid blocking a page in robots.txt if I need its noindex directive to be seen.
Use this as a model, not a script. Saying it in your own words is what makes it survive follow-up pressure.
Related questions
Reading is step one
Now say it out loud.
Understanding an answer and delivering it under pressure are different skills. Rehearse this topic in a live HTML mock interview and get scored feedback on depth, clarity and confidence.