ENGINEERING GUIDE · SEO FUNDAMENTALS

SEO Basics: How Search Engines Discover and Understand Websites

A practical introduction to crawling, indexing, relevance and the technical foundations every website needs.

SEO is not one setting. Search engines need to discover pages, fetch them, understand them and decide whether they are useful for a particular query. Technical SEO removes avoidable barriers; it does not guarantee rankings.

Crawling and discovery

Search engines discover URLs through links, sitemaps and other signals. Important pages should be reachable through sensible internal links. XML sitemaps provide an additional list of URLs you want crawled.

Indexing

Crawling a URL does not mean it will be indexed. Robots directives, canonicalisation, duplicate content and other signals affect indexing. Use noindex deliberately and avoid accidentally blocking important pages.

Relevance and content

Clear titles, useful headings, descriptive copy and meaningful links help machines and people understand a page. Write for the user first rather than trying to maximise keyword repetition.

Technical foundations

  • Use HTTPS consistently.
  • Provide a responsive viewport.
  • Give important pages unique titles and useful descriptions.
  • Use canonical URLs consistently.
  • Make important internal links crawlable.
  • Publish an accurate XML sitemap where appropriate.
  • Use structured data only when it accurately represents visible content.

The Ingenix SEO Checker turns many of these fundamentals into a repeatable audit.