Do 404 Errors Waste Crawl Budget?

"404 errors waste crawl budget" is one of the most repeated SEO claims. It is partly true, mostly misunderstood, and it leads people to make bad decisions, such as redirecting every missing URL to the homepage.

What crawl budget means

Crawl budget is the number of URLs a search engine is willing and able to crawl on your site in a given period. It depends on how fast your server responds and how much the search engine wants to crawl your content.

According to Google's documentation, crawl budget is mainly a concern for very large sites (hundreds of thousands to millions of pages) or sites with content that changes very frequently. A typical business site, blog or small store rarely has a crawl budget problem.

How search engines treat real 404s

A proper 404 or 410 is a clear signal that a page doesn't exist. Google keeps checking such URLs occasionally, but it reduces how often it crawls them over time. A 404 also doesn't lower the ranking of your other pages.

In other words, correctly returned 404s are cheap. They are not the main source of crawl waste.

What actually wastes crawl budget

Soft 404s. Pages that say "not found" but return 200 keep looking like real pages, so they keep getting crawled. See 404 vs Soft 404 vs 410 Explained.

Redirect chains. Every hop is an extra request. Long chains slow crawling and can be abandoned before the final page.

Infinite URL spaces. Faceted navigation, calendar pages, session IDs and endless parameter combinations generate countless URLs with little unique content.

Duplicate URL variants. Uppercase, trailing slashes and protocol variations multiply the same content; see our post on URL normalization.

Slow server responses. If your server responds slowly or with 5xx errors, search engines crawl less.

Common mistakes made in the name of crawl budget

• Redirecting all 404s to the homepage. This turns real 404s into soft 404s and creates more confusion. See Why Redirecting Every 404 to the Homepage Is a Mistake.
• Blocking 404 URLs in robots.txt. If crawlers can't access the URL, they can't see that it returns 404, so it may linger in reports longer.
• Returning 200 with an empty page to "avoid errors."

When 404s do deserve attention

404s matter when they come from your own site or valuable sources:

• internal links and navigation pointing to dead URLs,
• sitemaps listing removed pages,
• URLs with backlinks or real visitor traffic,
• important pages returning 404 by mistake.

These are problems for users and link value first, and crawl efficiency second. Prioritize them as described in How to Find and Prioritize Broken Links.

Summary

For most sites, real 404s don't waste meaningful crawl budget. Soft 404s, redirect chains, infinite parameters and duplicate URLs do. Return honest status codes, fix internal sources and redirect only URLs with real value. no404 helps you see which 404s receive real traffic, so you spend effort where it matters.