Large Sites

AEM is designed to support very large websites, but there are practical limits and architectural considerations that become increasingly important as a site grows. This guide summarizes the limits, explains where scaling challenges typically appear, and describes the recommended approach for structuring large sites.

Sizing considerations

Scaling up any site to a large number of pages should generally be governed by Google Search Console via crawl demand and budget. In other words, a site should only ever have as many pages as Google is willing to index.

AEM limits

Our platform enforces a number of site-specific limits to ensure reliable operation. The most relevant for large sites are:

Most customers never approach these limits. If your project is expected to exceed them, Adobe recommends considering architectural changes, as some limits can be increased on a case-by-case basis.

Challenges

Very large sites rarely fail because of a single hard limit. Instead, operations that work well for tens of thousands of pages become progressively harder to manage.

Large indexes

Indexes power features such as:

• listing pages
• search
• feeds
• filtered navigation

An individual index is limited to 50,000 indexed pages.

Even if your site is still within the limit, a large index:

• reindexes more slowly
• produces larger JSON payloads, introducing the need to add pagination logic
• does not isolate changes to one section

Large sitemaps

Search engines also expect sitemaps to remain reasonably sized.

Sitemaps are limited to:

• 50,000 URLs
• 50 MB

AEM adopts Google's sitemap limits to avoid indexing issues leading to SEO problems. If a site grows beyond this size, the sitemap won't grow further and search engines won't be able to crawl your entire site anymore.

Growing metadata

Bulk Metadata is extremely useful for applying metadata across large sections of a website.

Typical uses include:

• default images
• robots directives
• templates/themes
• title suffixes

For large sites, a single metadata sheet can become unmanageable and grow beyond proportions.

Large redirect lists

Redirects are typically managed using a central redirect sheet per site. While AEM supports up to 100,000 redirects per site, very large redirect lists can become increasingly difficult to maintain and review over time.

Common causes of large redirect lists include:

• legacy URL migrations
• frequent site restructures
• product renames
• consolidating multiple websites

If your site is expected to approach the recommended redirect limit, consider whether the content should be reorganized into multiple sites, each with its own redirect configuration.

Large JSON payloads

Indexes and metadata sheets are delivered as JSON. Very large indexes can eventually produce responses approaching the 6 MB payload limit, even though consumers typically only need a subset of the data in a single payload.

Reindexing

Whenever an index definition changes, the content needs to be reindexed.

Keeping indexes focused on a single business area reduces:

• reindex duration
• operational impact
• troubleshooting complexity

It is generally preferable to have several focused indexes rather than one very large catch-all index.

Structuring large sites

The recommended architecture depends on the size of your site.

Sites up to 50,000 pages

Most sites can be managed with a single:

• index
• sitemap
• metadata sheet

Sites between 50,000 to 1 million pages

For sites larger than 50,000 pages, Adobe still recommends keeping a single AEM site, but consider splitting supporting structures into logical sections.

This typically includes:

• multiple indexes for different content areas
• using pagination (limit and offset) when consuming rather than requesting the complete dataset.
• multiple sitemaps exposed through a single sitemap-index.xml added to your robots.txt
• optimized bulk metadata sheet leveraging URL patterns, or using additional metadata sheets to organize metadata by site section or business area
• optimized redirects by reviewing usage and removing obsolete redirects

For example, instead of one global index, sitemap, and metadata sheet, organize your site around countries, regions, or logical sections of your site, for example:

• Products
• Documentation
• Blog
• Support
• API Reference

This approach keeps each configuration smaller, easier to maintain, and faster to update and consume while preserving a single website and URL structure.

Sites approaching or exceeding 1 million pages

As your site approaches or exceeds the 1 million page limit, consider splitting it into multiple sites using the same code base.

A natural split is often along business boundaries such as:

• country or market
• language
• brand
• business unit
• product family

For example:

• products.example.com
• support.example.com
• developer.example.com
• blog.example.com

or separate sites under a common domain, depending on your information architecture.

If you split your content into multiple sites, you do not need to maintain multiple repositories.

AEM supports repoless sites, allowing multiple sites to share the same code base while each site maintains its own content source and configuration.

This architecture is intended primarily for very large websites that are approaching or exceeding the recommended site size limits, or for organizations that naturally operate multiple independent sites (for example, multiple brands, country sites, or regional websites).

Updates pushed to the shared GitHub repository automatically become available to every site using that codebase.

Summary

As your site grows larger, consider the following progression:

Following these practices keeps authoring, indexing, publishing, SEO, and ongoing maintenance manageable as content continues to grow.