Created: 21 Oct 2025, last update: 21 Oct 2025
Wildcards in Sitemap.xml and Sitecore XM Cloud
Sitecore has a convenient wildcard feature. However, when you already have a significant number of wildcard pages in use, you may later decide that it’s not ideal for these URLs to appear in your sitemap.xml as URLs containing “,-w-,” instead of the actual resolved wildcard URLs.
In the Sitecore search crawler configuration, you can ignore such URLs. But if they still appear in your sitemap.xml, search engines and other crawlers will pick them up causing 404 requests that show up in your monitoring and SEO reports. It’s better to ensure they’re not included in the sitemap.xml at all.
Replacing wildcard URLs in Sitemap.xml with real URLs
A good solution is to create multiple sitemap.xml files at different URLs, one for each type of wildcard you have. You can build APIs that generate these sitemap files dynamically using your own logic.
For example:
/api/sitemap → the original Sitecore sitemap
/api/functionality1/sitemap → your custom sitemap.xml containing wildcard URLs
You can then list all your sitemaps in robots.txt so that search engines can easily discover them.
Removing wildcard URLs from the main sitemap.xml
If you want to remove 404 producing URLs, all wildcard pages and their subpages need to be excluded from Sitecore’s default sitemap.xml.
Out of the box, you can do this by checking “Sitemap Navigation” in field “Check to hide in navigation” in the Navigation section available on every SXA page.
However, if you already have many wildcard pages, it’s worth automating this.
Below is a Sitecore PowerShell script that updates all wildcard and subpages to hide them from the sitemap navigation.
# Settings
$startPath = "/sitecore/content"
# Find all items with layout under the specified path
$criteria = @(
@{ Filter = "Equals"; Field = "_haslayout"; Value = "true" },
@{ Filter = "StartsWith"; Field = "_fullpath"; Value = $startPath },
@{ Filter = "Contains"; Field = "_fullpath"; Value = "*" }
)
# Search in the master index using the specified criteria
$props = @{ Index = "sitecore_master_index"; Criteria = $criteria }
Write-Host "Searching for items with layouts under the specified path..."
$WildcardUrlItems = Find-Item @props | Select-Object -Property ItemId
$total = $WildcardUrlItems.Count
Write-Host ("Total number of unique URLs found: {0}" -f $total)
# GUID to add (filter)
$filterGuid = "{A0E7FF57-6994-4B09-AA21-104239628D5A}"
$normFilter = $filterGuid.ToUpperInvariant()
foreach ($searchitem in $WildcardUrlItems) {
if (-not $searchitem.ItemId) {
Write-Host "Invalid item entry (no ItemId). Skipping."
continue
}
# Get all language versions of the item
$allLangItems = @(Get-Item -Path "master:" -ID $searchitem.ItemId -Language * -ErrorAction SilentlyContinue)
if (-not $allLangItems -or $allLangItems.Count -eq 0) {
Write-Host ("No language versions found for ItemId {0}. Skipping." -f $searchitem.ItemId)
continue
}
# Use the first language version to modify the shared field
$sourceItem = $allLangItems[0]
if (-not $sourceItem -or -not $sourceItem.Paths -or -not $sourceItem.Paths.Path) {
Write-Host "Item has no valid path or is null. Skipping."
continue
}
Write-Host ("Processing item: {0} (first language: {1})" -f $sourceItem.Paths.Path, $sourceItem.Language.Name)
# Safely read the field
$currentValue = ""
try { $currentValue = $sourceItem["NavigationFilter"] } catch { $currentValue = "" }
# Split smartly: both '|' and '}{' (when concatenated without a separator)
$values = @()
if (-not [string]::IsNullOrEmpty($currentValue)) {
# Normalize by inserting a pipe between glued GUIDs
$normalized = $currentValue -replace '}\{', '}|{'
# Split on '|', trim each value, and keep only valid GUIDs enclosed in braces
$values = $normalized -split '\|' |
ForEach-Object { $_.Trim() } |
Where-Object { $_ -match '^\{[0-9A-Fa-f-]{36}\}$' }
}
# Ensure uniqueness and consistency
$valuesNorm = @{}
foreach ($v in $values) {
$key = $v.ToUpperInvariant()
if (-not $valuesNorm.ContainsKey($key)) {
$valuesNorm[$key] = $v
}
}
# Add target GUID if not already present
if (-not $valuesNorm.ContainsKey($normFilter)) {
$valuesNorm[$normFilter] = $filterGuid
}
# Join clean values
$newValue = ($valuesNorm.Values -join '|')
# Skip if nothing changed
if ($newValue -eq $currentValue) {
Write-Host "No changes needed."
continue
}
# Edit the item
$edited = $false
$beganEdit = $false
try {
$sourceItem.Editing.BeginEdit()
$beganEdit = $true
$sourceItem.Fields["NavigationFilter"].Value = $newValue
$sourceItem.Fields["ChangeFrequency"].Value = "{1C2878D9-80A6-4760-B7EB-DBADE16AE8C8}"
$sourceItem.Editing.EndEdit()
$edited = $true
Write-Host ("NavigationFilter updated: {0}" -f $newValue)
} catch {
if ($beganEdit) {
try { $sourceItem.Editing.CancelEdit() } catch {}
}
Write-Host ("Error editing item {0}: {1}" -f $sourceItem.Paths.Path, $_.Exception.Message)
continue
}
}
Publishing
Initially, I added publishing directly to the script.
This caused a challenge: when you create a publish job for every single page, the publish queue quickly fills up.
A short delay (e.g., 45 seconds) between publishes helps, but publishing hundreds of pages this way becomes extremely slow.
In addition, it didn’t properly trigger sitemap.xml generation.
A better approach is to run the script first and then perform an incremental publish.
That way, you can update all items in one go and regenerate the sitemap.xml once, efficiently.
Using the API instead of PowerShell
Instead of PowerShell, you can also use the Sitecore API.
The GraphQL Authoring API is particularly suitable for this purpose, see the Sitecore Commander project for examples.
