Skip to main content

Sitemaps

Generate an XML sitemap with reciprocal hreflang alternates for your sluggable models. Bind a PolyslugUrlResolver — the package cannot know your URL structure, so this one class is yours to write — and register the types:

use Polyslug\Contracts\PolyslugUrlResolver;
use Polyslug\Contracts\Sluggable;

$this->app->bind(PolyslugUrlResolver::class, fn () => new class implements PolyslugUrlResolver
{
    public function url(Sluggable $model, string $locale): string
    {
        return route('pages.show', ['locale' => $locale, 'page' => $model->polyslugRouteKeyForLocale($locale)]);
    }
});
// config/polyslug.php
'sitemap' => ['types' => [\App\Models\Page::class, \App\Models\Article::class]],
php artisan polyslug:sitemap --path=public/sitemap.xml

Without --path the XML goes to standard output, which is convenient for piping or for inspecting a few entries during development.

To keep the file current, schedule the command and guard it against overlapping runs:

// routes/console.php
use Illuminate\Support\Facades\Schedule;

Schedule::command('polyslug:sitemap --path=public/sitemap.xml')
    ->daily()
    ->withoutOverlapping()
    ->onOneServer();

withoutOverlapping() keeps a slow run on a large table from starting twice, and onOneServer() keeps a multi-server deployment from writing the file on every host; the latter needs a cache store that supports atomic locks (Redis, database, DynamoDB or Memcached).

What it includes​

  • Only routable models and locales. Anything you hide via polyslugIsRoutable() stays out, and a model with no routable locale produces no entry at all.

  • Nothing that would answer 410 or 301. A record that reports polyslugIsGone(), and a superseded one whose successor is visible, are left out — the same precedence the canonical middleware applies — so the sitemap never submits an address the site itself refuses or redirects. A record whose successors come back to it, one that names itself included, is served at its own address and therefore listed.

  • One <url> per address, not per record. A record served under three locales produces three <url> entries, one per address. That is what search engines count as submitted: an <xhtml:link> is an annotation about an address, never a submission of it, so a locale that appeared only as an annotation was never announced at all. An address served under two codes is submitted once, with both codes among its alternates.

  • Reciprocal hreflang alternates, x-default included. Every one of those entries carries the same complete alternate set — one <xhtml:link> per routable locale, one for x-default, and a self-reference — because reciprocity is a property of the record rather than of the address. The set comes from hreflangLinks(), the one method that also feeds the page <head>, so the sitemap and the head cannot disagree about which addresses a record has. x-default is an alternate and never a <loc>: it is a second key over an address already in the set, and submitting it too would list that URL twice. See Multilingual slugs and hreflang.

    Changed in 0.15.0 and again since

    In 0.15.0 the command stopped building its alternates in its own loop, which had made <loc> take the alphabetically first locale while the head announced x-default, and had left x-default out of the sitemap entirely.

    Since then the entry itself changed shape: a record used to produce exactly one <url>, whose <loc> was the x-default address, with every other locale appearing only as an annotation. At N locales that left (N−1)/N of the addresses unsubmitted. Both are regenerated by the next run; nothing in your code changes, and a sitemap file written by an older version is not wrong, only shorter than it should be.

  • <lastmod> when a model offers one, and nothing else. <priority> and <changefreq> are documented as ignored, so the command emits neither. lastmod is the hint that is still read — while it stays accurate. Implement polyslugLastModified() to supply it:

    use DateTimeInterface;

    public function polyslugLastModified(): ?DateTimeInterface
    {
        return $this->updated_at;
    }

    It returns null by default, on purpose. A timestamp that moves on every write — a view counter, a cached column, a nightly re-import that touches every row — turns the field into noise, and the documented response is to disregard it for the whole site rather than for the one row. Wire it up where updated_at really does track the content, and leave it alone where it does not. The method lives on HasPolyslug rather than on the Sluggable contract, so a hand-written implementation keeps working unchanged.

  • Registered types only. Entries in polyslug.sitemap.types that are not Eloquent models implementing Sluggable are skipped rather than failing the run.

  • A type the resolver cannot address costs that type, not the document. polyslug.sitemap.types is configuration, and being a Sluggable model says nothing about whether anything routes it: a model that never had a route, or one whose route was renamed while the config stayed, is ordinary. Those records are skipped and the rest of the sitemap is written.

    Say it without throwing by adding canAddress() to your resolver — optional, so an existing implementation needs no change:

    public function canAddress(Sluggable $model): bool
    {
        return ! $model instanceof Milestone;   // carries a slug, nothing routes it
    }

    A declared refusal is silent. A resolver that throws instead is also survived — it has to be, because a resolver can fail for reasons nobody declared — but those records are counted and named at the end of the run. Only one of the two is a decision, and the output says which happened.

    Why the run no longer stops

    It used to. One throw ended the whole command: no <urlset>, no file, a red scheduled job — and a red scheduled job does not replace the file it was going to write, so the previous sitemap stayed where it was and aged silently. From the outside that is indistinguishable from a sitemap being kept current.

  • A database error ends the run, and the last sitemap stays. Only what the resolver throws is counted as a skip. A database error is not a record the resolver cannot address, wherever it is raised, so it fails the run with a non-zero exit code. Every file is written under a temporary name first and renamed into place once the whole set is complete, so a run that fails, part-way through a split set included, publishes nothing and the previous sitemap is still served. A replaced file keeps its permissions.

Large sites: splitting and the index​

The sitemap protocol caps one file at 50,000 URLs and 50 MB uncompressed. A file past either limit is rejected whole, not truncated.

You do not have to notice that. With --path, the command writes numbered parts beside it and puts a <sitemapindex> at --path itself as soon as either ceiling is reached:

public/sitemap.xml      <- the index, this is the URL you submit
public/sitemap-1.xml    <- 50,000 URLs
public/sitemap-2.xml    <- the rest

While everything fits, --path gets one ordinary <urlset> document and no index, exactly as before.

When a later run needs fewer parts, or fits one file again, it removes the parts the index it replaces named and the new set no longer writes, once every new file is in place. A file that index did not name stays, whatever it is called, and a run that fails removes nothing.

An index has to name each part by absolute URL, so the command needs to know where the files are served from. It takes app.url, or --base-url if the sitemaps live somewhere else (a CDN, a subdirectory):

php artisan polyslug:sitemap --path=public/sitemap.xml --base-url=https://cdn.example.com

If it needs an index and can determine neither, it fails rather than writing one with relative locations — an invalid index takes the whole submission down, where a missing one only delays it.

Both ceilings are configurable under polyslug.sitemap.max_urls and polyslug.sitemap.max_bytes. Lower them if a CDN or a search console you use wants smaller files. Raising them past the protocol produces documents engines reject.

When the records of a part carry a <lastmod>, the index names the latest of them for that part, so a search engine can tell which parts changed since it last read them.

The sitemap as a service​

The command covers one sitemap for the whole application, written to files. For anything else, call the service behind it, Polyslug\Sitemaps\Sitemap. It takes the sources of its records, the storage its parts are kept in and the root they are served under. That gives a multi-tenant application one sitemap per tenant, parts kept in a table instead of on a disk, and a rebuild of only the part that holds a changed record:

use Polyslug\Contracts\PolyslugUrlResolver;
use Polyslug\Sitemaps\DatabaseSitemapStorage;
use Polyslug\Sitemaps\Sitemap;
use Polyslug\Sitemaps\SluggableSource;

$pages = new SluggableSource('pages', [Page::class], app(PolyslugUrlResolver::class));

$sitemap = new Sitemap(
    sources: [$pages],
    storage: new DatabaseSitemapStorage('sitemap_parts', ['shop_id' => $shop->id]),
    root: 'https://'.$shop->domain.'/sitemaps',
);

$sitemap->build();                                  // every part of every source
$sitemap->rebuild('pages', [$pages->key($page)]);   // the part that holds $page

The ceilings come from polyslug.sitemap.max_urls and polyslug.sitemap.max_bytes unless the constructor is given maxUrls and maxBytes.

Parts cut by key​

Each source is cut into parts by the keys of its records, compared byte by byte. A part holds the records from the key it starts at up to the key the next part starts at, within both ceilings, and the addresses of one record always share a part. Because a part is a range of keys rather than a position, rebuild() reads only the range of the part that holds each key it is given: a new record lands in the part whose range holds its key, and a removed one leaves its part. A part that outgrows a ceiling, or empties while others of its source remain, has its source cut anew.

polyslug:sitemap --path is this service with one source over files, and it writes the same parts the service writes into any other storage.

Sources​

SluggableSource lists sluggable models and applies everything this page describes: the resolution gate, gone and superseded records, canAddress() and <lastmod>. Its key for a model comes from key($model), built from the position of the model's type and its primary key. An integer key is padded to twenty digits, so its byte order is its numeric order.

Anything else implements Polyslug\Contracts\SitemapSource: a name, which names the parts (products writes products-1.xml), and the records whose keys lie in a range, in the byte order of their keys. A read model with an address scheme of its own needs no Sluggable model:

use Polyslug\Contracts\SitemapSource;
use Polyslug\Sitemaps\SitemapRecord;

final class ProductPages implements SitemapSource
{
    public function __construct(private Shop $shop) {}

    public function name(): string
    {
        return 'products';
    }

    public function records(string $from = '', ?string $until = null): iterable
    {
        $pages = ProductPage::query()
            ->where('shop_id', $this->shop->id)
            ->where('key', '>=', $from)
            ->when($until !== null, fn ($query) => $query->where('key', '<', $until))
            ->orderBy('key');

        foreach ($pages->cursor() as $page) {
            // every address by locale or hreflang code, x-default among them where there is one
            yield new SitemapRecord($page->key, $page->addresses, $page->changed_at);
        }
    }
}

A key never changes, and the database has to sort the key column byte by byte, as a ULID or a UUID sorts everywhere and any column with a binary collation does (COLLATE "C" on PostgreSQL). Keys that come back in another order are noticed and leave the parts of that source without known ranges, so every rebuild of it reads the whole source. The result stays correct, and each rebuild costs a whole read.

Storage​

FileSitemapStorage is what the command uses: the parts beside a path and the index at the path, or the only part alone at the path. Files keep no record of where each part starts, so rebuild() writes a sitemap in files whole.

DatabaseSitemapStorage keeps a row for each part in a table of yours, so every node of a deployment serves the same sitemap without a shared disk:

Schema::create('sitemap_parts', function (Blueprint $table) {
    $table->id();
    $table->foreignId('shop_id');
    $table->string('source');
    $table->unsignedInteger('number');
    $table->text('starts_at')->nullable();
    $table->unsignedInteger('urls');
    $table->string('lastmod')->nullable();
    $table->longText('body');
    $table->unique(['shop_id', 'source', 'number']);
});

The scope, ['shop_id' => $shop->id] above, is matched on every read and written into every new row. Each build and rebuild is one transaction, and it reads the parts it plans from with FOR UPDATE, so two writes of one sitemap that already has parts wait for each other. Write a new sitemap's first parts from one process at a time, a queued job with WithoutOverlapping for one.

Any other store implements Polyslug\Contracts\SitemapStorage.

Serving the parts from a table​

Route::get('/sitemap.xml', fn () => response($sitemap->index())
    ->header('Content-Type', 'application/xml'));

Route::get('/sitemaps/{name}', fn (string $name) => response($sitemap->document($name) ?? abort(404))
    ->header('Content-Type', 'application/xml'));

index() names every part under the root, pages-1.xml, pages-2.xml and so on, each with its <lastmod>. document() answers with the part served under a name, or null. A sitemap whose parts need an index and that has no root refuses with CouldNotBuildSitemap, since a relative location is invalid in an index.

Memory​

Rows are read in keyset chunks of 500, so a large table is never loaded at once, and each chunk arrives with its slug rows eager-loaded and its tokens fetched in one query. The chunks follow the primary key whatever order a polyslugResolveQuery() applies, so every row the gate lets through is read exactly once. A run costs a few queries per 500 records rather than several per record. Each part is written as soon as it is full, so with --path the peak is one part, not the whole document. The token memo of a long-running process keeps at most 10,000 entries per memo and lets go of the older half beyond that. Without --path there is nowhere to flush to: the document is assembled in one piece and has to fit — roughly 40 MB of peak memory for 55,000 entries. That matters more than it sounds: the naive version of this command is the one that works in development and gets killed by the memory limit on the table that actually needed a sitemap.

When the resolver is missing​

Without a bound PolyslugUrlResolver the command cannot build a single URL, so it prints an error naming the contract and exits non-zero rather than writing an empty sitemap over a good one.

The same resolver powers short links and the laravel/head integration — bind it once. The URL resolver covers it in full, including what each feature does when it is missing.