Reliability
- Retries and backoff — exponential with full jitter, capped, Retry-After-aware
(
server.tries,server.backoff).no_retry_on_4xxkeeps a permanent400/410from being retried for hours while408/425/429still are, andretryableProblemTypes()retries a4xxwhose problem document names a type the delivery lists. See Sending. - Idempotency — a stable per-event id (the Standard Webhooks
webhook-id), preserved across redelivery, and two-tier inbound dedupe on receipt. - A time to keep trying, and ordered delivery —
retryForSeconds()retries until a time is up instead of a number of tries, andorderedBy()sends the deliveries to one URL under one key one at a time, each after the one before it succeeded (server.ordering.enabled). See Sending. - Circuit breaker (
platform.circuit_breaker.enabled, on by default) — afterplatform.circuit_breaker.thresholdconsecutive final failures an endpoint auto-disables and aPushery\Webhooks\Events\WebhookEndpointAutoDisabledevent fires. While an endpoint is still active a single successful delivery resets the streak, but once it is disabled it receives no further traffic and so does not self-recover — bring it back withWebhooks::enable($subscription)(which also clears the streak), typically wired to theWebhookEndpointAutoDisabledevent or an operator action. - Rate limiting (
platform.rate_limit.enabled, on by default) — a per-subscription outbound cap (platform.rate_limit.max_per_minute, default60) and an optional per-source inbound cap keep one slow endpoint from starving the queue. The outbound cap shapes traffic rather than dropping it: an over-limit delivery is logged, announced (Pushery\Webhooks\Events\WebhookDeliveryRateLimited) and enqueued with a delay, so a burst is spread across the following minutes instead of being lost. The schedule lives in the limiter store thatcache.limiternames, keyed by the subscription's owner and id, so the subscriptions of two tenants never share one, even where a database per tenant gives both the same id. - Lifecycle events — every delivery announces its fate. Which family to listen to depends on whether you run the Platform layer; see Events.
Retries need a real queue
Every retry in this package is a release() back onto the queue the job came from. The
sync connection has no worker, so a released job is never picked up again — the job runs
exactly once, and a delivery that fails in a retryable way ends there instead of using its
configured tries.
The package does not pretend otherwise. Such a delivery reaches a terminal state and reports
Pushery\Webhooks\Server\Exceptions\QueueCannotRetry, whose message leads with the
original transport failure and then names the queue — so the stored error still says
"certificate has expired", and the queue is given as the reason that failure became final
rather than as a replacement for it. webhooks:preflight warns when the server layer
resolves to sync.
Use database, redis or sqs wherever deliveries matter.
A health check for the delivery engine
With Laravel Health installed, register the package's check beside your others:
use Pushery\Webhooks\Health\DeliveryEngineCheck;
use Spatie\Health\Facades\Health;
Health::checks([
DeliveryEngineCheck::new(),
]);
It reads the delivery log of the last 24 hours and fails when more than half of those deliveries failed. It warns while an endpoint is switched off, whether the circuit breaker did it or somebody did by hand, or only when the breaker did, if you ask. Each part is configurable:
| method | default | what it changes |
|---|---|---|
window(int $hours) | 24 | how far back the check reads |
failWhenFailureRateAbove(float $percent) | 50 | the share of failed deliveries that fails the check |
warnWhenFailureRateAbove(float $percent) | off | a lower share that warns first |
minimumDeliveries(int $count) | 1 | the sample below which the rate is not judged |
warnWhenEndpointsAreDisabled(bool $warn) | true | whether a switched-off endpoint warns |
remedyForDisabledEndpoints(string $sentence) | none | a sentence appended to that warning, saying where an endpoint is switched back on |
warnOnlyAboutEndpointsTheBreakerDisabled() | off | the warning leaves out an endpoint somebody switched off by hand |
failWhenADeliveryIsPendingLongerThan(int $minutes) | off | how long a delivery may wait to be sent before the queue counts as stopped |
failWhenARetryIsOverdueAfter(int $minutes) | off | how old a delivery to a switched-on endpoint may get while it waits for its next attempt |
lookForStallsBack(int $hours) | one window | how far back the two limits above look for a delivery that is still waiting |
warnWhileAnEndpointIsFailing() | off | a switched-on endpoint with a failing health score warns |
remedyForFailingEndpoints(string $sentence) | none | a sentence appended to that warning, saying what to do before the breaker switches the endpoint off |
warnWhenPartitionsDrift() | off | rows in the default partition, or no partition for the next month, warn (PostgreSQL) |
The rate is the one the endpoint health score uses: failed deliveries over the deliveries that reached an attempted outcome. A pending delivery has no outcome yet and a refused one was never sent, because its endpoint was switched off while it waited in the queue, so neither counts toward the rate. A delivery waiting for its next attempt counts as failed until it resolves. The result lists every count, so the refused and pending ones stay visible even though the rate leaves them out.
Name the place an endpoint is switched back on. The package cannot know it: in one application the owning customer does it from their own settings, in another an operator does it from a back office. So the warning says what is off, and your application says where to go:
DeliveryEngineCheck::new()
->remedyForDisabledEndpoints('The owning company switches each one back on from the Webhooks tab of its settings.');
The sentence is appended to the warning, so it reaches the notification Laravel Health sends. Without it the warning reads as it always has.
Tell a stopped engine from a quiet one. A failure rate cannot see the engine stop: without a worker there are no outcomes, so there are no failures either, and a dead queue reads like a quiet night. Two limits catch it, and both are off until you set them:
DeliveryEngineCheck::new()
->failWhenADeliveryIsPendingLongerThan(15)
->failWhenARetryIsOverdueAfter(180);
The first fails when a delivery has waited longer than that to be sent at all. A delivery to an
endpoint over its outbound rate limit waits as pending until its window opens, so choose a limit
longer than the longest such wait you expect. The second fails when a delivery to a switched-on
endpoint is still waiting for its next attempt after that long, which is what a worker that stopped
between two attempts leaves behind. It counts from the delivery's creation, so choose it beyond the
whole retry schedule: every backoff and every honored Retry-After included. The retries of a
switched-off endpoint stopped on purpose and are not counted.
Keep reporting a jam that is older than the window. Both limits look back one window from the
limit. A worker that stopped leaves its last deliveries waiting, and once nothing new arrives they age
out of that window: the check turns green over a jam nobody cleared. lookForStallsBack() sets a
longer look-back for the two limits alone, so the rate still reads the window it always did:
DeliveryEngineCheck::new()
->failWhenADeliveryIsPendingLongerThan(15)
->lookForStallsBack(24 * 7);
Warn before the breaker switches an endpoint off. One endpoint that fails while the others
deliver keeps the rate of the whole engine low, so the rate does not see it. With
endpoint health scoring on,
warnWhileAnEndpointIsFailing() warns while a switched-on endpoint has a failing score, and the
result carries the count as failing_endpoints. remedyForFailingEndpoints() appends your sentence
to that warning, the way remedyForDisabledEndpoints() does to its own.
Hear about the partitions before retention stops. On PostgreSQL the delivery log is partitioned
by month, and webhooks:partition-maintenance keeps the months ahead provisioned. When the schedule
stops running it, the provisioned months run out, deliveries land in the default partition, and
retention stops dropping old months. warnWhenPartitionsDrift() warns while rows sit in the default
partition or the next month has no partition yet, and the result carries default_partition_rows
and next_month_partitioned. On MySQL there are no partitions and it reads nothing.
The warnings are reported together. A failure rate above the warning share, a switched-off
endpoint and a failing one are warnings of one rank, so the check names every one that applies: the
short summary reads like 1 endpoint off, 1 endpoint failing, and the message carries each sentence
with its own remedy. A switched-off endpoint therefore never hides the next one on its way to the
breaker. A failure still ends the check on its own: a stalled queue, an overdue retry or a rate
above failWhenFailureRateAbove().
Warn about the breaker only. An endpoint somebody switched off by hand is a decision rather than
news. warnOnlyAboutEndpointsTheBreakerDisabled() leaves it out of the warning, and the result still
counts both kinds. It switches the warning on as well, so of it and warnWhenEndpointsAreDisabled()
the later call decides: warnWhenEndpointsAreDisabled(false) after it silences the warning for the
breaker's endpoints too.
Every read is bounded on created_at, so on PostgreSQL only the monthly partitions it reaches are
scanned. The two limits read one look-back's worth of creation times, ending at the limit. Of the deliveries the result carries counts and the rate, never a
payload, a URL or an error text, because Laravel Health passes it on in notifications and keeps it
in its history. The only sentence the package did not write is the remedy you name.
The check needs the Platform layer: with platform.enabled off there is no delivery log, and it
fails with a message saying so rather than reporting an engine it never looked at.
Retention
php artisan webhooks:partition-maintenance (scheduled daily) ages the delivery log out past
platform.retention_months.
On PostgreSQL the log is monthly range-partitioned, so a whole old month is dropped as a
metadata operation, not a bulk DELETE, and the command also provisions upcoming partitions
and drains any delivery that landed in the catch-all default partition while the schedule was
behind — so a lapse in the cron cannot permanently stop either.
Each of its schema statements waits at most two seconds for its lock. Creating or dropping a partition conflicts with an open read of the log, and a statement waiting for such a lock would make every later read and write of the log wait behind it, for as long as that read lasts. So while a long transaction holds the log, the run gives up with PostgreSQL's lock timeout instead, and the next run does what it left; the months ahead are provisioned well in advance for that.
A month the command drains can still be receiving deliveries, when the lapse reached into the current month. Its last step pauses the log's writes: it moves the rows that arrived during the drain and attaches the month's partition in one transaction, and a delivery written in that moment waits for it and then lands in the new partition instead of failing. Measured on PostgreSQL 18, the pause lasted 42 ms after a drain of 200,000 rows and 235 ms after one of 1,000,000.
On MySQL the log is a flat table (kept so the ON DELETE CASCADE GDPR guarantee holds —
MySQL cannot have both a foreign key and partitioning), and retention is an indexed, chunked
DELETE over the same window; the same command, the same cutoff.
Below ~1M deliveries/month the difference is invisible; see Choosing your database.
Retention prunes rows. If you enabled large-payload offload, the content-addressed disk
objects are reclaimed separately — by a disk lifecycle policy, or by
php artisan webhooks:prune-orphaned-payloads. See the
command reference.
Forgetting an event before retention does
A deletion request does not wait for retention. Every delivery of one dispatch carries the same
event_id, so keep it with whatever the payload was about, and hand it back when the person it
describes asks to be forgotten:
use Pushery\Webhooks\Facades\Webhooks;
$eventId = Webhooks::dispatch('report.created', $payload)->first()?->event_id;
// later, answering the deletion request
$forgotten = Webhooks::forgetEvents([$eventId]);
It deletes the delivery-log rows of those events, redeliveries included, takes them out of an
external search index, and deletes their offloaded payload objects. A delivery of them still
waiting in the queue is not sent. The result says how far it went: deliveries and objects
count what was deleted, and objectsKept counts the objects left in place because another row
still points at them — offloaded bodies are content-addressed, so another event with the same bytes
shares one, and the object holds nothing that row does not. Inside a transaction the objects go
once it commits.
An id that is not an event id is refused with an InvalidArgumentException rather than answered
with nothing deleted. The hourly rollup behind the dashboard keeps its counts, which carry no
payload, and the Server layer's own log stores no payload at all.