- Begin with the failed operation, not a loop
- A completion standard worth keeping
Begin with the failed operation, not a loop
Suppose a collector requests a publisher’s feed and receives a rate-limit response. Immediately sending the same request again may reproduce the problem while adding more work for both systems. A useful recovery design begins by identifying what was attempted, whether it can safely be repeated and when the next attempt is allowed.
The goal is not to maximize the number of attempts. It is to obtain a trustworthy result without turning a temporary restriction into persistent pressure. Record the operation, destination, attempt time and observed response before choosing a retry policy.
Respect a supplied waiting period
HTTP Retry-After can express a delay in seconds or an HTTP date. Read both forms. A valid future date is a deadline to respect; it is not an invitation to poll repeatedly until the date arrives. When a server supplies no usable delay, apply an explicitly bounded local policy.
For example, an application might start with a one-minute delay, increase it after repeated failures and impose a maximum. These values are a policy choice, not universal protocol constants. Keep the policy visible in configuration so an operator can explain why a source was deferred.
One publisher may expose many feeds
Imagine ten feeds on one publisher’s hostname. Retrying each independently can create ten overlapping recovery loops even though the restriction applies to the same service. Model both per-source state and shared host pressure. A host cooldown can prevent a new request from a sibling feed while an earlier limit remains active.
Do not convert every failure into the same action. A timeout, an access denial, an absent resource and malformed XML are different observations. Preserve those distinctions. A protected destination should not become eligible simply because a retry policy failed to fetch it.
Make the next action inspectable
A useful status row has a last-attempt time, result, error category, consecutive failure count and next-attempt time. Operators should be able to distinguish an unhealthy source from a source deliberately waiting. Avoid labeling both as running.
A manual refresh should request new work without silently cancelling an upstream waiting period. Otherwise the refresh button becomes a way to defeat the very policy intended to protect the integration. If an override is necessary, make it a separate, audited operation with a stated reason.
Test the unhappy paths deterministically
Use a local fake server or mocked transport. Test an integer delay, a future HTTP date, an invalid header, a timeout, an oversized response and recovery after a successful request. Advance a controlled clock instead of making the test suite sleep for minutes.
For each case, check both the immediate return value and the persisted next action. A test that observes an error but never inspects scheduling state can miss the repeated-request defect that matters most in operation.
A completion standard worth keeping
The integration is ready for a longer trial when you can explain every attempted and deferred source, show that two workers do not multiply requests and demonstrate recovery after process interruption. Publish freshness measurements separately from request counts.
A successful retry establishes that this attempt obtained a result. It does not prove the content is new, relevant or accurate. Keep collection recovery separate from editorial approval, and keep the evidence that supports each decision.
Read beyond this page.
Recorded source-check date: 10 Sep 2026. A link is not, by itself, evidence that every claim has been independently verified.
Changes & version history
Version 3 · 11 Sep 2026
Scheduled release of checksum-bound AI-assisted editorial review
Version 2 · 10 Sep 2026
Checksum-bound editorial review scheduled for release
Version 1 · 10 Sep 2026
Source-linked private review edition