Your application is healthy. The database is responding. The dashboard is green. A user asks the AI assistant a question, and nothing comes back.This is an awkward kind of outage because the rest of the system may be working perfectly. The model service could be slow, rate limited, unavailable, or returning an answer that sounds convincing but is wrong. From the user's perspective, the feature has failed either way.I work on reliability for systems that support inventory and fulfillment operations. In one AI-assisted investigation project, the aim was to help users explore discrepancies through natural language instead of relying on an engineer to write every query. That kind of interface can make complex data much easier to use. It also adds a dependency between the person asking a question and the evidence they need to see.So the resilience question is bigger than "What if the model goes down?" It is: what can users still do safely when the model cannot help?An LLM can fail without going offlineTraditional service failures are often fairly clear. A request times out, a connection drops, or the server returns an error. LLM failures include those cases, plus a few that are harder to spot.A request might succeed but take too long to be useful. Rate limits might appear only when several teams are busy at once. A model change might alter how the assistant selects tools or interprets a familiar question. And an answer can be fluent while misstating what the underlying data shows.That last case matters most in business workflows. Imagine someone asking why an inventory count differs across two locations. If the assistant cannot access a required data source, a polished explanation is worse than a clear statement that the investigation is incomplete. The user may act on it.A system therefore needs to distinguish several outcomes: the AI is unavailable, the AI is slow, the necessary data is unavailable, and the AI produced an answer that could not be verified. Treating all four as a generic "try again" error leaves users guessing.Decide what the model is allowed to doThe first resilience decision happens before any outage. It is the boundary between the model and the rest of the application.For a narrow investigation workflow, I would rather give an assistant a small set of approved operations than let it generate unrestricted queries. One operation might trace an item's movement. Another might retrieve a discrepancy summary. Each can have defined inputs, access permissions, and limits.This helps during normal operation, but it also makes failure easier to contain. If the model misunderstands a request, it can select only from the operations the application permits. If a tool fails, the application knows which part of the investigation is missing. The model should not be able to compensate by inventing data or calling a more powerful tool.The same principle applies to actions. Reading a record and changing one are different levels of risk. An assistant might help explain an anomaly, while a separate rule or human approval determines whether anything should be changed. A provider outage should never cause the system to skip that boundary.Give each failure a deliberate fallback"Use another model" sounds like a resilience strategy, but it is only one possible fallback. A second provider may have different behavior, permissions, data handling requirements, and costs. Switching automatically can create a new failure mode if nobody has tested the resulting answers.I would start with the user's task. If the AI normally summarizes verified query results and the model is unavailable, the application may still be able to show those results in a structured view. If the AI normally routes a question to one of a few known investigations, the interface could let the user choose the investigation directly.When neither option is possible, the right fallback may be a clear stop: "I couldn't complete this investigation because the analysis service is unavailable. No conclusion was produced." Preserve the request so the user can retry, if doing so is appropriate, but do not present an old answer as a fresh one.The rule is simple: degrade the experience before you degrade the truth. A less convenient path to verified data is useful. A seamless path to an unsupported conclusion is not.Put a time budget around the whole requestTimeouts are easy to add one service at a time and surprisingly easy to get wrong across an entire workflow.Suppose an investigation needs to retrieve data, ask a model to interpret it, and format the result. If every step gets its own generous timeout, the user may wait far longer than the product can reasonably support. Retries can stretch that delay further, especially when the model provider is already overloaded.Set an overall request deadline first. Then allocate time within it for data retrieval, model processing, and the response. A retry should fit inside the remaining budget, use backoff, and occur only when the failure is likely to be temporary. If the deadline is reached, return the safest available result rather than leaving the interface spinning.Health checks need similar care. A running application container tells you little about whether a model-dependent investigation can finish. Monitor the dependencies that matter to the user's task, and distinguish a slow or rate-limited model from a failing data source. Those conditions call for different responses.Observe the investigation, not just the endpointA single "request failed" metric will not tell an engineer whether the model timed out, selected the wrong tool, received incomplete data, or returned an answer the application rejected.For an AI-assisted investigation, useful traces can record the request's path: which type of investigation was selected, which approved tools ran, whether they succeeded, how long each step took, and whether the final answer had sufficient evidence. Access to that telemetry should be controlled, especially if requests can contain sensitive business data.I would also watch the rate at which the system uses fallbacks. If structured results are suddenly appearing more often because model calls keep failing, the application may look available while its main experience has degraded. Users will notice before a simple uptime chart does.This is one reason AI reliability cannot be measured only by successful HTTP responses. A technically successful request that gives an unsupported answer has failed its real job.Test the bad day before launchA fallback that exists only in a design document is not a fallback. It needs to work while the provider is slow, the rate limit is exhausted, or a tool returns incomplete data.Before expanding an AI feature to a business-critical workflow, I would run tests that deliberately create those conditions. Delay model responses. Force a provider error. Remove access to one permitted tool. Feed the assistant ambiguous results and check whether it admits uncertainty. Change a model version in a test environment and compare its tool choices and answers on a fixed set of representative questions.These tests should examine what the user sees, not only whether the application stays up. Can they tell that an investigation is incomplete? Can they still reach verified data? Is an action blocked when its supporting evidence is missing?Formal evaluation and governance for the inventory investigation project described in my work are still areas to mature. That distinction matters. Designing a safer workflow is a useful start, but the claims you can make about its reliability should match what you have actually tested.Keep the resilience design proportionateEvery extra provider, cache, retry path, and fallback interface adds cost and complexity. More moving parts can create more incidents if a team cannot operate them confidently.The right design depends on the consequence of failure. An optional writing suggestion may need a short timeout and a friendly error. An assistant supporting an operational investigation needs stronger controls around data access, evidence, and uncertainty. A system that can change production resources needs stricter limits still.I would spend the complexity budget on the failure modes that matter most: keeping verified data accessible, preventing unsupported conclusions, and ensuring that AI cannot take an unsafe action. Add a second model provider only when testing shows it improves those outcomes enough to justify its operational burden.The goal is not to make an LLM dependency invisible. Users can handle an honest message that an AI feature is temporarily unavailable. What they cannot safely handle is a system that looks certain when its evidence, tools, or model have failed.