Pick the Architecture From the Job, Not From the Model

The model is a late decision

Most architecture conversations start in the wrong place.

Which model? Which vendor? Cloud or on-prem? Agents or not? A board paper fills with logos before anyone has named the job.

That is how companies buy a platform and still re-key emails.

A capable model is easy to access. The harder part is designing a safe place for it inside the operation. That means defining what it can read, what it is allowed to return, how its output is validated, and what happens when it gets something wrong.

Pick the architecture from the job. Then pick where inference runs. Then, and only then, pick a model that can do that job inside those constraints.

First: name the job

Almost every useful operational task is one of these, or a short sequence of them.

Extract. Turn messy text or a document into fields the system already has.

Classify. Put a case in a bucket: exception type, queue, urgency, language.

Retrieve and answer. Find approved material and answer from it. Cite. Refuse when the source is missing.

Generate. Draft a message, a summary, a checklist. A person still sends it.

Decide. Choose among known next steps. This is where people get excited, and where you should get strict.

Act. Write to the system of record, call a carrier API, move a state. This is not “the model.” This is software, after validation.

If a team cannot point to one of those words, they do not have a use case. They have a vibe. Stop and name the job before anyone mentions a model name.

Second: name the system of record

The model is not the place of truth.

An order lives in the ERP or TMS. Stock lives in the WMS. A ticket lives in the case system. A customer permission lives in the CRM. Other processes already read those rows.

If you cannot name the object you will read and the object you might write, you are not ready to choose an architecture. “We’ll connect our data” is not a name.

The system of record decides the schema. It decides the legal state transitions. It decides who may change a delivery address. The model does not get to rediscover those rules in prose.

The database is the contract. The model is the proposal.

Third: name what a wrong answer costs

This is the question that splits a copilot from a controlled step.

If the model summarises a long thread and a person still sends the email, a bad answer wastes time. Annoying. Survivable.

If the model writes a dock, a stock adjustment, a customs code, or a customer promise, a bad answer becomes everyone else’s fact.

Ask, out loud: can this process fail closed? Is there a queue? A named owner? A way to leave the official record untouched when confidence is low?

If the answer is no, do not put the model on the write path. Use it beside the work, or do not use it.

High cost plus no exception path is how “AI transformation” becomes a silent data incident.

Then choose the right pattern, not the vendor.

Once the job, the record, and the error cost are named, the pattern is usually obvious.

Rules only. If a deterministic table can do it, do not add a model. Dock type, opening hours, “READY cannot follow CANCELLED.” Software is cheaper and easier to explain.

Extract or classify, then a commit gate. The default for mail, PDFs, scanner notes. Structured output. Schema. Business rules. Confidence. Write or queue.

RAG over approved sources. For policy, SOP, specification. Retrieve first. Answer second. Show the passage. “I don’t know” when the passage is missing. A model that answers from memory is a liability.

Copilot beside the work. Drafts, explanations, checklists. The human still commits. Correct when the error cost is high and you are not ready to automate the write.

In-process model. The handler already owns the case, and inference becomes one controlled step inside that process. This is where higher volume can create real efficiency, provided the validation and escalation gates are strong enough.

Agent with tools. Multi-step, multiple systems, a plan that changes as it goes. Treat this as the exception, not the default. Allowlist the tools. No silent writes. A rule or a person before any commit. If you cannot draw the allowlist, you do not have an agent. You have hope.

Use case: turning an order email into an ERP update

Job: extract. Record: the order row. Error cost: medium. You can fail closed.

Architecture: in-process extraction. Mailbox event, load the thread and the candidate order, structured fields, validate, write or queue.

Not a copilot. The value is removing re-keying, not giving the clerk a nicer summary. Not an agent. There is one job and one write.

Not RAG. You are not answering a question. You are proposing fields.

Use case: answering “What is the current return procedure?”

Job: retrieve and answer. Record: the approved document set. Error cost: high if invented.

Architecture: RAG over sources a person has marked current. Cite the paragraph. If two versions conflict, or nothing matches, refuse.

Not extraction into the ERP. Not an agent that “goes and finds things.” The failure mode of a helpful intern is exactly what you are trying to avoid.

A copilot that searches the open web is the wrong product. The employee already has Google. They need the approved answer.

Use case: turning a warehouse exception note into an actionable case

Job: classify, maybe extract a location or quantity hint. Record: the WMS exception.

Error cost: high if it silently adjusts stock. Medium if it only routes.

Architecture: in-process classification on the existing exception object. Model proposes a type and a route. Software decides whether that route is legal. Stock writes stay deterministic.

If the note is vague, fail closed. “Maybe aisle 4, shift lead to confirm” is not a location update.

Use case: answering “Where is my shipment?” from live operational data

Job: retrieve. Record: TMS/WMS status events. Error cost: very high if invented.

Architecture: retrieve the official status. Optionally use a model to resolve a messy reference (“the Anderson order from last Thursday”) to an ID, then show the row.

Generate prose only as a skin on facts you already have. If the scan is missing, say the scan is missing. Do not complete the story.

This is the use case that most often gets the wrong architecture: a chat widget with a general model and a search plugin. That is how you get a confident Friday.

Use case: the tempting idea of an “autonomous planner”

Someone will ask for an agent that reads demand, moves appointments, rebooks carriers, and emails the customer.

Name the jobs. That is extract, decide, and act — across several systems of record. The error cost is the day’s network.

The honest architecture is usually: do not do this as one agent.

Split it. Extract the constraint. Show a proposed plan. Let existing optimisation or a planner accept it. Writes go through the TMS. The customer email is generated after the official appointment exists.

If you still want an agent, the allowlist is short and the commit gate is loud. Anything it cannot do with a named tool, it cannot do. Anything that changes a promise waits.

Autonomy is not a strategy. It is a privilege the process has to earn, one write path at a time.

Where inference runs is a workload decision

Only now is it useful to talk about cloud, EU, on-prem, or hybrid.

The question is not which logo is fashionable. It is what this workload requires: latency, volume, model capability, data class, residency, cost, who gets paged at 3am.

A cloud endpoint is often the right choice when the payload is small and tightly scoped, such as one email and one order, rather than an entire customer dataset.

A regional or EU-resident model is right when the work must stay in the EU. CodexCell uses Mistral for that constraint. Microsoft Foundry and AWS Bedrock are integrations into platforms you may already run. They are wiring. They are not a claim of sovereignty.

On-premises inference is right when the organisation must keep the bytes and the serving stack. It also transfers capacity, upgrades, and availability onto your own operations.

Hybrid is common: the system of record stays where it is; only the task payload travels; some jobs never leave the building.

None of these is “the architecture.” They are placements of one step.

Four questions that replace a strategy offsite

When a proposal arrives with a model name in the first sentence, ask these instead.

What happens if the model is wrong? If nobody can answer, it is not ready.

Who owns the exception? If the owner is “the team,” there is no owner.

Can you explain the write six months later? Prompt, payload, validation, actor, result. If not, you do not have an audit trail. You have a vibe.

Is Tuesday morning different? If the same person still copies the same email into the same second screen, you bought a tab.

Those four answers will choose the architecture more reliably than a vendor comparison slide.

The strategy is the boundary

You do not need a more autonomous model to start.

You need to know which job is being done, which row is authoritative, and what the process does when the answer is only probably right.

The model can be probabilistic.

The operation cannot be careless.

Pick the architecture from that fact, and the rest of the stack becomes a set of ordinary engineering choices. Pick the model first, and you will spend a year connecting a brain to a company that still works in another tab.