A delayed shipment creates a practical question: should someone expedite it, offer a discount, or wait? A useful answer needs to account for the order’s economics and explain the recommendation well enough for a person to challenge it.
FreightSense is my prototype for that workflow. It puts a rules-based calculation beside an LLM assessment, records both, and leaves the final decision with the operator.
What I built
The application combines a Python/FastAPI API, a small browser dashboard, a deterministic evaluation layer, Groq-hosted LLM calls and SQLite records. The repository also includes Docker and Cloud Run deployment configuration.
| Stage | Responsibility |
|---|---|
| Shipment input | Collect the order, shipping and estimated delivery details. |
| Deterministic evaluation | Calculate delay, financial exposure and a weighted risk score; apply recommendation rules. |
| LLM evaluation | Interpret the same context and return a structured assessment. |
| Comparison | Show whether the two recommendations agree. |
| Decision record | Store the evaluation and any later human overrides. |
Keep the arithmetic outside the prompt
The deterministic layer calculates delay and exposure before the model sees the request. Its risk score combines delay severity, historical context, financial exposure and margin. The recommendation itself follows separate business rules: no delay can mean NO_ACTION; a feasible and economically acceptable intervention can mean EXPEDITE; otherwise the result can be DISCOUNT or MONITOR.
That distinction matters. A risk score describes the situation. It does not, by itself, prove that paying to expedite is sensible. The code checks conditions such as the shipping mode and the estimated intervention cost.
The LLM receives these calculations as context. The application parses its response, checks the recommendation label and bounds the confidence value. It also handles several failure paths, including API errors and invalid JSON, by returning the deterministic result with the LLM marked unavailable.
Disagreement is useful information
A model disagreeing with the rules should be visible. FreightSense stores the two assessments and flags the mismatch instead of quietly replacing one with the other.
The override endpoint lets a person record a different decision, a reason and outcome notes. Multiple overrides are retained as separate entries. That history makes it possible to revisit why an operator intervened, rather than seeing only the latest choice.
What the prototype establishes
The repository demonstrates the complete evaluation-and-review path. It does not establish a measured reduction in shipping costs. The financial figures are estimates, and the model’s confidence is a self-reported value, not a calibrated probability of correctness.
For a production version, my next priorities would be durable shared storage, access control, stronger validation of model outputs, and a labelled evaluation set that includes expensive mistakes and ambiguous cases. Those are more useful next steps than adding another model to the comparison.
Implementation details: deterministic rules, LLM evaluation, and API routes.