Article
Local language models in corporate finance — when data must not leave the building
An accounting office’s client ledgers, payroll, contracts with counterparties — some data must never be pasted into a cloud chat, however good the model. A model running on your own machine solves that problem at the price of power and convenience. Below: when it is enough, what it costs and what it looks like in three uses I have built.
Author: Tomasz FordymackiPublished:
Cloud or your own server — a comparison
| Criterion | Local model (Ollama, 7–14 bn parameters) | Cloud model (e.g. Claude) |
|---|---|---|
| Data | never leave the machine — no processing agreements, no client questions | go to the provider; anonymisation and an agreement required; some clients will refuse |
| Reasoning quality | good on narrow tasks (classification, extraction); weak on long analyses | clearly better; handles multi-page documents and reasoning |
| Cost | hardware (a machine with lots of memory or a GPU: a few to a dozen thousand złoty once) + electricity | from tens of złoty a month at small scale; grows with document volume |
| Speed | seconds to a dozen seconds per document on an ordinary machine; seconds on a GPU | seconds, regardless of hardware |
| Maintenance | model and Ollama updates on your side; works offline | zero maintenance; dependence on the provider and its pricing |
| When I choose it | an accounting office, payroll, multi-client data, a client with a “nothing outside” policy | analyses, reports, translations, anonymised data, my own tools |
Three uses that work locally
- 1Classifying posting descriptions in the ledger controllerThe accounting-system export contains descriptions like “inv. 12/03 transp. serv. + pallets” — a rule cannot tell whether that is transport or packaging. The local model receives the description and the list of management lines and returns a proposal with a confidence; below the threshold it goes to the accountant. The data of the office’s fourteen companies never leaves its server, and the accountant approves a dozen cases a day instead of reading hundreds.
- 2A glossary with semantic search (RAG)The base of 176 financial terms behind FinGloss runs on a local embedding model and a local answer model: the question “how does net debt differ from liabilities” retrieves the nearest terms, and the answer cites the source. The model does not invent, because it answers only from what it found — that is the whole difference between RAG and “ask the chat”.
- 3Extracting fields from contracts and invoicesParties, amounts, payment terms, notice periods — from PDF to table, with a human sample check. In pre-transaction analysis that is the list of material contracts in an hour instead of two days; the target’s data stays on the machine, which matters once an NDA has been signed.
What a local model will not do well
- It will not write a results commentary you could send the board without rewriting — the sentences are correct but flat and sometimes wrong in detail.
- It will not read a hundred-page contract at once — small models have too short a context window; you must chunk it, which harms answers about the whole.
- It does not compute — like no language model does. Arithmetic in code, with a test, always.
- It will not replace anonymisation where data must reach a third party anyway (auditor, bank) — locality protects only that one stage.
Frequently asked questions
What hardware is needed?
For 7–8 billion-parameter models a machine with 16 GB of memory is enough — a short task answers in seconds. A 14-billion model wants 32 GB; a GPU (from about 12 GB of its own memory) speeds everything up several times. An office server with 64 GB will process an overnight job queue for a dozen companies.
Is a local model GDPR-compliant “by default”?
It solves one problem — no transfer of data to a third party. It does not solve the others: legal basis, retention, access, backups, logs containing content. The gate-and-zones architecture applies just the same; locality merely removes the processing agreement with a model provider from the list.
Which models?
It changes every few months, so I do not attach to names. The rule: for classification and extraction — the smallest model that passes a test on a hundred of your own examples; for embeddings — a multilingual model, because Polish posting descriptions fare worse in an English-only model. A test on your own data matters more than an online leaderboard.
Let’s talk about your situation
Write a few sentences about the company and the problem. I reply within two working days and the first conversation is free.
Write to me