Skip to content

AI Development · 02 Generative AI & LLMs

Generative AI that answers from your own knowledge

We build chatbots, copilots, and retrieval-augmented generation (RAG) systems on leading language models, then ground and fine-tune them in your own documents, data, and tone. Every answer is traceable to a source, every change is scored against an evaluation set before it ships, and every system is monitored once it is live.

You own the models, code and IP Weekly demos Reply within one business day

AI

Copilot · fine-tuned on your data

Live
Which invoices are overdue by more than 30 days?
14 invoices totalling 42,300 are overdue. Three accounts carry 68% of it — I’ve drafted reminders for review.

0.94

F1 score

86ms

Inference

12k/d

Requests

Generative AI & LLMs, in short

OlDevs designs and builds generative AI systems on leading large language models, including customer-facing chatbots, internal copilots, retrieval-augmented generation (RAG) systems, and domain fine-tuned models, for corporations, associations, government bodies, franchises, and startups in Vancouver and across Canada.Each system is grounded in the client's own documents and data, scored against an evaluation set before launch, and monitored in production for accuracy, cost, and safety. OlDevs has operated as a full-stack technology studio since 2014 and delivers this discipline as part of its AI Development service.

Key facts

01Discipline
02 of 6 in AI Development
02Core patterns
Chatbots, copilots, RAG, fine-tuning
03Model providers
OpenAI, Anthropic, Google, open-weight
04Grounding
Your documents, data, and tone
05Typical pilot
Four to eight weeks
06Ownership
Prompts, data, weights stay yours

02 · Generative AI & LLMs

Language models that know your domain.

What we deliver in Generative AI & LLMs

01

Customer-facing chatbots

Support and sales assistants that answer from your knowledge base, hand off to a person when they should, and speak in your brand's voice.

Support · Sales · Web chat · Handoff · Multilingual

02

Internal copilots

Assistants embedded in the tools your staff already use, drafting, summarising, and looking things up across policies, tickets, and CRM records.

Slack · Microsoft 365 · CRM · Drafting · Summaries

03

Retrieval-augmented generation (RAG)

Pipelines that clean, chunk, embed, and index your documents so the model answers with citations instead of guesses.

Vector search · Embeddings · Chunking · Hybrid search · Citations

04

Fine-tuning and domain adaptation

Supervised and preference fine-tuning on your own examples so the model adopts your terminology, formats, and judgement.

SFT · LoRA · Preference tuning · Open-weight models · Training datasets

05

Prompt engineering and evaluation

Structured prompts, guardrails, and a versioned evaluation set that catches regressions before your users do.

Prompt design · Guardrails · Evals · Regression tests · Red-teaming

06

Content and document generation

Generative workflows that produce reports, proposals, product copy, and structured documents from your data, with a human approval step where it matters.

Reports · Proposals · Product copy · Templates · Approval flow

Process

Our process for Generative AI & LLMs

  1. Discover

    We map the use case, the users, the questions they actually ask, and the sources that hold the answers, and agree what a good answer looks like.

  2. Data prep

    We collect, clean, and chunk your documents and data, set up embeddings and indexing, and build the first evaluation set from real queries.

  3. Build

    We run the model bake-off, design prompts and retrieval, fine-tune where your examples justify it, and iterate until the evaluation set passes.

  4. Integrate

    We connect the assistant to your website, app, Slack, Teams, or CRM with authentication, logging, and handoff to a person built in.

  5. Monitor

    We watch accuracy, cost, and safety in production, review flagged conversations with you, and re-index or retrain as your content changes.

How we work

How we approach Generative AI & LLMs

01

Model bake-off on real questions

We compare leading hosted and open-weight models on your actual queries for accuracy, latency, and cost before committing to one.

02

Grounded answers with citations

Every response is tied to retrieved sources, and the system says it does not know rather than inventing a fact.

03

Versioned evaluation set

We build a test set from real questions with agreed answers and score every prompt, model, or index change against it before release.

04

Everything under version control

Prompts, retrieval logic, and model configuration live in your repository, not in a dashboard nobody can audit.

05

Guardrails tested before launch

Prompt injection, personal-data leakage, and off-topic use are covered by guardrails and probed with adversarial prompts before go-live.

06

Traced in production

Every conversation is traced end to end so you can see token spend, latency, and answer quality per exchange and act on it.

Who it's for

Generative AI & LLMs for organisations that have to get it right.

Whether the audience is a customer, a member, a citizen, or your own team, the work has to hold up under real use.

Corporations

Internal copilots for support, HR, legal, and operations teams that need answers from thousands of documents without data leaving your tenant.

Associations & government

Member and citizen assistants that answer policy, programme, and eligibility questions in plain language, cite the official source, and meet accessibility requirements.

Franchises

One brand-consistent assistant that knows every location's hours, offers, and services, and routes each conversation to the right franchisee.

Entrepreneurs & startups

A product-ready AI feature built on the right model for your margins, with evaluation and monitoring in place so you can ship with confidence and iterate weekly.

12+

years shipping software, since 2014

60%

of support questions resolved without a human

95%+

of answers backed by a verified source citation

Selected work

Selected generative AI & LLMs work.

Client names are withheld; the problem and the outcome are the point.

01AI

A national commercial insurance brokerage with around 450 staff across six offices

Industry
Insurance & financial services
Services
AI development · Generative AI & LLMs · Full-stack development
Duration
14 weeks
“We were nervous about an AI giving confident wrong answers. What we got is a tool that says it cannot find something when it should, and shows its work when it can.”
— Chief Compliance Officer, national insurance brokerage

Challenge

Brokers answered policy-wording questions by searching a shared drive of 30,000 PDF wordings, endorsements and carrier bulletins, or by asking whichever senior colleague happened to be free. New hires took months to become confident, and the compliance team had no way to know which document version a broker had relied on when advising a client.

Solution

We built an internal assistant on a large language model that answers questions only from the brokerage's own document library and shows the exact passage and version it drew from. The retrieval layer indexes wordings, endorsements and bulletins as they land, keeps carrier-specific permissions intact, and declines to answer when it cannot find support in the source material. It runs inside the brokerage's own cloud tenancy, so no client or carrier data leaves their environment, and an audit log records every question, answer and citation. The full-stack build covers the ingestion pipeline, the broker-facing web app and the compliance review screens.

78%

Of wording questions resolved without escalating to a senior broker

11 min → 40 s

Median time to a cited answer

100%

Of answers traceable to a document version

Read the case study

02AI

A provincial electric utility in Manitoba with roughly 1,200 field and office staff

Industry
Energy and utilities
Services
Generative AI & LLMs · Natural Language · AI Development
Duration
5 months
“Our crews stopped phoning the control centre for procedure questions, and when the copilot answers they can see exactly which bulletin it came from.”
— Manager of operational readiness, provincial electric utility

Challenge

Operating procedures, safety bulletins and collective agreement clauses lived across SharePoint, a document management system and two decades of PDFs, so answering a straightforward question often meant asking a colleague who had been there long enough to remember. Crews in the field gave up and phoned the control centre instead. Anything an assistant said had to be traceable to the document it came from, and staff could only ever see the documents their role already permitted.

Solution

OlDevs built a retrieval system over the utility's own document set, chunked and indexed in PostgreSQL with pgvector, with answers generated by a large language model inside the utility's own tenancy so no content left their boundary. Every response cites the source document, section and revision date, and links straight to it. Permissions are read from existing Microsoft Entra ID groups at query time, so the copilot retrieves only what the person asking is already cleared to open.

6 wk

To first department live

−63%

Time spent locating policy answers

+41%

Weekly active internal users

Read the case study

FAQ

Generative AI & LLMs — questions we hear first.

A focused pilot, such as a support assistant grounded in one knowledge base, typically takes four to eight weeks from kickoff to a production release: one to two weeks of discovery and data preparation, two to four weeks of build and evaluation, and one to two weeks of integration and hardening. Larger programmes with fine-tuning, multiple data sources, or several integrations run twelve weeks or more. We agree the scope and dates before work starts and report progress weekly.

Three things: the documents or data the assistant should draw on (PDFs, help-centre articles, policies, product data, ticket history), a sample of the real questions people ask, and a stakeholder who can judge whether an answer is right. Access to the systems we will integrate with, such as your website, Slack, Teams, or CRM, comes later in the project. You do not need in-house AI expertise; we bring that.

We build on leading hosted models from OpenAI, Anthropic, and Google, and on open-weight models such as Llama, Mistral, and Qwen when data residency, cost, or fine-tuning calls for it. Around the model we use vector databases such as pgvector, Pinecone, and Weaviate, orchestration libraries such as LangChain and LlamaIndex, and cloud platforms including AWS Bedrock, Azure OpenAI, and Google Vertex AI, with Canadian regions where residency matters. We choose per project, based on a bake-off on your actual questions, and we are not tied to any vendor.

Before launch we build an evaluation set from real questions with agreed correct answers, and every change to prompts, retrieval, or model is scored against it for accuracy, groundedness, and refusal rate. After launch you get a dashboard and a monthly report covering answer quality, resolution or task-completion rate, user ratings, latency, and cost per conversation, plus a review of flagged conversations so the system keeps improving rather than drifting.

You do. Model-provider and cloud accounts are opened in your organisation's name with your billing, and we work as authorised users. All prompts, retrieval code, evaluation sets, and fine-tuned weights are delivered to your repository under your ownership, with the documentation to run them. Your data is used only to build your system; it is never used to train models for anyone else, and we configure each provider so your inputs are excluded from their training.

As an extension of it. We run a shared channel and a weekly review, pair with your developers on integration, and involve your subject-matter experts in scoring answers, since they know what right looks like. If you have an IT or security team, we bring them in early on data handling and access. At handover we document the system and train your team to maintain prompts, add documents, and read the monitoring, so you are not dependent on us unless you want to be.

It depends on the number of data sources, the integrations involved, whether fine-tuning is needed, and the volume of conversations you expect, so we scope each project individually rather than publishing a single rate. Ongoing model and hosting usage is billed by the providers directly to your own accounts, and we design for cost efficiency from the first prompt. Request a quote with a short description of your use case and we will reply within one business day with a proposed scope, timeline, and quote.

Still have a question? Ask us when you request a quote

Let’s connect

Let’s scope your generative AI & LLMs project.

Tell us what you’re building. We’ll reply within one business day with next steps and a tailored quote — no obligation.

We’ll only use your details to prepare your quote. No lists, no spam.

Call us Request a quote