The Answers Were Already There. Our AI Just Couldn't Find Them.
Our AI agent was escalating every property question to a human. The answers were already there — trapped in a Facebook group. Here's how we gave her access to them.
Here's a problem I didn't expect to spend a day on. Honestly, I figured it was a 30-minute job.
One of our clients, Creative Listing, has an AI agent named Anna that handles property inquiries. Questions come in through their portal, Anna tries to answer them, and when she can't, she escalates to a human.
That part worked. The problem was what she was escalating.
Questions like: "What's the zoning on this parcel?" "Is this property near schools?" "Does anyone know if this neighborhood has HOA fees?"
Not hard questions. But Anna didn't have the answers, even though they existed. They lived in the company's Facebook group, where the same topics had been asked and answered hundreds of times over the years.
Anna couldn't read the Facebook group. So she gave up and called for help.
The real gap wasn't intelligence. It was memory.
Anna is smart. She handles nuanced questions, maintains context, and responds in a friendly, clear way.
What she was missing wasn't intelligence. It was the right information.
The Facebook group had answers to hundreds of property questions, built up over years. That knowledge existed, it just wasn't in Anna's context. So from her perspective, the answer to every property question was: I don't know, let me escalate.
Once we saw this, the fix was obvious.
We gave Anna a memory.
Four steps:
- Extract questions and answers from the Facebook group posts
- Format them into structured knowledge chunks
- Embed them in our local vector database (Postgres + pgvector + Ollama, same stack we use for all our client brains)
- When a user asks Anna a question, run a hybrid search against the knowledge base and inject the top matches into her context
Now when someone asks about zoning on a specific parcel, Anna searches, finds three relevant posts that covered the same ground, and answers from those. She's not guessing, she's retrieving.
The escalations dropped. Anna started handling questions she couldn't have answered before, not because she got smarter, but because she could finally see what the community already knew.
This happens everywhere.
Every company I've talked to has this problem. The knowledge exists, it's just trapped in the wrong places.
Slack channels. Email threads. Old PDFs. Meeting notes nobody's touched since 2022. Support tickets that got resolved and then disappeared.
Your AI agent can't see any of that, so it escalates. Or hallucinates. Or gives a generic answer that doesn't fit the actual situation.
You don't need a smarter model. You need to give the one you have access to what you already know.
What this looks like in practice
You don't need to capture everything. Start with the highest-value stuff you already have:
- FAQ documents (if your team has answered the same question 50 times, those answers belong in the knowledge base)
- Community forums and Facebook groups where customers help each other
- Old support tickets, especially the ones with good resolution notes
- Meeting recordings and transcripts where real decisions got made
Extract it, chunk it, embed it. Then give your agent a search tool to query it at runtime.
Cost is low. We run Ollama locally for embeddings, which is free. Postgres with pgvector handles the search. The hard part is figuring out which knowledge is worth capturing, not building the pipeline.
The bigger picture
There are two ways AI deployments go wrong.
The first: deploy agent, agent fails, blame the AI.
The second: deploy agent, agent fails, ask why, fix the knowledge gap, agent succeeds.
The difference isn't which model you picked. It's whether you treat the knowledge layer as something worth maintaining or just a one-time setup you stop thinking about.
Most teams install the agent, write a prompt, and move on. Nobody asks what the agent doesn't know. Those gaps surface as escalations, hallucinations, and users who got the wrong answer and stopped trusting the tool.
The teams doing this well treat it like onboarding an employee. You wouldn't hire someone and give them zero context about how the business works.
Give your AI access to what you already know. A lot of "it's not smart enough" turns out to be "we never told it anything."
Running AI agents for your business and hitting the same wall? Talk to us.