Your Copilot Has No Memory — Why RAG Fails and LLM Wiki Fixes It

Last week, your copilot answered a question perfectly. Someone asked about a project timeline, and it pulled the right details connected the right dots gave you an answer that felt almost human. Today, someone asks the same question, slightly different wording. And copilot fumbles the exact connection it made yesterday. Here's the assumption everyone walks around with. Copilot remembers your organization. It doesn't. It searches. Every single time, from zero, that's the tension nobody talks about. Retrieval and memory are not the same thing. One finds information, the other builds on what it already knows. Microsoft built copilot on the first one and let the marketing imply the second. There's a pattern circulating right now. Born out of AI research circles, that treats knowledge as something you compile, not something you search for over and over. We'll name it in a few minutes. By the end of this episode, you'll understand why your copilot deployment feels sharp in demos and shallow in production and what structural change actually fixes it. Quick thing before we get into it. If you're into this channel because you want to understand the systems behind Microsoft 365, not just the button clicking, hit subscribe on M365FM podcast. This episode is exactly the kind of thing we do here. And I want to set the expectation right now. This isn't a copilot feature tour. We're not walking through settings menus. This is a structural diagnosis, the kind you do on a system that keeps almost working but never quite getting there. So if you came for prompt tips, this one's going to disappoint you. We're going under the interface into the architecture. That's where the real answer lives. The day one copilot promise. Think back to how copilot got pitched to you or to your leadership or in whatever deck convinced someone to write the check. Ask questions in plain English. Get answers grounded in your organization's own data. No more digging through SharePoint. No more pinging three people on teams to find one document. Just ask and copilot knows. Microsoft backed that pitch with numbers. 11 minutes saved per day per user. Do the math across a year and that's close to a full work week handed back to every employee. That's a real number and for a lot of organizations it's the number that justified the license cost. But here's what nobody set out loud during that pitch. Baked into asked questions and get grounded answers is an implicit promise. This thing gets better as you use it. You feed it more meetings, more documents, more decisions and it becomes sharper, more attuned to how your organization actually works. That's the story everyone heard, even if nobody wrote it on a slide. What actually ships is something different. It's a retrieval system wrapped in a chat interface. Dressed up just enough to feel like it's learning. It isn't. Every time someone types a question, copilot goes and looks, it doesn't recall. It searches, finds what looks relevant and generates a response from that. Ask again tomorrow and it searches again as if yesterday never happened. Now you might be thinking this sounds like a UX complaint, a minor annoyance. It's not. This is a governance and adoption problem and those are expensive in a completely different way than a clunky interface. When an assistant gives inconsistent answers to the same underlying question, people stop trusting it for anything that matters. They keep it around for quick summaries and calendar questions, sure. But the moment a decision actually rides on the answer, they go back to asking a person. That's not a bug report. That's an organization quietly deciding the tool isn't reliable enough to lean on. And once that decision gets made, even informally, adoption stores. Not because people didn't try copilot because they tried it, got burned once or twice by an inconsistent answer and adjusted their behavior around it. Nobody files a ticket for that. It just shows up months later as flat usage numbers and a leadership team asking why the tool they paid for isn't delivering the return they were promised. So the gap between the pitch and the product isn't cosmetic. It's the difference between a tool that compounds in value and one that just sits there, static, no matter how long you've had it deployed, to actually see where that gap comes from. You have to look at what's happening mechanically every single time someone types a question into copilot because the behavior we just described isn't random. It's the direct, predictable result of how the system is built underneath the chat window. What drag actually does? So let's name it directly. Our Ag stands for retrieval augmented generation. Strip the jargon and here's what that actually means. The system retrieves some information, then uses that information to help generate an answer. That's the whole concept. Retrieval, then generation, glued together. Here's how it works mechanically, step by step. First, your documents get chunked. Every SharePoint file, every Teams transcript, every policy PDF gets sliced into smaller pieces, usually a few hundred words at a time. Second, each of those chunks gets converted into something called an embedding, which is just a mathematical fingerprint of what that chunk means. Third, when someone types a question, that question also gets turned into a fingerprint and the system searches for chunks whose fingerprints look similar. Fourth, whichever chunk score highest gets stuffed into a prompt alongside the original question. Fifth, the language model reads all of that and generates an answer. Now pay attention to that word chunk because it's doing more work than people give it credit for. Documents get sliced into fragments with no awareness of the whole. A 40-page project plan doesn't get read as a project plan. It gets cut into 15 or 20 disconnected pieces. Each one judged purely on whether it resembles the question being asked. The system never holds the full document in mind. It holds fragments and it bets that the right fragments to stick together will look enough like a coherent answer. This is a search engine with a language model bolted on the end. It's not a thinking system and it was never built to be one. The intelligence you're seeing when co-pilot gives you a good answer isn't co-pilot understanding your organization. It's a language model doing a genuinely impressive job of stitching together whatever fragments got handed to it. That's a real skill. It's just not memory and it's not comprehension of the whole. If you want a simple way to picture this, think about typing a question into SharePoint Search. You type in a phrase, SharePoint goes and finds documents that contain similar words or concepts and it hands you a list of results. Ragn does almost exactly that. Except instead of handing you the list, it reads the top few results and writes you a paragraph summarizing them. Same search, same fragments, just a nicer wrapper on the output. And look, for a single look up, this works fine. Genuinely fine. If someone asks, what's our current PTO policy? Ragn finds the chunk that talks about PTO, generates a clean summary, and everyone moves on with their day. One question, one search, one answer, no complaints. The failure doesn't show up there. It shows up the moment you need continuity. The moment a question depends on something the system already figured out five minutes ago or yesterday or last month. Because Ragn has no concept of already figured out. It only has search again. And that's exactly where the cracks start to show. The stateless trap. Let's put a real word on what we just described because it matters. That behavior is called statelessness. Every query starts from zero, no matter how many times you've asked something related before, no matter how obviously connected today's question is to yesterday's. The system has no concept of before. There's only this query right now treated like the first one it's ever seen. Walk through what that actually looks like inside co-pilot. Someone on a project team asks, what's the status of the client onboarding for Meridian? Co-pilot searches, finds the right team's thread, the right sharepoint update, maybe a status email from last week and gives a genuinely good answer. Clean, accurate, useful. The person moves on with their day, impressed. Tomorrow, that same person asks a follow-up, something like, what's blocking us from finishing that onboarding? Notice what just happened in that sentence. That onboarding. A human would hear this and instantly connected to yesterday's conversation, no clarification needed, co-pilot doesn't. It has no idea that onboarding refers to Meridian because it never stored the fact that Meridian came up yesterday. So it searches again from scratch, hoping the word onboarding alone points it back to the right documents. Sometimes it gets lucky, sometimes it grabs the different clients onboarding thread entirely and now you've got a confidently wrong answer standing right next to yesterday's confidently right one. Here's the part that actually costs you. Nothing compounds. There's no accumulated understanding of the Meridian project building up over these two conversations, no growing sense of who's involved, what decisions got made, what the blockers have historically been. Each question is an island. You could ask co-pilot about the same project 50 times over three months and it would know exactly as much on question 50 as it did on question one because it never carried anything forward between them. Now compare that to how an actual colleague would handle it, say the person who handled Meridian's onboarding last month, the answer to what's blocking us comes wrapped in memory. They remember the client pushed back on the integration timeline two weeks ago. They remember a decision got made in a meeting to reprioritize. They don't need to redrive any of that because they were there and they kept it. Every follow-up question they answer builds on the last one, gets sharper, gets more useful over time. That's what a second conversation with a person actually feels like and it's exactly what a second conversation with co-pilot doesn't. So name the real cost here plainly because it's not what people assume. The cost isn't wrong answers, not usually. Rags often technically accurate on any single question you throw at it. The cost is shallow answers, ones that never deep in no matter how long you've been using the tool, no matter how much history exists between you and the system. You get competence without growth, fluency without accumulation. And here's the thing worth sitting with before we go further. This isn't a training problem, it's not a prompt problem and it's not a rollout problem you can fix with better change management. It's what happens structurally every single time you take a search mechanism and ask it to do memory's job. Search was never built to remember, it was built to find. And no amount of clever phrasing changes what the system was designed to do underneath. Why this isn't a prompting problem? At this point, someone in IT is going to have a reflex reaction to everything we just said. They're going to blame the prompts. Maybe the rollout was rushed, maybe users just don't know how to ask questions the right way yet and a few training sessions will close the gap. That instinct makes sense. It's usually where the blame lands first because it's the easiest thing to fix without touching the underlying system. Rewrite the prompt library, run a lunch and learn, tell people to be more specific and show a better phrasing helps at the margins. But it doesn't touch the actual problem and there's a number that makes this hard to ignore. Research on structured knowledge systems shows organizations using layered knowledge with LLMs cut AI error rates by more than 60% compared to standard rag setups, 60%. That's not a rounding error and it's not the kind of gap you close by teaching people to phrase their questions better. Think about what that number actually implies. It's not measuring how well people ask questions. It's measuring what the system does before a question ever gets typed. 60% fewer errors didn't come from smarter users. It came from a completely different relationship between the AI and the knowledge underneath it, one where the knowledge had already been organized, connected and reconciled before anyone opened a chat window. So here's the reframe and it's worth sitting with because it changes where you point your effort. You can't prompt your way out of an architecture that has no place to store what it learns. It doesn't matter how carefully worded the question is. If the system searches fresh every time, then even a perfect prompt just retrieves a fresh set of chunks and generates a fresh answer, disconnected from anything that came before. A better prompt gets you a better single search. It doesn't get you memory because memory was never something a prompt controls. That's the part worth naming plainly. The fix doesn't live in the interface. It doesn't live in how questions get asked. It lives below all of that in how knowledge gets prepared and represented before Copilot ever touches it. If the knowledge sitting behind the assistant is a pile of raw documents waiting to be chunked on demand, no amount of prompt engineering changes what happens next. The system is built to search and search is what it will keep doing no matter how the question gets phrased, which raises the actual question worth spending the rest of this episode on. Not how do we ask Copilot better questions. But what does the system actually look like when it's built for memory instead of search because that system exists and it works on a completely different principle than the one copilot ships with today? Enter the LLM Wiki. Here's that system. It didn't come out of Redmond and it didn't come out of a product road map. It surfaced from AI research circles and the person most associated with articulating it clearly is Andre Kapati, one of the more influential voices in how people think about working with these models day to day. He wasn't building an enterprise product. He was solving his own problem, building a personal knowledge base for research he cared about and the pattern he landed on is the one worth stealing. The core move is simple to say and genuinely different in practice. Instead of searching raw documents every single time a question comes in and AI reads the sources once and builds a structured interlinked knowledge base out of them. Once, not once per question. Once, period, then it keeps that structure around and refers back to it. Picture three layers because this is where it stops being an abstract idea and starts being something you could actually build. Layer one is raw sources. These are your original documents whatever they are and they stay read only. Nobody touches them, nothing re-rides them. They're ground truth sitting there untouched the same way a source document should. Layer two is the Wiki itself. This is where the actual work happens. It's a set of pages, the AI writes. One per concept, one per entity, one per recurring topic, all cross linked to each other the way a real Wiki works. Not a database of chunks. Pages written in something like plain language meant to be read as a whole. Layer three is the schema. Think of it as the rule book. It tells the AI how to structure new pages, how to decide when something updates an existing page versus when it deserves a new one, how to format things consistently so the Wiki doesn't turn into chaos as it grows. Now here's the part people get wrong on first hearing this. They assume the Wiki is just static storage, a folder where documents get copied and organized once. It's not, it's maintained. Every time a new source comes in, the AI reads it and does real work with it. It updates existing pages if the new source adds detail to something already compiled. It creates new pages if the source introduces a genuinely new concept that didn't exist in the Wiki yet. And if the new source says something that contradicts what's already there, it doesn't just override quietly. It flags it, that flag is the whole point and we'll come back to why in a bit. So say the line plainly, because it's the whole argument in one sentence. Knowledge gets compiled once then kept current. That's the exact opposite of what Ragn does. Ragn rebuilds understanding from scratch on every query. A Wiki builds it once and then just maintains it, the way you'd maintain anything that's actually alive and growing instead of something you keep rebuilding from nothing. To see why this distinction actually matters for co-pilot and not just as a nice idea for personal research projects, you need to put the two systems side by side and watch where the actual work happens in each one. Ragn vs LLM Wiki side by side. Put the two systems next to each other and ask one question, where does the thinking actually happen? Not the answering, the thinking. The part where raw information turns into something organized enough to be useful. That question tells you almost everything you need to know about why one system compounds and the other doesn't. In Ragn, the thinking happens at query time. Every single question triggers the whole process from the beginning. Fresh search across the embeddings, fresh chunking logic deciding which fragments matter. Fresh synthesis, where the model reads whatever got pulled and stitches it into an answer on the spot. None of that work exists until the question shows up. The system has zero understanding sitting in reserve. It manufactures understanding, on demand every time and then throws it away the moment the answer gets delivered. In the LLM Wiki, the thinking happens at ingestion time. That's the entire structural difference and it's easy to miss because it sounds small. The synthesis is already done before anyone asks the question. When a new source comes in, the AI does the hard part right then. Reading it, deciding what it means, connecting it to what's already compiled, updating pages, flagging conflicts. By the time someone actually types a question into the system, there's nothing left to figure out. The system isn't generating understanding. It's retrieving understanding that already exists, fully formed, sitting on a page. That difference shows up in a number worth paying attention to. For focused knowledge basis, LLM Wiki can cut token usage by roughly 95% compared to naive rag document loading. 95% that's not a marginal efficiency gain that's a different order of magnitude. Here's why that number exists and it's not magic. Rag spends tokens researching and restitching every single time. It has to pull multiple chunks, feed all of them into the prompt alongside the question, and let the model do the work of reconciling fragments that were never written to sit next to each other. All of that costs tokens every query, forever. A Wiki skips that entirely. You're not researching and restitching. You're reading a page that was already organized, already coherent, already written, to be read as a whole. Less raw material has to get shoved into the model's context window because someone or something already did the organizing work in advance. Now, let's be honest about where this doesn't apply because overselling it here would undercut the whole argument. Rag stays the default for large, relatively static repositories and that's completely fine. If you're sitting on hundreds of thousands of documents that don't change much and get searched broadly across a huge range of topics, Rag's search first design is doing exactly what it's built for. The Wiki pattern isn't trying to replace that. Where the Wiki pattern actually shines is below roughly 50 to 100,000 tokens of compiled knowledge. That's a bounded focused domain. Not everything your organization has ever produced, a specific slice of it, the stuff people ask about again and again compiled once in Keptrap. None of this stays theoretical for long though because there's a version of exactly this tension already sitting inside Microsoft 365 right now, quietly shaping how co-pilot behaves every time someone opens a chat window. Where co-pilot's retrieval actually lives, let's get concrete about where all of this actually plays out because up to now we've been talking architecture in the abstract. Inside Microsoft 365, co-pilot indexes SharePoint OneDrive, Teams Exchange, and whatever other connected sources your organization has plugged in, it respects your permission structure while it does this. So someone only sees what they're already allowed to see. That parts real and it matters. But look at what that list actually is. It's a description of Rag at Enterprise Scale. Microsoft calls it Graph Grounding. And the name makes it sound like something new, something built specifically for this moment. It isn't. Strip the branding off and its retrieval same as we've been describing this whole episode, just dressed in Enterprise clothing. Co-pilot reaches into Graph, pulls whatever content looks relevant to the question and generates an answer from those fragments. That's the identical mechanism running against a much bigger, much messier pile of source material. Now, the permission model deserves real credit here because it solves a genuinely hard problem. Making sure someone in finance doesn't accidentally see HR's compensation planning docs through a chat window that's not trivial and Microsoft built real infrastructure to handle it. But notice exactly what that infrastructure solves. It solves the access problem, who's allowed to see what. It does absolutely nothing about the memory problem we spent the last several sections on. Permissions decide whether a document is eligible to be retrieved. They say nothing about whether the system remembers retrieving it or builds on it or connects it to the conversation from yesterday. Access and memory are two entirely separate questions and solving one doesn't touch the other. Here's where it gets interesting though. Microsoft's own guidance quietly admits something worth pausing on. Admins can now register specific SharePoint sites as knowledge sources for Co-pilot. Read that carefully. That feature only makes sense if not all content is equally trustworthy or equally structured or equally worth trusting Co-pilot to pull from. If every SharePoint site were equally reliable, there'd be no reason to let admins flag some of them as special. The existence of that toggle is Microsoft tacitly admitting the obvious. Some of what's sitting in your tenant is good, organized, current, and some of it is stale, duplicated, or just noise. And Co-pilot, left to its own devices, treats all of it the same. That single feature is Microsoft nudging your organization toward curation without ever fully naming the underlying issue out loud. It's a quiet acknowledgement wrapped in an admin setting. Nobody in the documentation says, our retrieval system can't tell good content from bad content, so we built your workaround. But that's functionally what registering a knowledge source does. It's damage control for a structural limitation presented as a feature. And curation, as a starting point, is genuinely useful. But it's not the same thing as compilation, and the difference between those two words is exactly where this conversation needs to go next. Curation isn't compilation. So let's draw the line clearly, because it's easy to conflate these two ideas. And conflating them is exactly how organizations end up thinking they've solved the memory problem when they've only solved the access problem twice. Marking a SharePoint site as a trusted knowledge source tells Co-pilot where to look. That's all it does. It's a pointer. It says, when you're searching, wait this location more heavily than that one. What it doesn't do, what it can't do, is tell Co-pilot what it learned from that site. There's no learning happening in the act of registering a knowledge source. There's just a narrower, more confident search. And that's the part we're sitting with. Curated sources still get retrieved fresh, chunked fresh, synthesized fresh every single time someone asks a question. Registring a site as trusted doesn't change the mechanism underneath it at all. It changes which pile of documents get searched. It doesn't change the fact that it's still a search starting from nothing every time. You've made the haste-axe more and more reliable. You haven't given the system a memory of what's in it. The wiki pattern goes one step further, and this is the actual structural difference. It doesn't just point at good sources and say, look here first. It processes those sources into standing knowledge. It reads them once and turns them into something that already exists before the next question arrives. That's not a better pointer. That's a different kind of object entirely. Here's the planist way to say it. Curation is picking better ingredients. You've gone through the pantry, thrown out, what's expired, kept what's fresh, and told the kitchen, cook from the shelf, not that one. That's real and it matters. But compilation is actually cooking the dish once and keeping it in the fridge ready to serve. Nobody's back in the kitchen chopping and searing from scratch every time someone's hungry. The work happened once. What's left is just serving it. Without that second step, even perfectly curated content still forces co-pilot to start over on every question. You can register every sharepoint site in your tenant as a trusted knowledge source, get the pantry as clean as it's ever going to be and co-pilot will still chunk those documents fresh, still search them fresh, still synthesize an answer fresh every single time. Curation narrows what gets searched. It doesn't touch how the searching works or what happens to the understanding once the answer's been delivered. It evaporates, same as always, waiting for the next question to rebuild it from nothing. So curation is a real improvement and worth doing regardless of anything else in this episode, but it's not the fix. It's step one of a process that most organizations stop halfway through mistaking a cleaner haystack for an actual memory. This is exactly where the week is three-layer structure stops being an abstract idea from AI research circles and starts being something you could genuinely build inside your own tenant. The three layers applied to co-pilot. So let's put actual Microsoft 365 nouns onto those three layers because that's where this stops sounding like a research idea and starts sounding like something you could sketch on a whiteboard this week. Layer one raw sources. In your tenant, that's your SharePoint libraries, your Teams meeting transcripts, exchange threads that carry real decisions buried in reply chains. Viva engage posts where someone announced a policy change nobody bothered to document properly. All of it stays exactly where it is. Nobody touches it, nobody edits it, nobody deletes it to clean things up. It's ground truth, sitting untouched, the same way it should sit, whether or not you ever build anything on top of it. Layer two, the wiki itself. This is the part worth picturing concretely, not a folder of documents. A maintained set of pages, one per project, one per client, one per recurring decision, your organization keeps having to re-explain to somebody new. Cross-linked, so the page about a client connects to the page about the project, connects to the page about the compliance interpretation that shaped how that project got scripted. This is the layer that didn't exist an hour ago in your tenant and does exist now once someone builds it. Layer three, the schema. Think of this as the rule book, the thing that decides how any of this actually happens when a new Teams transcript comes in, does it update an existing client page or does it spin up a new one? How does a page get formatted? So it's consistent with every other page in the wiki. And critically, what happens when something in that transcript contradicts what's already written down? The schema is what answers those questions before anyone has to answer them manually every single time. Now here's the thing, none of this should feel foreign to anyone who spent time in Microsoft 365 governance. Map it onto what you already understand. Retention labels already decide what happens to content based on rules you defined in advance. Content types already give structure to what would otherwise be undifferentiated files. Metadata schemas already tell you what fields a document needs before it counts as complete. This isn't a new discipline landing on your desk out of nowhere. It's an extension of information architecture work. Your organization has probably already been doing just aimed at a different output. Instead of the schema governing how a document gets filed, it's governing how a wiki page gets written and maintained. But here's the part that doesn't happen automatically and it's worth being blunt about it. Someone or something has to actually do the compiling. The three layers we just walked through don't assemble themselves the moment you understand the concept. Current copilot deployments don't ship with a background process that reads your SharePoint libraries and writes structured pages out of them. That work is real work. It has to be built or run or triggered by something that treats it as its job. Right now, in a standard Microsoft 365 environment, nothing is doing that job. Copilot searches it doesn't compile the wiki layer if it exists at all exists because somebody decided to build it, not because it came in the box. Which puts a fairly obvious question on the table and it's the one worth answering next. If this layer doesn't ship by default and it doesn't build itself, who or what is actually supposed to build it inside a Microsoft ecosystem? Because the three layer structure only matters if there's a real answer to that question, not just a diagram that looks clean on a slide. Who builds the wiki layer? Let's be honest about something before we go any further because glossing over it would make the rest of this episode sound like a feature announcement. This isn't a Microsoft ship capability today. There's no toggle in the admin center labeled turn on compiled knowledge. What we've been describing is a pattern, something you'd implement on top of your existing knowledge state, not something that arrives with your next licensing renewal. So what does implementing it actually look like? Picture an AI agent and if you've spent any time in Copilot Studio, the shape of this will feel familiar. Not unlike how a Copilot Studio agent gets built to watch a trigger and take an action or how a custom graph connected agent gets set up to reach into specific data sources on a schedule. Same underlying capability pointed at a different job. Instead of answering a user's question in the moment, this agent's entire purpose is reading sources and writing structured pages out of them. It goes into a SharePoint library, reads what's there, decides what belongs on an existing page and what deserves a new one and writes accordingly. Nobody's chatting with it. It's not waiting for a prompt. It's just working quietly in the background whether or not anyone's logged in that day. That distinction is worth sitting with because it's where agente orchestration actually earns its name instead of just being a buzzword. This isn't a chatbot answering questions on demand. It's a background process running on its own schedule, compiling and maintaining knowledge, whether or not a single person asks it anything that day. The value isn't in the conversation. It's in the fact that the conversation never has to start from zero because something already did the work before the question showed up. And here's the part that should feel encouraging rather than daunting. None of the infrastructure to build this is missing. Copilot Studio already supports custom connectors, reaching into whatever data source you pointed at. It already supports scheduled flows, running processes on a timer without a human kicking them off manually. Those two capabilities connectors and schedules are most of what you need to build the compiling half of the system. The pieces exist in your tenant right now. They're just not assembled this way by default because nobody's shipped a template that says, "Here's how you wire a connector and a schedule together to build a wiki instead of just automating a workflow." Which means the real barrier here isn't technical capability. It's a shift in how IT thinks about what Copilot even is. Right now, most organizations think of Copilot as one thing, a chat interface. You type, it answers. That's the whole mental model. But once you see the wiki pattern clearly, Copilot stops being one system and starts being two. There's the compiler, the background agent quietly reading, writing and maintaining structured pages. And there's the responder, the familiar chat interface that answers questions, except now it's answering from compiled pages instead of raw fragments. Two systems doing two different jobs working together. Most IT teams today have only built or bought the second one. Once you see it that way, as two systems instead of one, a different question starts to matter more than efficiency ever did. Because splitting the compiler from the responder doesn't just make answers cheaper or faster. It changes something more fundamental about whether people trust what Copilot tells them in the first place. The trust problem, RAAG, can't solve. There's data behind this, and it's worth putting a real number on it before we go further. Over 60% of enterprise IT leaders said they intended to expand generative AI for knowledge discovery. That's not a small commitment. That's leadership teams planning to lean harder into this budgeting forward, telling their boards it's the next phase of the rollout. And then in a lot of those same organizations adoption stalls, not because people forgot to use the tool, because somewhere along the way, the answer started feeling inconsistent. And inconsistency is quietly fatal to whether people keep coming back. Here's what's worth understanding about that inconsistency, because it isn't random noise you can chalk up to bad luck or a model having an off day. It's structural. Ask the same underlying question two different ways, and RAAG doesn't just phrase the answer differently. It retrieves an entirely different set of chunks, different search, different fragments pulled from the index different material handed to the model to synthesize from. Two people asking about the same policy, one saying, what's our remote work policy and the other saying, can I work from home three days a week? Might get two answers that don't even agree with each other. Because under the hood, they triggered two separate searches that happened to land on different documents. Users notice this fast. And here's the part that should worry you more than a formal complaint would. They don't complain loudly. Nobody files a ticket that says co-pilot gave me inconsistent answers. What actually happens is quieter and harder to track. Someone gets burned once, maybe twice, on a question that mattered. And they just stop asking co-pilot the harder questions. They still use it for quick summaries, for drafting an email, for things with no real stakes attached. But the moment a real decision rides on the answer, they go find a person instead. That shift happens silently one person at a time. And it never shows up as a complaint. It shows up months later as usage numbers that look fine on paper, but never touch the questions that actually mattered. Now compare that to what a compiled wiki page does. Ask it the same underlying question five different ways, phrased five different ways, worded by five different people. You get the same answer every time. Because the synthesis already happened once carefully before any of those five people ever typed anything. There's no fresh search rolling the dice on which fragments get pulled. There's one page already reconciled, already settled, and every version of the question just gets pointed at it. This is the part worth naming directly, because it's easy to file under nice to have and move past. Consistency isn't a polish feature. It's a trust mechanism. It's the actual thing that decides whether someone is willing to let a tools answer carry weight in a real decision, or whether they keep it around for the easy stuff and quietly root anything important through a person instead. Once trust breaks on the hard questions, it doesn't come back just because the interface gets a new code of paint. And trust is only half the payoff here. The other half shows up somewhere you might not expect. And how much redundant work is happening across your organization right now. Work that exists purely because nobody trusted search to surface the answer that already existed. Killing redundant knowledge creation? There's another number worth putting on the table and it points at something most organizations never think to measure. Early adopters of AI-based knowledge discovery report a 25% to 40% reduction in redundant knowledge creation. That's a strange phrase, redundant knowledge creation. So let's unpack what it actually means. Because once you see it, you start noticing it everywhere. Here's the mechanism, and it's simpler than it sounds. When nobody trusts search to surface the right answer, people don't just give up. They rebuild. Someone needs a document explaining how a process works. They run a search, get nothing solid, and instead of digging further, they just write it again from scratch. Except that document already exists. It's sitting in a team's thread from four months ago or buried on page three of an old SharePoint site. Nobody thinks to check anymore. The information isn't missing. It's unfindable, which functionally amounts to the same thing. And unfindable information gets recreated over and over by different people who have no idea. They're duplicating work that already happened. This is where a compiled weeky changes the math. Instead of a search turning up nothing useful or five slightly different fragments that half answer the question, it surfaces the existing answer directly with a citation pointing back to where it came from. Someone doesn't spend an afternoon reconstructing a document that already lives in the company's collective memory. They find it in minutes because the weeky already did the work of deciding that this concept exists and here's where it lives. And this connects back to something we touched on earlier, but it's worth stating in its sharpest form here. Ragtreeze every piece of content is equally hard to find every single time. It doesn't matter if someone asked the exact same question last week and got a great answer. The system starts over searching the same haystack with the same odds of missing what it's looking for, a wiki flips that completely. It treats concepts as already found permanently. Once something's compiled onto a page, it stays found. Nobody has to get lucky with their search terms ever again. Frame the business case here in the planist terms possible because this is where a lot of the argument in this episode stops being architectural and starts being financial. Less duplicated effort isn't a technical when you put on a slide about system design. It's hours, actual hours given back to people doing actual work instead of quietly rebuilding something that already existed somewhere in the tenant, multiply that across a few hundred employees over a year and you're not talking about a nice efficiency gain. You're talking about a meaningful chunk of your organization's collective time spent solving a problem that had already been solved. But none of this holds up if the material feeding the wiki in the first place is a mess and that's the next real constraint worth being honest about. Garbage in, still garbage out. Let's be direct about something the whole wiki pattern depends on because it's the part that's easiest to skip past when you're excited about the architecture. The wiki doesn't fix bad source material. It organizes what you give it. It doesn't audit whether what you gave it is true, current or even still relevant. That job never belonged to the compiler and it never will. Here's what that means in practice. If your share point is full of outdated policy documents, documents nobody archived properly, documents that describe a process your organization stopped using two reogs ago, the wiki will read every one of them faithfully and compile them into clean, confident looking pages. That's the uncomfortable part. The output won't look messy or uncertain. It'll look organized, it'll look authoritative and it'll be wrong presented with the same polish as everything else in the wiki that happens to be right. This is exactly where content lifecycle management stops being optional and it's worth pointing out that Microsoft has been saying this since its own January 2024 Copilot readiness guidance that guidance wasn't subtle about it. Content needs owners, it needs version control, it needs a process for retiring things that are no longer accurate instead of letting them sit indefinitely next to whatever replaced them. None of that is new advice invented for this episode. It's been sitting in Microsoft's own documentation for a while mostly ignored because it sounded like housekeeping rather than something that determines whether your AI investment actually works. So say it plainly because this is the part organizations keep skipping. Structured knowledge work starts before any AI touches your content retention labels still matter, content type still matter, clear ownership. So someone's actually responsible for knowing whether a document is still true, still matters. None of that becomes less important once you add a wiki layer on top. If anything, it becomes the entire foundation the wiki is standing on. And here's the sharper version of the risk the one worth sitting with. The wiki pattern raises the stakes on curation because it makes bad content look more authoritative, not less. A messy pile of raw documents, at least looks messy. People approach it with some skepticism. A beautifully compiled wiki page doesn't invite that same skepticism. It reads like something's already been verified, organized and settled even when the underlying source was garbage the whole time. So assuming your source material actually is solid, here's what happens next. Once new content starts arriving and the system has to decide what to do with it. When bad knowledge looks authoritative, here's what that means in practice. If your share point is full of outdated policy documents, documents nobody archived properly, documents that describe a process your organization stopped using two reoggs ago, the wiki will read every one of them faithfully and compile them into clean, confident looking pages. That's the uncomfortable part. The output won't look messy or uncertain. It'll look organized, it'll look authoritative and it'll be wrong, presented with the same polish as everything else in the wiki that happens to be right. This is exactly where content lifecycle management stops being optional. And it's worth pointing out that Microsoft has been saying this since its own January 2024 co-pilot readiness guidance. That guidance wasn't subtle about it. Content needs owners. It needs version controlled. It needs a process for retiring things that are no longer accurate instead of letting them sit indefinitely next to whatever replaced them. None of that is new advice invented for this episode. It's been sitting in Microsoft's own documentation for a while, mostly ignored because it sounded like housekeeping rather than something that determines whether your AI investment actually works. So say it plainly, because this is the part organizations keep skipping. Structured knowledge work starts before any AI touches your content. Retention labels still matter, content types still matter. Clear ownership, so someone's actually responsible for knowing whether a document is still true still matters. None of that becomes less important once you add a wiki layer on top. If anything, it becomes the entire foundation the wiki is standing on. And here's the sharper version of the risk the one worth sitting with. The wiki pattern raises the stakes on curation because it makes bad content look more authoritative, not less. A messy pile of raw documents at least looks messy. People approach it with some skepticism. A beautifully compiled wiki page doesn't invite that same skepticism. It reads like something's already been verified, organized and settled, even when the underlying source was garbage the whole time. So assuming your source material actually is solid, here's what happens next. Once new content starts arriving and the system has to decide what to do with it. So picture what actually happens when a new source lands because this is where the wiki pattern stops being a diagram and starts being a repeatable process. A document shows up, maybe a policy update, maybe a new team's transcript from a client call. The system reads it once, not skims it, not chunks it into fragments for later retrieval. Reads it, the way you'd read something if your job was to understand it well enough to explain it to someone else. Key concepts get pulled out during that read. What's this document actually about? What does it claim? What does it connect to? From there, one of two things happens. And this is worth being precise about because it's the whole mechanism. If the new source adds detail to something already compiled, an existing page gets updated. Say there's already a page about a client's onboarding process, and this new transcript adds a wrinkle nobody had documented yet. That page gets richer. It doesn't get duplicated. It doesn't sit next to an old version creating confusion. It gets updated in place. But if the source introduces something genuinely new, something that doesn't map onto any page that already exists, a new page gets created. And whichever of those two things happens, related ideas get linked. The new page connects to the client it belongs to, the project it touches, the policy that shaped it, nothing sits alone. Here's the part that matters most, though. And it's the part, rag has no equivalent for it all. What happens when the new information contradicts something already sitting in the wiki? Say the old page claims a process works one way, and the new source says it changed. The system doesn't just override the old claim and move on like nothing happened. It flags it. Both versions get preserved with a note that something shifted, and a signal that someone should look at this before treating either version as settled. This flagging behavior isn't a nice to have detailed buried in the mechanics. It matters enormously for actual enterprise use, because businesses change constantly. And most of that change happens quietly. In a policy update here, a strategy pivot there, a decision that got reversed without anyone sending out a formal memo about it. When a policy update contradicts an old assumption that's been floating around for months, that contradiction doesn't get buried under a pile of documents that all look equally current. It gets surfaced directly to whoever's supposed to notice it. Now hold that against what rag does with the exact same situation, because the contrast is where this really lands. A stale document sits in your share point next to the document that replaced it. Someone asks a question, and rag retrieves both. Not one, both, as two equally weighted chunks, stitched together into an answer with absolutely no signal about which one is actually current. The model doesn't know one superseded the other. It just sees two fragments that seem relevant and does its best to synthesize something coherent out of material that's quietly contradicting itself. Nobody flagged anything. Nobody got a signal that something needed a second look. The contradiction just sits there, silently corrupting whatever answer comes out the other end. That difference, flagging versus silent averaging, points at something bigger than a technical detail about how updates get handled. It's a concept worth naming directly, because once you see it, you can't unsee it in every rag deployment you've ever looked at. Contradiction as a feature, not a bug. Most systems, when they run into contradictory information, treat it as noise. Something to smooth over, average out, resolve into a single, clean answer, so nobody has to look at the mess underneath. The wiki pattern does something almost nobody expects the first time they see it. It treats contradiction as a signal, worth preserving, not a problem to hide. Think about what a contradiction actually represents inside a business. It's rarely a mistake. Most of the time, it's a policy shift somebody made three months ago and never fully announced, or a strategy pivot that quietly replaced last year's plan, or a decision that got reversed after a client pushed back, and only half the team ever heard about the reversal. Contradictions in an organizational context are usually just changed that hasn't finished propagating yet. Here's where rag runs into a wall it can't get passed. A stateless system has no way to represent this used to be true, now it isn't. There's no mechanism for that. Rag just retrieves, whichever chunk seems more relevant at the moment the question gets asked, and depending on wording, embedding similarity, whatever's closest in the vector space, it might hand you the old policy or the new one. Neither answer is technically wrong. The document exists. It's just that rag has no concept of existed, but got superseded. Every chunk is equally present, equally retrievable, equally weighted, forever, regardless of whether it's still true. A compiled wiki does something structurally different. It can hold both the old state and the new state on the same page, with a note attached explaining when the change happened and why, if that reason is known, not too competing documents floating in a search index with no relationship to each other. One page, with history built into it. The old assumption is still there, but it's marked as superseded, dated, explained. Nothing gets erased, nothing gets hidden, it just gets sequenced. This is the actual difference between an assistant that answers and an assistant that understands your organization's history. Answering means producing something plausible when asked. Understanding history means knowing that what's true today used to be something else and knowing why the shift happened and being able to tell you that story instead of just picking aside and hoping it's the right one. Understanding history like that is one thing, making it usable across an entire enterprise, not just one project page or one client file, but thousands of them, spanning years, across every team, is an entirely different problem. And it's the one worth turning to next. The scale question. So let's be honest about the limit here, because this pattern doesn't scale the way you'd wanted to, just by wishing hard enough. Personal scale wikis, the kind cup he's been describing, work well around 100 or so source documents. That's the range where one person or one small compiling agent can keep everything coherent, cross-linked and current without the whole thing turning into its own management problem. Your enterprise content estate is not that. It's not 100 documents, it's not 1,000. Most organizations are sitting on content spread across years, teams and systems that nobody's fully audited, let alone compiled. So where does that leave the pattern once your past personal scale? The honest answer is pointing towards 2026 outlooks that keep landing on the same conclusion. Roughly 75% of enterprise applications are moving toward hybrid architectures, ones that combine wiki style compilation with rag and agentic search, rather than picking one and walking away from the other. That's not a compromise bone out of indecision. It's a recognition that these two systems are solving different problems, and neither one covers the other's job. Here's why hybrid actually makes sense, once you stop thinking of this as a competition. Rag remains the default for large, relatively static repositories, the sprawling stuff that doesn't get asked about constantly, but still needs to be searchable when someone does need it. Wiki's excel somewhere else entirely, the stable, high value knowledge that gets asked about again and again, the stuff worth the effort of compiling because people keep coming back to it. Frame it as a division of labor because that's really what it is. Rag handles breadth, it covers the long tail, everything that exists, but doesn't get touched often enough to justify compiling it by hand. The wiki handles depths on the narrow set of things that actually matter enough to deserve careful, maintained synthesis. Neither one is trying to do the other's job. That's the whole point. Worth a cost reality check here, too, because none of this is free either direction. MVP Rag systems typically run 15 to $40,000 to stand up, covering the vector infrastructure, the embedding pipeline, the retrieval tooling, the wiki layer isn't free, but its cost doesn't show up the same way. It shows up as ongoing maintenance, not infrastructure. You're not buying a bigger system. You're paying for the compiling and upkeep that keeps the wiki accurate as your organization keeps changing. Once you see it that way, breadth handled one way, depth handled another, the question stops being which architecture do we pick, and starts being something much more practical. How does this hybrid framing actually change what a rollout looks like inside a real Microsoft 365 environment? What hybrid looks like for co-pilot? So picture what this actually looks like once it's running, not as a diagram, but as something sitting inside your tenant doing two different jobs at once. Co-pilot with two tiers, one tier is the broad, graph grounded retrieval you already have today, reaching across the long tail of everything in SharePoint teams exchange all of it, searchable the way it's always been searchable. The other tier is a compiled wiki layer, sitting quietly underneath, built specifically for the topics your organization asks about again and again, and that second tier only earns its keep on specific kinds of content. So it's worth naming what actually belongs there, onboarding knowledge, the stuff every new hire needs and every team ends up re-explaning verbally because nobody trusts the search results. Product positioning, the language your organization has already agreed on, that keeps drifting because three different people describe it three different ways, depending on who's asked. Recurring client questions, the ones that show up every quarter with new phrasing, but the same underlying ask. Compliance interpretations, where getting it slightly wrong isn't a minor inconvenience. Project histories, the decisions and reversals that shaped how something got built, the kind of context a new team member has no way of finding without asking someone who's been there for years. Notice what all of those have in common. They're exactly the categories where consistency and trust matter most, the ones we spent real time on earlier in this episode, and they're exactly the categories where redundant recreation waste the most time, people rebuilding something that already exists because search let them down once and they stop trusting it. That's not a coincidence. Those two lists overlap almost completely, which is precisely why this is where compiling earns its cost instead of just adding complexity for its own sake. Here's the mechanic and it's worth describing plainly, instead of dressing it up as something more dramatic than it is. An agent process runs periodically, not constantly, not on every query, just on a schedule, reading through relevant, share point libraries and teams content tied to whichever domain it's responsible for. It compiles what it finds into structured pages, updating what already exists, creating new pages where something genuinely new shows up. And then when someone asks Copilot a question that touches one of those compiled topics, Copilot's studio surfaces those pages preferentially, not exclusively, not instead of graph grounding entirely, just weighted higher, trusted more, pulled first when the topic matches. Worth being clear about what this isn't because it's easy to oversell a pattern like this once you've spent an episode building the case for it. This isn't a rip and replace of Copilot. You're not tearing out graph grounding and swapping in something else. It's an added layer. Sitting alongside what already exists, changing what grounding actually means specifically for the topics your organization keeps asking about. Everything else still works the way it always has. The long tail is still there, still searchable, still handle the old way, because most of your content doesn't need this kind of treatment and never will. But none of this runs itself. Somebody has to decide what counts as a recurring topic worth compiling in the first place. Somebody has to define how those pages get formatted, what triggers an update, when a contradiction gets flagged instead of quietly resolved. And that's a governance question. One IT leader's can't really skip once they've decided this layer is worth building at all. The governance layer nobody's talking about. Somebody has to own this. Not in a vague, it will figure it out sense, but a real role with a real name attached to it. Someone has to own the schema, the actual rules for what gets compiled and what doesn't, how a page gets formatted, so it matches every other page in the wiki. And critically, when a contradiction gets escalated to an actual human, instead of just sitting flagged in a system nobody's watching. Without that owner, the schema drifts, the pages get inconsistent and the whole thing slowly turns into exactly the mess it was supposed to replace. Here's the good news, and it's worth saying plainly because this part tends to get overcomplicated. This role isn't new, it maps directly onto jobs that already exist inside most Microsoft 365 environments. Information architects already think about how content gets structured. Records managers already think about lifecycle, about what stays, what gets retired, what needs a review cycle. Content owners already carry responsibility for whether something's accurate. None of these people need a new job title. They need this added to a job they're already doing because the skill underneath it is identical. Deciding what belongs where and who's accountable when it's wrong. There's already a preview of where this is heading and it's sitting inside Pervue right now. The data loss prevention controls, Microsoft built for co-pilot's web searches, the ones that stop sensitive data from leaking into a prompt, while still letting co-pilot ground its answer in internal sources, that's the same instinct applied to a narrower problem. Guard rails around what an AI-driven system is allowed to see and by extension, what it's allowed to compile. Once you're running an agent that reads across your SharePoint libraries and writes structured pages, you need the same kind of thinking just aimed at compilation instead of search. So frame this the right way because how you frame it changes whether it actually gets funded. This isn't a brand new discipline landing on IT's desk out of nowhere. It's an extension of governance work that's probably already underway in some form. If your organization already has content lifecycle policies, retention rules, ownership models, you already have the starting point for a Wiki schema. You're not building from zero. You're extending something that exists. And this is really the dividing line between organizations that get real value out of this pattern and organizations that build something impressive looking and then watch it decay within a year. The ones that benefit most treat this as a knowledge management initiative that happens to have an AI component, not an AI initiative that happens to touch knowledge, that ordering matters more than it sounds like it should. One puts governance first and lets the technology serve it. The other bolts governance on afterward once something's already gone wrong. With the mechanics covered and the governance question at least named honestly, it's worth stepping back from the how and naming what's actually different here underneath all of it. What actually changes? So here's the shift stated as plainly as it deserves to be stated. Copilot stops being a search box with better manners, one that phrases things politely and summarizes nicely, but still starts from zero every time and starts becoming a system that actually has somewhere to put what it learns. That's the whole difference, not a smarter model, not a better prompt, a place for knowledge to live once it's been figured out instead of a mechanism that refigures it out on command forever. And that difference doesn't show up on day one. It shows up over time, which is exactly why so many organizations miss it during a pilot. A stateless system is the same tool on day 500 as it was on day one. Ask at the same category of question a year from now and it goes through the identical motions it went through in week one, search, chunk, retrieve, synthesize, forget. Nothing about that process improves with age, it doesn't get faster, it doesn't get more accurate, it just repeats. A compiled system by contrast is measurably different a year in because every project page, every client history, every resolved contradiction from month three is still sitting there in month 14, ready to be built on instead of rediscovered. That's not a theoretical claim. It shows up in the numbers we've already touched on earlier in this episode around structured knowledge and it's worth tying back to the sharpest version of them. Enterprise knowledge graphs and structured content approaches have delivered results like 287% ROI over three years with multi million dollar annual benefits reported in some organizations. Those aren't numbers from a system that got asked more questions, they're numbers from a system that got smarter at answering the same questions because the underlying knowledge kept compounding instead of resetting. And that's really the mechanism worth naming directly because it's easy to miss if you're only looking at usage dashboards. Compounding knowledge, not repeated search, is what actually saves time at scale. Repeated search just means people are using the tool more often. It says nothing about whether the tool is getting better. Compounding knowledge means the thousandth question benefits from everything the system learned answering the first 999. That's a completely different kind of value and it's the kind drag structurally can't produce no matter how much you use it. So say the quiet part plainly because it's the sentence this whole episode has been building toward. An assistant that gets smarter is a fundamentally different asset than one that just gets asked more questions. One compounds the other just accumulates traffic. Confusing the two is exactly how organizations end up with impressive adoption metrics and a tool that still can't hold a coherent memory of its own history, which naturally raises the only question left worth answering. Not whether this is worth doing but where an organization actually starts, small and deliberate instead of trying to compile everything at once. What if you start today? So pick one thing, not the whole knowledge estate, not every department, one recurring high value domain. Maybe it's a single product line that support keeps asking about. Maybe it's one recurring client type, the kind of account whose questions repeat every quarter with new phrasing. Maybe it's a single compliance area where getting the answer wrong actually costs something. The domain doesn't matter as much as the discipline of picking just one. Here's what a first 90 days actually looks like without dressing it up as bigger than it is. Start by identifying the raw sources, whatever SharePoint libraries, Teams, Threads and old policy docs already touch this domain, then define a schema and keep it lightweight on purpose. Not a governance framework spanning 50 pages, just enough rules to say what a page looks like and how contradictions get flagged, run an initial compilation pass against that schema. Then test it against real questions, phrase the way actual people phrase them, not the clean version you'd write in a demo script. Picture the end of that pilot next to where you started. Before, co-pilot retrieves five slightly different chunks about the same policy and depending on how someone phrased the question, they get a different one of those five. After, it answers from one compiled current page every time, no matter how the question gets asked. That's the whole test, not whether the answer sounds smart, whether it's the same answer twice, worth connecting this back to the research we've been leaning on all episode because it matters that this isn't theoretical. The organization's reporting a 25 to 40% reduction in redundant knowledge creation, the ones seeing meaningfully lower error rates. They didn't get there by rebuilding everything at once. They got their one domain at a time, exactly like this. Small-bounded, tested against real use before expanding, and there's a thread here worth planting without pulling on a too hard right now. The moment you run this pilot, you're going to hit real questions about who owns the schema, who decides when a contradiction gets escalated, who's accountable when a page goes stale. That's governance. And it's worth its own conversation rather than a rushed afterthought here. So that's the technical case and the strategic case, sitting next to each other, time to bring them together. Key transformation. Here's the transformation, stated as plainly as it deserves. Copilot's value ceiling isn't set by the model underneath it. It's set by whether your organization's knowledge gets compiled or just searched over and over forever. Rags will keep giving you fast, plausible, forgettable answers. So that's what it's built to do, and it'll keep doing it well. A wiki layer gives you something structurally different, an assistant that remembers what your organization has already figured out instead of refiguring it out every time someone asks. And here's the part worth sitting with on your way out of this episode. The tools inside Microsoft 365 to build this exist today. Connectors. Scheduled flows. Copilot studio agents. All of it's already sitting in your tenant. They're just not assembled this way by default. So here's your challenge for this month. Pick one recurring knowledge domain, and go find out honestly whether Copilot is searching it or actually compiling it, not what the vendor deck says, what actually happens when someone asks. Here's the test, and it's simple enough to run this week. Ask the same question two or three different ways. If the answer changes depending on how it's phrased, you've found a retrieval problem, not a copilot problem. That distinction is the whole episode, boiled down to one homework assignment. If this changed how you think about what's actually running under your copilot deployment, subscribe to M365FM podcast. Next time we're going deeper into the governance side of this, who owns the schema, who owns the wiki, and what happens when that ownership isn't clear. And if you want more people, more IT pros, more decision makers, to run into this conversation about fixing Copilot's memory problem, leave a review. It's genuinely how this finds the people who need it. One last thing, connect with me, Mirko Peters on LinkedIn. I want to know what recurring knowledge domain you'd compile first and what you think the next episode should cover.