← Back to writing

I Went Looking for ChatGPT's Memory

·6 min read·

It started with four mistakes. Four mistakes in a row, in a few hours.

I talk to ChatGPT almost every morning. Out loud, about whatever I'm working on. I've been building things with technology for more than twenty years, and I can do things today that I couldn't have imagined doing on my own a few months ago. So I'm a fan. Keep that in mind.

On this one day it kept getting things wrong. They were things about how we work together. I'd tell it: remember this, don't make that mistake again. It would agree. And a little later it would make the mistake again. I can't even tell you now what each one was. Something in our workflow. What stayed with me was the question.

How is this possible? I asked for it. It said yes. And it still did it.

This wasn't normal for ChatGPT. So I did what I've done with every broken system in my life. I went looking for the bug.

#The black box

I opened the memory, and I couldn't find anything.

I mean, there was text there. But it was a condensed, rewritten version of what I'd said, in somebody else's words, following rules I can't see. The thing I'd asked it to remember wasn't there in any form I could check.

And I don't even know if that's all of it! Maybe there's more somewhere else. Maybe there's a vector database behind it. Who knows? Yes, there are settings. Yes, you can export your data. I still couldn't answer the simplest question: what did you know when you made that mistake?

You could say, so ask ChatGPT to audit its own memory. Think about that for a second. I'd be asking the black box to tell me what's inside the black box. It makes no sense.

There was a second problem. I work with Claude too. Whatever ChatGPT remembered, in whatever form, Claude couldn't see. I was working with two AIs every day and they couldn't read each other's notes.

I wanted a solution.

#Files and folders

Files and folders. Yes, that's it.

I know how that sounds. Twenty years of building systems and my big idea is folders. But I wanted something I could open. Something I could read with my own eyes, fix by hand, and carry somewhere else the day I want to leave. So I put plain text files in a private GitHub repository. GitHub is still someone else's server, I know. But every change has a history, I can copy the whole thing to my laptop with one command, and it's live, so any agent I allow can reach it from anywhere. Vector databases are great for plenty of things. I wanted a memory I could read.

The files were the obvious part. The problem was that I now had two AIs from two different companies reading and writing the same folder, with no protocol. Who writes what? How does Claude know ChatGPT already answered? What stops one of them from doing whatever it wants in there? And I didn't want to guide every conversation.

So I designed one. Maybe I designed it poorly! But it works. It's a small draft: rules for how we record things, who's who, who's responsible for what, and what I call contracts, written agreements about how an agent behaves in the space. The rule I love most is that nobody rewrites an old entry. If something changed, you add a new one. You can guess why that one matters to me.

Then I ran tests between ChatGPT and Claude. And they pretty much honored the contracts! Same files, same rules, two different models, and they kept to their roles and picked up each other's work.

I started calling it the shared space.

#Last month

That was 26 days ago. I know it's 26 days because the first entry has a timestamp on it.

The log has more than 900 entries now. And I can't explain to you how much the three of us, Claude, ChatGPT and me, have done in these weeks compared to any month before them. It's a huge leap. Everything moves quicker and everything's written down. We make better decisions and we make them faster. My output has tripled, at least. That's me comparing this month to every month before it.

The repeat mistakes dropped, a lot, especially in those morning conversations. Something I tell ChatGPT in the morning, Claude can read that afternoon. And when I ask how we got to where we are, the answer exists, with the reasons, in order.

The biggest part for me is the talking. I can say a thought out loud far faster than I can type it, so I get much more across in a voice conversation than I ever could at a keyboard. I talk to ChatGPT on my phone, it reaches the same live files, and the agents write it down and file it. I didn't want to spend my time managing files. I wanted to keep working.

And then this happened. We were in the middle of talking about one thing, and ChatGPT brought up something from last month. It connected what we were discussing to something from weeks before. Last month! And I hadn't asked it to look.

I've had a lot of good answers from AI. This one landed differently, because today's conversation had a past. I started to see what this could be.

#The chain

Think about a decision you made six months ago. Why did you make it? What did you know at the time? What had you already tried? If you worked through it in a chat window, good luck.

Now every decision I make goes on top of the ones before it. It's written down, with the reasons, one thing after another, a logical chain you can build on. Without the documentation that chain doesn't exist in any way you can reach.

This is what I'm so excited about. I see so much potential just from documenting the journey this way. I think there's something very promising here for every business owner and every person trying to build something over time with AI. Which, in my opinion, is everyone.

What if every business decision could carry its own reasons, evidence, risks and history forward? Somebody who joins you next year could read why you did what you did. A new agent could read it on day one. When something goes wrong you could go back and see what you believed at the time, and what you missed. You don't start from a blank screen anymore. I find that a huge relief.

I still make the decisions. I still have to check what the agents find, and they can be wrong in a file just as easily as in a chat. The difference is I can find it now.

If you want to try this, the folder has to be live. A folder on your laptop can't do it, because the whole idea is to talk out loud with your agents, and ChatGPT on your phone can't reach your laptop. Put it somewhere all of them can get to, GitHub or something like Dropbox.

And you need rules. Put two agents in a folder with nothing to stop them and they can do whatever they want in it, and you'll end up guiding every conversation. Write down who's allowed to do what, and that nobody rewrites an old entry. Mine started as a small draft, and yours can too. At the end of your next real conversation, ask your agent to record what you decided and why. Then open a conversation with a different agent and ask it what you decided.

A few weeks ago I had four mistakes and a black box. Today I have a live folder I can open from anywhere, and a record of what we decided and why. I can't wait to see what it looks like in a year.

Share