Building SoPhi · Article 05

The memory that can’t lie: crash-safe sessions with nothing but JSONL

Shukri Shukriev · · Also on Medium

Illustration for The memory that can’t lie: crash-safe sessions with nothing but JSONL

Every commercial aircraft carries a device engineered around one grim assumption: the plane might not survive, but the last few minutes of what happened to it should. A flight data recorder doesn’t hold a “current state of the flight” that gets overwritten every second — it writes forward, continuously, onto a medium built to physically survive fire and impact, so that whatever the recorder captured up to the moment everything stopped is still there afterward, intact, readable, unambiguous about what it does and doesn’t know.

ADR-005 is dated June 27th, 2026 — the first hard decision insophi-core, before there was a loop, a tool, or a model call to speak of. Before an agent can plan anything, it needs a memory that survives it dying mid-sentence. Not "usually survives." Not "recovers most of the time gracefully." Survives, the way a black box is supposed to survive, because the alternative — a memory that quietly lies about what it remembers — is worse than no memory at all. This is the story of how that memory got built, and the place where the decision as written and the decision as shipped quietly disagree with each other.

The document that lies about itself

The obvious first draft of a session format is one JSON document per conversation. Load it, mutate it, write it back. It’s simple, and it’s wrong in a way that only shows up at the worst possible moment: if the process dies while writing that file — a crashed JVM, a killed container, a laptop lid slammed shut — you don’t get a truncated conversation. You get a corrupted one. Files.writeText() doesn't guarantee the old bytes are gone before
the new ones land; a half-written file is often invalid JSON, which means the next load() doesn't lose the last message, it throws on all of them. A single mutable document is a logbook you erase and rewrite from page one every time something happens — which is fine, right up until the moment you're mid-erasure when the lights go out.

ADR-005’s answer was to stop treating a session as a document and start treating it as a log, in the old, literal sense of the word — a ship’s log, a flight recorder’s tape. One line, one entry, appended, never rewritten:

Kotlin
@Serializable
data class SessionEntry(    val id: String,
    val parentId: String? = null,
    val role: EntryRole,
    val content: String,
    val timestamp: Long,
    val metadata: Map<String, String> = emptyMap()
)

The pointer that builds a tree for free

That parentId is doing more work than it looks like. It's not just "the previous line" — it's an explicit pointer, which means the file isn't really a list; it's a tree wearing a list's clothing. AgentSession doesn't know about files at all; it just tracks a tip and walks parent links backwards to reconstruct the active branch:

Kotlin
fun branch(): List<SessionEntry> {
  val byId = _entries.associateBy { it.id }
  val result = ArrayDeque<SessionEntry>()
  var current: SessionEntry? = tip

  while (current != null) {
    result.addFirst(current)
    current = current.parentId?.let { byId[it] }
  }

  return result.toList()
}

Which means branching — the thing every other session format has to bolt on later, usually with a “conversation forking” feature flag and a migration — is free. You don’t fork a JSONL-tree session. You just append() a new entry whose parentId points at some entry that isn't the current tip, and checkout() to make it the tip:

Kotlin
fun checkout(entryId: String) {
  require(_entries.any { it.id == entryId }) {
    "Entry not found: $entryId"
  }

  _tipId = entryId
}

Old branches don’t get deleted. They just stop being the tip — the way an investigator pulling a black box doesn’t get “the flight,” they get every recorded second of it, and decide afterward which stretch matters. /checkout in the CLI is a two-line function calling straight into this — there was never a "branching feature" to build, because the data structure already had branches in it from the day it was designed.

The gap the ADR admits to

Here’s the part I like most about this story, and the reason it’s worth telling instead of just linking the ADR. ADR-005 describes atomicity as “appending a line is atomic on all major filesystems,” which is true and also not what actually does. Go look at FileSessionManager.save() and you won't find an append call anywhere:

Kotlin
override fun save(session: AgentSession) {
  validateId(session.id)
  val file = sessionsDir.resolve("${session.id}.jsonl")

  atomicWrite(file, session.entries.joinToString("\n") {
    json.encodeToString(it) })
    ...
  }
}

// Write-in-place would corrupt the file if the process dies mid-write;
// stage to a temp file in the same directory and atomically rename over.
private fun atomicWrite(target: Path, content: String) {
  val tmp = Files.createTempFile(sessionsDir, ".${target.fileName}", ".tmp")
  try {
    tmp.writeText(content)

    try {
      Files.move(tmp, target, StandardCopyOption.ATOMIC_MOVE, StandardCopyOption.REPLACE_EXISTING)        } catch (e: AtomicMoveNotSupportedException) {            Files.move(tmp, target, StandardCopyOption.REPLACE_EXISTING)
    }

  ...
}

Every save rewrites the whole file to a temp path and atomically renames it over the original. That’s not the streaming, one-line-at-a-time durability the ADR describes — it’s closer to a photographer developing a new print in the darkroom before ever swapping it into the frame on the wall, rather than painting new brushstrokes directly onto a canvas that’s already hanging. Different mechanism, same guarantee: a crash mid-write leaves the
old file intact, never a half-written one — at the cost of the ADR’s own stated benefit, real streaming append. The ADR even flags this honestly in its Consequences section as a known gap, filed away for a “Phase 7” that would make saves incremental. The design goal and the shipped code disagree, on purpose, and the disagreement is written down instead of quietly forgotten. That’s worth more than getting it right the first time.

Legible over clever

A flight recorder’s tape isn’t encrypted, isn’t compressed into something only the manufacturer’s proprietary tool can open, and isn’t a black box in the sense of being opaque — it’s called that for the color of the casing, not the readability of what’s inside. JSONL entries carry the same instinct: they’re lines you can grep. Nobody needed a database, a schema registry, or a custom viewer to debug a session — a text editor works. The honest gaps — no compaction yet, no streaming append yet — are written down as
consequences, not discovered later as surprises. It's the same value its sibling decision, ADR-006, made a bet on the very next day, just applied to a different problem: how the agent acts instead of how it remembers.

What’s next?

The next article in this series covers building tools, or as we call them: The agent hands that don’t freeze

Find the full source on GitHub and connect with me on LinkedIn.

All articles in the series