Fix Kiro post-February storage discovery (getagentseal#339)

iamtoruk · iamtoruk · commit 6e24f826bd53 · 2026-05-24T02:25:09.000-07:00
Adds extensionless session index and nested execution file discovery
while preserving legacy .chat support. Modern execution JSON is parsed
for identifiers, timestamps, model IDs, conversation text, structured
tools, and token estimates. Also fixes dedup key poisoning on invalid
timestamps in the legacy parser.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -73,6 +73,11 @@
   because the cache shortcut only merged cost/calls. Per-provider periods now
   always do a full parse. Also floors `maxCost` at 0.01 to avoid NaN bar
   widths in ActivitySection and ModelsSection. (#362)
+- **Kiro post-February 2026 storage discovery.** The Kiro provider now keeps
+  legacy `.chat` support while also discovering extensionless session index
+  files and nested execution files. Modern execution JSON is parsed for
+  identifiers, timestamps, model IDs, conversation text, structured tools, and
+  estimated token usage. Thanks @ozymandiashh. Closes #329. (#339)
 
 ### Fixed (macOS menubar)
 - **Per-provider refresh latency.** Switching provider tabs took ~24s on heavy
diff --git a/docs/providers/kiro.md b/docs/providers/kiro.md
@@ -4,7 +4,7 @@ Kiro IDE chat history.
 
 - **Source:** `src/providers/kiro.ts`
 - **Loading:** eager (`src/providers/index.ts:7`)
-- **Test:** `tests/providers/kiro.test.ts` (328 lines)
+- **Test:** `tests/providers/kiro.test.ts`
 
 ## Where it reads from
 
@@ -16,29 +16,38 @@ VS Code-style globalStorage at `kiro.kiroagent`:
 | Windows | `%APPDATA%/Kiro/User/globalStorage/kiro.kiroagent` |
 | Linux | `~/.config/Kiro/User/globalStorage/kiro.kiroagent` |
 
-Sessions are `.chat` files under hash-named subdirectories. Discovery is in `kiro.ts:215-247`; the path-resolution helpers it uses start at `kiro.ts:164`.
+Sessions are under hash-named workspace subdirectories. Discovery keeps backward compatibility with legacy `.chat` files and also scans the post-February 2026 extensionless format:
+
+- `<workspace-hash>/<execution-id>.chat` legacy session files
+- `<workspace-hash>/<session-hash>` extensionless session index files
+- `<workspace-hash>/<session-hash>/<execution-hash>` extensionless execution files inside session directories
 
 ## Storage format
 
-JSON `.chat` files (`kiro.ts:153`).
+Kiro has two known JSON formats:
+
+- Legacy `.chat` files with `{ chat, metadata, executionId }`
+- Modern extensionless execution files with identifiers/timestamps at the top level plus conversation fields such as `messages`, `conversation`, `chat`, `transcript`, `entries`, `events`, or direct prompt/response fields
+
+Session index files with `{ executions: [...] }` are discovered but skipped during parsing because they do not contain conversation content.
 
 ## Caching
 
 None.
 
 ## Deduplication
 
-Per `executionId` (`kiro.ts:104`).
+Modern files deduplicate per session/execution pair. Legacy `.chat` files deduplicate per workflow/execution pair.
 
 ## Quirks
 
-- **Workspace hash resolution** is non-trivial. The parser tries `workspace.json` first; if that fails, it base64-decodes the directory name to recover the workspace path (`kiro.ts:198-213`).
-- **Model ID normalization.** Kiro stores models like `claude-1.2`; the parser rewrites the dot to a hyphen so they match `claude-1-2` in the pricing snapshot (`kiro.ts:65-67`). Add new versions here when Kiro ships them.
-- **Tool name extraction is regex-driven.** Kiro embeds tool calls inside the message text as `<tool_use><name>...</name>` (`kiro.ts:69-78`). Brittle but unavoidable until Kiro emits structured tool data.
-- Token counts are estimated via char count (`CHARS_PER_TOKEN = 4`, `kiro.ts:9`, `:108-109`).
+- **Workspace hash resolution** is non-trivial. The parser tries `workspace.json` first; if that fails, it base64-decodes the directory name to recover the workspace path.
+- **Model ID normalization.** Kiro stores models like `claude-1.2`; the parser rewrites the dot to a hyphen so they match `claude-1-2` in the pricing snapshot. Add new versions here when Kiro ships them.
+- **Tool name extraction accepts text and structured calls.** Kiro can embed tool calls inside message text as `<tool_use><name>...</name>` or expose structured `toolCalls` / `tool_calls` / `tools` entries.
+- Token counts are estimated via char count (`CHARS_PER_TOKEN = 4`).
 
 ## When fixing a bug here
 
 1. If the bug is "wrong workspace", check the base64 fallback path. Some users name their workspaces with characters that are not valid base64.
-2. If the bug is "missing model in pricing", add the model to the normalization map at `kiro.ts:65-67` and verify against `tests/providers/kiro.test.ts`.
-3. If the bug is "tools missing", look at the regex at `kiro.ts:69-78`. Kiro changes its envelope occasionally.
+2. If the bug is "missing model in pricing", add the model to the normalization map and verify against `tests/providers/kiro.test.ts`.
+3. If the bug is "tools missing", check both text-envelope extraction and structured tool-call extraction. Kiro changes its envelope occasionally.
diff --git a/src/providers/kiro.ts b/src/providers/kiro.ts
@@ -1,5 +1,6 @@
-import { readdir, readFile, stat } from 'fs/promises'
-import { basename, join } from 'path'
+import type { Dirent } from 'fs'
+import { readdir, readFile } from 'fs/promises'
+import { basename, dirname, extname, join } from 'path'
 import { homedir } from 'os'
 
 import { readSessionFile } from '../fs-utils.js'
@@ -8,6 +9,8 @@ import type { ToolCall } from '../types.js'
 import type { Provider, SessionSource, SessionParser, ParsedProviderCall } from './types.js'
 
 const CHARS_PER_TOKEN = 4
+const MIN_REASONABLE_TIMESTAMP_MS = 1_000_000_000_000
+const MODERN_CONVERSATION_KEYS = ['messages', 'conversation', 'chat', 'transcript', 'entries', 'events']
 
 const modelDisplayNames: Record<string, string> = {
   'claude-sonnet-4-6': 'Sonnet 4.6',
@@ -63,6 +66,8 @@ type KiroChatFile = {
   }
 }
 
+type KiroModernExecution = Record<string, unknown>
+
 function normalizeModelId(raw: string): string {
   return raw.replace(/(\d+)\.(\d+)/g, '$1-$2')
 }
@@ -78,6 +83,98 @@ function extractToolNames(content: string): string[] {
   return tools
 }
 
+function asRecord(value: unknown): Record<string, unknown> | null {
+  return value && typeof value === 'object' && !Array.isArray(value) ? value as Record<string, unknown> : null
+}
+
+function stringField(record: Record<string, unknown> | null, names: string[]): string {
+  if (!record) return ''
+  for (const name of names) {
+    const value = record[name]
+    if (typeof value === 'string' && value.trim()) return value.trim()
+  }
+  return ''
+}
+
+function timeField(record: Record<string, unknown> | null, names: string[]): number | string | undefined {
+  if (!record) return undefined
+  for (const name of names) {
+    const value = record[name]
+    if (typeof value === 'number' || typeof value === 'string') return value
+  }
+  return undefined
+}
+
+function parseKiroTimestamp(value: number | string | undefined): Date | null {
+  if (value === undefined) return null
+
+  let parsed: number | string = value
+  if (typeof value === 'string') {
+    const trimmed = value.trim()
+    if (!trimmed) return null
+    parsed = /^-?\d+(\.\d+)?$/.test(trimmed) ? Number(trimmed) : trimmed
+  }
+
+  if (typeof parsed === 'number') {
+    if (!Number.isFinite(parsed)) return null
+    const ms = parsed < MIN_REASONABLE_TIMESTAMP_MS ? parsed * 1000 : parsed
+    const date = new Date(ms)
+    return Number.isNaN(date.getTime()) || date.getTime() < MIN_REASONABLE_TIMESTAMP_MS ? null : date
+  }
+
+  const date = new Date(parsed)
+  return Number.isNaN(date.getTime()) || date.getTime() < MIN_REASONABLE_TIMESTAMP_MS ? null : date
+}
+
+function textField(record: Record<string, unknown> | null, names: string[]): string {
+  if (!record) return ''
+  for (const name of names) {
+    const text = extractText(record[name])
+    if (text) return text
+  }
+  return ''
+}
+
+function extractText(value: unknown): string {
+  if (typeof value === 'string') return value
+  if (Array.isArray(value)) return value.map(extractText).filter(Boolean).join('\n')
+  const record = asRecord(value)
+  if (!record) return ''
+  for (const key of ['content', 'text', 'message', 'value', 'parts']) {
+    const text = extractText(record[key])
+    if (text) return text
+  }
+  return ''
+}
+
+function messageRole(value: unknown): string {
+  const record = asRecord(value)
+  if (!record) return ''
+  return stringField(record, ['role', 'type', 'author']).toLowerCase()
+}
+
+function extractStructuredToolNames(value: unknown, text: string, options: { includeDirectName?: boolean } = {}): string[] {
+  const tools = extractToolNames(text)
+  const record = asRecord(value)
+  if (!record) return tools
+
+  if (options.includeDirectName ?? true) {
+    const directName = stringField(record, ['toolName', 'name'])
+    if (directName) tools.push(toolNameMap[directName] ?? directName)
+  }
+
+  for (const key of ['toolCalls', 'tool_calls', 'tools']) {
+    const entries = record[key]
+    if (!Array.isArray(entries)) continue
+    for (const entry of entries) {
+      const name = stringField(asRecord(entry), ['name', 'toolName', 'tool_name'])
+      if (name) tools.push(toolNameMap[name] ?? name)
+    }
+  }
+
+  return tools
+}
+
 function parseChatFile(data: KiroChatFile, sessionId: string, project: string, seenKeys: Set<string>): ParsedProviderCall[] {
   const results: ParsedProviderCall[] = []
   const { chat, metadata } = data
@@ -107,14 +204,14 @@ function parseChatFile(data: KiroChatFile, sessionId: string, project: string, s
 
   const dedupKey = `kiro:${sessionId}:${data.executionId}`
   if (seenKeys.has(dedupKey)) return results
-  seenKeys.add(dedupKey)
 
   const outputTokens = Math.ceil(totalOutputChars / CHARS_PER_TOKEN)
   const inputTokens = Math.ceil(pendingUserMessage.length / CHARS_PER_TOKEN)
   const costUSD = calculateCost(modelId, inputTokens, outputTokens, 0, 0, 0)
-  const tsDate = metadata.startTime ? new Date(metadata.startTime) : null
-  if (!tsDate || isNaN(tsDate.getTime()) || tsDate.getTime() < 1_000_000_000_000) return results
+  const tsDate = parseKiroTimestamp(metadata.startTime)
+  if (!tsDate) return results
   const timestamp = tsDate.toISOString()
+  seenKeys.add(dedupKey)
 
   results.push({
     provider: 'kiro',
@@ -140,23 +237,132 @@ function parseChatFile(data: KiroChatFile, sessionId: string, project: string, s
   return results
 }
 
+function parseModernExecution(data: KiroModernExecution, sourcePath: string, seenKeys: Set<string>): ParsedProviderCall[] {
+  const results: ParsedProviderCall[] = []
+  if (Array.isArray(data['executions'])) return results
+
+  const metadata = asRecord(data['metadata'])
+  const modelObj = asRecord(data['model'])
+  let modelId = normalizeModelId(
+    stringField(data, ['modelId', 'modelID', 'modelName', 'model']) ||
+    stringField(modelObj, ['id', 'name']) ||
+    stringField(metadata, ['modelId', 'modelID', 'modelName']),
+  )
+  if (modelId === 'auto' || !modelId) modelId = 'kiro-auto'
+
+  const executionId = stringField(data, ['executionId', 'id']) || basename(sourcePath)
+  const sessionId = stringField(data, ['sessionId', 'conversationId', 'workflowId']) ||
+    stringField(metadata, ['workflowId', 'sessionId']) ||
+    basename(dirname(sourcePath)) ||
+    executionId
+
+  let inputChars = 0
+  let outputChars = 0
+  let pendingUserMessage = ''
+  const allTools: string[] = []
+  let hasOutputActivity = false
+  const directInput = textField(data, ['prompt', 'input', 'userMessage', 'user_message', 'request'])
+  const directOutput = textField(data, ['response', 'output', 'assistantMessage', 'assistant_message', 'result'])
+  const directTools = extractStructuredToolNames(data, directOutput, { includeDirectName: false })
+
+  if (directInput) {
+    inputChars += directInput.length
+    pendingUserMessage = directInput.slice(0, 500)
+  }
+
+  if (directOutput) {
+    outputChars += directOutput.length
+    hasOutputActivity = true
+  }
+
+  if (directTools.length > 0) {
+    hasOutputActivity = true
+    allTools.push(...directTools)
+  }
+
+  for (const key of MODERN_CONVERSATION_KEYS) {
+    const messages = data[key]
+    if (!Array.isArray(messages)) continue
+
+    for (const message of messages) {
+      const text = extractText(message)
+      const role = messageRole(message)
+      const tools = extractStructuredToolNames(message, text)
+
+      if (role === 'human' || role === 'user') {
+        if (!text) continue
+        inputChars += text.length
+        pendingUserMessage = text.slice(0, 500)
+      } else if (role === 'bot' || role === 'assistant' || role === 'ai' || role === 'model') {
+        if (text) outputChars += text.length
+        if (text || tools.length > 0) hasOutputActivity = true
+        allTools.push(...tools)
+      } else if (role === 'tool' || role === 'system') {
+        if (text) inputChars += text.length
+        allTools.push(...tools)
+      }
+    }
+    break
+  }
+
+  if (!hasOutputActivity) return results
+
+  const dedupKey = `kiro:${sessionId}:${executionId}`
+  if (seenKeys.has(dedupKey)) return results
+
+  const rawStartTime = timeField(data, ['startTime', 'createdAt', 'timestamp']) ??
+    timeField(metadata, ['startTime', 'createdAt', 'timestamp'])
+  const tsDate = parseKiroTimestamp(rawStartTime)
+  if (!tsDate) return results
+
+  const inputTokens = Math.ceil(inputChars / CHARS_PER_TOKEN)
+  const outputTokens = Math.ceil(outputChars / CHARS_PER_TOKEN)
+  const costUSD = calculateCost(modelId, inputTokens, outputTokens, 0, 0, 0)
+  seenKeys.add(dedupKey)
+
+  results.push({
+    provider: 'kiro',
+    model: modelId,
+    inputTokens,
+    outputTokens,
+    cacheCreationInputTokens: 0,
+    cacheReadInputTokens: 0,
+    cachedInputTokens: 0,
+    reasoningTokens: 0,
+    webSearchRequests: 0,
+    costUSD,
+    tools: [...new Set(allTools)],
+    bashCommands: [],
+    timestamp: tsDate.toISOString(),
+    speed: 'standard',
+    deduplicationKey: dedupKey,
+    userMessage: pendingUserMessage,
+    sessionId,
+  })
+
+  return results
+}
+
 function createParser(source: SessionSource, seenKeys: Set<string>): SessionParser {
   return {
     async *parse(): AsyncGenerator<ParsedProviderCall> {
       const content = await readSessionFile(source.path)
       if (content === null) return
 
-      let data: KiroChatFile
+      let data: unknown
       try {
         data = JSON.parse(content)
       } catch {
         return
       }
 
-      if (!data.chat || !data.metadata) return
+      const record = asRecord(data)
+      if (!record) return
 
-      const sessionId = data.metadata.workflowId ?? basename(source.path, '.chat')
-      const calls = parseChatFile(data, sessionId, source.project, seenKeys)
+      const metadata = asRecord(record['metadata'])
+      const calls = Array.isArray(record['chat']) && metadata
+        ? parseChatFile(record as unknown as KiroChatFile, stringField(metadata, ['workflowId']) || basename(source.path, '.chat'), source.project, seenKeys)
+        : parseModernExecution(record, source.path, seenKeys)
       for (const call of calls) {
         yield call
       }
@@ -232,19 +438,30 @@ async function discoverSessions(agentDir: string, workspaceStorageDir: string):
     const wsPath = join(agentDir, wsHash)
     const project = await resolveWorkspaceProject(agentDir, workspaceStorageDir, wsHash)
 
-    let files: string[]
+    let entries: Dirent[]
     try {
-      const entries = await readdir(wsPath)
-      files = entries.filter(f => f.endsWith('.chat'))
+      entries = await readdir(wsPath, { withFileTypes: true })
     } catch {
       continue
     }
 
-    for (const file of files) {
-      const filePath = join(wsPath, file)
-      const s = await stat(filePath).catch(() => null)
-      if (!s?.isFile()) continue
-      sources.push({ path: filePath, project, provider: 'kiro' })
+    for (const entry of entries) {
+      if (entry.name.startsWith('.')) continue
+      const entryPath = join(wsPath, entry.name)
+      if (entry.isFile() && (entry.name.endsWith('.chat') || extname(entry.name) === '')) {
+        sources.push({ path: entryPath, project, provider: 'kiro' })
+        continue
+      }
+
+      if (!entry.isDirectory()) continue
+
+      const childEntries = await readdir(entryPath, { withFileTypes: true }).catch(() => [])
+      for (const child of childEntries) {
+        if (child.name.startsWith('.')) continue
+        if (!child.isFile()) continue
+        if (extname(child.name) !== '') continue
+        sources.push({ path: join(entryPath, child.name), project, provider: 'kiro' })
+      }
     }
   }
 
diff --git a/tests/providers/kiro.test.ts b/tests/providers/kiro.test.ts
