kiro-cli-langfuse

Langfuse observability integration for Kiro CLI. Automatically traces AI agent activity during terminal coding sessions using Kiro CLI Hooks.

Features

• All 5 Kiro CLI hook types supported (AgentSpawn, UserPromptSubmit, PreToolUse, PostToolUse, Stop)
• Traces grouped by conversation
• Sessions grouped by workspace (cwd)
• Dynamic tagging (tool type, model)
• Completion status scoring
• Non-blocking error handling

Supported Hooks

Kiro CLI Hook Type	What it traces
AgentSpawn	Agent initialization
UserPromptSubmit	User prompts and queries
PreToolUse	Tool invocations before execution
PostToolUse	Tool results and duration
Stop	Agent completion with status scores

Setup

1. Copy to your project

cp -r kiro-cli-langfuse/.kiro/agents/ your-project/.kiro/agents/
cp -r kiro-cli-langfuse/hooks/ your-project/hooks/

2. Install dependencies

cd your-project/hooks && npm install

3. Configure Langfuse credentials

cp .env.example .env
# Edit .env with your Langfuse keys

Get your keys from Langfuse Cloud or your self-hosted instance.

4. Use the agent

The hooks are defined in .kiro/agents/langfuse-observer.json. This agent includes "tools": ["*"] so it retains access to all standard Kiro CLI tools (file read/write, bash, etc.) alongside the Langfuse hooks. Switch to this agent in Kiro CLI, or merge the hooks field into your existing agent config:

# Switch to the langfuse-observer agent
/agent swap langfuse-observer

# Or merge hooks into your default agent config

How It Works

Kiro CLI triggers hook event
    -> Pipes JSON to hook-handler.js via stdin
    -> Parses hook_event_name, cwd, prompt, tool_name, etc.
    -> Creates/updates Langfuse trace for the conversation
    -> Routes to event-specific handler
    -> Creates spans, generations, scores in Langfuse
    -> Flushes data before exit

Key Difference from IDE Hooks

Kiro CLI hooks differ from Kiro IDE hooks:

	Kiro IDE	Kiro CLI
Hook definition	.kiro/hooks/*.kiro.hook files	hooks field in .kiro/agents/*.json
Hook data	Env vars + stdin	JSON via stdin only
Hook types	10 types (file ops, tasks, manual, etc.)	5 types (spawn, prompt, pre/post tool, stop)
Event names	prompt_submit, agent_stop, etc.	agentSpawn, userPromptSubmit, etc.
Tool names	read, write, shell	fs_read, fs_write, execute_bash

Trace Hierarchy

• Trace — one per conversation
  • Session — grouped by cwd
  • Events — agent spawn, agent stop
  • Generations — user prompts
  • Spans — tool use (pre/post)
  • Scores — completion status (0-1)

Prerequisites

• Node.js v18+
• Kiro CLI
• A Langfuse account (cloud or self-hosted)

Project Structure

.kiro/
  agents/
    langfuse-observer.json     # Agent config with hook definitions
hooks/
  hook-handler.js              # Main entry point
  package.json
  lib/
    langfuse-client.js         # Langfuse SDK wrapper
    handlers.js                # Event-specific handlers
    utils.js                   # Shared utilities
  test/
    helpers.js                 # Mock factories
    utils.test.js              # Utility function tests
    handlers.test.js           # Hook handler tests
    langfuse-client.test.js    # Client wrapper tests
    e2e.test.js                # End-to-end flow tests
    integration/
      setup.js                 # Docker test helpers
      harness.js               # Shared before/after setup
      trace-creation.test.js   # Trace creation & session tests
      hook-flow.test.js        # Full hook flow & event tests
      scoring.test.js          # Completion & custom score tests
docker-compose.test.yml        # Langfuse v3 test stack

Testing

Unit tests

cd hooks && npm test

Integration tests (Docker)

cp .env.test.example .env.test
docker compose --env-file .env.test -f docker-compose.test.yml up --wait
cd hooks && npm run test:integration
docker compose -f docker-compose.test.yml down -v

Customizing

Filter specific tools

Edit .kiro/agents/langfuse-observer.json and add a matcher to the preToolUse/postToolUse hooks:

{
  "hooks": {
    "preToolUse": [
      {
        "matcher": "fs_write",
        "command": "node hooks/hook-handler.js"
      }
    ]
  }
}

Supported matchers:

• fs_write or write — file write tool
• fs_read or read — file read tool
• execute_bash or shell — shell commands
• use_aws or aws — AWS CLI tool
• @git — all tools from git MCP server
• @git/status — specific MCP tool
• * — all tools

Merge into existing agent

Add the hooks field from langfuse-observer.json into your own agent config:

{
  "name": "my-agent",
  "hooks": {
    "agentSpawn": [{ "command": "node hooks/hook-handler.js" }],
    "userPromptSubmit": [{ "command": "node hooks/hook-handler.js" }],
    "preToolUse": [{ "command": "node hooks/hook-handler.js" }],
    "postToolUse": [{ "command": "node hooks/hook-handler.js" }],
    "stop": [{ "command": "node hooks/hook-handler.js" }]
  }
}

References

• Kiro CLI Hooks Documentation
• Kiro CLI Agent Configuration
• Langfuse Documentation