Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Migrate persistence layer from Exposed + HikariCP to SQLDelight 2 (KMP support) #679

Description

@haiphucnguyen

Replace the current JVM-only Exposed + HikariCP + sqlite-jdbc persistence stack in shared/ with SQLDelight 2, to enable sharing the local SQLite persistence layer across Kotlin Multiplatform targets (Android, iOS, Desktop)

Motivation

  • shared is currently a kotlin("jvm") module. org.jetbrains.exposed.v1.jdbc.* and sqlite-jdbc are JVM-only (JDBC + JNI), so this code cannot run on iOS or JS/WASM targets.
  • SQLDelight 2 has first-class KMP drivers (sqlite-driver on JVM, AndroidSqliteDriver, NativeSqliteDriver for iOS/Native) backed by SQLite's native C API directly - no JDBC bridging required outside the JVM.
  • Compile-time SQL validation against the schema (.sq files checked at build time) catches query/schema mismatches earlier than Exposed's runtime DSL.
  • Removes reliance on HikariCP connection pooling, which provides little benefit for a single-writer SQLite file and can be replaced by a single SqlDriver instance.

Current state

Schema is defined as raw SQL DDL in DatabaseManager.kt - 17 tables, each created via CREATE TABLE IF NOT EXISTS plus idempotent ALTER TABLE ... ADD COLUMN migration blocks wrapped in try/catch.

~13 repository classes under shared/src/main/kotlin/io/askimo/core/**/repository/:
ChatMessageRepository, ChatSessionRepository, ChatMessageAttachmentRepository, ChatDirectiveRepository, SessionMemoryRepository, UserMemoryRepository, ProjectRepository, ResourceSegmentRepository, ModelClassificationRepository, UserProfileRepository, PlanExecutionRepository, SkillRunHistoryRepository, LlmUsageRepository
Custom Exposed column type: SQLiteInstantColumnType for Instant mapping.
Connection/pooling managed by DatabaseManager (HikariCP) + SQLiteDataSourceFactory.
Integration tests: *IT.kt under cli/src/test/kotlin/io/askimo/core/chat/repository/ (ChatMessageRepositoryIT, ChatSessionRepositoryIT, ChatDirectiveRepositoryIT, ConversationSyncRepositoryIT, SessionMemoryRepositoryIT, ProjectRepositoryIT) plus ResourceSegmentRepositoryTest in shared.

Known challenges

Dynamic/conditional query building - ChatMessageRepository.searchMessages(...) and getMessagesPaginated(...) build WHERE/ORDER BY clauses conditionally at runtime (optional startTime, endTime, projectId, cursor-based pagination direction). SQLDelight .sq files are static; this needs either multiple named query variants or a fallback to hand-built SQL via SqlDriver.executeQuery for these specific methods.

JOIN + custom row mapping - e.g. loadAttachmentsForMessageIds (LEFT JOIN messages/attachments) and getAllBookmarkedWithSessions (INNER JOIN → Pair<ChatMessage, ChatSession>) require custom SQLDelight result mappers.

Column type adapters - SQLiteInstantColumnType and enum mapping (MessageRole) need to be re-implemented as SQLDelight ColumnAdapter<T, String> instances configured on the generated Database object.

Migrations - convert existing idempotent ALTER TABLE + try/catch pattern into SQLDelight's versioned .sqm migration files.
Upsert - bulkUpsert (INSERT ... ON CONFLICT) ports directly to SQLite's native ON CONFLICT DO UPDATE syntax in .sq, no functional loss expected.

Proposed approach (incremental, not big-bang)

  • Add SQLDelight 2 Gradle plugin + JVM sqlite-driver dependency to shared (or a new shared-db module if we also restructure for KMP source sets).
  • Port schema DDL from DatabaseManager.kt into .sq files (one per table/domain area), plus versioned .sqm migration files replacing the manual ALTER TABLE blocks.
  • Implement ColumnAdapters for Instant and MessageRole (and other enums).
  • Migrate repositories one at a time behind existing public interfaces, starting with simpler ones:
    UserMemoryRepository, UserProfileRepository, ModelClassificationRepository, SkillRunHistoryRepository, LlmUsageRepository
    ChatDirectiveRepository, SessionMemoryRepository, PlanExecutionRepository
    ProjectRepository, ChatMessageAttachmentRepository, ResourceSegmentRepository
    ChatSessionRepository, ChatMessageRepository (dynamic search/pagination - highest complexity)
  • Replace DatabaseManager's HikariCP-based connection management with a single SQLDelight SqlDriver instance per database file.
    Update/port all *IT.kt integration tests and ResourceSegmentRepositoryTest to run against the new repositories.
    Once all repositories are migrated, remove Exposed/HikariCP/sqlite-jdbc dependencies from shared/build.gradle.kts.
    (Follow-up, separate issue) Restructure shared module to kotlin("multiplatform") and add Android/iOS source sets + drivers.

Acceptance criteria

  • All existing repository methods have SQLDelight-backed equivalents with identical public signatures/behavior (no breaking changes to callers in cli/desktop).

  • All existing integration tests pass against the new implementation.

  • No regression in sync-related flows (markSynced, getUnsyncedMessages, bulkUpsert) since these back cross-device sync.
    Exposed, HikariCP, and sqlite-jdbc dependencies fully removed from shared/build.gradle.kts.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requesthelp wantedExtra attention is needed

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions