Thanks to visit codestin.com
Credit goes to github.com

Skip to content

eloqdata/eloqdoc-migration

Repository files navigation

EloqDoc Migration Tool

A command-line tool for migrating MongoDB databases to EloqDoc, supporting index, role, user, and data migration.

Features

  • Schema Migration: Create databases and collections structure in target before migrating data
  • Index Migration: Migrate MongoDB indexes with full compatibility checking for EloqDoc
  • Role Migration: Migrate custom roles with dependency resolution (built-in roles are automatically skipped)
  • User Export: Export user accounts to JSON file (passwords cannot be automatically migrated)
  • Data Migration: Bulk copy of collection data (coming soon)
  • Idempotent: Safe to run multiple times - skips existing indexes and roles
  • Dry Run Mode: Preview changes without making modifications
  • Selective Migration: Choose which databases and tasks to migrate

Installation

cd eloqdoc-migration
go build -o eloqdoc-migration main.go

Usage

Basic Usage

# Migrate schema, indexes and roles (default)
./eloqdoc-migration import \
  --source "mongodb://user:pass@source-host:27017/admin" \
  --target "mongodb://user:pass@target-host:27017/admin"

Migrate Specific Tasks

# Migrate only schema (databases and collections)
./eloqdoc-migration import \
  --source $SOURCE_URI \
  --target $TARGET_URI \
  --tasks schema

# Migrate only indexes
./eloqdoc-migration import \
  --source $SOURCE_URI \
  --target $TARGET_URI \
  --tasks index

# Migrate schema, indexes and roles
./eloqdoc-migration import \
  --source $SOURCE_URI \
  --target $TARGET_URI \
  --tasks schema,index,role

Migrate Specific Databases

# Migrate only specified databases
./eloqdoc-migration import \
  --source $SOURCE_URI \
  --target $TARGET_URI \
  --databases mydb1,mydb2,mydb3

Dry Run

# Preview migration without making changes
./eloqdoc-migration import \
  --source $SOURCE_URI \
  --target $TARGET_URI \
  --dry-run \
  --verbose

Command-Line Options

Flag Description Default
--source Source database URI (required) -
--target Target database URI (required) -
--tasks Comma-separated tasks: schema,index,role,user,data schema,index,role
--databases Comma-separated database names all non-system DBs
--dry-run Preview changes without applying false
--verbose Enable verbose logging false
--log-file Write logs to file -
--parallel Number of concurrent database migrations 1

Index Compatibility

Fully Supported ✅

  • Single-Field Index
  • Compound Index
  • Multikey Index
  • Text (Full-Text) Index
  • Geospatial Index (2d, 2dsphere)
  • Sparse Index
  • TTL Index
  • Partial Index
  • Covering Index

Partially Supported ⚠️

  • Unique Index: May have limitations, migration will log warnings

Not Supported ❌

  • Wildcard Indexes
  • Hash Index
  • Hidden Index
  • Clustered Index
  • Time-Series Index

Unsupported indexes are automatically skipped with warnings in the migration report.

MongoDB Atlas Compatibility

Shared Clusters (M0/M2/M5) Limitations

  • User Export: The usersInfo command is restricted on Atlas shared clusters. You may see "permission denied" errors when trying to export users. This is an Atlas limitation, not a tool defect. The tool will skip user export for affected databases and report it as a warning.

Unsupported Roles & Actions

Some Atlas-specific roles and actions are not supported in EloqDoc:

  • Atlas Managed Roles: atlasAdmin, backup, restore
  • Cloud-Specific Actions: Actions related to Atlas-specific features (e.g., search indexes, triggers)

Idempotency

The tool is designed to be idempotent:

  • Indexes: Checks if index already exists before creation
  • Roles: Checks if role already exists before creation
  • Safe Re-runs: Can be executed multiple times safely

This allows you to:

  • Retry failed migrations
  • Resume interrupted migrations
  • Incrementally add new indexes/roles

Examples

Migrate indexes, roles, and export users

./eloqdoc-migration import \
  --source "mongodb+srv://user:[email protected]/" \
  --target "mongodb://eloqdoc:27017" \
  --tasks index,role,user \
  --verbose

Note: User export creates a users_export.json file with all user information. Passwords cannot be automatically migrated and must be manually reset on the target database.

Migration Reports

After each migration, a detailed JSON report is automatically generated with the filename format migration_report_YYYYMMDD_HHMMSS.json.

The report includes:

  • Migration timestamp and duration
  • Source and target connection info (masked)
  • Tasks executed and databases migrated
  • Statistics for each task (total, success, failed, skipped)
  • Detailed error messages for failed operations
  • Warnings for partially supported features

Example report structure:

{
  "timestamp": "2025-12-24T14:30:00Z",
  "duration": "2m30s",
  "source_uri": "mongodb+srv://user...net/",
  "target_uri": "mongodb://eloqdoc...",
  "tasks": ["schema", "index", "role"],
  "results": {
    "index": {
      "stats": {
        "total": 45,
        "success": 42,
        "failed": 2,
        "skipped": 1
      },
      "errors": [
        {
          "database": "mydb",
          "collection": "users",
          "name": "wildcard_idx",
          "error": "Unsupported index type"
        }
      ]
    }
  }
}
./eloqdoc-migration import \
  --source "mongodb+srv://user:[email protected]/admin" \
  --target "mongodb://admin:[email protected]:27017/admin" \
  --verbose

Migrate specific database with verbose output

./eloqdoc-migration import \
  --source "mongodb://localhost:27017" \
  --target "mongodb://eloqdoc:27017" \
  --databases production \
  --verbose \
  --log-file migration.log

Preview migration changes

./eloqdoc-migration import \
  --source "mongodb://localhost:27017" \
  --target "mongodb://eloqdoc:27017" \
  --dry-run \
  --verbose

Migration Output

The tool provides detailed output:

[INFO] Starting index migration
[INFO] Found databases to migrate: 3
[INFO] Migrating database: mydb
[INFO]   Collection: users (5 indexes)
[INFO]     ✓ Created index: email_1 (unique)
[WARN]     ⚠ Unique constraint may have limitations
[INFO]     ✓ Created index: username_1
[INFO]     ✓ Created index: created_at_1 (ttl)
[WARN]     ⊗ Skipped index: search_idx (wildcard not supported)
[INFO]     ✓ Created index: location_2dsphere (geospatial)

=== Migration Summary ===
Task: INDEX
  Total:   18
  Created: 15
  Skipped: 2
  Failed:  1
  Warnings: 3
  Duration: 2.3s

Error Handling

  • Non-fatal errors are logged and migration continues
  • Fatal errors stop the migration and return error code
  • Detailed error messages include database, collection, and index/role names
  • Warnings are provided for partially supported features

Development

Project Structure

eloqdoc-migration/
├── cmd/                    # CLI commands
├── internal/
│   ├── config/            # Configuration
│   ├── connection/        # Database connections
│   ├── migrator/          # Migration logic
│   └── models/            # Data models
├── pkg/
│   └── logger/            # Logging utilities
└── main.go                # Entry point

Building

# Build for current platform
go build -o eloqdoc-migration main.go

# Build for Linux
GOOS=linux GOARCH=amd64 go build -o eloqdoc-migration-linux main.go

# Build for macOS
GOOS=darwin GOARCH=amd64 go build -o eloqdoc-migration-darwin main.go

# Build for Windows
GOOS=windows GOARCH=amd64 go build -o eloqdoc-migration.exe main.go

License

Copyright © 2025 EloqData

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Packages

No packages published