AI-Powered Data Mapper for Automated Workflows
Transforma is a powerful data transformation platform that leverages AI to automatically generate JavaScript functions for mapping data between different schemas. Transforma intelligently creates optimized transformation logic that handles validation, error cases, and complex data mapping scenarios.
- 🤖 AI-Powered Code Generation: Automatically generates transformation functions using OpenAI
- 🔄 Schema-to-Schema Mapping: Supports JSON Schema or loosly defined JSON if you're feeling spicy
- ⚡ High-Performance Execution: Built-in caching system for compiled transformation functions
- 🛡️ Robust Validation: Comprehensive input/output validation with detailed error handling
- 📊 Execution Analytics: Track transformation usage and performance metrics
- 🔍 Search & Pagination: Easily manage and discover existing data maps
- 🏗️ Monorepo Architecture: Scalable architecture with shared packages and clear separation of concerns
- Schema Definition: You provide input and output schemas
- AI Generation: Transforma uses OpenAI to generate a JavaScript transformation function
- Validation: The generated function includes robust validation logic
- Execution: Input data is processed through the generated function with performance caching
- Analytics: Execution metrics are tracked for monitoring and optimization
Input Schema:
{
"type": "object",
"properties": {
"user_name": {"type": "string"},
"user_email": {"type": "string"},
"registration_date": {"type": "string"}
},
"required": ["user_name", "user_email"]
}Output Schema:
{
"type": "object",
"properties": {
"name": {"type": "string"},
"email": {"type": "string"},
"registeredAt": {"type": "string"}
},
"required": ["name", "email"]
}Generated Function:
function transform(inputObject) {
if (!inputObject || typeof inputObject !== 'object') {
throw new Error("Error: Input must be an object");
}
if (inputObject.user_name == null) {
throw new Error("Error: user_name is required");
}
if (inputObject.user_email == null) {
throw new Error("Error: user_email is required");
}
return {
name: inputObject.user_name,
email: inputObject.user_email,
registeredAt: inputObject.registration_date || null
};
}The easiest way to run Transforma is using Docker Compose, which sets up all services including the database.
-
Clone the repository
git clone https://github.com/zjcompt/transforma.git cd transforma -
Create secrets directory and files
mkdir secrets echo "your_database_password_here" > secrets/db_password.txt echo "your_openai_api_key_here" > secrets/open_api_key.txt
-
Build and start all services
docker-compose up --build
- Node.js 22+
- PostgreSQL database
- OpenAI API key
-
Clone the repository
git clone https://github.com/zjcompt/transforma.git cd transforma -
Install dependencies
npm install
-
Set up environment variables Create a
.envfile inapps/engine/:PORT=3000 ADDRESS=localhost (defaults to localhost) POSTGRES_DB='transforma' (defaults to transforma) POSTGRES_USER=your_postgres_user POSTGRES_PASSWORD=your_postgres_password POSTGRES_HOST='localhost' (defaults to localhost) POSTGRES_PORT=5432 (defaults to 5432) OPENAI_API_KEY=your_openai_api_key_here OPENAI_MODEL=your_desired_model (defaults to 4.1) EXECUTION_CACHE_SIZE=100 (defaults to 100)
-
Start development servers
# Start all applications npm run dev
Transforma is built as a modern TypeScript monorepo using Turborepo:
transforma/
├── apps/
│ ├── engine/ # Fastify API backend
│ └── manager/ # React frontend (coming soon)
└── packages/
├── imports/ # Shared TypeScript interfaces
├── eslint-config/ # Shared ESLint configurations
└── typescript-config/ # Shared TypeScript configurations
Backend (Engine)
- Framework: Fastify with TypeScript
- Database: PostgreSQL
- AI Integration: OpenAI API
- Validation: AJV (JSON Schema validation)
- Logging: Pino with pretty printing
Frontend (Manager)
- Framework: React 19, shadcn/ui, Tailwind
- Build Tool: Vite
- Development: Hot Module Replacement (HMR)
Development Tools
- Monorepo: Turborepo for build orchestration
- Package Manager: npm workspaces
- Linting: ESLint with shared configurations
- Formatting: Prettier
http://localhost:3000/api/v1
POST /mapRequest Body:
{
"name": "User Profile Transformation",
"type": "jsonSchema",
"inputSchema": "{\"type\":\"object\",\"properties\":{\"firstName\":{\"type\":\"string\"},\"lastName\":{\"type\":\"string\"}},\"required\":[\"firstName\",\"lastName\"]}",
"outputSchema": "{\"type\":\"object\",\"properties\":{\"fullName\":{\"type\":\"string\"}},\"required\":[\"fullName\"]}"
}Response:
{
"id": "123e4567-e89b-12d3-a456-426614174000",
"name": "User Profile Transformation",
"type": "jsonSchema",
"inputSchema": "...",
"outputSchema": "...",
"javascript": "function transform(inputObject) { ... }",
"timesRan": 0,
"createdAt": "2024-01-01T00:00:00.000Z"
}POST /map/{id}/executeRequest Body:
{
"input": {
"firstName": "John",
"lastName": "Doe"
}
}Response:
{
"output": {
"fullName": "John Doe"
}
}GET /map?page=1&limit=10&search=profileGET /map/{id}DELETE /map/{id}PUT /map/{id}Request Body:
{
"name": "User Profile Transformation V2",
"type": "jsonSchema | json",
"inputSchema": "...",
"outputSchema": "..."
}Response:
{ ... Updated map object }
GET /map/{id}/runs?page=1&limit=10This project is licensed under the MIT License - see the LICENSE file for details.
- 🐛 Issues: GitHub Issues
- 📧 Contact: Zach Compton
- 📚 Documentation: Coming soon
- Complete React frontend
- OpenAPI Documentation
- Support for additional data formats (CSV, XML, YAML)
- Real-time transformation preview
- Batch processing capabilities
- Plugin system for custom transformations
- API rate limiting and authentication
- Docker containerization
Made with ❤️ by Zach Compton