Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Convert CSV files to custom JSON formats using simple templates.

max9159/csv_2_json

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

CSV to JSON Array Converter

Convert CSV files to custom JSON formats using simple templates. Perfect for data transformation or API integration.

🚀 Why Use This Tool?

  • Zero Code Required - Just configure templates and run
  • Custom JSON Formats - Create any JSON structure you need
  • Batch Processing - Handle multiple CSV files at once
  • Data Type Conversion - Auto-convert to string, int, float, bool, timestamp
  • Template Reuse - Save and reuse templates for different projects

⚡ Quick Start

# 1. Process all CSV files (uses default template)
python csv_to_json.py

# 2. Process specific file
python csv_to_json.py --input-file your_file.csv

# 3. Use custom template
python csv_to_json.py --template api_format

That's it! Your JSON files will be in export_json/ folder.

🎨 How Templates Work

Simple concept: Map CSV columns to JSON structure using {{type:column_name}} syntax.

Example: CSV to JSON

Your CSV:

name,age,email
John,25,[email protected]

Template:

{
  "full_name": "{{string:name}}",
  "years_old": "{{int:age}}",
  "contact": "{{string:email}}"
}

Result:

[
  {
    "full_name": "John",
    "years_old": 25,
    "contact": "[email protected]"
  }
]

Data Types

  • {{string:column}} - Text
  • {{int:column}} - Numbers
  • {{float:column}} - Decimals
  • {{bool:column}} - True/False
  • {{unixtimestamp:column}} - Unix timestamp

📂 Setup

your_project/
├── csv_to_json.py         # Main script
├── template/import_template.json    # Your templates
├── import_csv/                      # Put CSV files here
└── export_json/                     # JSON output goes here

🔧 Commands You'll Use

# Setup: Create template config
python csv_to_json.py --create-config

# Process: Convert all CSV files  
python csv_to_json.py

# Check: List available templates
python csv_to_json.py --list-templates

# Custom: Use specific template
python csv_to_json.py --template your_template_name

🎯 Template Options

Mix Static + Dynamic Values

{
  "user_name": "{{string:name}}",
  "status": "active",
  "user_id": "{{int:id}}",
  "profile_url": "https://example.com/user/{{string:username}}"
}

Nested JSON Structures

{
  "user": {
    "personal": {
      "name": "{{string:full_name}}",
      "age": "{{int:age}}"
    },
    "contact": {
      "email": "{{string:email}}"
    }
  }
}

🚀 Getting Started

  1. Place CSV files in import_csv/ folder
  2. Run setup: python csv_to_json.py --create-config
  3. Edit templates in template/import_template.json (optional)
  4. Convert files: python csv_to_json.py
  5. Get results from export_json/ folder

❓ Common Issues

Problem Solution
"Column not found" Check column names match CSV headers exactly
Empty JSON output Verify CSV has headers and proper format
Template not found Run --list-templates to see available options
Date conversion fails Use {{timestamp:column}} for date fields

🔍 Data Verification

Verify your CSV→JSON conversions are 100% accurate!

Quick Verification

# Verify all files
python tests/verify_csv_json.py

# Verify single file
python tests/verify_csv_json.py --file ETH_USDC_Blacklist.csv

What Gets Verified ✅

  • Row Count Matching: CSV rows = JSON records
  • Data Transformations: All {{type:column}} mappings work correctly
  • Type Conversions: String→Int, DateTime→Unix timestamp, etc.
  • Static Values: Fixed values in templates (like "status": "active")
  • Template Integrity: Every CSV field correctly mapped to JSON

Sample Output

🔍 CSV to JSON Data Integrity Verification
============================================================
📁 Found 9 CSV files to verify

📋 Verifying: ETH_USDC_Blacklist.csv → ETH_USDC_Blacklist.json
✓ CSV loaded: 314 rows
✓ JSON loaded: 314 records  
✓ Row counts match: 314 records
✓ Using template with 4 mapped fields
✅ Data verification PASSED - All 314 records match perfectly

📊 VERIFICATION SUMMARY
============================================================
Total files verified: 9
✅ Successful: 9
❌ Failed: 0

📈 DATA STATISTICS:
Total CSV rows processed: 6,837
Total JSON records created: 6,837

🎉 ALL VERIFICATIONS PASSED! Your data integrity is perfect.

When to Use Verification

  • After first conversion - Ensure everything works correctly
  • Template changes - Verify new mappings work as expected
  • New data files - Check compatibility with existing templates
  • Before production - Final quality assurance step
  • Troubleshooting - Identify specific data issues

Verification Options

# Verify all CSV/JSON pairs
python tests/verify_csv_json.py

# Verify specific file
python tests/verify_csv_json.py --file your_file.csv

# Use custom template for verification
python tests/verify_csv_json.py --template custom_template.json

💡 Tips

  • Test first: Use --input-file single.csv to test templates
  • Check headers: Column names in template must match CSV exactly
  • Backup data: Keep original CSV files safe
  • Start simple: Use --create-config for basic setup
  • Verify always: Run verification after any template changes

Ready to convert your CSV files to custom JSON formats! 🚀

About

Convert CSV files to custom JSON formats using simple templates.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages