📌 Developed by XI TJKT 2 | 2024/2025 | ❗ Any commercial use or unauthorized exploitation is prohibited
Release:
██╗ ██╗ ████████╗ ███████╗
██║ ██║ ╚══██╔══╝ ██╔════╝
██║ ██║ ██║ ███████╗
██║ ██║ ██║ ██╔════╝
██║ ███████╗ ██║ ███████╗
╚═╝ ╚══════╝ ╚═╝ ╚══════╝
-------------------------------
ILTE - Indigenous Language Translator Engine
The Indigenous Language Translator Engine (ILTE) now offers four distinct versions, each tailored to different translation needs:
- 🌱 ILTE-ALT (Optimized for Speed) – A lightweight, dictionary-based translator optimized for fast, low-resource translations.
- 🧠 ILTE-ZS (Hybrid, Multi-Processing) – Combines dictionary-based rules, RBMT, FST, semantic matching, and zero-shot translation while efficiently handling large text files.
- 🧐 ILTE-ADV (AI-Powered, Context-Aware) – An advanced, AI-driven translation engine that integrates context awareness, semantic similarity, and zero-shot learning.
- 🔮 ILTE-ATI (Advanced Attention & Iterative Processing) – The most sophisticated version with hierarchical normalization, iterative refinement, attention-based translation, and multi-level candidate selection.
- ✅ Dictionary-Based Lookup for direct translations.
- ✅ Basic Stemming for Indonesian (ID) & English (EN).
- ✅ Levenshtein Distance Matching for closest word lookup.
- ✅ Automated Confidence Scoring for accuracy estimation.
- ✅ Structured DOCX Report Generation.
- ✅ Low Memory Usage – Optimized for lower-end machines.
- ⚡ Dictionary + RBMT + FST + Semantic Matching + Zero-Shot Translation.
- ⚖️ Handles Large Files Efficiently via chunking & batch multi-processing.
- 🛠️ Optimized Resource Management, cleans memory and GPU after processing.
- 🔄 Auto-Parallelized Translation Pipeline.
- ⏳ Faster Preprocessing, No Unnecessary Computation.
- 🧠 Contextual Translation using IndoBERT & Sentence Transformers.
- 🔍 Zero-Shot Learning for Handling Unknown Words.
- 📚 Pattern-Based Learning & Semantic Matching.
- 🛠️ Enhanced Translation Confidence Metrics.
- ⚡ Leverages GPU Acceleration for Faster Processing.
- ✨ Hierarchical Normalization for Better Preprocessing.
- 🔄 Iterative Translation for Context Awareness.
- 📚 Attention-Based Translation for Multi-Level Candidate Generation.
- ⚖️ Refined Confidence Scoring & Adaptive Refinement.
- ✅ Full Formatting Preservation in DOCX Reports.
- 🚀 Optimized for Dynamic, Multi-Stage Translation Processes.
| Feature | Model Used |
|---|---|
| Translation (ID-EN, EN-ID) | Helsinki-NLP/opus-mt-id-en, Helsinki-NLP/opus-mt-en-id |
| Stemming | Sastrawi (Indonesian), SnowballStemmer (English) |
| Fuzzy Matching | Levenshtein Distance |
| Feature | Model Used |
|---|---|
| Dictionary-Based Lookup | JSON-based dictionary |
| Rule-Based Translation (RBMT, FST) | Custom FST Rules |
| Semantic Similarity | sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 |
| Zero-Shot Translation | facebook/mbart-large-50-many-to-many-mmt |
| Feature | Model Used |
|---|---|
| Contextual Embeddings | cahya/bert-base-indonesian-1.5G |
| Semantic Matching | sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 |
| Zero-Shot Classification | typeform/distilbert-base-uncased-mnli |
| Translation (ID-EN, EN-ID) | Helsinki-NLP/opus-mt-id-en, Helsinki-NLP/opus-mt-en-id |
| Feature | Model Used |
|---|---|
| Hierarchical Normalization | Regex + Dynamic Stemming |
| Contextual Translation | sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 |
| Iterative Processing | Multi-Level Candidate Refinement |
| Translation (ID-DYK, DYK-ID) | Enhanced Dictionary Lookup + Semantic Matching |
| Feature | ILTE-ALT | ILTE-ZS | ILTE-ADV | ILTE-ATI |
|---|---|---|---|---|
| Translation Approach | Dictionary | Hybrid | AI-Based | Attention-Based + Iterative |
| Processing Speed | Fast | Moderate | Slower | Balanced |
| Handling Large Files | Struggles | Efficient Chunking | Slower | Optimized Processing |
| Memory Usage | Low | Moderate | High | Optimized |
| Context Awareness | None | Partial | Strong | 🔮 Very Strong |
| Idiomatic Expressions | Limited | Rule-Based | AI-Based | AI + Attention |
| Parallelization | Minimal | Yes | DataLoader | Thread + Process Pool |
| Zero-Shot Capability | No | Yes | Yes | Yes |
| Best Use Case | Fast translation | Large text processing | Context-Aware | High-Accuracy, AI-Powered |
python engine_ALT.pypython engine_ZS.pypython engine_ADV.pypython engine_ATI.pyChoose the version that best suits your needs and contribute to indigenous language preservation. 🚀
- ✅ ALT: For lightweight, dictionary-based translations.
- ✅ ZS: For handling large files efficiently with hybrid translation techniques.
- ✅ ADV: For AI-powered, context-aware translations.
- ✅ ATI For advanced attention towards content and context.