Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit 828d059

Browse files
committed
fix: skip triplet JSON export for large datasets (>100K)
JSON.stringify fails on 1M+ triplets. Training succeeded (33.3% improvement) but export crashed. Now skips export when >100K triplets. Co-Authored-By: claude-flow <[email protected]>
1 parent 85417b8 commit 828d059

1 file changed

Lines changed: 7 additions & 3 deletions

File tree

scripts/train-ruvllm.js

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1257,9 +1257,13 @@ async function main() {
12571257
contrastiveResult.finalLoss = finalContrastiveLoss;
12581258
contrastiveResult.improvement = contrastiveImprovement;
12591259

1260-
// Export contrastive training data
1261-
const contrastiveOutDir = contrastiveTrainer.exportTrainingData();
1262-
console.log(` Training data exported to: ${contrastiveOutDir}`);
1260+
// Export contrastive training data (skip for large datasets to avoid JSON string limit)
1261+
if (contrastiveTrainer.getTripletCount() < 100000) {
1262+
const contrastiveOutDir = contrastiveTrainer.exportTrainingData();
1263+
console.log(` Training data exported to: ${contrastiveOutDir}`);
1264+
} else {
1265+
console.log(` Skipping triplet export (${contrastiveTrainer.getTripletCount()} triplets too large for JSON)`);
1266+
}
12631267

12641268
// -----------------------------------------------------------------------
12651269
// Phase 2: Task head training via TrainingPipeline

0 commit comments

Comments
 (0)