ALBERT model Pretraining and Fine Tuning using TF2.0

[ECCV 2024] M3DBench introduces a comprehensive 3D instruction-following dataset with support for interleaved multi-modal prompts.

mlm-pipeline is a cloud architecture that preprocesses the masked language model (mlm)

Minimal Learning Machine implementation using the scikit-learn API.

This repository contains the code for pretraining a BERT model on domain-specific data.

Repository for the ACL 2023 paper "Deriving Language Models from Masked Language Models"

Just practising deploying MLM locally on my laptop

This repository extends a basic MLM implementation to allow for efficiently conditioning on chained previous texts, in a tree; for e.g., a Reddit thread.

BERT and GPT Model Implementation with Training Procedures

This is a guide on how to train a new language from scratch with Transformers. this scripts use Oscar Corpus as the dataset and a MLM task model is trained for Farsi Language

Fine-tuned chemical language model for predicting molecular lipophilicity in drug design. Explores parameter-efficient fine-tuning strategies (LoRA, BitFit, IA3), layer freezing techniques, and influence-based data selection. Balances accuracy and computational efficiency for molecular property prediction tasks.

Training multilingual language models on African languages including Amharic, Fulani, Hausa, Somali, Swahili, and Yoruba.

大模型视力表

A Chinese Typo Checker based on BERT MLM.

NLP based solution for Hangman

BERT Pretraining Logic Implementation in Tensorflow2

Causal and Mask Language Modeling with 🤗 Transformers