Feasibility of training a Turkmen LLM from scratch on 100MB data #930
|
Hello everyone,
To clarify my expectations: I only need the model to answer questions based on the information contained in this training data. I do not expect it to have general knowledge or capabilities beyond what is in the dataset. Is this feasible? Can I effectively get a working model by following this pipeline with such a small amount of data? |
Replies: 2 comments 1 reply
|
Hi there, I think this sounds somewhat feasible, but in the case of such a small dataset plus
maybe try a RAG setup first. You could try one of these notebooks. Not totally from scratch but should be easy to use: https://github.com/rasbt/RAGs |
|
Dear Sebastian, |
|
did you conclude this project? dm [email protected] we can work on it open source tutkmen only llm i have many usecases for it so it will actully will improve peoples lives. |
Hi there, I think this sounds somewhat feasible, but in the case of such a small dataset plus
maybe try a RAG setup first. You could try one of these notebooks. Not totally from scratch but should be easy to use: https://github.com/rasbt/RAGs