Codestin Search App

tom-odon / NGrams Public

Notifications You must be signed in to change notification settings
Fork 0
Star 1

Extract unigrams and bigrams from text, and utilize their frequencies to create models for rating perplexity of new data as well as generating new sentences.

1 star 0 forks Branches Tags Activity

Star

Notifications

Branches Tags

Name		Name	Last commit message	Last commit date
Latest commit History 3 Commits
HG-heldout50.txt		HG-heldout50.txt
HG-test.txt		HG-test.txt
HG-train.txt		HG-train.txt
HG-wordtypes.txt		HG-wordtypes.txt
NGrams.py		NGrams.py
TESTOUTPUT.txt		TESTOUTPUT.txt
WriteUp.docx		WriteUp.docx
data-output.txt		data-output.txt
data-small.txt		data-small.txt
data-vsmall-output.txt		data-vsmall-output.txt
data-vsmall.txt		data-vsmall.txt
ngramprob-HG-new.txt		ngramprob-HG-new.txt
ngramprob-tiny-new.txt		ngramprob-tiny-new.txt
output-HG-test-new.txt		output-HG-test-new.txt
output-tiny-test.txt		output-tiny-test.txt
output-tiny-train.txt		output-tiny-train.txt
tiny-test.txt		tiny-test.txt
tiny-train.txt		tiny-train.txt
tiny-wordtypes.txt		tiny-wordtypes.txt
~$riteUp.docx		~$riteUp.docx

About

Extract unigrams and bigrams from text, and utilize their frequencies to create models for rating perplexity of new data as well as generating new sentences.