Thanks to visit codestin.com
Credit goes to hanlp.hankcs.com

Toggle navigation sidebar

HanLP Documentation

Introduction

Tutorial
Install
Configuration
Data Format
Annotations
Contributing Guide
Live Demo

Python API

hanlp
- hanlp
- common
  - structure
  - vocab
  - transform
  - dataset
  - component
  - torch_component
- components
  - mtl
    - MultiTaskLearning
    - tasks
  - classifiers
  - eos
  - tokenizers
    - transformer
    - multi_criteria
  - lemmatizer
  - taggers
    - transformer_tagger
    - rnn_tagger
  - ner
  - parsers
  - srl
    - span_rank
    - span_bio
  - pipeline
  - sts
- pretrained
  - mtl
  - eos
  - tok
  - pos
  - ner
  - dep
  - constituency
  - srl
  - sdp
  - amr
  - amr2text
  - sts
  - word2vec
  - glove
  - fasttext
  - mlm
- datasets
  - eos
    - eos
    - resources
  - tok
  - pos
    - resources
  - ner
    - tsv
    - json
    - resources
  - dep
    - conll
    - resources
  - srl
    - conll2012_dataset
    - resources
  - con
    - constituency_dataset
    - resources
- utils
  - io_util
- layers
  - embeddings
    - embedding
    - word2vec
    - fasttext
    - char_cnn
    - char_rnn
    - transformer
  - transformers
    - encoder
    - tokenizer
  - decoders
    - linear_crf
    - biaffine_ner
hanlp_common
- document
- conll
- configurable
- constant
hanlp_restful
hanlp_trie
- trie
- dictionary

Java API

1.x API
Java RESTful API

Golang API

Golang RESTful API

References

References

Theme by the Executable Book Project

Toggle navigation

tokenizers

tokenizers¶

Tokenizers.

transformer
multi_criteria

previous

eos

next

transformer

By hankcs
© Copyright 2020-2025, hankcs.