Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[Model]Transformer#186

Merged
yzh119 merged 80 commits into
masterfrom
transformer
Dec 7, 2018
Merged

[Model]Transformer#186
yzh119 merged 80 commits into
masterfrom
transformer

Conversation

@yzh119

@yzh119 yzh119 commented Nov 28, 2018

Copy link
Copy Markdown
Member

Update the model transformer.

Universal Transformer would be uploaded soon.
Something I'm not sure are:

  1. whether I should put the codes in dataset to dgl.data?
  2. I am also not sure where to put the codes in scripts.

TODOs:

  1. MultiGPU support
  2. MultiGPU loss

@QipengGuo

Copy link
Copy Markdown
Contributor
  • Put the link of dataset and Transformer paper in the Readme.
  • More comments, such as the class SimpleLossCompute (examples/pytorch/transformer/loss/__init__.py), why the input argument "criterion" could be a list or instance (the support of Multi-GPU)?
  • the function "clones" in examples/pytorch/transformer/modules/layers.py maybe not necessary.
  • the class Embeddings (examples/pytorch/transformer/modules/embedding.py) could be replaced by the torch.nn.Embedding with the "max_norm" argument?
  • Does the variable "N" in dgl/examples/pytorch/transformer/scripts/build_vocab.py used?
  • Keep the translated text after testing, I think the user wants to see it.

@jermainewang

Copy link
Copy Markdown
Member
  1. I saw data files in the folder. You could pack them in a zip, upload it to some cloud storage (current solution, improved later) and download it on demand. You can use the download and the unzip function.
  2. The data provider related codes can stay in the examples folder.
  3. What are the scripts used for? If they are used to preprocess data, I will also pack them in the same zip, and run it using your python script.

@yzh119

yzh119 commented Nov 29, 2018

Copy link
Copy Markdown
Member Author

@jermainewang

  1. The reason i don't use download function is that one of the dataset required in my task was stored in google drive, which is not supported, thus i use another download function copied from torchtext.
  2. The scripts are used for both preprocessing and evaluating.

@QipengGuo

Copy link
Copy Markdown
Contributor

package dependencies: requests, tqdm
The model is still training, it looks good.

@yzh119

yzh119 commented Dec 1, 2018

Copy link
Copy Markdown
Member Author

Data and script related files are supposed to be downloaded from s3. I'll fix this.

@yzh119 yzh119 requested a review from QipengGuo December 6, 2018 14:57

@QipengGuo QipengGuo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good job

@VoVAllen

VoVAllen commented Dec 7, 2018

Copy link
Copy Markdown
Collaborator

Tutorial and README finished.

@jermainewang jermainewang self-requested a review December 7, 2018 06:35
@yzh119 yzh119 merged commit 9f32554 into master Dec 7, 2018
@yzh119 yzh119 deleted the transformer branch December 7, 2018 07:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants