Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Annotations#29

Merged
sbassi merged 28 commits into
masterfrom
annotations
Oct 20, 2017
Merged

Annotations#29
sbassi merged 28 commits into
masterfrom
annotations

Conversation

@eseiver

@eseiver eseiver commented Oct 18, 2017

Copy link
Copy Markdown
Collaborator

This addresses #28 and updates the filename structure for annotation DOIs so that they do not depend on the journal name. Getting the journal name required going into the article XML or querying online.
Example DOI: '10.1371/annotation/008b05a8-229b-4aca-94ae-91e6dd5ca5ba'
Previous filename: 'allofplos_xml/journal.one.correction.008b05a8-229b-4aca-94ae-91e6dd5ca5ba.xml'
New filename: 'allofplos_xml/plos.correction.008b05a8-229b-4aca-94ae-91e6dd5ca5ba.xml'

In addition, the filename regex has been updated to this new format.

If you have an existing copy of the corpus and want to transform the annotation articles to the new format, I wrote a gist to aid with that process.

I accidentally created this from the prodteam branch instead of master so there are a lot of useless old commits. My apologies.

All transform functions are passing the unit tests.

When this PR is merged, before closing please tag me so I can be sure to update the filenames on the corpus server.

@eseiver eseiver requested a review from sbassi October 18, 2017 23:15

@sbassi sbassi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code looks good. Will run it locally before merging.

@sbassi

sbassi commented Oct 19, 2017

Copy link
Copy Markdown
Contributor

Now I found an error, will take a look later:

Traceback (most recent call last):
File "plos_corpus.py", line 1060, in
main()
File "plos_corpus.py", line 1042, in main
dois_needed_list = get_dois_needed_list()
File "plos_corpus.py", line 316, in get_dois_needed_list
local_article_list = [filename_to_doi(article) for article in listdir_nohidden(directory, '.xml')]
File "plos_corpus.py", line 316, in
local_article_list = [filename_to_doi(article) for article in listdir_nohidden(directory, '.xml')]
File "plos_corpus.py", line 130, in filename_to_doi
return doi
UnboundLocalError: local variable 'doi' referenced before assignment

@sbassi

sbassi commented Oct 19, 2017

Copy link
Copy Markdown
Contributor

The problem is in the method filename_to_doi(). When the flow doesn't enter in any of the 3 if, this error is occurs. I will print out the filename that is being processed to see which one trigger this.

@sbassi

sbassi commented Oct 19, 2017

Copy link
Copy Markdown
Contributor

This guy is the culprit:
allofplos_xml/journal.pone.correction.dc398db7-e121-4bce-82d1-feac6d9bc68f.xml

@sbassi

sbassi commented Oct 19, 2017

Copy link
Copy Markdown
Contributor

I found that the REGEX was targeted to newarticledir but the file is in the corpusdir.
Will try by removing the dir in the validate_file function. BTW, what is validating is the file name, not the file, so I will rename it accordingly.

@sbassi

sbassi commented Oct 19, 2017

Copy link
Copy Markdown
Contributor

Pushed some changes, what is left: url_to_path(), change this method because annot_prefix is not 'plos.correction.' anymore

@eseiver

eseiver commented Oct 19, 2017

Copy link
Copy Markdown
Collaborator Author

Hi @sbassi, if you read the PR, my changes were designed to change the annotation files to a new structure. The reason your file failed is because it's the old structure: journal.p[journal code].correction, whereas now it's plos.correction, which does not require discovering the journal name. Would you please revert your changes? Thanks!

@sbassi

sbassi commented Oct 19, 2017

Copy link
Copy Markdown
Contributor

I don't get it, if it is now plos.correction, why do we have a file called journal.pone.correction.dc398db7-e121-4bce-82d1-feac6d9bc68f.xml?

@eseiver

eseiver commented Oct 19, 2017

Copy link
Copy Markdown
Collaborator Author

'annotation' DOI files deleted, redownloaded onto allofplos server, zip file recreated!

@sbassi sbassi merged commit 56c675d into master Oct 20, 2017
@sbassi sbassi deleted the annotations branch October 20, 2017 05:34
@eseiver eseiver added this to the 0.8.3 milestone Jan 19, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants