The First Workshop on Fact Extraction and Verification

To reproduce our FEVER shared task results:

1. switch to the takuma-dev branch
2. run bash initial_setup.sh (This will download several files and take some time.)
3. move to jack directory and install dependencies according to the README. (i.e., python3 -m pip install -e .[tf])
4. move to fever-baselines directory and install dependencies (i.e., pip install -r requirements.txt)

After step 2, fever, jack, fever-baselines directory should be at the same level (these should be in the same directory).

python3 pipeline.py --config configs/submission_config.json --model [arbitrary name]

Output files will be generated under results/[arbitrary name]. submission.json and test_submission.json correspond to the output for development and test set for each. (Note that the score displayed after running this particular model is not valid, since we included development set for the training even though the score is calculated based on the development set.)

1. create new configuration file configs/config.json and reflect your directory structure
2. run python3 pipeline.py --config configs/config.json --model [arbitrary name]

1. remove index files in data directory
2. run python3 doc_ir_model.py to create a document index and retrieval model
3. run python3 line_ir_model.py to create a line index and retrieval model

Configuration files can have a parent, which is specified by parent_config attribute.

Attributes that are not specified in the child config file are inherited from the parent. Usually you do not need to modify the parent config.

Note that currently a child cannot have its child. (Having a grand-parent or grand-child is not supported)

Information Retrieval module.

Data format conversion module (to let jack handle our data).

Recognizing Textual Entailment module (training).

Recognizing Textual Entailment module (inference).

Aggregation module

Reranking module

Evaluation module

UCL Machine Reading Group: Four Factor Framework For Fact Finding (HexaF)