Repository for OncoKB, a precision oncology knowledge base.
The core of OncoKB Annotation service.
Please confirm your running environment is:
- Java version: 8
- MySQL version: 8.0.36
cp -r core/src/main/resources/properties-EXAMPLE core/src/main/resources/properties
# Otherwise you will not see logs
# You can also do -Dlog4j.configuration=file:core/src/main/resources/properties/log4j.properties
# if you want to skip this step
cp core/src/main/resources/properties-EXAMPLE/log4j.properties core/src/main/resources/log4j.properties
- database.properties
- jdbc.driverClassName : We use mysql as database. Here, it will be com.mysql.jdbc.Driver
- jdbc.url: Database url
- jdbc.username & jdbc.password: MySQL user name and password
- config.properties
- oncotree.api: OncoTree service. Default: http://oncotree.info/api/
enterprise
: includes public and private APIspublic-api
: includes only public API endpointsimage-build
: uses Jib to generate a docker image and push to DockerHub
NOTE: We deprecated the legacy
public
andcurate
profiles.
mvn clean install -P <profile(s)> -DskipTests=true
The WAR file is under /web/target/
- Download VSCode extension
Community Server Connector
- In VSCode sidebar, find
Servers
dropdown. - Right click
Community Server Connector
and chooseCreate New Server
. - Either download Tomcat 8 on local machine or let CSC download for you.
- Right click Tomcat server and choose
Add Deployment
. This is the WAR file generated in the previous step. - Edit the Tomcat server and add
"vm.install.path": "/path/to/java8"
. - Set
"args.override.boolean": "true"
to ensure VM override arguments are applied. - Append
-Dlog4j.configuration=\"file:/path/to/log4j.properties\"
to"args.vm.override.string"
so Tomcat loads the correct Log4j configuration. - Start Tomcat server. Make sure to
Publish Server (Full)
to keep server synchronized with WAR file (if changes were made). - Test the endpoint by making a request to
http://localhost:8080/app/api/v1/info
OncoKB™ is a precision oncology knowledge base developed at Memorial Sloan Kettering Cancer Center that contains biological and clinical information about genomic alterations in cancer. OncoKB uses Genome Nexus to annotate genomic change to protein change using OncoKB picked transcripts. By default, the API requests are sent to www.genomenexus.org for GRCh37 and grch38.genomenexus.org for GRCh38. However, you can choose to use a local version of Genome Nexus by following the instructions for Option A, otherwise follow instructions for Option B.
OncoKB docker compose file consists of the following services:
- OncoKB: provides variant annotations
- OncoKB Transcript: serves OncoKB metadata including gene, transcript, sequence, etc.
- Genome Nexus: provides annotation and interpretation of genetic variants in cancer
- GRCh37 (optional):
- gn-spring-boot: the backend service responsible for aggregating variant annotations from various sources
- gn-mongo: variants fetched from external resources and small static data are cached in the MongoDB database
- gn-vep: is a spring boot REST wrapper service for VEP using GRCh37 data
- GRCh38 (optional):
- gn-spring-boot-grch38: same as
gn-spring-boot
service, however the VEP URL points togn-vep-grch38
- gn-mongo-grch38: contains static data relevant to GRCh38
- gn-vep-grch38: a spring boot REST wrapper service for VEP using GRCh38 data
- gn-spring-boot-grch38: same as
- GRCh37 (optional):
For this option, you need to download the VEP cache, which is used in the gn-vep
and gn-vep-grch38
services. We have pre-downloaded the VEP data and saved them to our AWS S3 Bucket. If interested, here are the instructions we followed to download the Genome Nexus VEP Cache.
-
OncoKB requires a MySQL server and the
oncokb
andoncokb-transcript
databases imported. This step must be completed before continuing the installation process. Reach out to [email protected] to get access to the data dump. How to setup MySQL Server -
Follow the Genome Nexus VEP instructions for downloading and setting up the VEP MySQL server.
-
Run docker-compose to create containers.
docker-compose --profile genome-nexus up -d
Note: The --profile argument is used as a way to selectively enable services. Services with the genome-nexus profile will only be spun up when the profile is specified.
- OncoKB requires a MySQL server and the
oncokb
andoncokb-transcript
databases imported. This step must be completed before continuing the installation process. Reach out to [email protected] to get access to the data dump. - Remove
-Dgenome_nexus.grch37.url
and-Dgenome_nexus.grch38.url
properties from theoncokb
service. - Run docker-compose to spin up oncokb and oncokb-transcript services
docker-compose up -d
Warning
To reduce costs, future Docker Hub images for oncokb
and oncokb-transcript
will be hosted under the MSKCC Docker Hub organization. You can find their
new locations here:
Warning
OncoKB uses a native password for MySQL. If you use MySQL 8.1 or above you must configure native passwords enabled as they no longer are enabled by default.
oncokb | oncokb-transcript | gn-spring-boot | gn-mongo | genome-nexus-vep | MySQL | Redis | Oncokb Data |
---|---|---|---|---|---|---|---|
v3.x.x | v0.9.4 | v1.x.x | v0.x | v0.0.1 | v8.0.x | v7.x.x | v4.x.x |
v4.x.x | v0.9.4 | v1.x.x | v0.x | v0.0.1 | v8.0.x | v7.x.x | v5.x.x |
- Continue to use
mvn test
command that connects to a local MySQL server defined in database.properties file. Note that some test cases will insert dummy data, so make sure the database is not used for any downstream workflows. - Run tests using docker-compose by running
sh unit_test/scripts/run_docker_test.sh
. Make sure to add a mysql data dump file underunit_test/mysql_dumps
The docker compose file has a pre-generated oncokb-transcript JWT token, which is required to make API requests to the oncokb-transcript service. To generate the JWT token, go to the https://jwt.io/ website and follow these instructions:
- Add the
auth
key and set it toROLE_ADMIN
to grant roles. Thepayload
section should look something like this:{ "sub": "1234567890", "name": "John Doe", "auth":"ROLE_ADMIN", "iat": 1516239022 }
- In the
Verify Signature
section, check the boxsecret base64 encoded
. Copy and paste the oncokb-transcript base64 secret into the input box.- You can also change the default base64 secret used for encoding by generating a base64 string and add the environment variable,
JHIPSTER_SECURITY_AUTHENTICATION_JWT_BASE64_SECRET: <new-base64-string>
, to oncokb-transcript.
- You can also change the default base64 secret used for encoding by generating a base64 string and add the environment variable,
- Replace
-Doncokb_transcript.token
with the JWT token you generated.
OncoKB predownloads VEP data and saves it to AWS S3 bucket. These steps are for OncoKB developers and show how to download and upload new Ensembl VEP data to S3. However, you can follow along and save VEP data to your own S3 bucket.
- Change Ensembl image in genome-nexus-vep Dockerfile to desired version
- Follow instructions to download VEP cache files and FASTA files for GRCh37 and GRCh38.
- After downloading your directory should like:
VEP_CACHE/
├─ homo_sapiens/
│ ├─ 98_GRCh37/
│ ├─ 98_GRCh38/
- Zip the files
tar cf 98_GRCh37.tar homo_sapiens/98_GRCh37
tar cf 98_GRCh38.tar homo_sapiens/98_GRCh38
- Go to AWS S3 webpage and under
oncokb/gn-vep-data/
, create two folders:
98_GRCh37/
98_GRCh38/
- Upload
tar
files to corresponding S3 folders - Make the two S3 folders (
oncokb/gn-vep-data/98_GRCh37/
andoncokb/gn-vep-data/98_GRCh38/
) publicly accessible - Update
gn-vep
andgn-vep-grch38
services indocker-compose.yml
Modify environment variable to point to the new FASTA file
gn-vep
VEP_FASTAFILERELATIVEPATH=homo_sapiens/98_GRCh37/Homo_sapiens.GRCh37.75.dna.primary_assembly.fa.gz
gn-vep-grch38
VEP_FASTAFILERELATIVEPATH=homo_sapiens/98_GRCh38/Homo_sapiens.GRCh38.dna.toplevel.fa.gz
- Modify Dockerfile line in genome-nexus-vep to use the new Ensembl VEP image. As of 4/28/2023, genome-nexus-vep uses
ensemblorg/ensembl-vep:release_98.3
. - Push new genome-nexus-vep image to DockerHub
- Change the image for both
gn-vep
andgn-vep-grch38
to the image built in step 7.
-
Create a Docker network
docker network create mysql-test-network
-
Run MySQL container and attach to the network
docker run --name my-mysql -e MYSQL_ROOT_PASSWORD=root -p 3307:3306 --network mysql-test-network -d mysql:8
We suggest using MySQL 8 since that is what we use in production. -
If you already have a MySQL Docker container running, follow the steps to add network (otherwise skip)
docker stop my-mysql
docker network connect mysql-test-network my-mysql
docker start my-mysql
- Update docker-compose file
# Add networks to each service
services:
oncokb:
networks:
- mysql-test-network
...
oncokb-transcript:
networks:
- mysql-test-network
...
# Declare external network that MySQL container is connected to
networks:
mysql-test-network:
external: true
- Now, the container is connected to the
mysql-test-network
network, and other containers on the same network can communicate with it using its container name (my-mysql).
#JDBC URL
jdbc:mysql://my-mysql:3306/oncokb_core?useUnicode=yes&characterEncoding=UTF-8&useSSL=false
-
Follow MySQL documentation
-
On MacOS and Windows, you can use host.docker.internal to point docker containers to MySQL server running on local machine.
#JDBC URL
jdbc:mysql://host.docker.internal:3306/oncokb_core?useUnicode=yes&characterEncoding=UTF-8&useSSL=false
The best way is to send an email to [email protected] so all our team members can help.