Thanks to visit codestin.com
Credit goes to github.com

Skip to content

regexpr/polyglot-db

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

PolyG-DBP - [PolyG]lot - [D]ata[B]ase[P]erformance

Big Data Traineeship SuSe 2018

See https://dbs.uni-leipzig.de/study/ss_2018/bigdprak

Contributor

  • Hyeon Ung Kim (LastMinuteHero)
  • Tim Niehoff (regexpr)

Teacher

  • J. Zschache

Requirements

  • Maven
  • MongoDB
  • Neo4J
  • Mongo Connector + Neo4j doc manager

You can install mongo connector and neo4j doc manger with

pip install -r utils/pip-requirements.txt   

Please visit our Wiki to find out which Versions of the requiremental software we have used: https://github.com/regexpr/polyglot-db/wiki/

Installing

git clone https://github.com/regexpr/polyglot-db
cd polyglot-db/PolyG-DBP/
mvn install

Run

  1. Run one Neo4j database in the background:
neo4j/bin/neo4j console

2.a Run one mongod replica set in the background

mongod --replSet exSet 

2.b For the first time, initialize the replica set

mongo
rs.initiate()
  1. Run PolyG-DBP
cd polyglot-db/PolyG-DBP/target
java -jar PolyG-DBP-0.1.jar

Usage

  • java -jar PolyG-DBP-0.1.jar help: Displays this help;
  • java -jar PolyG-DBP-0.1.jar list: lists all queries provided by PolyG-DBP.;
  • java -jar PolyG-DBP-0.1.jar QUERY [Options] Benchmark with the given query. Example
    java -jar PolyG-DBP-0.1.jar q1
  • java -jar PolyG-DBP-0.1.jar custom [Options] Benchmark with a custom query from PolyG-DBP. You will be asked to specify your queries for Mongo and Neo4j afterwards. Example: java -jar PolyG-DBP-0.1.jar custom.

Options (can be given in any order)

  • -i, --inputPath to the directory with JSON file(s). Example:
java -jar PolyG-DBP-0.1.jar q1 -i yelp
  • -nb, --neo4jAddressBolt1Adress of the neo4j instance with the bolt address. Example:
java -jar PolyG-DBP-0.1.jar q1 -nb localhost:7687
  • -nr, --neo4jAddressRemoteAdress of the neo4j instance with the remote address. Example:
java -jar PolyG-DBP-0.1.jar q1 -nr localhost:7474
  • -m, --mongoAddressAdress of the mongodb instance. Example:
java -jar PolyG-DBP-0.1.jar q1 -m localhost:27017
  • -md, --mongoDatabaseName of the mongodb database. Example:
java -jar PolyG-DBP-0.1.jar q1 -md yelp
  • -r, --reduceImport just a certain amount of lines of each input JSON. Example:
java -jar PolyG-DBP-0.1.jar q1 -r 300

Available Queries for Yelp Dataset for both Databases:

  • q1: Output all business names and ids a rated with minumum of
  • q2: Output the average stars of all businesses
  • q3: Output the average stars of all businesses that grouped by category
  • q4: Output the average stars of all businesses grouped by category descending order
  • q5: Output all businesses that are in the category Cannabis Tours and return the average of all stars grouped by all the categories that they are in
  • q6: Output all businesses that were reviewed more than 700 times
  • q7: The same query as above, but this time we count the reviews in the reviews collection
  • qa: Run all queries above.

Furthermore

  • Why not testing the efficiency of the neo4j data model created by mongo-connector + neo4j doc manager? You can import the YELP dataset into the Neo4j directly by inserting the commands given in utils/Neo4jImportYelpJsons.txt into a cypher shell.

About

Big Data Praktikum

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 2

  •  
  •  

Languages