This command prompt application StixToNeoDB, loads STIX-2 objects and relations from json and zip files into a Neo4j graph database.
See also StixLoader for a prototype graphical user interface to convert STIX-2 data.
The OASIS open standard Structured Threat Information Expression STIX-2 is a language for expressing cyber threat and observable information.
Neo4j "is a highly scalable native graph database that leverages data relationships as first-class entities, helping enterprises build intelligent applications to meet today’s evolving data challenges." In essence, a graph database and processing engine that is used here for storing Stix objects and their relationships.
StixToNeoDB converts files of STIX-2 domain objects (SDO) and relationships (SRO) to Neo4j nodes and relations using the Java Neo4j API. This allows for creating a new database or for adding new nodes and relations to an existing Neo4j graph database. The file types include files containing STIX-2 bundles in JSON format and zip files.
Once the Stix objects are in a Neo4j graph database you can use the built-in tools to visualise and analyse the data. Other tools such as the Tinkerpop framework and Spark GraphX can easily link to the neo4j data for very large data sets processing. You can also use these export tools to export the data into GraphML, Cypher statements, CSV and binary formats. See also how-2-convert for some explanations on how to convert Stix objects into those formats.
To load your Stix objects data into a Neo4j graph database, simply type at the prompt:
java -jar stixtoneodb-4.0.jar -f stix_file db_dir
or
java -jar stixtoneodb-4.0.jar -x stix_file db_dir
A stixtoneodb-4.0.jar file is in the distrib directory ready for use (uncompress the file first).
With the option -f the input file stix_file must be a file containing the Stix objects data that you want to convert, and db_dir is the location path to the Neo4j database directory. The input file can be a text file containing a single bundle in json format or a zip file containing one or more bundle files (.zip). Only .json and .stix files in the zip file are processed.
If db_dir is absent, the default output directory will be in the current directory with the name stixdb. If the database already exists, the data will be added to it, otherwise a new neo4j database will be created. An existing database must not be "opened" by another process during processing.
The -x option is for the experimental processing of large file one line at a time. The input file must contain a Stix object on one line ending with a new line. Similarly if the input file is a zip file, each zip file entry must contain Stix objects on one line ending with a new line.
Note that StixToNeoDB will try to "skip" errors in the objects and relations of the input file, e.g. references to non existent objects. The log of the processing can be found in application.log in the logs directory.
To view the data in Neo4j, launch the Neo4j Community Edition application, select your db_dir as the database location and click start. Once the status is "Started", open a browser on http://localhost:7474.
To compile from source and assemble the application and all its dependencies into a single fat jar file, use SBT and type:
sbt assembly
This will produce stixtoneodb-5.0.jar in the ./target/scala-2.12 directory.
Depends on ScalaStix, StixToNeoLib and the associated Neo4j Community jar file.
See also the build.sbt file.
Java 8 is required and Neo4j Community should be installed to process the results.
work in progress