Beware! This is proposal of pretty huge feature that need to be discussed first.

One of the main ideas behind mwdb.cert.pl service is to share our malware analysis capabilities with other researchers. It would be nice addition if people setting up their own Malwarecage instances could easily synchronize it with our service e.g. by submitting sample to both repositories and enhancing their own analytics with data from mwdb.cert.pl.

Below is a short proposal of feature implementation that could make it easier to connect and synchronize two instances.

Server implementation

Remote setup & authentication

Malwarecage instance administrator adds API key and URL of remote instance to the local configuration e.g.:

[malwarecage]
...
remotes = mwdb.cert.pl

[remote:mwdb.cert.pl]
url = https://mwdb.cert.pl/api
api_key = ey....

Local instance behaves as a proxy that uses provided key for remote authentication, so local users with appropriate rights can operate on the remote instance with the identity and rights of the configured API key.

List of available remotes

List of remotes can be exposed via endpoint:

GET /api/remote/

Object synchronization

Then, let's add two endpoints:

• Pull object from remote to local instance /api/remote/<remote_name>/<type>/<id>/pull
• Push object to remote from local instance /api/remote/<remote_name>/<type>/<id>/push

Additional options:

• scope - which elements we want to pull/push along with object itself
  • tags
  • attributes
  • comments
  • relationships (with already existing objects in the target instance)
• share_with indicating with whom the target object should be shared (similar to upload)

That will allow to directly transfer complete object state between instances. Transfering whole tree of objects could be time-consuming and requires introduction of asynchronic requests (processed by external worker), so it's preferred to limit this feature to the synchronization of single object.

Implementation Note: Object push/pull must occur within single transaction

Remote API access

/api/remote/<remote_name>/api/<path:path>

We can implement a bit more universal approach and set-up a real proxy for remote API, allowing to do various actions in both repositories using the same client.

mwdblib example:

mwdb = Malwarecage(api_url="http://mwdb.tailcall.net/api/")
# Get local file by hash
file = mwdb.query_file("1234...")
# Connect to the remote API
certpl = mwdb.remote("mwdb.cert.pl")
# Push file to the remote, getting remote-bound MalwarecageFile instance
remote_file = certpl.push(file)
# Add tag to the file on the remote side (PUT http://mwdb.tailcall.net/api/remote/mwdb.cert.pl/api/file/1234.../tag)
remote_file.add_tag("feed:tailcall")

API key-level ACL

Unfortunately, API keys are currently as powerful as the user account so it can be used for:

• getting profile details
• triggering request for a new password
• managing API keys (e.g. removing itself )

This limitation is pretty hard to overcome if we want to expose the "remote" for multiple users in our local instance. That's why we need to limit the API key rights.

Proposed implementation is to:

• extend the set of 'capabilities' with:
  • manage_profile that gives rights to local account management which includes:
    • auth.* endpoints
    • api_key.* endpoints
  • manage_quick_query that limits adding/removing Quick Queries which is a part of personalization:
• turn on these capabilities by default in the public group
• allow to specify the deny list of capabilities that are turned off for API key. Capabilities mentioned above will be banned by default.

The final API key rights are evaluated as follows:

caps = sum(group.caps for group in user.groups) - api_key.denied_caps

If local administrator doesn't want to let the local users e.g to remove some comments on the remote sides, local admin can ban adding_comments for API key used for remote authentication.

Client implementation

API proxying enables us to directly reuse the client functions for interaction with remote instance.

We can expose the list of remotes somewhere and allow users to easily switch between the local and remote instance.

That type of switching would be a bit challenging because we need to consider:

• nested routing to prepend path with information about the "remote"
• adapt navbar and links to stay within the "remote mode":
  • allow to navigate to the Samples/Configs/Blobs/Upload/Search/Statistics within "remote mode"
  • disable Admin, About, Profile and Logout
  • allow to quit the "remote view" with single button
• consider difference between local and remote capabilities - possibly some parts of view must be hidden in "remote mode" due to API key caps.

List of remote-enabled views:

• Recent Samples/Configs/Blob/Search
• Sample/Config/Blob view
• Upload view (?)
• Statistics view (?)

Object detailed view should implement additional buttons for pulling/pushing current object. After successful push/pull, we should be redirected to the target side.