Diffgram - Supervised Learning Data Platform

Diffgram is annotation and training data software.

What is Diffgram Exactly?

The domain is "Training Data". It's one integrated system that handles everything in the new Training Data abstractions world. It integrates with adjacent tools.

Give me a Compass - What's the Stack look like?

Let's think of it like a loose analogy to LAMP, or MEAN stacks. One example - use a pre-processing tool like Lightly, then do annotation in Diffgram, and model training with Determined AI. This is like a "LDD" stack: lightly diffgram determined-ai .

You can use Diffgram with your choice of surrounding tools - the ones shown are examples and optional.

Who are the end users?

• The largest group of end users is data annotators and subject matter experts.
• Data Scientists or similar sets up the Schema (labels and attributes), the dataset structure, etc.
• An admin or project manager sets up the human annotation pipelines.
• A software engineer sets up the overall system, data permissions, and maintaining a large deployment.

Note, to just get started a single user may play all roles.

What is this a drop in replacement for?

Diffgram is a drop in replacement for most of the functions of the following systems: Labelbox, CVAT, SuperAnnotate, Label Studio (Heartex), V7 Labs (Darwin), BasicAI, SuperbAI, Kili-Technology, HastyAI, Dataloop, Keymakr.

If you see any missing features, bugs etc please report them ASAP to diffgram/issues. See Contribution Guide for more.

More on Understanding Diffgram High Level

Quickstart

Try Diffgram Online (Hosted Service, No Setup.)

Diffgram Dev Installer Quickstart

Requires Docker and Docker Compose

git clone https://github.com/diffgram/diffgram.git
cd diffgram
python install.py
# Follow the installer instruction and 
# After install:  View the Web UI at: http://localhost:8085

Other Useful Links For Starting Out

• Install Guide Compute Engine
• Updating Existing Installation
• Development Install Docs
• Production Install Docs
• Helm Chart for Kubernetes Clusters

Benefits

1. Flexible deploy and many integrations - run Diffgram anywhere in the way you want.
2. Scale every aspect - from volume of data, to number of supervisors, to ML speed up approaches.
3. Fully featured - 'batteries included'.

Docs

Docs

• Getting Started Plan
• Videos
• Cookbook (Advanced)

Support & Community

1. Open an issue (Technical, bugs, etc)
2. Chat on Discord
3. Forum (Coming Soon)

Security issues: Do not create a public issue. Email [email protected] with the details. Docs

Vision

1. Application: Support all popular media types for raw data; all popular schema, label, and attribute needs; and all annotation assist speed up approaches
2. Support all popular training data management and organizational needs
3. Integrate with all popular 3rd party applications and related offerings
4. Support modification of source code
5. Run on any hardware, any cloud, and anywhere

Technical Direction

Features

Overview Image and Video Annotation.

• Segmentation
• Video Annotation
• Versioning
• Streaming
• Security and Privacy
• Speed Up with AI Userscripts
• Open Core (This Repo!)
• Integrations

Speed Ups & AI

Latest AI + More

• Examples
• Userscripts Overview

Integrations

• Diffgram Python SDK
• Diffgram API Any language
• AWS - Amazon Storage
• GCP Google Storage
• Azure - (Select during install - not available as separate connection yet)
• Scale AI
• Datasaur
• Labelbox
• Submit a pull request! We want your integration here too

Note for initial open core release Actions Hooks are not yet available. Please see Diffgram.com and use them there if needed.

Contributing

We welcome contributions! Please see our contributing documentation.

Architecture & Design Docs

We plan to release more internal architecture docs over time. Please see the general docs in the mean time.