Make stream processing easier

A magical framework that make stream processing easier!

The original intention of StreamX is to make stream processing easier. StreamX focuses on the management of development phases and tasks. Our ultimate goal is to build a one-stop big data solution integrating stream processing, batch processing, data warehouse and data laker.

• Scaffolding
• Out-of-the-box connectors
• Support maven compilation
• Configuration
• Multi version flink support(1.12.x,1.13.x,1.14.x, 1.15.x)
• Scala 2.11 / 2.12 support
• restapi support.
• All Flink deployment mode support(Remote/K8s-Native-Application/K8s-Native-Session/YARN-Application/YARN-Per-Job/YARN-Session)
• start, stop, savepoint, resume from savepoint
• Various companies and organizations use StreamX for production and commercial products.
• Flame graph
• Notebook
• Project configuration and dependency version management
• Task backup and rollback
• Manage dependencies
• UDF
• Flink SQL Connector
• Flink SQL WebIDE
• Catalog、Hive
• Full support from task development to deployment
• ...

Streamx consists of three parts,streamx-core,streamx-pump and streamx-console

streamx-core is a framework that focuses on coding, standardizes configuration, and develops in a way that is better than configuration by convention. Also it provides a development-time RunTime Content and a series of Connector out of the box. At the same time, it extends DataStream some methods, and integrates DataStream and Flink sql api to simplify tedious operations, focus on the business itself, and improve development efficiency and development experience.

streamx-pump is a planned data extraction component, similar to flinkx. Based on the various connector provided in streamx-core, the purpose is to create a convenient, fast, out-of-the-box real-time data extraction and migration component for big data, and it will be integrated into the streamx-console.

streamx-console is a stream processing and Low Code platform, capable of managing Flink tasks, integrating project compilation, deploy, configuration, startup, savepoint, flame graph, Flink SQL, monitoring and many other features. Simplify the daily operation and maintenance of the Flink task.

Our ultimate goal is to build a one-stop big data solution integrating stream processing, batch processing, data warehouse and data laker.

• Apache Flink
• Apache YARN
• Spring Boot
• Mybatis
• Mybatis-Plus
• Flame Graph
• JVM-Profiler
• Vue
• VuePress
• Ant Design of Vue
• ANTD PRO VUE
• xterm.js
• Monaco Editor
• ...

Thanks to the above excellent open source projects and many outstanding open source projects that are not mentioned, for giving the greatest respect,Thanks to Apache Flink for creating a great project! Thanks to the Apache Zeppelin project for the early inspiration.

click Document for more information

Various companies and organizations use StreamX for research, production and commercial products. Are you using this project ? you can add your company

We have received some precious honors, which belong to everyone who contributes to StreamX, Thank you !

You can submit any ideas as pull requests or as GitHub issues.

If you're new to posting issues, we ask that you read How To Ask Questions The Smart Way (This guide does not provide actual support services for this project!), How to Report Bugs Effectively prior to posting. Well written bug reports help us help you!

Thank you to all the people who already contributed to StreamX!

Are you enjoying this project ? 👋

If you like this framework, and appreciate the work done for it to exist, you can still support the developers by donating ☀️ 👊

WeChat Pay	Alipay

Welcome individuals and enterprises to sponsor, your support will help us better develop the project

Thanks to JetBrains for supporting us free open source licenses.

Thank you to all our backers!

StreamX enters the high-speed development stage, we need your contribution.