Thanks to visit codestin.com
Credit goes to github.com

Skip to content

BarunW/Flink-HTTPSourceConnector

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This Flink-HTTPSourceConnector is flink source wrapped around java http client 
for rest API endpoints
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

JAVA  :   VERSION 11
FLINK :  1.20.1

HOW TO USE:
Put the httpSource folder/dir to your project 
```
    import httpSource.HTTPSource;
    
    StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment(config);

    env.setParallelism(1);
    env.enableCheckpointing(1000 * 10) // 10 sec;

    HTTPSource<YourPojo> source = HTTPSource.<YourPojo>builder()
                .addEndpoint("https://whatever_your_endpoint/")
                .setDeserializer(new YourSchemaDeserializer())
                .addHeader("", "") // Some headers cannot be set in java httpClient check out for that
                .setHTTPMethod("POST") // GET, PUT, PATCH
                .setHTTPRequestBodyPublisher("") // Takes String as input for HttpRequest.BodyPublishers.ofString()
                .setBoundedness(Boundedness.CONTINUOUS_UNBOUNDED)
                .setPollingInterval(5000L * 6) //  30 second
                .build();

   DataStream<VehicleUpdateDataSchema> stream = env.fromSource(
                source,
                WatermarkStrategy.noWatermarks(),
                "source"
    ); 

```



About

This flink source is for rest endpoints

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages