Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Add RemoteProcessor Service#476

Merged
mylibrar merged 14 commits intoasyml:masterfrom
mylibrar:suqi-forteservice
Jun 14, 2021
Merged

Add RemoteProcessor Service#476
mylibrar merged 14 commits intoasyml:masterfrom
mylibrar:suqi-forteservice

Conversation

@mylibrar
Copy link
Collaborator

This PR fixes #475.

Description of changes

  • Add Forte service in pipeline.py:
    • A Forte pipeline can be serialized into an intermediate representation that is dumped to a YAML file by calling Pipeline().save().
    • Each pipeline has a FastAPI app that can be used to open an end point for remote service by calling Pipeline().serve().
    • Forte can directly load an intermediate representation from file path to initialize a pipeline and create a remote service by calling forte.serve().
  • Add a RemoteProcessor:
    • This will use the service created by Forte pipeline on a specified end point.
    • It wraps up the input DataPack to a POST request and update the input according to the response from service.
    • The service pipeline must have a RawDataDeserializeReader.

Possible influences of this PR.

For pylint checking, Pipeline class has reached the maximum number of public methods (20/20)

Test Conducted

A unit test remote_processor_test.py is provided to verify the intermediate representation and RemoteProcessor.

@codecov
Copy link

codecov bot commented Jun 14, 2021

Codecov Report

Merging #476 (cf64a40) into master (0075013) will increase coverage by 1.57%.
The diff coverage is 56.35%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #476      +/-   ##
==========================================
+ Coverage   79.34%   80.91%   +1.57%     
==========================================
  Files         150      206      +56     
  Lines        9736    14237    +4501     
==========================================
+ Hits         7725    11520    +3795     
- Misses       2011     2717     +706     
Impacted Files Coverage Δ
forte/data/base_pack.py 79.14% <ø> (-0.96%) ⬇️
forte/data/base_reader.py 87.59% <ø> (ø)
forte/data/batchers.py 85.02% <ø> (+7.70%) ⬆️
forte/data/caster.py 94.73% <ø> (ø)
forte/data/common_entry_utils.py 100.00% <ø> (ø)
forte/data/container.py 77.27% <ø> (-7.22%) ⬇️
forte/data/converter/__init__.py 100.00% <ø> (ø)
forte/data/converter/converter.py 95.69% <ø> (ø)
forte/data/converter/feature.py 87.00% <ø> (ø)
forte/data/data_pack.py 79.79% <ø> (-2.03%) ⬇️
... and 310 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 17fe0ba...cf64a40. Read the comment docs.

Copy link
Member

@hunterhector hunterhector left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, not many comments. Probably we can move the __dict__ operation inside DataPack now, but still, keep the current implementation. We can open another PR to implement the update interface carefully with test cases.

@mylibrar mylibrar merged commit 6257156 into asyml:master Jun 14, 2021
@mylibrar mylibrar deleted the suqi-forteservice branch June 14, 2021 20:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Forte as a remote service

2 participants