Multiple Service using keploy (Api chaining)

Project Proposal
Mentors:

Gourav Kumar
Shivam Sourav Jha
By:

Ahmed Mohamed Ahmed Lotfy

 Table of Contents
1. Overview 3
2. Goals & Ideas 3
3. Personal Details 3
4. About Yourself 4
5. Commitment 6
6. Contributions so far 6
7. Qualification Tasks: 7
Proposal 7
Overview 7
Detailed 8
Project Plan - Preliminary Plan: 13
8. Major Milestones 14
9. Additional Information: 14
1. Overview

This project will implement Contract Testing for Keploy, which empowers
seamless interactions between services by emphasizing contract testing. This
approach ensures all communications adhere to predefined expectations,
boosting the reliability of interconnected systems.

Contract testing verifies interactions between microservices, applications, or

systems. It guarantees proper communication through pre-defined agreements
(contracts). This enables early detection of issues and reduces the complexity
of end-to-end testing.

2. Goals & Ideas

a. Keploy Support: Integrate contract testing capabilities into Keploy to enable
advanced testing functionalities, enhancing its ability to validate
microservice interactions effectively.
b. One-Go Testing: Implement functionality within Keploy for recording tests
and mocks for all related microservices simultaneously. This feature would
capture both ingress and egress traffic, utilizing process identification (PID)
for filtering. The primary goal is to ensure that tests and mocks remain
synchronized across all services, thereby maintaining consistency and
accuracy in testing environments.
c. Automated Change Management for API Chains: When using Keploy with
services m1, m2, and m3, consider a scenario where m2 undergoes a
change, such as the addition of a new field. When this change is
incorporated into m2's test cases, it could potentially disrupt the mock
interactions expected by m1, necessitating updates to m1's tests to align with
the new behavior. This process ensures that mocks are consistently
refreshed to reflect the most current service interactions. Keploy facilitates
this dynamic updating mechanism, streamlining the maintenance of test
cases and mocks across services. Consequently, it simplifies end-to-end
(E2E) testing, making it more efficient and reducing the complexity typically
associated with testing interconnected services.
3. Personal Details
● Name:​Ahmed Mohamed Ahmed Lotfy
● Course:​Computer Engineering at Faculty of engineering Cairo University
● Email:​​[email protected]
● Github:​Link
● LinkedIn: Link
● Phone:​01097865159
● Current Country: Egypt
● Link to Resume / CV:​CV

4. About Yourself
a. Please describe yourself, including your development background
and specific expertise.

● My name is Ahmed & I’m in my last year in my college.I’m a backend

engineer who worked in companies like Ejada and took internships in
companies like VMware, Master Micro, FGS.
● I took three trainings with ITI in topics “Red Hat system administration”,
“Flutter”,”MEAN Stack”.
● I worked with many programming languages like
Go,C++,C,Java,Python,Javascript,Typescript and many frameworks like
Angular, Expressjs, Flutter.
● Here are the relevant college courses:
❖ - Introduction to ❖ - Image Processing
Programming (C++) & Computer Vision
❖ - Programming (Py) - Pattern
Techniques (C++)
❖ - Advanced Recognition &
Programming Neural Networks
Techniques (Java) - (Py) * - Database
Data Structures and Design &
Algorithms (C++) Management (SQL,
❖ - Design & Analysis C#) - Cryptography
of Algorithms (C++) - & Cybersecurity *
Computer Graphics ❖ - Logic Design I & II
(C++, OpenGL) *
❖ - Operating Systems ❖ - Microprocessors I
❖ - Software & II
Engineering ❖ - Embedded
 Systems* Number Theory
❖ - Computer ❖ - Signal Processing
Architecture ❖ - Statics
❖ Some notes: ❖ - Dynamics
❖ - Calculus I, II & III. ❖ - Physics I, II & III.
❖ - Algebra ❖ - Mechanical
❖ - Linear Algebra Engineering
❖ - Analytic Geometry
(Robotics &
❖ - Numerical
methods Thermodynamics) -
❖ - Probability & Control Engineering*
Statistics - Discrete ❖ - VLSI Design
Mathematics -
b. Why are you interested in the Keploy project(s) you stated above?

● I’m interested in that Keploy project as Firstly, the integration of contract

testing capabilities into Keploy aligns perfectly with my professional
background and interests.
● I am enthusiastic about leveraging my expertise in software testing
methodologies to contribute to the project's goal of validating microservice
interactions effectively. By incorporating contract testing, we can not only
enhance the quality of interconnected systems but also streamline the
development process by identifying potential issues early on.
● The idea of implementing functionality within Keploy for recording tests
and mocks for all related microservices simultaneously excites me greatly.
I see immense potential in this feature to revolutionize the testing
process, ensuring consistency and accuracy across different testing
environments. As someone who values efficiency and precision in
software development, I am eager to contribute to the development of this
feature and witness its transformative impact on testing practices.

c. Have you participated in an open-source project before? If so, please

send us URLs to your profile pages for those projects or some other
demonstration of the work that you have done in open-source. If not,
why do you want to work on an open-source project in GSoC this
summer with Keploy?
● No, I haven't participated for OSS and I want to join Keploy this summer
as contributing to an open-source project like Keploy allows me to
collaborate with a diverse community of developers, mentors, and
contributors from around the world. This collaborative environment fosters
learning, sharing of ideas, and collective problem-solving, which are
invaluable experiences for personal and professional growth.
● Participation in Keploy's open-source project offers the chance to address
practical challenges in software development. By contributing to the
 project, one can directly impact the reliability and efficiency of software
systems, benefiting both developers and end-users. GSoC offers a
platform to connect with industry professionals, mentors, and fellow
students who share similar interests and goals. Through networking
opportunities provided by the program, participants can build meaningful
relationships, receive mentorship, and potentially explore future career
opportunities in the tech industry.

5. Commitment
a. Are you planning any vacations during the GSoC period?
■ I have no planned absence days for the duration of the GSoC project
period. If any Emergency occurs, then I will inform the Mentor.
B. How many classes are you taking during the GSoC period?
● I will have my final exams for weeks 1-2.
C. Do you have any other employment during the GSoC period?
● No.
D. How many hours per week do you expect to work on the project and
what hours do you tend to work?

● Time zone: UTC+02:00. I plan to work around 30-35 hours/week.

I am fully committed for the whole period except for the first 3
weeks when I will have my final exams. During which, I plan to
only work on the simpler tasks of the project as explained in the
project schedule.
● My preferred working hours are from 9 AM to 6 PM.

6. Contributions so far
a. PRs Merged:
■ #1590: [feature]: hide complete test summary in case of app not running
#1590
■ #1585: [fix]: bug while keploy test in user app #1585
b. PRs Unmerged:
■ #1584: Colorise keploy logo #1584
■ #1609: [fix]: update keploy even it has the latest version #1609
■ #1640: Addition of version argument to installation script #1640
■ #1660: fix compare diffs in case of array of json #1660
■ #1699: permissions for keploy folder #1699
 ■ #1702: Validate some paths #1702
c. Documentation contributions:
■ #337: (add): env field in debugger guide docs #337
d. Other contributions [GSoC related issues]:
■ #1541: [GSoC]: test multiple services with keploy #1541
■ #1543: [GSoC]: Build an app with multiple services, demonstrating
contract testing #1543

7. Qualification Tasks:
a. You can see my work on these issues:
■ [GSoC]: Build an app with multiple services, demonstrating contract
testing #1543
■ [GSoC]: test multiple services with keploy #1541
■ [GSoC]: Add support in Keploy to read from server initiated calls #1572

Proposal

Overview
● Provide a brief overview of your proposed solution and objectives you want to achieve
with your chosen project.
Overview: My proposed solution focuses on enhancing microservice testing capabilities within
Keploy, a platform designed for API chaining and contract testing. By integrating advanced
testing functionalities into Keploy, we aim to streamline the validation of microservice
interactions and improve the reliability of interconnected systems.

Objectives:

1. Integrate Contract Testing: Implement contract testing capabilities into Keploy to enable
comprehensive validation of microservice interactions. This will involve defining and
enforcing predefined contracts to ensure seamless communication between services.
2. One-Go Testing Functionality: Develop a feature within Keploy to enable recording tests
and mocks for all related microservices simultaneously. This functionality will capture
both ingress and egress traffic, maintaining consistency and accuracy across testing
environments.
3. Automated Change Management: Implement automated change management for API
chains to facilitate dynamic updates to tests and mocks. This will ensure that tests
remain synchronized with any changes in service behaviors, simplifying maintenance
and reducing the complexity of end-to-end testing.
 4. Documentation and Training: Provide comprehensive documentation and training
materials to support users in leveraging the enhanced testing capabilities of Keploy. This
will ensure that developers can effectively utilize the platform to improve the reliability of
their microservice architectures.

● Please include programming languages, tools and technologies you plan to use.
- I will mainly focus on Go as it’s the programming language for keploy’s core.

Detailed

1. Integrate Contract Testing:

Referred to this issue: [GSoC]: Build an app with multiple services, demonstrating
contract testing #1543

Let’s divide it into two phases:

1. Producing the contract (consumer-driven contract):

● Define Consumer Expectations: after writing automated tests that
define how the consumer service expects to interact with the provider API. These
tests typically involve sending requests (e.g., GET, POST) with specific data and
asserting the expected responses (status code, response body), Keploy will listen
to this network interchange like what Keploy performs in record mode (outgoing
and incoming channels) and create the contract instead of creating a mock and a
testcase.
○ Contract Design:

■ Interaction Design:
 ■ Request Design (may be changed later):

■ Response Design(may be changed later):

● Implementation Sample:
○ Java code to save the contract file as a json (may be
changed to yaml later).
2. Verification with Provider:
a. I will make a feature on Keploy that sets up mock
consumer interactions that mimic the contract (similar to
the replay feature on Keploy). This allows them to verify
their implementation adheres to the agreed-upon behavior
before deployment.
 b. Matching will be done by the reflect package as done on
Keploy.

2. One-Go Testing:
1. First Solution:
a. Base Case: we have two services which doesn’t
depend on each others and we want to record
them:

i.
 ii. Each service or let’s call it App has its own PID
to capture data (network traffic, function calls)
related to it, this done on Hook.
iii. Each App has its recorder with PID (sequence
number) which is passed to hook instead of
this.

iv. Incoming Channel (incomingChan):

● Content: It now receives only those test
cases generated by the specific
microservice under test, as filtered by
PID. This eliminates irrelevant test cases
from other microservices, ensuring a
focused test dataset.
v. Outgoing Channel (outgoingChan):
● Content: It now receives only those
mocks intended for the specific
microservice under test, as filtered by
PID. This prevents mocks from being
applied to unintended services, ensuring
accurate and isolated testing.
vi. Functions InsertTestCase and InsertMock
are required to be synchronized as they
will be called many times concurrently, so
we will use Mutexes to ensure only one
goroutine can access shared data or
resources at a time.
vii. Also the shared data structures among apps
like MockCountMap must be synchronized
(using sync.Map or using sync/atomic for
atomic operations).

b.If there is two services which one of

them is dependent on the other:
i. Race condition on writing mocks and test
cases will be solved with pid filtering &
synchronization as lets say that s1
requested s2 so mock for s2 will be
generated and test case also and on the
same time the incoming channel of s2 would
capture test case for s2 and generate it
which could make a race condition but the
Mutexes on both functions will solve this.
c. If Number of services >2:
 i. No problem would occur if we solved for n=2

d. In case of testing multiple apps:

i. Each app will interact with its test cases
and mocks only. We will edit the Start and
BootReplay functions to run multiple apps
like what we did in record.
e. Sample of code for recording (not necessary to
be the actual implementation).
2. Automated Change Management for API Chains:
1. First solution:
● We can do this by the concept of contracts, let’s take an
example:
Imagine you have an e-commerce application where two
services interact:
○ Product Service (m1): This service retrieves
product information from a database and sends it to
the client application.
○ Inventory Service (m2): This service manages
product stock levels and communicates them to the
Product Service.
○ If Product Service requests Inventory Service to
get products test case and mock are generated but
the problem is when there is a major change
happen in Inventory Service like adding a field this
change won’t impact the interaction with Product
Service as there is an existing mock for Product
Service (which no longer reflects the new
response format from the Inventory Service) and
it thought that everything is ok but when doing e2e
testing between the two services it will fail.
○ The solution is when there is any change in the
provider service like Inventory Service in our case
it will unverify all the contracts with the consumers
so that they will be notified for the change in the
mocks and change their test cases accordingly or
we can implement a feature to automatic change
the test-cases to match the expected mock by
getting the different attributes by matching
techniques like reflect and update the test-case.
○ Sample of what pact does when there is any
change in mock and there is a service dependent
on it:
■ This is the contract generated between
product (consumer) and retailer( the provider)

■ So any change any field name in provider test

will cause the contract to be unverifed
 ■ And consumer is notified by it’s outdated
testcase

■ The contract must be somewhere shareable

between consumer and provider services.

2. Second Solution:
a. While recording if there any incoming requests
we will search for all mocks contains this
service as a provider ( we could add a
dependent_service parameter in the mock) and
match the response with the expected response on
the mock:
i. If Not matched, update it.
ii. If matched , update the timestamp of the
updated mock.
iii. Example: initial recording between s1 and
s2 causing generation of s2 mock for s1.Any
subsequent recording will cause to check
 the validate all dependent mocks

Project Plan - Preliminary Plan:

(Community Bonding Period)

- Understand Existing Codebase.
- Discuss with the mentor the best way to go about the implementation.

Week Number Start Date End Date Tasks to be completed

Week 1-3 May 27 June 16 Understanding more about the project

and the best practices done in go.
Note: I will have my final exams for weeks 1-3.
That is why I picked simple tasks for them. (~ 8-10
hours/week).
Week 4-6 June 17 July 7 Integrate Contract Testing into Keploy
First Evaluation
Week 7 July 8 July 14 Testing, Bug fixes, and Documentation
Week 8-11 July 15 August 11 Implement “One-Go Testing” feature
Week 11-13 August August 25 Implement “Automated Change Management for
12 API Chains”
& Testing, Bug fixes, and Documentation
Final Evaluation for Medium Projects

8. Major Milestones
a. Week 6 : Finishing the contract testing.
b. Week 11: Implementation of the "One-Go Testing" feature.
c. Week 13: Implementation of the "Automated Change Management for API
Chains" feature.
9. Additional Information:
a. I've already implemented contract testing using spring boot so it would be good
to implement it into Keploy.
b. I worked for companies like Ejada,VMware so I know how to work in a team.
c. I took 3 training with ITI in these topics: MEAN stack,Red hat system
administration,Flutter
d. I had many experiences in backend, frontend and testing development.
e. Here is some of my projects:
■ RedditX Backend: Reddit clone
■ Alpha-Pet: full-stack web-application using Angularjs,Node.js,Express.js
and MySQL which is a mimic of Vezeeta.
■ OS Scheduler: Scheduling the processes by the operating system is one
of the most important jobs of the OS. We will apply some scheduling
algorithm in this project, like SRTN algorithm, RoundRobin algorithm,
HPF algorithm.
■ Book Store: full-stack web-application using Angularjs, Node.js,
Express.js, MongoDB and used ngBootstrap, NodeMailer, Json Web
Tokens, Font Awesome, Paypal Checkout, Animate.css
■ Logic Simulator : Simulator like Logisim.

f. Keploy is the only project I focused on and interested in.

Thank You