Thanks to visit codestin.com
Credit goes to www.scribd.com

0% found this document useful (0 votes)
39 views186 pages

Numerical Methods For Solving Nonlinear Equations

This document is a reprint of a special issue focused on numerical methods for solving nonlinear equations, edited by Maria Isabel Berenguer and Manuel Ruiz Galán. It includes articles that discuss various iterative methods and their convergence properties for approximating polynomial zeros and other nonlinear problems. The content is aimed at advancing the understanding and application of numerical techniques in mathematics, with contributions from multiple authors in the field.

Uploaded by

perrot louis
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
39 views186 pages

Numerical Methods For Solving Nonlinear Equations

This document is a reprint of a special issue focused on numerical methods for solving nonlinear equations, edited by Maria Isabel Berenguer and Manuel Ruiz Galán. It includes articles that discuss various iterative methods and their convergence properties for approximating polynomial zeros and other nonlinear problems. The content is aimed at advancing the understanding and application of numerical techniques in mathematics, with contributions from multiple authors in the field.

Uploaded by

perrot louis
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 186

Special Issue Reprint

Numerical Methods for


Solving Nonlinear Equations

Edited by
Maria Isabel Berenguer and Manuel Ruiz Galán

mdpi.com/journal/mathematics
Numerical Methods for Solving
Nonlinear Equations
Numerical Methods for Solving
Nonlinear Equations

Editors
Maria Isabel Berenguer
Manuel Ruiz Galán

Basel • Beijing • Wuhan • Barcelona • Belgrade • Novi Sad • Cluj • Manchester


Editors
Maria Isabel Berenguer Manuel Ruiz Galán
Department of Applied Department of Applied
Mathematics, University of Mathematics, University of
Granada Granada
Granada, Spain Granada, Spain

Editorial Office
MDPI
St. Alban-Anlage 66
4052 Basel, Switzerland

This is a reprint of articles from the Special Issue published online in the open access journal
Mathematics (ISSN 2227-7390) (available at: https://www.mdpi.com/si/mathematics/Numerical
Methods for Solving Nonlinear Equations).

For citation purposes, cite each article independently as indicated on the article page online and as
indicated below:

Lastname, A.A.; Lastname, B.B. Article Title. Journal Name Year, Volume Number, Page Range.

ISBN 978-3-0365-9214-5 (Hbk)


ISBN 978-3-0365-9215-2 (PDF)
doi.org/10.3390/books978-3-0365-9215-2

© 2023 by the authors. Articles in this book are Open Access and distributed under the Creative
Commons Attribution (CC BY) license. The book as a whole is distributed by MDPI under the terms
and conditions of the Creative Commons Attribution-NonCommercial-NoDerivs (CC BY-NC-ND)
license.
Contents

About the Editors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

Petko D. Proinov and Milena D. Petkova


On the Convergence of a New Family of Multi-Point Ehrlich-Type Iterative Methods for
Polynomial Zeros
Reprinted from: Mathematics 2021, 9, 1640, doi:10.3390/math9141640 . . . . . . . . . . . . . . . . 1

Li Zhang, Jin Huang, Hu Li and Yifei Wang


Extrapolation Method for Non-Linear Weakly Singular Volterra Integral Equation with Time
Delay
Reprinted from: Mathematics 2021, 9, 1856, doi:10.3390/math9161856 . . . . . . . . . . . . . . . . 17

Marı́a Isabel Berenguer and Manuel Ruiz Galán


An Iterative Algorithm for Approximating the Fixed Point of a Contractive Affine Operator
Reprinted from: Mathematics 2022, 10, 1012, doi:10.3390/math10071012 . . . . . . . . . . . . . . . 37

Juan Zhang and Xiao Luo


Gradient-Based Optimization Algorithm for Solving Sylvester Matrix Equation
Reprinted from: Mathematics 2022, 10, 1040, doi:10.3390/math10071040 . . . . . . . . . . . . . . . 47

Michael I. Argyros, Ioannis K. Argyros, Samundra Regmi and Santhosh George


Generalized Three-Step Numerical Methods for Solving Equations in Banach Spaces
Reprinted from: Mathematics 2022, 10, 2621, doi:10.3390/math10152621 . . . . . . . . . . . . . . . 61

Ioannis K. Argyros, Samundra Regmi, Stepan Shakhno and Halyna Yarmola


A Methodology for Obtaining the Different Convergence Orders of Numerical Method under
Weaker Conditions
Reprinted from: Mathematics 2022, 10, 2931, doi:10.3390/math10162931 . . . . . . . . . . . . . . . 89

R. H. Al-Obaidi and M. T. Darvishi


Constructing a Class of Frozen Jacobian Multi-Step Iterative Solvers for Systems of Nonlinear
Equations
Reprinted from: Mathematics 2022, 10, 2952, doi:10.3390/math10162952 . . . . . . . . . . . . . . . 105

Khaled Ben Amara, Maria Isabel Berenguer and Aref Jeribi


Approximation of the Fixed Point of the Product of Two Operators in Banach Algebras with
Applications to Some Functional Equations
Reprinted from: Mathematics 2022, 10, 4179, doi:10.3390/math10224179 . . . . . . . . . . . . . . . 119

Gholamreza Farahmand, Taher Lotfi, Malik Zaka Ullah and Stanford Shateyi
Finding an Efficient Computational Solution for the Bates Partial Integro-Differential Equation
Utilizing the RBF-FD Scheme
Reprinted from: Mathematics 2023, 11, 1123, doi:10.3390/math11051123 . . . . . . . . . . . . . . . 137

Obadah Said Solaiman, Rami Sihwail, Hisham Shehadeh, Ishak Hashim and Kamal Alieyan
Hybrid Newton–Sperm Swarm Optimization Algorithm for Nonlinear Systems
Reprinted from: Mathematics 2023, 11, 1473, doi:10.3390/math11061473 . . . . . . . . . . . . . . . 151

v
About the Editors
Maria Isabel Berenguer
Maria Isabel Berenguer is an Associate Professor in the Department of Applied Mathematics at
the University of Granada, Spain; since 2015, she has also been a member of the university’s Institute
of Mathematics. She is a notable reviewer, and has published a several articles in well-known
journals with high JCR ranking. She has also actively participated in different projects with national
or regional funding. Additionally, she has trained young researchers and tutored international
doctoral students during their term abroad at the University of Granada. Currently, she is a
member of the Editorial Board of MDPI’s journal Mathematics. Her research interests include applied
mathematics, numerical analysis, fixed point theory and inverse problems.

Manuel Ruiz Galán


Manuel Ruiz Galán received his Ph.D. in 1999 from the University of Granada, Spain. Currently,
he serves as a Full Professor in the Mathematics Department at the University of Granada, Spain. He
has authored more than 60 research papers and book chapters, particularly on topics related to convex
and numerical analysis and their applications. He has been a member and principal investigator in
several nationally funded projects (Spanish Government). He has served as a Guest Editor for several
Special Issues of journals such as Optimization and Engineering, Frontiers in Psychology, Mathematical
Problems in Engineering, and the Journal of Function Spaces and Applications. Additionally, he is a
member of the Editorial Board of MDPI’s journal Mathematics and the journal Minimax Inequalities
and its Applications.

vii
Preface
Many problems that emerge in areas such as medicine, biology, economics, finance, or
engineering can be described in terms of nonlinear equations or systems of such equations, which can
take different forms, from algebraic, differential, integral or integro-differential models to variational
inequalities or equilibrium problems. For this reason, nonlinear problems are one of the most
interesting fields of study in pure and applied mathematics.
However, there is a lack of direct methods that can facilitate the effective resolution of nonlinear
problems, and hence, research interest in their numerical treatment has further consolidated. This
Special Issue have collated manuscripts that address the recent advancements in the aforementioned
area. It contains 10 articles accepted for publication among the 24 submitted.

Maria Isabel Berenguer and Manuel Ruiz Galán


Editors

ix
mathematics

Article
On the Convergence of a New Family of Multi-Point
Ehrlich-Type Iterative Methods for Polynomial Zeros
Petko D. Proinov * and Milena D. Petkova

Faculty of Mathematics and Informatics, University of Plovdiv Paisii Hilendarski, 24 Tzar Asen,
4000 Plovdiv, Bulgaria; [email protected]
* Correspondence: [email protected]

Abstract: In this paper, we construct and study a new family of multi-point Ehrlich-type iterative
methods for approximating all the zeros of a uni-variate polynomial simultaneously. The first member
of this family is the two-point Ehrlich-type iterative method introduced and studied by Trićković
and Petković in 1999. The main purpose of the paper is to provide local and semilocal convergence
analysis of the multi-point Ehrlich-type methods. Our local convergence theorem is obtained by an
approach that was introduced by the authors in 2020. Two numerical examples are presented to show
the applicability of our semilocal convergence theorem.

Keywords: multi-point iterative methods; iteration functions; polynomial zeros; local convergence;
error estimates; semilocal convergence

MSC: 65H04

Citation: Proinov, P.D.; Petkova,


M.D. On the Convergence of a New
Family of Multi-Point Ehrlich-Type 1. Introduction
Iterative Methods for Polynomial This work deals with multi-point iterative methods for approximating all the zeros of
Zeros. Mathematics 2021, 9, 1640. a polynomial simultaneously. Let us recall that an iterative method for solving a nonlinear
https://doi.org/10.3390/ equation is called a multi-point method if it can be defined by an iteration of the form
math9141640

x ( k +1) = ϕ ( x ( k ) , x ( k −1) , . . . , x ( k − N ) ), k = 0, 1, 2, . . . ,
Academic Editors: Maria Isabel
Berenguer and Manuel Ruiz Galán where N is a fixed natural number, and x (0) , x (−1) , . . . , x (− N ) are N + 1 initial approxima-
tions. In the literature, there are multi-point iterative methods for finding a single zero of
Received: 17 June 2021 a nonlinear equation (see, e.g., [1–7]). This study is devoted to the multi-point iterative
Accepted: 8 July 2021 methods for approximating all the zeros of a polynomial simultaneously (see, e.g., [8–11]).
Published: 12 July 2021
Let us recall the two most popular iterative methods for simultaneous computation
of all the zeros of a polynomial f of degree n ≥ 2. These are Weierstrass’ method [12] and
Publisher’s Note: MDPI stays neutral
Ehrlich’s method [13].
with regard to jurisdictional claims in
Weierstrass’ method is defined by the following iteration:
published maps and institutional affil-
iations.
x ( k +1) = x ( k ) − W f ( x ( k ) ), k = 0, 1, 2, . . . , (1)

where the function W f : D ⊂ Kn → Kn is defined by W f ( x ) = (W1 ( x ), . . . , Wn ( x )) with

Copyright: © 2021 by the authors. f ( xi )


Wi ( x ) = (i = 1, . . . , n), (2)
Licensee MDPI, Basel, Switzerland. a0 ∏ ( x i − x j )
This article is an open access article j  =i
distributed under the terms and
conditions of the Creative Commons where a0 ∈ K is the leading coefficient of f and D denotes the set of all vectors in Kn with
Attribution (CC BY) license (https:// pairwise distinct components. Weierstrass’ method (1) has second order of convergence
creativecommons.org/licenses/by/ (provided that f has only simple zeros).
4.0/).

Mathematics 2021, 9, 1640. https://doi.org/10.3390/math9141640 https://www.mdpi.com/journal/mathematics


1
Mathematics 2021, 9, 1640

Ehrlich’s method is defined by the following fixed point iteration:

x ( k +1) = T ( x ( k ) ), k = 0, 1, 2, . . . , (3)

where the iteration function T : Kn → Kn is defined by T ( x ) = ( T1 ( x ), . . . , Tn ( x )) with

f ( xi )
Ti ( x ) = xi − (i = 1, . . . , n). (4)
1
f  (x i) − f ( xi ) ∑
j  =i
xi − x j

Ehrlich’s method has third order convergence. In 1973, this method was rediscovered
by Aberth [14]. In 1970, Börsch-Supan [15] constructed another third-order method for
simultaneous computing all the zeros of a polynomial. However in 1982, Werner [16]
proved that both Ehrlich’s and Börsch-Supan’s methods are identical.
In 1999, Trićković and Petković [9] constructed and studied a two-point version of
Ehrlich’s method. They√ proved that the two-point Ehrlich-type method has the order of
convergence r = 1 + 2.
In the present paper, we introduce an infinite sequence of multi-point Ehrlich-type
iterative methods. We note that the first member of this family of iterative methods is the
two-point Ehrlich-type method constructed in [9]. The main purpose of this paper is to pro-
vide a local and semilocal convergence analysis of the multi-point Ehrlich-type methods.
Our local convergence result (Theorem 2) contains the following information: con-
vergence domain; a priori and a posteriori error estimates; convergence order of every
method of the family. For instance, we prove that for a given natural number N, the order
of convergence of the Nth multi-point Ehrlich-type method is r = r ( N ), where r is the
unique positive solution of the equation

1 + 2( t + . . . + t N ) = t N +1 . (5)

It follows from this√result that the first iterative method ( N = 1) has the order of
convergence r (1) = 1 + 2 which coincides with the above mentioned result of Trićković
and Petković. We √note that each method of the new family has super-quadratic convergence
of order r ∈ [1 + 2, 3). The semilocal convergence result (Theorem 4) states a computer-
verifiable initial condition that guarantees fast convergence of the corresponding method
of the family.
The paper is structured as follows: In Section 2, we introduce the new family of
multi-point iterative methods. Section 3 contains some auxiliary results that underlie the
proofs of the main results. In Section 3, we present a local convergence result (Theorem 2)
for the iterative methods of the new family. This result contains initial conditions as well as
a priori and a posteriori error estimates. In Section 5, we provide a semilocal convergence
result (Theorem 4) with computer verifiable initial conditions. Section 6 provides two
numerical examples to show the applicability of our semilocal convergence theorem and
the convergence behavior of the proposed multi-point iterative methods. The paper ends
with a conclusion section.

2. A New Family of Multi-Point Ehrlich-Type Iterative Methods


Throughout the paper (K, | · |) stands for a valued field with a nontrivial absolute
value | · | and K[z] denotes the ring of uni-variate polynomials over K. The vector space
Kn is equipped with the product topology.
For a given vector u ∈ Kn , ui always denotes the ith component of u. For example, if F
is a map with values in Kn , then Fi ( x ) denotes the ith component of the vector F ( x ) ∈ Kn .
Let us define a binary relation # on Kn as follows [17]

u#v ⇔ ui = v j for all i, j ∈ In with i = j.

2
Mathematics 2021, 9, 1640

Here and throughout the paper, In is defined by

In = {1, 2, . . . , n}.

Suppose f ∈ K[z] is a polynomial of degree n ≥ 2. A vector ξ ∈ Kn is called a root


vector of the polynomial f if
n
f ( z ) = a0 ∏ ( z − ξ i ) for all z ∈ K,
i =1

where a0 ∈ K. It is obvious that f possesses a root vector in Kn if and only if it splits over K.
In the following definition, we introduce a real-value function of two vector variables
that plays an essential role in the present study.

Definition 1. Suppose f ∈ K[z] is a polynomial of degree n ≥ 2. We define an iteration function


Φ : DΦ ⊂ Kn × Kn → Kn of two vector variables as follows:

f ( xi )
Φi ( x, y) = xi − (i = 1, . . . , n), (6)
1
f  (x i) − f ( xi ) ∑ xi − y j
j = i

where DΦ is defined by
 
1
DΦ = ( x, y) ∈ Kn × Kn : x # y, f  ( xi ) − f ( xi ) ∑ = 0 for i ∈ In . (7)
j = i
xi − y j

Now the two-point Ehrlich-type root-finding method introduced by Trićković and


Petković [9] can be defined by the following iteration

x ( k +1) = Φ ( x ( k ) , x ( k −1) ) , k = 0, 1, . . . (8)

with initial approximations x (0) , x (−1) ∈ Kn .

Theorem 1 (Petković√and Trićkovic [9]). The convergence order of the two-point Ehrlich-type
method (8) is r = 1 + 2 ≈ 2.414.

Based on the function Φ, we define a sequence (Φ( N ) )∞ N =1 of vector-valued functions


such that the Nth function Φ( N ) is a function of N + 1 vector variables.

Definition 2. We define a sequence (Φ( N ) )∞


N = 0 of iteration functions

Φ ( N ) : D N ⊂ Kn × . . . × Kn → Kn
  
N +1

recursively by setting Φ(0) ( x ) = x and

Φ( N ) ( x, y, . . . , z) = Φ( x, Φ( N −1) (y, . . . , z)). (9)

3
Mathematics 2021, 9, 1640

The sequence ( D N )∞
N = 0 of domains is defined also recursively by setting D0 = K and
n



D N = ( x, y, . . . , z) ∈ Kn × . . . × Kn : (y, . . . , z) ∈ D N −1 , x # Φ( N −1) (y, . . . , z)
⎩   
N +1
⎫ (10)
1 ⎬
and f  ( xi ) − f ( xi ) ∑ = 0 for i ∈ In .
j = i xi − Φ j
( N −1)
(y, . . . , z) ⎭

Clearly, the iteration function Φ(1) coincides with the function Φ.

Definition 3. Let N be a given natural number, and x (0) , x (−1) , . . . , x (− N ) ∈ Kn be N + 1


initial approximations. We define the Nth iterative method of an infinite sequence of multi-point
Ehrlich-type methods by the following iteration

x ( k +1) = Φ ( N ) ( x ( k ) , x ( k −1) , . . . , x ( k − N ) ) , k = 0, 1, . . . . (11)

Note that in the case N = 1, the iterative method (11) coincides with the two-point
Ehrlich-type method (8).
In Section 4, we present a local convergence theorem (Theorem 2) for the methods (11)
with initial conditions that guarantee the convergence to a root vector of f . In the case
N = 1, this result extends Theorem 1 in several directions.
In Section 5, we present a semilocal convergence theorem (Theorem 4) for the fam-
ily (11), which is of practical importance.

3. Preliminaries
In this section, we present two basic properties of the iteration function Φ defined in
Definition 1, which play an important role in obtaining the main result in Section 4.
In what follows, we assume that Kn is endowed with the norm · ∞ defined by

u ∞ = max{|u1 |, . . . , |un |}

and with the cone norm · : Kn → Rn defined by

u = (|u1 |, . . . , |un |),

assuming that Rn is endowed with the component-wise ordering defined by

u v ⇔ ui ≤ vi for all i ∈ In .

Furthermore, for two vectors u ∈ Kn and v ∈ Rn , we denote by u/v the vector


 
u | u1 | |un |
= ,··· , .
v v1 vn

We define a function d : Kn → Rn by d(u) = (d1 (u), . . . , dn (u)) with

di (u) = min |ui − u j | (i = 1, . . . , n).


j = i

Lemma 1 ([11]). Suppose x, y, ξ ∈ Kn and ξ is a vector with pairwise distinct components.

| xi − y j | ≥ (1 − E( x ) − E(y)) |ξ i − ξ j | for all i, j ∈ In , (12)

4
Mathematics 2021, 9, 1640

where the function E : Kn → R+ is defined by


 
x − ξ
E( x ) =  
 d(ξ )  . (13)

Lemma 2. Suppose f ∈ K[z] is a polynomial of degree n ≥ 2, which splits over K, and ξ ∈ Kn is


a root vector of f . Let x, y ∈ Kn be two vectors such that x # y. If f ( xi ) = 0 for some i ∈ In , then

f  ( xi ) 1 1 − τi
−∑ = , (14)
f ( xi ) j = i
x i − y j x i − ξi

where τi ∈ K is defined by

yj − ξ j
τi = ( xi − ξ i ) ∑ ( xi − ξ j )( xi − y j )
. (15)
j = i

Proof. Since ξ is a root vector of f , we obtain


 
f  ( xi ) 1 n
1 1 1 1 1
−∑ ∑ xi − ξ i j∑ xi − ξ i j∑
= − = + +
f ( xi ) x − yj
j = i i j =1
x − yj
= i i = i
xi − ξ j xi − y j

1 yj − ξ j 1 − τi
xi − ξ i j∑
= − = ,
= i
( x i − ξ j )( x i − y j ) x i − ξi

which proves (14).

Define the function σ : D ⊂ Kn × Kn → R+ by

( n − 1) E ( x ) E ( y )
σ ( x, y) = (16)
(1 − E( x ))(1 − E( x ) − E(y)) − (n − 1) E( x ) E(y)

with domain
D = {( x, y) ∈ K × Kn : (1 − E( x ))(1 − E( x ) − E(y)) > (n − 1) E( x ) E(y) and E( x ) + E(y) < 1},
n
(17)
where E : Kn → R+ is defined by (13).

Lemma 3. Let f ∈ K[z] be a polynomial of degree n ≥ 2 with n simple zeros in K, and let ξ ∈ Kn
be a root vector of f . Suppose x, y ∈ Kn are two vectors such that ( x, y) ∈ D. Then:
(i) ( x, y) ∈ DΦ ;
(ii) Φ( x, y) − ξ σ ( x, y) x − ξ ;
(iii) E(Φ( x, y)) ≤ σ( x, y) E( x ),
where the functions Φ, E and σ are defined by (6), (13) and (16), respectively.

Proof. (i) According to (17), we have E( x ) + E(y) < 1. Then it follows from Lemma 1 that

| xi − y j | ≥ (1 − E( x )) d j (ξ ) > 0 (18)

for every j = i. This yields x # y. In view of (7), it remains to prove that

1
f  ( xi ) − f ( xi ) ∑ = 0 (19)
j = i
xi − y j

5
Mathematics 2021, 9, 1640

for i ∈ In . Let i ∈ In be fixed. We shall consider only the non-trivial case f ( xi ) = 0. In this
case, (19) is equivalent to
f  ( xi ) 1
− ∑ = 0. (20)
f ( xi ) x − yj
j = i i

On the other hand, it follows from Lemma 2 that (20) is equivalent to τi = 1, where τi is
defined by (15). By Lemma 1 with y = ξ, we obtain

| xi − ξ j | ≥ (1 − E( x )) di (ξ ) > 0 (21)

for every j = i. From (15), (18) and (21), we obtain

|y j − ξ j |
|τi | ≤ | xi − ξ i | ∑ | xi − ξ j || xi − y j |
(22)
j = i

1 | xi − ξ i | |y j − ξ j |

(1 − E( x ))(1 − E( x ) − E(y)) di (ξ ) ∑ d j (ξ )
j = i
( n − 1) E ( x ) E ( y )
≤ < 1.
(1 − E( x ))(1 − E( x ) − E(y))

This implies that τi = 1 which proves the first claim.


(ii) The second claim is equivalent to

|Φi ( x, y) − ξ i | ≤ σ( x, y) | xi − ξ i | (23)

for all i ∈ In . If xi = ξ i , then (23) holds trivially. Let xi = ξ i . Then, it follows from (21) that
f ( xi ) = 0. It follows from (6), (20) and (14) that
  −1
f  ( xi ) 1
Φi ( x, y) − ξ i = xi − ξ i − −∑ (24)
f ( xi ) x
j = i i
− yj
xi − ξ i τ
= xi − ξ i − = − i ( x i − ξ i ).
1 − τi 1 − τi

By (24) and the estimate (22), we obtain

|τi | |τi |
|Φi ( x, y) − ξ i | = |x − ξi | ≤ |x − ξi |
|1 − τi | i 1 − |τi | i
( n − 1) E ( x ) E ( y )
≤ |x − ξi |
(1 − E( x ))(1 − E( x ) − E(y)) − (n − 1) E( x ) E(y) i

= σ( x, y) | xi − ξ i |.

Therefore, (23) holds, which proves the second claim.


(iii) By dividing both sides of the last inequality by di (ξ ) and taking the max-norm,
we obtain the third claim.

Lemma 4. Let f ∈ K[z] be a polynomial of degree n ≥ 2 with n simple zeros in K, and let ξ ∈ Kn
be a root vector of f . Suppose x, y ∈ Kn are two vectors satisfying

2
max{ E( x ), E(y)} ≤ R = √ , (25)
3+ 8n − 7

where the function E : Kn → R+ is defined by (13). Then:


(i) ( x, y) ∈ D;

6
Mathematics 2021, 9, 1640

E( x ) E(y)
(ii) σ( x, y) ≤ ;
R2

E ( x )2 E ( y )
(iii) E(Φ( x, y)) ≤ .
R2

Proof. It follows from (25) that E( x ) + E(y) ≤ 2R < 1 and

(1 − E( x ))(1 − E( x ) − E(y)) − (n − 1) E( x ) E(y) ≥ (1 − R)(1 − 2R) − (n − 1) R2 > 0. (26)

Hence, it follows from (17) that ( x, y) ∈ D which proves the claim (i). It is easy to show
that R is the unique positive zero of the function φ, defined by

( n − 1) t2
φ(t) = . (27)
(1 − t)(1 − 2t) − (n − 1)t2

Then, from (16) and (26), we obtain

( n − 1) E ( x ) E ( y )
σ( x, y) ≤
(1 − R)(1 − 2R) − (n − 1) R2
( n − 1) R2 E( x ) E(y)
=
(1 − R)(1 − 2R) − (n − 1) R2 R2
E( x ) E(y) E( x ) E(y)
= φ( R) = , (28)
R2 R2
which proves (ii). The claim (iii) follows from Lemma 3 (iii) and claim (ii).

4. Local Convergence Analysis


In this section, we present a local convergence theorem for the multi-point iterative
methods (11). More precisely, we study the local convergence of the multi-point Ehrlich-
type methods (11) with respect to the function of the initial conditions E : Kn → R+ defined
by (13), where ξ ∈ Kn is a root vector of a polynomial f ∈ K[z].

Definition 4. We define a sequence (σN )∞


N = 1 of functions σN : D N ⊂ K
n
× .
. . × Kn → R by
 
N +1

σN ( x, y, . . . , z) = σ ( x, Φ( N −1) (y, . . . , z)), (29)

where σ is defined by (16). The domain D N is defined by

D N ={( x, y, . . . , z) : x ∈ Kn , (y, . . . , z) ∈ D N −1 ,
(1 − E( x ))(1 − E( x ) − E(Φ( N −1) (y, . . . , z))) > (n − 1) E( x ) E(Φ( N −1) (y, . . . , z)),
E( x ) + E(Φ( N −1) (y, . . . , z)) < 1},

and D N is defined by (10).

Lemma 5. Let f ∈ K[z] be a polynomial of degree n ≥ 2 with n simple zeros in K and ξ ∈ Kn be


a root vector of f . Assume N ≥ 1 and ( x, y, . . . , z) ∈ D N . Then:
(i) ( x, y, . . . , z) ∈ D N ;
(ii) Φ( N ) ( x, y, . . . , z) − ξ σN ( x, y, . . . , z) x − ξ ;

(iii) E(Φ( N ) ( x, y, . . . , z)) ≤ σN ( x, y, . . . , z) E( x ),


where Φ( N ) and σN are defined by (9) and (29), respectively.

7
Mathematics 2021, 9, 1640

Proof. Applying Lemma 1 with y = Φ( N −1) (y, . . . , z), we obtain (i). It follows from Defini-
tion 2, Lemma 3 (ii) and Definition 4 that

Φ( N ) ( x, y, . . . , z) − ξ = Φ( x, Φ( N −1) (y, . . . , z)) − ξ


σ ( x, Φ( N −1) (y, . . . , z)) x − ξ = σN ( x, y, . . . , z) x − ξ ,

which proves (ii). From Definition 2, Lemma 3 (iii) and Definition 4, we obtain

E(Φ( N ) ( x, y, . . . , z)) = E(Φ( x, Φ( N −1) (y, . . . , z)))


≤ σ( x, Φ( N −1) (y, . . . , z)) E( x ) = σN ( x, y, . . . , z) E( x ),

which proves (iii).

Lemma 6. Let f ∈ K[z] be a polynomial of degree n ≥ 2 with n simple zeros in K, and let ξ ∈ Kn
be a root vector of f . Assume N ≥ 1 and x, y, . . . , t, z are N + 1 vectors in Kn such that

2
max{ E( x ), E(y), . . . , E(z)} ≤ R = √ , (30)
3+ 8n − 7

where the function E : Kn → R+ is defined by (13). Then:


(i) ( x, y, . . . , t, z) ∈ D N ;
E ( x ) E ( y )2 . . . E ( t )2 E ( z )
(ii) σN ( x, y, . . . , t, z) ≤ ;
R2N
E ( x )2 E ( y )2 . . . E ( t )2 E ( z )
(iii) E(Φ( N ) ( x, y, . . . , t, z)) ≤ .
R2N

Proof. The proof goes by induction on N. In the case N = 1, Lemma 6 coincides with
Lemma 4. Suppose that for some N ≥ 1 the three claims of the lemma hold for every N + 1
vectors x, y, . . . , t, z ∈ Kn satisfying (30). Let x, y, . . . , t, z ∈ Kn be N + 2 vectors satisfying

max{ E( x ), E(y), . . . , E(t), E(z)} ≤ R.

We must prove the following three claims:

( x, y, . . . , t, z) ∈ D N +1 , (31)

E ( x ) E ( y )2 . . . E ( t )2 E ( z )
σN +1 ( x, y, . . . , t, z) ≤ , (32)
R 2( N +1)
E ( x )2 E ( y )2 . . . E ( t )2 E ( z )
E(Φ( N +1) ( x, y, . . . , z) ≤ . (33)
R 2( N +1)
By induction assumption, we obtain (y, . . . , t, z) ∈ D N . By induction assumption (ii) and
(30), we obtain

E( x ) + E(Φ( N ) (y, . . . , t, z)) ≤ E( x ) + E(y)2 . . . E(t)2 E(z)/R2N ≤ 2R < 1. (34)

By induction assumption, we also have

(1 − E( x ))(1 − E( x ) − E(Φ( N ) (y, . . . , z))) − (n − 1) E( x ) E(Φ( N ) (y, . . . , z))


(35)
> (1 − R)(1 − 2R) − (n − 1) R2 > 0

8
Mathematics 2021, 9, 1640

The inequalities (34) and (35) yield ( x, y, . . . , z) ∈ D N +1 , which proves (31). From Defini-
tion 4, Lemma 4 (ii) and induction assumption (ii), we obtain

σN +1 ( x, y, . . . , z) = σ ( x, Φ( N ) (y, . . . , z)) ≤ E( x ) E(Φ( N ) (y, . . . , z)/R2


≤ E( x ) E(y)2 . . . E(t)2 E(z)/R2( N +1) ,

which proves (32). Claim (33) follows from Lemma 5 (ii) and claim (32).

Now we are ready to state the first main result in this paper.

Theorem 2. Suppose f ∈ K[z] is a polynomial of degree n ≥ 2 which has n simple zeros in K,


ξ ∈ Kn is a root vector of f , and N ∈ N. Let x (0) , x (−1) , . . . , x (− N ) ∈ Kn be initial approximations
such that
2
max E( x (k) ) < R = √ , (36)
− N ≤ k ≤0 3 + 8n − 7
where the function E : Kn → R+ is defined by (13). Then the multi-point Ehrlich-type iteration (11)
is well defined and converges to ξ with order r and error estimates
k + N +1 − r k + N
x ( k +1) − ξ λr x (k) − ξ for all k ≥ 0, (37)
k + N −r N
x (k) − ξ λr x (0) − ξ for all k ≥ 0, (38)
where r = r ( N ) is the unique positive root of the Equation (5), and λ is defined by
 1/rk+ N
E ( x (k) )
λ = max . (39)
− N ≤ k ≤0 R

Proof. First, we will show that the iterative sequence ( x (k) )∞


k =− N generated by (11) is well
defined and the inequality
ν+ N
E ( x ( ν ) ) ≤ R λr (40)
holds for every integer ν ≥ − N. The proof is by induction. It follows from (39) that (40)
holds for − N ≤ ν ≤ 0. Suppose that for some k ≥ 0 the iterates x (k) , x (k−1) , . . . , x (k− N ) are
well defined and
ν+ N
E ( x ( ν ) ) ≤ R λr for all k − N ≤ ν ≤ k. (41)
We shall prove that the iterate x (k+1) is well defined and that it satisfies the inequality (40)
with ν = k + 1. It follows from (39) that 0 ≤ λ < 1. Hence, from (41) we obtain

max E( x (ν) ) ≤ R.
k− N ≤ ν≤k

Then by (11), Lemma 6 (iii), (41) and the definition of r, we obtain

E ( x ( k +1) ) = E(Φ( N ) ( x (k) , x (k−1) . . . , x (k− N ) ))


 2
≤ E( x (k) ) E( x (k−1) ) . . . E( x (k− N +1) ) E( x (k− N ) )/R2N
 k + N k + N −1 
k +1 2 r k N −1 k + N +1
λ = R λr (1+2r+...+2r +2r ) = Rλr
k N
≤ R λr λr · · · λr ,

which completes the induction. By Lemma 6 (ii), (40) and the definition of r, we obtain the
following estimate
 2
σN ( x (k) , x (k−1) , . . . , x (k− N ) ) ≤ E( x (k) ) E( x (k−1) ) · · · E( x (k− N +1) ) E( x (k− N ) )/R2N
k+ N
 k + N −1 
k +1 2 r k N −1 k + N +1 − r k + N
λ = λr (1+2r+...+2r +r ) = λr
k N
≤ λr λr · · · λr .

9
Mathematics 2021, 9, 1640

From (11), Lemma 5 (ii) and the last estimate, we obtain

x ( k +1) − ξ = Φ ( N ) ( x ( k ) , x ( k −1) , . . . , x ( k − N ) ) − ξ
σN ( x (k) , x (k−1) , . . . , x (k− N ) ) x (k) − ξ
k + N +1 − r k + N
λr x (k) − ξ ,

which proved the a posteriori estimate (37). The a priori estimate (38) can be easily proved
by induction using the estimate (37). Finally, the convergence of the sequence x (k) to a root
vector ξ follows from the estimate (38).

Remark 1. It can be proved that the sequence r ( N ), N = 1, 2, . . ., of orders of the multi-point


Ehrlich-type methods (11) is an increasing sequence which converges to 3 as N → ∞. In Table 1,
one can see the order of convergence r = r ( N ) for N = 1, 2, . . . , 10.

Table 1. Values of the convergence order r = r ( N ) for N = 1, 2, . . . , 10.

N 1 2 3 4 5 6 7 8 9 10
r( N ) 2.41421 2.83117 2.94771 2.98314 2.99446 2.99816 2.99939 2.99979 2.99993 2.99998

5. Semilocal Convergence Analysis


In this section, we present a semilocal convergence result for the multi-point Ehrlich
type methods (11) with respect to the function of initial conditions E f : D ⊂ Kn → R+
defined by  
 W (x) 
 f 
E f (x) =   , (42)
 d( x ) 

where the function W f : D ⊂ Kn → Kn is defined by (2). We note that in the last decade,
this is the most frequently used function to set the initial approximations of semilocal
results for simultaneous methods for polynomial zeros. (see, e.g., [10,11,17–22]).
The new result is obtained as a consequence from the local convergence Theorem 2 by
using the following transformation theorem:

Theorem 3 (Proinov [19]). Let K be an algebraically closed field, f ∈ K[z] be a polynomial of


degree n ≥ 2, and let x ∈ Kn be a vector with pairwise distinct components such that
 
 W (x)  R (1 + R )
 f 
  < , (43)
 d( x )  (1 + 2R)(1 + nR)


where 0 < R ≤ 1/( n − 1 − 1). Then f has only simple zeros in K and there exists a root vector
ξ ∈ Kn of f such that  
x − ξ
 
 d(ξ )  < R. (44)

Each iterative method for finding simultaneously all roots of a polynomial f ∈ K[z] of
degree n ≥ 2 is an iterative method in Kn . It searches the roots ξ 1 , . . . , ξ n of the polynomial
f as a vector ξ = (ξ 1 , . . . , ξ n ) ∈ Kn . We have noticed in Section 2 that such a vector ξ is
called a root vector of f . Clearly, a polynomial can have more than one vector of the roots.
On the other hand, we can assume that the vector root is unique up to permutation.
A natural question arises regarding how to measure the distance of an approximation
x ∈ Kn to the zeros of a polynomial. The first step is to identify all vectors whose
components are the same up to permutation. Namely, we define a relation of equivalence

10
Mathematics 2021, 9, 1640

≡ on Kn by x ≡ y if the components of x and y are the same up to permutation. Then


following [11,20], we define a distance between two vectors x, y ∈ Kn as follows

ρ( x, y) = min x − v ∞. (45)
v≡y

Note that ρ is a metric on the set of classes of equivalence. For simplicity, we shall identify
equivalence classes with their representatives.
In what follows, we consider the convergence in Kn with respect to the metric ρ.
Clearly, if a sequence x (k) in Kn is convergent to a vector x ∈ Kn with respect to the norm
· , then it converges to x with respect to the metric ρ. The opposite statement is not true
(see [11]).
Before formulating the main result, we recall a technical lemma.

Lemma 7 ([11]). Let x, ξ, ξ ∈ Kn be such that ξ ≡ ξ. Then there exists a vector x ∈ Kn such that
x ≡ x and    
x − ξ x − ξ
 
  = 
 d(ξ )  . (46)
 d(ξ )  ∞

Now we can formulate and prove the second main result of this paper.

Theorem 4. Suppose K is an algebraically closed field, f ∈ K[z] is a polynomial of degree


n ≥ 2 and N ∈ N. Let x (0) , x (−1) , . . . , x (− N ) ∈ Kn be initial approximations satisfying the
following condition:

2(5 + 8n − 7)
max E f ( x (k) ) < Rn = √ √ , (47)
−N ≤ k ≤ 0 (2n + 3 + 8n − 7)(7 + 8n − 7)

where the function E f is defined by (42). Then the polynomial f has only simple zeros and the
multi-point Ehrlich-type iteration (11) is well defined and converges (with respect to the metric ρ)
to a root vector ξ of f with order of convergence r = r ( N ), where r is the unique positive solution
of the Equation (5).

Proof. The condition (47) can be represented in the form


 
 W (x)  R (1 + R )
 f 
max   < , (48)
− N ≤ k ≤ 0  d( x )  (1 + 2 R)(1 + n R)

where R is defined in (36). From Theorem 3 and the inequality (48), we conclude that f has
n simple zeros in K and that there exist root vectors ξ (0) , ξ (−1) , . . . ξ (− N ) ∈ Kn such that
 
 x (k) − ξ (k) 
 
max   < R. (49)
− N ≤ k ≤ 0  d ( ξ (k ) 

Let us put ξ (0) = ξ. Since ξ (0) , ξ (−1) , . . . ξ (− N ) are root vectors of f , then ξ (k) ≡ ξ for all
k = 0, −1, . . . , − N. It follows from Lemma 7 that there exist vectors x (0) , x (−1) , . . . , x (− N )
such that x (k) ≡ x (k) and (49) can be represented in the form
 
 x (k) − ξ 
 
max   < R. (50)
− N ≤ k ≤ 0  d(ξ ) 

It follows from Theorem 2 and inequality (50) that the multi-point iterative method (11)
with initial approximations x (0) , x (−1) , . . . , x (− N ) is well defined and converges to ξ. Hence,
the iteration (11) with initial approximations x (0) , x (−1) , . . . , x (− N ) converges with respect
to the metric ρ to the root vector of f .

11
Mathematics 2021, 9, 1640

The following criterion guarantees the convergence of the methods (11). It is an


immediate consequence of Theorem 4.

Corollary 1 (Convergence criterion). If there exists an integer m ≥ 0 such that


 
Em = max E f ( x (m) ), E f ( x (m−1) ), . . . , E f ( x (m− N ) ) < Rn , (51)

then f has only simple zeros and the multi-point Ehrlich-type iteration (11) converges to a root
vector ξ of f .

The next result is an immediate consequence of Theorem 5.1 of [19]. It can be used as
a stopping criterion of a large class of iterative methods for approximating all zeros of a
polynomial simultaneously.

Theorem 5 (Proinov [19]). Suppose K is an algebraically closed field, f ∈ K[z] is a polynomial


of degree n ≥ 2 with simple zeros, and ( x (k) )∞
k =0 is a sequence in K consisting of vectors with
n

pairwise distinct components. If k ≥ 0 is such that



E f ( x (k) ) < μn = 1/(n + 2 n − 1), (52)

then the following a posteriori error estimate holds:

ρ( x (k) , ξ ) ≤ ε k = α( E f ( x (k) )) W f ( x (k) ∞, (53)

where the metric ρ is defined by (45), the function E f is defined by (42), and the function α is
defined by 
α(t) = 2/(1 − (n − 2)t + (1 − (n − 2)t)2 − 4t). (54)

6. Numerical Examples
In this section, we present two numerical examples in order to show the applicability
of Theorem 4. Using the convergence criterion (51), we show that at the beginning of the
iterative process it can be proven numerically that the method is convergent under the
given initial approximations.
We apply the first four methods of the family (11) for calculating simultaneously all
the zeros of the selected polynomials. In each example, we calculate the smallest m > 0
that satisfies the convergence criterion (51). In accordance with Theorem 5, we use the
following stop criterion

E f ( x (k) ) < μn and ε k < 10−12 , (55)

where μn and ε k are defined by (52) and (53), respectively. To see the convergence behavior
of the methods, we show in the tables ε k+1 in addition to ε k .
In both examples, we take the same polynomials and initial approximations as in [11],
where the initial approximations are chosen quite randomly. This choice gives the oppor-
tunity to compare numerically the convergence behavior of the multi-point Ehrlich-type
methods with those of the multi-point Weierstrass-type methods which are studied in [11].
To present the calculated approximations of high accuracy, we implemented the
corresponding algorithms using the programming package Wolfram Mathematica 10.0
with multiple precision arithmetic.

Example 1. The first polynomial is

f (z) = z3 − (2 + 5i )z2 − (3 − 10i )z + 15i (56)

12
Mathematics 2021, 9, 1640

with zeros −1, 3 and 5i (marked in blue in Figure 1). For N ∈ {1, 2, 3, 4}, the initial approximations
x (0) , x (−1) , . . . , x (− N ) in C3 are given in Table 2, where

a = (5 + i, 7 − i, −4.5i ), b = (1, −2.7, 4.5i ), c = (−5i, 2, 8),

u = (−10, −5i, 8), v = (i, 3 + i, 8).


In the case N = 3, the initial approximations are marked in red in Figure 1.

Table 2. Initial approximations for Example 1.

N x(−4) x(−3) x(−2) x(−1) x (0)


1 − − − a b
2 − − a b c
3 − a b c u
4 a b c u v

The numerical results for Example 1 are presented in Table 3. For instance, for the
multi-point Ehrlich-type method (11) with N = 3, one can see that the convergence condi-
tion (51) is satisfied for m = 6 which guarantees that the considered method is convergent
with order of convergence r = 2.94771. The stopping criterion (55) is satisfied for k = 6
and at the sixth iteration the guaranteed accuracy is 10−16 . At the next seventh iteration,
the zeros of the polynomial f are calculated with accuracy 10−47 .

Table 3. Convergence behavior for Example 1 (Rn = 0.125, τn = 0.171573).

N m E f ( x(m) ) k E f ( x(k) ) εk ε k +1 r
1 4 0.036247 5 0.000039 9.06336 × 10−14 1.52321 × 10−32 2.41421
2 5 0.001957 5 0.001957 5.97453 × 10−17 5.45631 × 10−48 2.83117
3 6 0.076062 6 0.076062 2.46336 × 10−16 1.05897 × 10−47 2.94771
4 7 0.083021 7 0.083021 6.50717 × 10−17 3.80803 × 10−51 2.98314

In Figure 1, we present the trajectories of the approximations generated by the first


six iterations of the method (11) for N = 3. We observe how each initial approximation,
moving along a bizarre trajectory, finds a zero of the polynomial.

Example 2. The second polynomial is

f (z) = z7 − 28 z6 + 322 z5 − 1960 z4 + 6769 z3 − 13132 z2 + 13068 z − 5040 (57)

with zeros 1, 2, 3, 4, 5, 6, 7 (marked in blue in Figure 2). For given N ∈ {1, 2, 3, 4}, the initial
approximations x (k) ∈ Cn (k = − N, . . . , −1, 0) are chosen with Aberth initial approximations
as follows:
 
(k) a π 3
xν = − 1 + Rk exp (iθν ), θν = 2ν − , ν = 1, . . . , n, (58)
n n 2

where a1 = −28, n = 7, Rk = R + 2 − k and R = 13.7082. In the case N = 3, the initial approx-


imations are marked in red in Figure 2.

13
Mathematics 2021, 9, 1640

  





Figure 1. Trajectories of the approximations for Example 1 (N = 3).


The numerical results for Example 2 are presented in Table 4. For example, for the
multi-point Ehrlich-type method (11) with N = 3, the convergence condition (51) is satisfied
for m = 7 and the stopping criterion (55) is satisfied for k = 8 which guarantees an accuracy
10−22 . At the next ninth iteration, the zeros of the polynomial f are calculated with accuracy
10−65 . In Figure 1, we present the trajectories of the approximations generated by the first
seven iterations of the method (11) for N = 3. One can see that the trajectories are quite
regular in the case of Aberth’s initial approximations.

Table 4. Convergence behavior for Example 2 (Rn = 0.125, τn = 0.171573).

N m E f ( x(m) ) k E f ( x(k) ) εk ε k +1
1 18 0.00526 21 3.48544 × 10−10 4.73454 × 10−16 1.25695 × 10−38
2 6 0.01689 8 7.85062 × 10−6 4.23967 × 10−17 1.06658 × 10−48
3 7 0.01348 8 0.00038 1.12167 × 10−22 6.66169 × 10−65
4 14 0.03215 14 0.03215 6.61642 × 10−24 4.98369 × 10−71

14
Mathematics 2021, 9, 1640





      








Figure 2. Trajectories of the approximations for Example 2 (N = 3).

7. Conclusions
In this paper, we introduced a new family of multi-points iterative methods for
approximating all the zeros of a polynomial simultaneously. Let us note that the first
member of this family is the two-point Ehrlich-type√ method introduced in 1999 by Trićković
and Petković [9]. Its convergence order is r = 1 + 2 .
We provide a local and semilocal convergence analysis of the new iterative methods.
Our local convergence result (Theorem 2) contains the following information for each
method: convergence order; initial conditions that guarantee the convergence; a priori and
a posteriori error estimates. In√particular, each method of the family has super-quadratic
convergence of order r ∈ [1 + 2, 3). Our semilocal convergence result (Theorem 4) can be
used to numerically prove the convergence of each method for a given polynomial and
initial approximation.
Finally, we would like to note that the local convergence theorem was obtained by a
new approach developed in our previous article [11]. We believe that this approach can be
applied to obtain convergence results for other multi-point iterative methods.

Author Contributions: The authors contributed equally to the writing and approved the final
manuscript of this paper. Both authors have read and agreed to the published version of the manuscript.
Funding: This research was supported by the National Science Fund of the Bulgarian Ministry of
Education and Science under Grant DN 12/12.
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: Not applicable.
Conflicts of Interest: The authors declares no conflict of interest.

15
Mathematics 2021, 9, 1640

References
1. Stewart, G.W. On the convergence of multipoint iterations. Numer. Math. 1994, 68, 143–147. [CrossRef]
2. Abu-Alshaikh, I.; Sahin, A. Two-point iterative methods for solving nonlinear equations. Appl. Math. Comput. 2006, 182, 871–878.
[CrossRef]
3. Ignatova, B.; Kyurkchiev, N.; Iliev, A. Multipoint algorithms arising from optimal in the sense of Kung-Traub iterative procedures
for numerical solution of nonlinear equations. Gen. Math. Notes 2011, 11, 45–79.
4. Argyros, I.K.; González, D. Unified majorizing sequences for Traub-type multipoint iterative procedures. Numer. Algorithms
2013, 64, 549–565. [CrossRef]
5. Tiruneh, A.T.; Ndlela, W.N.; Nkambule, S.J. A two-point Newton method suitable for nonconvergent cases and with super-
quadratic convergence. Adv. Numer. Anal. 2013, 687382. [CrossRef]
6. Petković, M.S.; Neta, B.; Petković, L.D.; Džunić, J. Multipoint methods for solving nonlinear equations: A survey.
Appl. Math. Comput. 2014, 226, 635–660. [CrossRef]
7. Amat, S.; Argyros, I.; Busquier, S.; Hernández-Verón, M.A.; Rubio, M.J. A unified convergence analysis for some two-point type
methods for nonsmooth operators. Mathematics 2019, 7, 701. [CrossRef]
8. Kanno, S.; Kjurkchiev, N.V.; Yamamoto, T. On some methods for the simultaneous determination of polynomial zeros. Jpn. J.
Indust. Appl. Math. 1996, 13, 267–288. [CrossRef]
9. Tričković, S.B.; Petković, M.S. Multipoint methods for the determination of multiple zeros of a polynomial. Novi Sad J. Math.
1999, 29, 221–233.
10. Proinov, P.D.; Petkova, M.D. Convergence of the two-point Weierstrass root-finding method. Jpn. J. Indust. Appl. Math. 2014,
31, 279–292. [CrossRef]
11. Proinov, P.D.; Petkova, M.D. Local and semilocal convergence of a family of multi-point Weierstrass-type root-finding methods.
Mediterr. J. Math. 2020, 17, 107. [CrossRef]
12. Weierstrass, K. Neuer Beweis des Satzes, dass jede ganze rationale Function einer Veränderlichen dargestellt werden kann als ein
Product aus linearen Functionen derselben Veränderlichen. Sitzungsber. Königl. Akad. Wiss. Berlin 1891, II, 1085–1101. [CrossRef]
13. Ehrlich, L. A modified Newton method for polynomials. Commun. ACM 1967, 10, 107–108. [CrossRef]
14. Aberth, O. Iteration methods for finding all zeros of a polynomial simultaneously. Math. Comput. 1973, 27, 339–344. [CrossRef]
15. Börsch-Supan, W. Residuenabschätzung für Polynom-Nullstellen mittels Lagrange-Interpolation. Numer. Math. 1969/70,
14, 287–296. [CrossRef]
16. Werner, W. On the simultaneous determination of polynomial roots. In Iterative Solution of Nonlinear Systems of Equations
(Oberwolfach, 1982); Springer: Berlin, Germany; New York, NY, USA, 1982; Volume 953, pp. 188–202.
17. Proinov, P.D.; Vasileva, M.T. On the convergence of high-order Ehrlich-type iterative methods for approximating all zeros of a
polynomial simultaneously. J. Inequal. Appl. 2015, 2015, 336. [CrossRef]
18. Proinov, P.D. General convergence theorems for iterative processes and applications to the Weierstrass root-finding method.
J. Complex. 2016, 33, 118–144. [CrossRef]
19. Proinov, P.D. Relationships between different types of initial conditions for simultaneous root finding methods. Appl. Math. Lett.
2016, 52, 102–111. [CrossRef]
20. Proinov, P.D.; Ivanov, S.I. Convergence analysis of Sakurai-Torii-Sugiura iterative method for simultaneous approximation of
polynomial zeros. J. Comput. Appl. Math. 2019, 357, 56–70. [CrossRef]
21. Cholakov, S.I.; Vasileva, M.T. A convergence analysis of a fourth-order method for computing all zeros of a polynomial
simultaneously. J. Comput. Appl. Math. 2017, 321, 270–283. [CrossRef]
22. Ivanov, S.I. A unified semilocal convergence analysis of a family of iterative algorithms for computing all zeros of a polynomial
simultaneously. Numer. Algorithms 2017, 75, 1193–1204. [CrossRef]

16
mathematics

Article
Extrapolation Method for Non-Linear Weakly Singular Volterra
Integral Equation with Time Delay †
Li Zhang 1 , Jin Huang 1, *, Hu Li 2 and Yifei Wang 1

1 School of Mathematical Sciences, University of Electronic Science and Technology of China,


Chengdu 611731, China; [email protected] (L.Z.); [email protected] (Y.W.)
2 School of Mathematics, Chengdu Normal University, Chengdu 611130, China; [email protected]
* Correspondence: [email protected]
† This work was supported by the Program of Chengdu Normal University (CS18ZDZ02).

Abstract: This paper proposes an extrapolation method to solve a class of non-linear weakly singular
kernel Volterra integral equations with vanishing delay. After the existence and uniqueness of
the solution to the original equation are proved, we combine an improved trapezoidal quadrature
formula with an interpolation technique to obtain an approximate equation, and then we enhance
the error accuracy of the approximate solution using the Richardson extrapolation, on the basis of
the asymptotic error expansion. Simultaneously, a posteriori error estimate for the method is derived.
Some illustrative examples demonstrating the efficiency of the method are given.

Keywords: weakly singular kernel Volterra integral equation; proportional delay; improved trape-
zoidal quadrature formula; Richardson extrapolation; posteriori error estimate

Citation: Zhang, L.; Huang, J.; Li, H.;


Wang, Y. Extrapolation Method for 1. Introduction
Non-Linear Weakly Singular Volterra
Delay functional equations are often encountered in biological processes, such as
Integral Equation with Time Delay.
the growth of the population and the spread of an epidemic with immigration into the
Mathematics 2021, 9, 1856. https://
population [1,2], and a time delay can cause the population to fluctuate. In general, some
doi.org/10.3390/math9161856
complicated dynamics systems are also modeled by delay integral equations since the
delay argument could cause a stable equilibrium to become unstable. The motivation of
Academic Editors: Alicia Cordero
our work is twofold: one of the reasons is based on the first-kind delay Volterra integral
Barbero and Maria Isabel Berenguer
equation (VIE) of the form [3]
Received: 15 June 2021  t
Accepted: 3 August 2021 k (t, s)y(s)ds = f (t), t ∈ I := [0, T ],
Published: 5 August 2021
qt

which was discussed and transformed into the second-kind equivalent form
Publisher’s Note: MDPI stays neutral
 t
with regard to jurisdictional claims in ∂k(t, s)
k (t, t)y(t) − qk(t, qt)y(qt) + y(s)ds = f  (t),
published maps and institutional affil-
qt ∂t
iations.

if k (t, t) = 0 for t ∈ I, the normal form was given by


 t  qt
y(t) = f (t) + y(qt) + K1 (t, s)y(s)ds + K (t, s)y(s)ds, t ∈ I.
Copyright: © 2021 by the authors. 0 0
Licensee MDPI, Basel, Switzerland.
There has been some research [4–6] to the following form
This article is an open access article
distributed under the terms and  t  qt
conditions of the Creative Commons y(t) = f (t) + K1 (t, s)y(s)ds + K (t, s)y(s)ds, t ∈ I.
0 0
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).

Mathematics 2021, 9, 1856. https://doi.org/10.3390/math9161856 https://www.mdpi.com/journal/mathematics


17
Mathematics 2021, 9, 1856

Another source of motivation comes from the weakly singular delay VIE [7–9]
 qt
K (t, s)
y(t) = f (t) + G (s, y(s))ds, t ∈ [0, 1],
0 (qt − s)λ

where λ ∈ (0, 1), K (t, s) is smooth and G (s, y(s)) is a smooth non-linear function. However,
there has not yet been investigated for the case where two integral terms are presented, the
first integral term is the weakly singular Volterra integral and the second integral terms not
only has weak singularity in the left endpoint but also its upper limit is a delay function,
which is challenging to calculate. It is the aim of this paper to fill this gap.
With theoretical and computational advances, some numerical methods for delay
differential equations [10–13], delay integral equations [14], delay integral–differential
equations [15–18], and fractional differential equations with time delay [19–22] have been
investigated widely. Here, we consider the following non-linear weakly singular kernel
VIE with vanishing delay
 t  θ (t)
y(t) = f (t) + sλ k1 (t, s; y(s))ds + sμ k2 (t, s; y(s))ds, t ∈ I, (1)
0 0

where θ (t) := qt, q ∈ (0, 1), λ, μ ∈ (−1, 0), f (t), k1 (t, s; y(s)), k2 (t, s; y(s)) are r (r ≥ 1,
r ∈ N) times continuously differentiable on I, D × R, Dθ × R, respectively,
D := {(t, s) : 0 ≤ s ≤ t ≤ T } and Dθ := {(t, s) : 0 ≤ s ≤ θ (t) ≤ θ ( T ), t ∈ I }. Ad-
ditionally, k i (t, s; y(s)) (i = 1, 2) satisfy the Lipschitz conditions with respect to y(s) on the
domains, respectively. That is, for fixed s and t, there are two positive constants L j ( j = 1, 2)
which are independent of s and t, such that

|k j (t, s; y(s)) − k j (t, s; v(s))| ≤ L j |y(s) − v(s)|. (2)

Then, Equation (1) possesses a unique solution (see Theorem 1). In this paper, we consider
the case where the solution is smooth.
Some numerical investigations of delay VIE have been conducted, such as discontinu-
ous Galerkin methods [23], collocation methods [24–26], the iterative numerical method [27],
and the least squares approximation method [28]. In [29], an h p version of the pseudo-
spectral method was analyzed, based on the variational form of a non-linear VIE with
vanishing variable delays. The algorithm increased the accuracy by refining the mesh
and/or increasing the degree of the polynomial. Mokhtary et al. [7] used a well-conditioned
Jacobi spectral Galerkin method for a VIE with weakly singular kernels and proportional de-
lay by solving sparse upper triangular non-linear algebraic systems. In [8], the Chebyshev
spectral-collocation method was investigated for the numerical solution of a class of weakly
singular VIEs with proportional delay. An error analysis showed that the approximation
method could obtain spectral accuracy. Zhang et al. [9] used some variable transformations
to change the weakly singular VIE with pantograph delays into new equations defined on
[−1, 1], and then combined it with the Jacobi orthogonal polynomial.
The extrapolation method has been used extensively [30,31]. We apply the extrapola-
tion method for the solution of the non-linear weakly singular kernel VIE with proportional
delay. We prove the existence of the solution to the original equation using an iterative
method, while uniqueness is demonstrated by the Gronwall integral inequality. We obtain
the approximate equation by using the quadrature method based on the improved trape-
zoidal quadrature formula, combining the floor technique and the interpolation technique.
Then, we solve the approximate equation through an iterative method. The existence
of the approximate solution is validated by analyzing the convergence of the iterative
sequence, while uniqueness is shown using a discrete Gronwall inequality. In addition,
we provide an analysis of the convergence of the approximate solution and obtain the
asymptotic expansion of the error. Based on the error asymptotic expansion, the Richardson
extrapolation method is applied to enhance the numerical accuracy of the approximate
solution. Furthermore, we obtain the posterior error estimate of the method. Numerical

18
Mathematics 2021, 9, 1856

experiments effectively support the theoretical analysis, and all the calculations can be
easily implemented.
This paper is organized as follows: In Section 2, the existence and uniqueness of the
solution for (1) are proven. The numerical algorithm is introduced in Section 3. In Section 4,
we prove the existence and uniqueness of the approximate solution. In Section 5, we
provide the convergence analysis of the approximate solution. In Section 6, we obtain
the asymptotic expansion of error, the corresponding extrapolation technique is used for
achieving high precision, and a posterior error estimate is derived. Numerical examples
are described in Section 7. Finally, we outline the conclusions of the paper in Section 8.

2. Existence and Uniqueness of Solution of the Original Equation


In this section, we discuss the existence and uniqueness of the solution of the original
equation. There are two cases, 0 ≤ t ≤ T ≤ 1 and 1 < t ≤ T, that we will discuss in
the following.

Lemma 1 ([32]). Let y(t) and g(t) be non-negative integrable functions, t ∈ [0, T ], A ≥ 0,
satisfying
 t
y(t) ≤ A + g(s)y(s)ds,
0
then, for all 0 ≤ t ≤ T,
t
g(s)ds
y(t) ≤ Ae 0 .

Theorem 1. f (t), k1 (t, s; y(s)), k2 (t, s; y(s)) are r (r ≥ 1, r ∈ N) times continuously differ-
entiable on I, D × R, Dθ × R, respectively. Additionally, assume that k i (t, s; y(s)) (i = 1, 2)
satisfies the Lipschitz conditions (2), respectively. Then, Equation (1) has a unique solution.

Proof. We first construct the sequence {yn (t), n ∈ N} as follows:

y0 ( t ) = f ( t ),
 t  qt
yn (t) = f (t) + sλ k1 (t, s; yn−1 (s))ds + sμ k2 (t, s; yn−1 (s))ds.
0 0

Let b = max |y1 (t) − y0 (t)|, L = max{ L1 , L2 }, γ = min{λ, μ}.


0≤ t ≤ T
• Case I. For 0 ≤ s ≤ t ≤ T ≤ 1, by means of mathematical induction, when n = 1,
 t   qt   
        
|y2 (t) − y1 (t)| =  sλ k1 t, s; y1 (s) − k1 t, s; y0 (s) ds + sμ k2 t, s; y1 (s) − k2 t, s; y0 (s) ds
0 0
 t       qt     
λ 
≤ s k1 t, s; y1 (s) − k1 t, s; y0 (s) ds + sμ k2 t, s; y1 (s) − k2 t, s; y0 (s) ds
0 0
 t    t  
≤ L1 s y1 ( s ) − y0 ( s )  +
λ
L2 sμ y1 (s) − y0 (s)ds
0 0
 t (3)
λ μ
≤ (s Lb + s Lb)ds
0
 t
≤ 2Lb sγ ds
0
t γ +1
= 2Lb .
γ+1

Suppose that the following expression is established when n = k,

(2L)k−1
|yk (t) − yk−1 (t)| ≤ b t(k−1)(γ+1) . (4)
( k − 1 ) ! ( γ + 1) k −1

Let n = k + 1; then,

19
Mathematics 2021, 9, 1856

   t λ       qt  
   
 y k +1 ( t ) − y k ( t )  ≤ s k1 t, s; yk (s) − k1 t, s; yk−1 (s) ds + sμ k2 t, s; yk (s) − k2 t, s; yk−1 (s) ds
0 0
 t    t  
≤ L1 s λ  y k ( s ) − y k −1 ( s )  + L2 sμ yk (s) − yk−1 (s)ds
0 0
 t  
≤ 2L s yk (s) − yk−1 (s)ds
γ
0
(2L)k
≤b t k ( γ +1) ,
k!(γ + 1)k

that is, the recurrence relation is established when n = k + 1, then the inequality (4) is
also established. Next, we prove that the sequence yn (t) is a Cauchy sequence,
       
 y n ( t ) − y n + m ( t )  ≤  y n +1 ( t ) − y n ( t )  +  y n +2 ( t ) − y n +1 ( t )  + · · · +  y n + m ( t ) − y n + m −1 ( t ) 
(2L)n (2L)n+m−1
≤b t n ( γ +1) + · · · + b t(n+m−1)(γ+1)
n!(γ + 1) n ( n + m − 1) ! ( γ + 1) n + m −1
n + m +1
2L i i(γ+1) 1
≤b ∑ (
γ+1
)T
i!
.
i=n


The term ∑ ( γ2L i i (γ+1) 1 is convergent, so the Cauchy sequence { y }
+1 ) T i! n n∈N is con-
i =0
vergent uniformly to y(t). Thus, y(t) is the solution to Equation (1), the existence
is proved.
• Case II. For 1 < s ≤ t ≤ T, the process is similar. Let γ
 = max{λ, μ}, when n = 1,

tγ+1
|y2 (t) − y1 (t)| ≤ 2Lb . (5)
+1
γ

Suppose that the following expression is established when n = k,

(2L)k−1
|yk (t) − yk−1 (t)| ≤ b t(k−1)(γ+1) . (6)
 + 1) k −1
( k − 1) ! ( γ

Let n = k + 1. Then, we have

  (2L)k
 y k +1 ( t ) − y k ( t )  ≤ b tk(γ+1) ,
k!(γ + 1) k

i.e., the recurrence relation is established when n = k + 1, such that the inequality (6)
is also established. For the sequence yn (t),

  n + m +1
yn (t) − yn+m (t) ≤ b ∑ ( 2L )i T i(γ+1) 1 .
i=n
+1
γ i!


Since the term ∑ ( γ2L i i (γ
+1) 1 is convergent, so the Cauchy sequence { y }
+1 ) T i! n n∈N is
i =0
convergent uniformly to y(t). Thus, y(t) is the solution to Equation (1), the existence
is proved.

20
Mathematics 2021, 9, 1856

Now, we prove that the solution to Equation (1) is unique. Let y(t) and v(t) be two dis-
tinct solutions to Equation (1), and denote the difference between them by
w(t) = |y(t) − v(t)|. We obtain
 t       qt      

w(t) =  sλ k1 t, s; y(s) − k1 t, s; v(s) ds + sμ k2 t, s; y(s) − k2 t, s; v(s) ds
0 0
 t       qt     
λ 
≤ s k1 t, s; y(s) − k1 t, s; v(s) ds + sμ k2 t, s; y(s) − k2 t, s; v(s) ds
0 0
 t  qt
λ μ
≤ L1 s w(s)ds + L2 s w(s)ds
0 0
 t
≤ ( Lsλ + Lsμ )w(s)ds.
0

Let g(s) = Lsλ + Lsμ , then g(s) is a non-negative integrable function, according to
Lemma 1. We obtain w(t) = 0, i.e., y(t) = v(t), the solution to Equation (1) is unique.

3. The Numerical Algorithm


In this section, we first provide some essential lemmas which are useful for the deriva-
tion of the approximate equation. Next, the discrete form of Equation (1) is obtained by
combining an improved trapezoidal quadrature formula and linear interpolation. Finally,
we solve the approximate equation using an iterative method. The process does not have
to compute the integrals; hence, the method can be implemented easily.

3.1. Some Lemmas


Lemma 2 ([32]). Let u ∈ C3 (0, 1) and z = βx + (1 − β)y with β ∈ [0, 1], x, y ∈ [0, T ]. Then,

β (1 − β )
u(z) = βu( x ) + (1 − β)u(y) − ( x − y)2 u (z) + O(( x − y)3 ). (7)
2

Proof. The Taylor expansion of function u( x ) at the point z is

u( x ) = u( βx + (1 − β) x )
= u( βx + (1 − β)y + (1 − β)( x − y))
= u(z + (1 − β)( x − y)) (8)
(1 − β )2
= u(z) + (1 − β)( x − y)u (z) + ( x − y)2 u (z) + O(( x − y)3 ).
2
Similarly, the Taylor expansion of function u(y) at point z is

β2
u(y) = u(z − β( x − y)) = u(z) − β( x − y)u (z) + ( x − y)2 u (z) + O(( x − y)3 ), (9)
2
combining (8) with (9), the proof is completed.

(b− a)
Lemma 3 ([33,34]). Let g(t) ∈ C2r [ a, b] ( r ≥ 1,  r ∈ N ), G ( t ) = ( b − t ) λ g ( t ), h = N ,
b
and tk = a + kh for k = 0, · · · , N, as for the integral a G (t)dt. Then, the error of the modified
trapezoidal integration rule

N −1
h
TN ( G ) = G (t0 ) + h ∑ G (tk ) − ζ (−λ) g(b)h1+λ , (10)
2 j =1

has an asymptotic expansion

21
Mathematics 2021, 9, 1856

r −1
 r −1
B2j (2j−1) 2
g ( j ) ( b ) h j + λ +1
EN ( G ) = ∑ (2j)!
G ( a)h2j + ∑ (−1) j ζ (−λ − j)
( j!)
+ O(h2r ), (11)
j =1 j =1

where −1 < λ < 0, ζ is the Riemann–Zeta function and B2j represents the Bernoulli numbers.

3.2. The Approximation Process


In this subsection, we describe the numerical method used to find the approxi-
mate solution to Equation (1). Let y(t) have continuous partial derivatives up to 3 on I,
f (t), k1 (t, s; y(s)), k2 (t, s; y(s)) are four times continuously differentiable on I,
D × R, Dθ × R, respectively. Let y(ti ), yi denote the exact solution and approximate
solution when t = ti , respectively. We divide I = [0, T ] into N subintervals with a uniform
step size h = N T
, ti = ih, i = 0, 1, · · · , N. Let t = ti in Equation (1). Then,
 t    qti  
i
y ( ti ) = f ( ti ) + sλ k1 ti , s; y(s) ds + sμ k2 ti , s; y(s) ds
0 0
 t    t    qti  
i [qi ]
λ
= f ( ti ) + s k1 ti , s; y(s) ds + sμ k2 ti , s; y(s) ds + sμ k2 ti , s; y(s) ds (12)
0 0 t[qi]

= f (ti ) + I1 + I2 + I3 ,

where [qi ] denotes the maximum integer less than qi. According to Lemma 3, we have
 t     i −1   h  
i
I1 = sλ k1 ti , s; y(s) ds ≈ −ζ (−λ)k1 ti , t0 ; y(t0 ) h1+λ + h ∑ tλk k1 ti , tk ; y(tk ) + tiλ k1 ti , ti ; y(ti ) . (13)
0 k =1
2
For I2 and I3 , there are two cases.
• Case I. If [qi ] = 0, then

I2 = 0;
 qt     qt   (14)
i
I3 = sμ k2 ti , s; y(s) ds ≈ −ζ (−μ)(qti )1+μ k2 ti , t0 ; y(t0 ) + i (qti )μ k2 ti , qti ; y(qti ) .
0 2

• Case II. If [qi ] ≥ 1, we obtain


⎧    
⎪ 1+ μ k t , t ; y ( t ) + h t μ k t , t ; y ( t ) ,
⎨−ζ (−μ)h 2 i 0 0 2 1 2 i 1 1 [qi ] = 1,
I2 ≈   [qi ]−1
μ   μ  

⎩−ζ (−μ)h1+μ k2 ti , t0 ; y(t0 ) + h ∑ tk k2 ti , tk ; y(tk ) + 2h t[qi] k2 ti , t[qi] ; y(t[qi] ) , [qi ] > 1.
k =1

qti − t[qi]  μ    
I3 ≈ t[qi] k2 ti , t[qi] ; y(t[qi] ) + (qti )μ k2 ti , qti ; y(qti ) . (15)
2
y(qti ) can be represented by linear interpolation of the adjacent points y(t[qi] ) and y(t[qi]+1 ).
For the node ti = ih, i = 0, 1, · · · , N, since [qi ] ≤ qi ≤ [qi ] + 1, we obtain
t[qi] ≤ qti ≤ t[qi]+1 ; according to Lemma 2, there exists β i ∈ [0, 1] such that
qti = β i t[qi] + (1 − β i )t[qi]+1 . The value of β i = 1 + [qi ] − qi can be calculated easily.
Then, the approximate expression of y(qti ) is

y(qti ) ≈ β i y(t[qi] ) + (1 − β i )y(t[qi]+1 ). (16)

Then, (15) can be written as

qti − t[qi]  μ    
I3 ≈ t[qi] k2 ti , t[qi] ; y(t[qi] ) + (qti )μ k2 ti , qti ; β i y(t[qi] ) + (1 − β i )y(t[qi]+1 ) . (17)
2
The approximation equations are as follows

22
Mathematics 2021, 9, 1856

• Case I. When [qi ] = 0,

y0 = f ( t0 );
  i −1   h  
yi ≈ f (ti ) − ζ (−λ)k1 ti , t0 ; y0 h1+λ + h ∑ tλk k1 ti , tk ; yk + tiλ k1 ti , ti ; yi (18)
k =1
2
1+ μ
  qti  
− ζ (−μ)(qti ) k2 ti , t0 ; y0 + (qti )μ k2 ti , qti ; β i y[qi] + (1 − β i )y[qi]+1 .
2

• Case II. When [qi ] ≥ 1,

y0 = f ( t0 );
  i −1   h  
yi ≈ f (ti ) − ζ (−λ)k1 ti , t0 ; y0 h1+λ + h ∑ tλk k1 ti , tk ; yk + tiλ k1 ti , ti ; yi
k =1
2
    (19)
h μ
− ζ (−μ)h1+μ k2 ti , t0 ; y0 + δi + t[qi] k2 ti , t[qi] ; y[qi]
2
qti − t[qi]  μ    
+ t[qi] k2 ti , t[qi] ; y[qi] + (qti )μ k2 ti , qti ; β i y[qi] + (1 − β i )y[qi]+1 ,
2
where ⎧

⎨0, [qi ] = 1,
δi ≈ [qi ]−1
μ  

⎩h ∑ t k k 2 ti , t k ; y k , [qi ] ≥ 2.
k =1

3.3. Iterative Scheme


Now, the solution of the approximate equation can be solved by an iterative algorithm.
Iterative algorithm
Step 1. Take sufficiently small  > 0 and set ỹ0 = f (t0 ), i := 1.
Step 2. Let ỹ0i = ỹi−1 , m := 0, then we compute yim+1 (i ≤ N ) as follows:
• Case I. When [qi ] = 0,

y0 = f ( t0 );
  i −1   h  
yim+1 ≈ f (ti ) − ζ (−λ)k1 ti , t0 ; ỹ0 h1+λ + h ∑ tλk k1 ti , tk ; ỹk + tiλ k1 ti , ti ; yim (20)
k =1
2
1+ μ
  qti  +1

− ζ (−μ)(qti ) k2 ti , t0 ; ỹ0 + (qti )μ k2 ti , qti ; β i ỹ[qi] + (1 − β i )ym
[qi ]+1
.
2

• Case II. When [qi ] ≥ 1,

y0 = f ( t0 );
i −1
h
yim+1 ≈ f (ti ) − ζ (−λ)k1 (ti , t0 ; ỹ0 )h1+λ + h ∑ tλk k1 (ti , tk ; ỹk ) + 2 tiλ k1 (ti , ti ; yim )
k =1
    (21)
h μ
− ζ (−μ)h1+μ k2 ti , t0 ; ỹ0 + δi + t[qi] k2 ti , t[qi] ; ỹ[qi]
2
qti − t[qi]  μ  +1

+ t[qi] k2 (ti , t[qi] ; ỹ[qi] ) + (qti )μ k2 ti , qti ; β i ỹ[qi] + (1 − β i )ym
[qi ]+1
,
2
where 
0, [qi ] = 1,
δi ≈ [qi ]−1 μ  
h ∑k=1 tk k2 ti , tk ; ỹk , [qi ] ≥ 2.

Step 3. If −|yim+1 yim |


≤ , then let ỹi := yim+1
and i := i + 1, and return to step 2. If
otherwise, let m := m + 1, and return to step 2.

23
Mathematics 2021, 9, 1856

Remark 1. In Section 3.2, we considered the regularity of k i (t, s; y(s))(i = 1, 2) only up to 


r=2
in Lemma 3, since the desired accuracy has been obtained, and it is sufficient for the subsequent
convergence analysis and extrapolation algorithm.

4. Existence and Uniqueness of the Solution to the Approximate Equation


In this section, we investigate the existence and uniqueness of the solution to the
approximate equation. We first introduce the following discrete Gronwall inequality.

Lemma 4 ([35,36]). Suppose that the non-negative sequence {wn }, n = 0, · · · , N, satisfy

n −1
wn ≤ h ∑ Bk wk + A, 0 ≤ n ≤ N, (22)
k =1

where A and Bk , k = 1, · · · , N are non-negative constants, h = 1/N, when h max wk ≤ 1


2 then
0≤ k ≤ N
we have
N
max wn ≤ A exp(2h
0≤ n ≤ N
∑ Bk ).
k =1

Theorem 2. Let f (t), k1 (t, s; y(s)), k2 (t, s; y(s)) are four times continuously differentiable on
I, D × R, Dθ × R, respectively. Additionally, y(t) has continuous partial derivatives up to 3 on
I and k i (t, s; y(s)) (i = 1, 2) satisfy Lipschitz conditions (2). Assume that h is sufficiently small,
then the solution to Equation (21) exists and is unique.

Proof. We discuss the existence of the approximate solution under two cases.
• Case I. When [qi ] = 0,

 m +1  
m  h λ
 m −1 

y − y =  t k 1 ( t i , t i ; y m
) − k 1 ( t i , t i ; y ) 
i i i i i
2
h  
≤ L1 tiλ yim − yim−1 .
2

When h is sufficiently small, such that L1 2h tiλ ≤ 12 , then |yim+1 − yim | ≤ 12 |yim − yim−1 |
holds. Therefore, the iterative algorithm is convergent and the limit is the solution to
the approximation equation. The existence of approximation is proved when [qi ] = 0.
Now, we prove the uniqueness of approximation. Suppose yi and xi are both solutions
to Equation (20). Denote the absolute differences as wi = |yi − xi |. We have

w0 =0,
  i −1  
wi ≤ − ζ (−λ)k1 (ti , t0 ; y0 ) − k1 (ti , t0 ; x0 )h1+λ + h ∑ tλk k1 (ti , tk ; yk ) − k1 (ti , tk ; xk )
k =1
h    
+ tiλ k1 (ti , ti ; yi ) − k1 (ti , ti ; xi ) − ζ (−μ)(qt1 )1+μ k2 (ti , t0 ; y0 ) − k2 (ti , t0 ; x0 )
2
qt  
+ i (qti )μ k2 (ti , qti ; β i y[qi] + (1 − β i )y[qi]+1 ) − k2 (ti , qti ; β i x[qi] + (1 − β i ) x[qi]+1 )
2
i −1
h qt
≤ L1 h ∑ tλk wk + L1 tiλ wi + L2 i (qti )μ ( β i w[qi] + (1 − β i )w[qi]+1 )
k =1
2 2
i −1
h qti
≤ Lh ∑ tλk wk + L 2 tiλ wi + L 2
(qti )μ (1 − β i )w1 . (23)
k =1

24
Mathematics 2021, 9, 1856

where L = max{ L1 , L2 }. When h is sufficiently small, such that L1 2h tiλ ≤ 12 , we have

i −1
wi ≤2Lh ∑ tλk wk + Lhtiλ wi + Lqti (qti )μ (1 − βi )w1
k =1
 i −1
≤ 2Lhtλk + Lh(qti )μ (1 − β i ) w1 + 2Lh ∑ tλk wk
k =2
i −1
=h ∑ Bk wk ,
k =1

where 
2Ltλk + L(qti )μ (1 − β i ), j = 1,
Bk =
2Lhtλk , j = 2, · · · , i − 1.
According to Lemma 4 with A = 0, we have wi = 0, i.e., yi = xi , the solution of
Equation (20) is unique.
• Case II. For [qi ] > 1, we consider the following cases.
(1) The first situation is [qi ] + 1 = i, namely, when i ≤ 1− 1
q , we have
 m +1  h   
y  
i − yim  = tiλ k1 (ti , ti ; yim ) − k1 (ti , ti ; yim−1 ) 
2
qti − t[qi]     
 −1 
+ (qti )μ k2 ti , qti ; β i ỹ[qi] + (1 − β i )ym
[ qi ]+ 1 − k2 ti , qti ; β i ỹ[qi] + (1 − β i )ym
[ qi ]+ 1 
2
h   qti − t[qi]  
≤ L1 tiλ yim − yim−1  + L2 (qti )μ (1 − β i )ym m −1 
[qi ]+1 − (1 − β i ) y[qi ]+1
2 2
h  
≤ L tiλ + (qti )μ (1 − β i ) yim − yim−1 .
2
 
Let the step size h be small enough, such that L 2h tiλ + (qti )μ (1 − β i ) ≤ 12 . Then,
we can determine that |yim+1 − yim | ≤ 12 |yim − yim−1 | holds.
(2) The second situation is [qi ] + 1 < i, namely, when i > 1− 1
q , we obtain

 m +1  
m  h λ
 m −1 

y − y =  t k 1 ( t i , t i ; y m
) − k 1 ( t i , t i ; y ) 
i i i i i
2
h  
≤ L1 tiλ yim − yim−1 
2
h  
≤ L tiλ yim − yim−1 .
2

Let L 2h tiλ ≤ 12 for a sufficiently small h, then |yim+1 − yim | ≤ 12 |yim − yim−1 | holds.
The above two situations show that the iterative algorithm is convergent and that the
limit is the solution to Equation (21).
Next, we prove that the solution to Equation (21) is unique. Suppose yi and xi are both
solutions to Equation (21). Denote the differences as w i = |yi − xi |, i = 1, · · · , N. Then,
we have

25
Mathematics 2021, 9, 1856

0 =0;
w
  i −1  
i ≤ − ζ (−λ)k1 (ti , t0 ; y0 ) − k1 (ti , t0 ; x0 ) h1+λ + h ∑ tλk k1 (ti , tk ; yk ) − k1 (ti , tk ; xk )
w
k =1
h    
+ tiλ k1 (ti , ti ; yi ) − k1 (ti , ti ; xi ) − ζ (−μ)h1+μ k2 (ti , t0 ; y0 ) − k2 (ti , t0 ; x0 )
2
[qi ]−1  h μ 
μ
+ h ∑ tk |k2 (ti , tk ; yk ) − k2 (ti , tk ; xk ) + t[qi] |k2 (ti , t[qi] ; y[qi] ) − k2 (ti , t[qi] ; x[qi] )
k =1
2
qti − t[qi] μ  
+ (t k2 (ti , t[qi] ; y[qi] ) − k2 (ti , t[qi] ; x[qi] ) (24)
2   [qi]   
+ (qti )μ k2 ti , qti ; β i y[qi] + (1 − β i )y[qi]+1 ) − k2 ti , qti ; β i x[qi] + (1 − β i ) x[qi]+1 ) 
i −1 [qi ]−1
h μ h μ
≤h ∑ tλk L1 wk + 2 tiλ L1 wi + h ∑ k + t[qi] L2 w
t k L2 w
2
[qi]
k =1 k =1
qti − t[qi]  μ 
+ [qi] + (qti )μ L2 ( β i w
t[qi] L2 w [qi] + (1 − β i )w
[qi]+1 )
2
i −1 [qi ]−1
γ h γ γ h γ
≤ Lh ∑ tk wk + L 2 ti wi + Lh ∑ k + L t[qi] w
tk w
2
[qi]
k =1 k =1
h γ  
+ L tγ[qi] w
[qi] + t[qi] β i w
[qi] + (1 − β i )w
[qi]+1 .
2

(1) The first situation is [qi ] + 1 = i (i.e., when i ≤ 1


1−q ). Then, (24) entails
[qi ]−1 [qi ]−1
γ γ h γ γ h γ
i ≤ Lh
w ∑ k + Lht[qi] w
tk w [qi] + L ti w
2
i + Lh ∑ k + L t[qi] w
tk w
2
[qi]
k =1 k =1
h γ γ  
+L  + t[qi] β i w
t w [qi] + (1 − β i )w
[qi]+1 (25)
2 [qi] [qi]
[qi ]−1  h γ 
h γ h γ
=2Lh ∑ tγk w γ
k + (2Lht[qi] + L β i t[qi] )w
[qi] + L ti + L t[qi] (1 − β i ) w[qi]+1 .
k =1
2 2 2
 γ γ 
By letting h be so small that L 2h ti + L 2h t[qi] (1 − β i ) ≤ 12 , we can easily derive

[qi ]−1 i −1
γ γ γ
i ≤ 4Lh
w ∑ tk wk + (4Lht[qi] + Lhβ i t[qi] )w
[qi] = h ∑ Bk wk ,
k =1 k =1

where  γ
4Ltk , j = 1, · · · , [qi ] − 1,
Bk = γ γ
4Lt[qi] + Lβ i t[qi] , j = [qi ].

According to Lemma 4 with A = 0, we have wi = 0, and the solution of Equation (21)


is unique.
(2) The second situation is [qi ] + 1 < i (i.e., when i > 1−
1
q ). Then, (24) can imply

26
Mathematics 2021, 9, 1856

[qi ]−1 i −1
γ γ γ γ h γ
i ≤ Lh
w ∑ k + Lht[qi] w
tk w [qi] + Lht[qi]+1 w
[qi]+1 + Lh ∑  k + L ti w
tk w
2
i
k =1 k =[qi ]+2

h γ 
[qi ]−1
γ h γ γ  
+ Lh ∑ tk w
2
[qi] + L t[qi] w
k + L t[qi] w
2
[qi] + t[qi] β i w
[qi] + 1 − β i )w
[qi]+1
k =1
(26)
[qi ]−1  h   h γ 
γ γ
=2Lh ∑ k
tk w + 2Lht[qi] + L β i tγ[qi] w γ
[qi] + Lht[qi]+1 + L t[qi] (1 − β i ) w[qi]+1
k =1
2 2
i −1
γ h γ
+ Lh ∑  k + L ti w
tk w
2
i .
k =[qi ]+2

γ
Letting h be so small that L 2h ti ≤ 12 , then
[qi ]−1   i −1
γ γ γ γ γ γ
i ≤4Lh
w ∑ k + (4Lht[qi] + Lhβ i t[qi] )w
tk w [qi] + 2Lht[qi]+1 + Lht[qi] (1 − β i ) w[qi]+1 + 2Lh ∑ k
tk w
k =1 k =[qi ]+2
n −1
=h ∑ Bk wk ,
k =1

where ⎧ γ

⎪ 4Ltk , j = 1, · · · , [qi ] − 1,

⎪ γ γ
⎨ 4Lt[qi] + Lβ i t[qi] , j = [qi ],
k =
B
⎪ γ γ

⎪ 2Lt[qi]+1 + Lt[qi] (1 − β i ), j = [qi ] + 1,

⎩ γ
2Ltk , j = [qi ] + 2, · · · , i − 1.
According to Lemma 4 with A = 0, we have w i = 0, i.e., yi = xi , the solution
of Equation (21) is unique. Combining the above situations, the proof of Theorem 2
is completed.

5. Convergence Analysis
In this section, we will discuss errors caused by the process of obtaining discrete
equations using a quadrature formula and interpolation technique and the errors caused
by solving the discrete equation using iterative algorithms. According to the quadrature
rule, Equation (12) can be expressed as

y ( t0 ) = f ( t0 ),
  i −1   h  
y(ti ) = f (ti ) − ζ (−λ)k1 ti , t0 ; y(t0 ) h1+λ + h ∑ tλk k1 ti , tk ; y(tk ) + tiλ k1 ti , ti ; y(ti ) + E1,i
k =1
2

  [qi ]−1 (27)


μ   h μ  
− ζ (−μ)h1+μ k2 ti , t0 ; y(t0 ) + h ∑ tk k2 ti , tk ; y(tk ) + t[qi] k2 ti , t[qi] ; y(t[qi] ) + E2,i
k =1
2
qti − t[qi]  μ    
+ t[qi] k2 ti , t[qi] ; y(t[qi] ) + (qti )μ k2 ti , qti ; β i y(t[qi] ) + (1 − β i )y(t[qi]+1 ) + E3,i .
2
From Lemmas 2 and 3, the remainders are
  
   [k1 ti , s; y(s) ] s=0
E1,i 
= [k1 ti , s; y(s) ] s=0 ζ (−λ − 1)h 2+ λ
+ ζ (−λ − 2)h3+λ + O(h4+λ )
2!
= T1 (ti )h2+λ + O(h3+λ ),

27
Mathematics 2021, 9, 1856


 [k2 (ti , s; y(s))] s=0
E2,i = [k2 (ti , s; y(s))] s=0 ζ (−μ − 1)h2+γ + ζ (−μ − 2)h3+μ + O(h4+μ )
2!
= T2 (ti )h2+μ + O(h3+μ ),
β(1 − β) 2    
E3,i = − h y (qti )(qti )γ k2 ti , qti ; β i y(t[qi] ) + (1 − β i )y(t[qi]+1 )
2
(qti − t[qi] )2  qti ∂2    μ
+ k 2 t i , s; y ( s ) s ds + O(h3 )
12 t[qi] ∂s2

(qti − t[qi] )2 − h2  qti ∂2    


= T3 (ti )h2 + k2 ti , s; y(s) sμ ds + O(h3 )
12 t[qi] ∂s 2

= T3 (ti )h2 + O(h3 ),


where
  
T1 (ti ) = [k1 ti , s; y(s) ] s=0 ζ (−λ − 1),
  
T2 (ti ) = [k2 ti , s; y(s) ] s=0 ζ (−μ − 1),
   
β(1 − β)  1 qti ∂2  
T3 (ti ) = − u (qti )(qti )μ k2 ti , qti ; β i y(t[qi] ) + (1 − β i )y(t[qi]+1 ) + k2 ti , s; y(s) sμ ds.
2 12 t[qi] ∂s2

In order to investigate the error between the exact solution and the approximate
solution of Equation (1), we first give the following theorem.

Theorem 3. Under the conditions of Theorem 2, y(ti ) is the exact solution of Equation (1) when
t = ti and yi is the solution of discrete Equation (19) at ti . Assume that h is sufficiently small, then,
the absolute error denote by e1,i = |y(ti ) − yi | has the estimate

max |e1,i | ≤ O(h2+γ ).


1≤ i ≤ N

Proof. Subtracting (19) from (27),


 
e1,0  =0,
    i −1  
e1,i  = − ζ (−λ)k1 (ti , t0 ; y(t0 )) − k1 (ti , t0 ; y0 ) h1+λ + h ∑ tλ k1 (ti , tk ; y(tk )) − k1 (ti , tk ; yk )
k
k =1
h    
+ tiλ k1 (ti , ti ; y(ti )) − k1 (ti , ti ; yi ) − ζ (−μ)h1+μ k2 (ti , t0 ; y(t0 )) − k2 (ti , t0 ; y0 )
2
[qi ]−1   h μ  
μ
+ h ∑ tk k2 (ti , tk ; y(tk )) − k2 (ti , tk ; yk ) + t[qi] k2 (ti , t[qi] ; y(t[qi] )) − k2 (ti , t[qi] ; y[qi] )
k =1
2
qti − t[qi]  μ  
+ t[qi] k2 (ti , t[qi] ; y(t[qi] )) − k2 (ti , t[qi] ; y[qi] ) (28)
2
   
+ (qti )μ k2 ti , qti ; β i y(t ) + (1 − β i )y(t
[qi ] ) − k2 (ti , qti ; β i y + (1 − β i )y
[qi ]+1 [qi ] )
[qi ]+1

+ T1 (ti )h2+λ + T2 (ti )h2+μ + T3 (ti )h2 + O(h3+γ )


i −1 [qi ]−1
h μ h μ
=h ∑ tλk L1 |e1,k | + 2 tiλ L1 |e1,i | + h ∑ tk L2 |e1,k | + t[qi] L2 |e1,[qi] |
2
k =1 k =1
qti − t[qi]  μ

+ t[qi] L2 |e1,[qi] | + (qti )μ L2 | β i e1,[qi] + (1 − β i )e1,[qi]+1 |
2
+ T1 (ti )h2+λ + T2 (ti )h2+μ + T3 (ti )h2 + O(h3+γ ).

28
Mathematics 2021, 9, 1856

Letting h be so small, that 2h tiλ L1 ≤ 12 , it is easy to derive

i −1
|e1,i | ≤ A + h ∑ Bj |e1,j |, 0 ≤ i ≤ N,
j =1

where
A = 2| T1 (ti )h2+λ + T2 (ti )h2+μ + T3 (ti )h2 + O(h3+γ ) = O(h2+γ ).
By Lemma 4, we have
max |e1,i | ≤ O(h2+γ ).
1≤ i ≤ N

The proof is complete.

Next, we evaluate the error arising from the iterative process.

Theorem 4. Under the conditions of Theorem 2, yi is the solution of Equation (19) and yi is the
approximate solution of Equation (1), and yi is defined by (21). The absolute error is denoted by
e2,i = |yi − yi |. Assume that h is sufficiently small, then, there exist two positive constants, C1 and
C2 , which are independent of h = N T
, such that

C1 h, [qi ] + 1 = i,
vi ≤
C2 h, [qi ] + 1 ≤ i.

Proof. Subtracting (21) from (19), we have e2,0 = 0. We consider two cases.
(1) The first case is [qi ] + 1 = i (i.e., when i ≤ 1
1−q ). Then, we have

i −1 [qi ]−1
h μ h μ
e2,i = h ∑ tλk L1 e2,k + 2 tiλ L1  + h ∑ tk L2 e2,k + t[qi] L2 e2,[qi]
2
k =1 k =1
qti − t[qi]  μ 
+ t[qi] L2 e2,[qi] + (qti )μ L2 ( β i e2,[qi] + (1 − β i ))
2
i −1 [qi ]−1
μ h μ
≤h ∑ tλk Le2,k + h ∑ tk Le2,k + t[qi] Le2,[qi]
2
(29)
k =1 k =1
h μ  h h 
+ t L + (qti )μ β i e2,[qi] + (qti )μ L(1 − β i ) + tiλ L 
2 [qi] 2 2
i −1 1 
1
=h ∑ Bk e2,k + (qti )μ L(1 − β i ) + tiλ L h.
k =1
2 2

According to Lemma 4, we have e2,i ≤ C1 h.


(2) The second case is [qi ] + 1 ≤ i (i.e., when i > 1
1−q ). Then, we obtain

i −1 [qi ]−1
h μ h μ
e2,i = h ∑ tλk L1 e2,k + 2 tiλ L1  + h ∑ tk L2 e2,k + t[qi] L2 e2,[qi]
2
k =1 k =1
qti − t[qi]  μ 
+ t[qi] L2 e2,[qi] + (qti )μ L2 ( β i e2,[qi] + (1 − β i )e2,[qi]+1 )
2
i −1 [qi ]−1
h μ h μ
≤h ∑ tλk Le2,k + 2 tiλ L + h ∑ tk Le2,k + t[qi] Le2,[qi]
2
(30)
k =1 k =1
h μ 
+ t Le + (qti )μ L( β i e2,[qi] + (1 − β i )e2,[qi]+1 )
2 [qi] 2,[qi]
i −1
1
=h ∑ Bk e2,k + tiλ Lh.
k =1
2

29
Mathematics 2021, 9, 1856

According to Lemma 4, we have e2,i ≤ C2 h.

Theorem 5. Under the conditions of Theorem 2, y(ti ) is the exact solution of Equation (1), yi is
the approximate solution of Equation (1) when t = ti , we have

C1 h + O(h2+γ ), [qi ] + 1 = i,
|y(ti ) − yi | ≤ + γ
C2 h + O(h 2 ), [qi ] + 1 ≤ i.

Proof. By Theorems 3 and 4, the absolute error between y(ti ) and yi has the expression

|y(ti ) − yi | = |y(ti ) − yi + yi − yi |


(31)
≤ |y(ti ) − yi | + |yi − yi |.

We obtain the conclusion of Theorem 5.

6. Extrapolation Method
In this section, we first describe the asymptotic error expansion and then present an
extrapolation technique for achieving high precision. Finally, a posterior error estimate is
derived.

Theorem 6. Let f (t), k1 (t, s; y(s)), k2 (t, s; y(s)) are four times continuously differentiable on
I, D × R, Dθ × R, respectively. Additionally, y(t) has continuous partial derivatives up to 3 on I
and k i (t, s; y(s)) (i = 1, 2) satisfy Lipschitz conditions (2). There exist functions Ŵi (t)(i = 1, 2, 3)
independent of h, such that we have the following asymptotic expansions:

yi = y(ti ) + Ŵ1 (ti )h2+λ + Ŵ2 (ti ))h2+μ + Ŵ3 (ti )h2 + O(h3+γ ), −1 < λ < 0, −1 < μ ≤ 0. (32)

Proof. Assume that {Ŵk (t), k = 1, 2, 3} satisfy the auxiliary delay equations
 t  qt
Ŵk (t) = Wk (t) + sλ k1 (t, s; y(s))Ŵk (s)ds + sμ k2 (t, s; y(s))Ŵk (s)ds,
0 0

and Ŵk (ti ), i = 1, · · · , N satisfy the approximation equations


i −1
Ŵk (ti ) = − ζ (−λ)h1+λ k1 (ti , t0 ; y(t0 ))Ŵk (t0 ) + h ∑ tλk k1 (ti , tk ; y(tk ))Ŵk (tk )
k =1
h  
+ tiλ k1 ti , ti ; y(ti ) Ŵk (ti ) − ζ (−μ)h1+μ k2 (ti , t0 ; y(t0 ))Ŵk (t0 )
2
[qi ]−1
μ h μ (33)
+h ∑ tk k2 (ti , tk ; y(tk ))Ŵk (tk ) + t[qi] k2 (ti , t[qi] ; y(t[qi] )Ŵk (ti )
2
k =1
qti − t[qi]  μ 
+ t[qi] k2 (ti , t[qi] ; y(t[qi] ))Ŵk (t[qi] ) + (qti )μ k2 ti , qti ; β i y(t[qi] )Ŵk (t[qi] )
2

+ (1 − β i )y(t[qi]+1 )Ŵk (t[qi]+1 ) + Wk (ti ).

The analysis procedure is similar to the proof of Theorem 3. We obtain

max |Ŵk (ti ) − W (ti )| ≤ Lh2+γ .


1≤ i ≤ N

Let  
Ei = ei − W1 (ti )h2+λ + W2 (ti )h2+μ + W3 (ti ) h2 .

Then, we obtain

30
Mathematics 2021, 9, 1856

i −1
h
Ei = − ζ (−λ)h1+λ k1 (ti , t0 ; y(t0 )) E0 + h ∑ tλk k1 (ti , tk ; y(tk ))Ek + 2 tiλ k1 (ti , ti ; y(ti ))Ei
k =1
[qi ]−1
μ h μ
− ζ (−μ)h1+μ k2 (ti , t0 ; y(t0 )) E0 + h ∑ tk k2 (ti , tk ; y(tk )) Ek + t[qi] k2 (ti , t[qi] ; y(t[qi] ) E[qi]
k =1
2
qti − t[qi]  μ  
+ t[qi] k2 (ti , t[qi] ; y(t[qi] )) E[qi] + (qti )μ k2 ti , qti ; β i y(t[qi] ) E[qi] + (1 − β i )y(t[qi]+1 ) E[qi]+1 .
2
According to Lemma 4, there exists a constant d such that

max | Ei | ≤ dh3+γ .
1≤ i ≤ N

The asymptotic expansion is

yi = y(ti ) + Ŵ1 (ti )h2+λ + Ŵ2 (ti ))h2+μ + Ŵ3 (ti )h2 + O(h3+γ ).

From Theorem 6, we consider the Richardson extrapolation method to achieve


higher accuracy.
Extrapolation algorithm
Step 1. Assume γ = min (λ, μ) = λ, and halve the step length to obtain
 2+ λ   2+ μ  2   
h h h h h 3+ λ
yi2 = y(ti ) + Ŵ1 (ti ) + Ŵ2 (ti ) + Ŵ3 (ti ) +O . (34)
2 2 2 2

Then, the term Ŵ1 (ti )h2+λ can be removed.


h
22+λ yi2 − yih
y1,h
i = = y(ti ) + Ŵ2 (ti ))h2+μ + Ŵ3 (ti )h2 + O(h3+λ ). (35)
22 + λ − 1

Step 2. To eliminate Ŵ2 (ti ))h2+μ , we apply Richardson h2+μ extrapolation:


  2+ μ  2   
1, 2h h h h 3+ λ
yi = y(ti ) + Ŵ2 (ti ) + Ŵ3 (ti ) +O . (36)
2 2 2

Combining (35) and (36), we have

1, h
22+μ yi 2 − y1,h
y2,h = i
= y(ti ) + Ŵ3 (ti )h2 + O(h3+λ ). (37)
i 22 + μ − 1
A posterior asymptotic error estimate is
h h h h
 h   22+λ y 2 − yh yh − y 2   22+λ y 2 − yih   yh − y 2 
 2     
yi − y(ti ) =  2+λi i
− y(ti ) + 2i+λ i  ≤  2+λi − y(ti ) +  2i+λ i 
2 −1 2 −1 2 −1 2 −1
h
(38)
   yh − y 2 
   i i 
= y1,h − y ( t i )  +   + O ( h2 )
i 22+λ − 1
h
h
yih −yi2
The error yi2 − y(ti ) is bounded by 22+λ −1
, which is important for constructing
adaptable algorithms.

31
Mathematics 2021, 9, 1856

7. Numerical Experiments
In this section, we illustrate the performance and accuracy of the quadrature method
using the improved trapezoid formula. For ease of notation, we define
E 
h
Eh = |y(ti ) − yih |, Ek,i = |y(ti ) − yik,h | (k = 1, 2), Rate = log2 ,
Eh
2

where yih is the approximate solution of Equation (1), yik,h is the approximate solution of
k-th extrapolation, Ek,i is the absolute error between the exact solution and the approximate
solution of k-th extrapolation when t = ti . The procedure was implemented in MATLAB.

Example 1. Consider the following equation


 t  qt
y(t) = f (t) − sλ sin(y(s))ds + (t + s) sin(y(s))ds, t ∈ [0, T ], (39)
0 0

with T = 1, λ = − 12 , and q = 0.95. The exact solution is given by y(t) = t and f (t) is determined
by the exact solution.
Applying the algorithm with N = 24 , 25 , 26 , 27 , 28 , the numerical results at t = 0.4 are
presented in Table 1, the CPU time(s) are 0.34, 0.55, 0.98, 1.62, and 3.01 s, respectively. By
comparing Eh and E1,i , we can observe that the accuracy was improved and the extrapolation
algorithm was effective. In the third column, the rate values show that the convergence order was
consistent with the theoretical analysis.

Table 1. Numerical results at t = 0.4 of Example 1.

N Eh Rate E1,i Posteriori Errors


24 3.32 × 10−4 — — —
25 1.18 × 10−4 21.50 4.70 × 10−6 1.17 × 10−4
26 4.01 × 10−5 21.55 2.04 × 10−6 4.21 × 10−5
27 1.35 × 10−5 21.57 9.90 × 10−7 1.46 × 10−5
28 4.51 × 10−6 21.58 4.21 × 10−7 4.92 × 10−6

Example 2. Consider the following equation


 t  qt
y(t) = f (t) − sλ (t2 + s)(y(s))2 ds + sμ sin(y(s))ds, t ∈ [0, T ], (40)
0 0

where T = 1, λ = μ = − 12 , q = 0.8 and the analytical solution is y(t) = t. Then, f (t) is


determined by the exact solution.
By applying the numerical method for N = 24 , 25 , 26 , 27 , 28 , the obtained results at t = 0.2
are shown in Table 2. By comparing Eh and E1,i , we can observe that the accuracy was improved,
proving that the extrapolation algorithm is effective. The results verified the theoretic convergence
order, which is O(h1.5 ).

Table 2. Numerical results at t = 0.2 of Example 2.

N Eh Rate E1,i Posteriori Errors


24 6.57 × 10−4 — — —
25 2.30 × 10−4 21.51 3.13 × 10−6 2.33 × 10−4
26 8.03 × 10−5 21.52 1.58 × 10−6 8.19 × 10−5
27 2.81 × 10−5 21.52 5.38 × 10−7 2.86 × 10−5
28 9.82 × 10−6 21.52 1.56 × 10−7 9.97 × 10−6

32
Mathematics 2021, 9, 1856

Example 3. We consider the following equation


 t  qt
y(t) = f (t) − sλ (t + s) sin(y(s))ds + sμ (t + s)(y(s))2 ds, t ∈ [0, T ], (41)
0 0

where T = 1, λ = − 13 , μ = − 41 , q = 0.9, and the analytical solution is y(t) = t. Then, f (t) is


determined by the exact solution.
By applying the numerical method for N = 24 , 25 , 26 , 27 , and 28 , the obtained results at
t = 0.4 are shown in Table 3. As λ was not equal to μ, we first applied the Richardson h2+λ
extrapolation, and then adopted the Richardson h2+μ extrapolation. By comparing Eh , E1,i and
E2,i , these results verify the theoretical results, and we can see that the extrapolation improved the
accuracy dramatically. When N = 8, 16, 32, 64, 128, the CPU time(s) are 1.43, 2.41, 3.99, 17.46, and
21.36 s, respectively. The exact solution and the approximation when N=8 are plotted in Figure 1.

Table 3. Numerical results at t = 0.4 of Example 3.

N Eh Rate E1,i E2,i Posteriori Errors


24 9.36 × 10−5 — — — —
25 3.23 × 10−5 21.53 4.12 × 10−6 — 2.82 × 10−5
26 1.06 × 10−5 21.61 6.00 × 10−7 8.89 × 10−7 9.99 × 10−6
27 3.41 × 10−6 21.63 1.17 × 10−7 8.70 × 10−8 3.30 × 10−6
28 1.10 × 10−6 21.65 2.31 × 10−8 1.68 × 10−8 1.07 × 10−6

1.1
N=23
1 exact solution

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2
0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Figure 1. The absolute errors and the approximations when N = 23 .

8. Conclusions
In this paper, by using the improved trapezoidal quadrature formula and linear inter-
polation, we obtained the approximate equation for non-linear Volterra integral equations
with vanishing delay and weak singular kernels. The approximate solutions were obtained
by an iterative algorithm, which possessed a high accuracy order O(h2+γ ). Additionally,
we analyzed the existence and uniqueness of both the exact and approximate solutions.
The significance of this work was that it demonstrated the efficiency and reliability of the
Richardson extrapolation. The computational findings were compared with the exact solu-
tion: we found that our methods possess high accuracy and low computational complexity,
and the results showed good agreement with the theoretical analysis. For future work, we
can apply this method for solving two-dimensional delay integral equations.

Author Contributions: Conceptualization, J.H. and L.Z.; methodology, J.H. and L.Z.; validation,
J.H. and H.L.; writing—review and editing, L.Z. and Y.W. All authors have read and agreed to the
published version of the manuscript.

33
Mathematics 2021, 9, 1856

Funding: This research was funded by the Program of Chengdu Normal University, grant number
CS18ZDZ02.
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: Not applicable.
Acknowledgments: The authors would like to thank the editor and referees for their careful com-
ments and fruitful suggestions.
Conflicts of Interest: The authors declare no conflict of interest.

References
1. Avaji, M.; Hafshejani, J.; Dehcheshmeh, S.; Ghahfarokhi, D. Solution of delay Volterra integral equations using the Variational
iteration method. J. Appl. Sci. 2012, 12, 196–200. [CrossRef]
2. Williams, L.R.; Leggett, R.W. Nonzero Solutions of Nonlinear Integral Equations Modeling Infectious Disease. Siam J. Math. Anal.
1982, 13, 121. [CrossRef]
3. Volterra, V. On some questions of the inversion of definite integrals. (Sopra alcune questioni di inversione di integrali definiti).
Ann. Mat. Pura Appl. 1897, 25, 139–178. [CrossRef]
4. Yang, K.; Zhang, R. Analysis of continuous collocation solutions for a kind of Volterra functional integral equations with
proportional delay. J. Comput. Appl. Math. 2011, 236, 743–752. [CrossRef]
5. Xie, H.; Zhang, R.; Brunner, H. Collocation methods for general Volterra functional integral equations with vanishing delays.
SIAM J. Sci. Comput. 2011, 33, 3303–3332. [CrossRef]
6. Ming, W.; Huang, C.; Zhao, L. Optimal superconvergence results for Volterra functional integral equations with proportional
vanishing delays. Appl. Math. Comput. 2018, 320, 292–301. [CrossRef]
7. Mokhtary, P.; Moghaddam, B.P.; Lopes, A.M.; Machado, J.A.T. A computational approach for the non-smooth solution of
non-linear weakly singular Volterra integral equation with proportional delay. Numer. Algorithms 2019, 83, 987–1006. [CrossRef]
8. Gu, Z.; Chen, Y. Chebyshev spectral-collocation method for a class of weakly singular Volterra integral equations with proportional
delay. J. Numer. Math. 2014, 22, 311–342. [CrossRef]
9. Zhang, R.; Zhu, B.; Xie, H. Spectral methods for weakly singular Volterra integral equations with pantograph delays. Front.
Math. China 2014, 8, 281–299. [CrossRef]
10. Brunner, H. Collocation Methods for Volterra Integral and Related Functional Differential Equations; Cambridge University Press:
Cambridge, UK, 2004.
11. Zhang, G.; Song, M. Impulsive continuous Runge—Kutta methods for impulsive delay differential equations. Appl. Math.
Comput. 2019, 341, 160–173. [CrossRef]
12. Shu, H.; Xu, W.; Wang, X.; Wu, J. Complex dynamics in a delay differential equation with two delays in tick growth with diapause.
J. Differ. Equ. 2020, 269, 10937–10963. [CrossRef]
13. Fang, J.; Zhan, R. High order explicit exponential Runge—Kutta methods for semilinear delay differential equations. J. Comput.
Appl. Math. 2021, 388, 113279. [CrossRef]
14. Song, H.; Xiao, Y.; Chen, M. Collocation methods for third-kind Volterra integral equations with proportional delays. Appl. Math.
Comput. 2021, 388, 125509.
15. Brunner, H. The numerical solution of weakly singular Volterra functional integro-differential equations with variable delays.
Commun. Pure Appl. Anal. 2006, 5, 261–276. [CrossRef]
16. Huang, C. Stability of linear multistep methods for delay integro-differential equations. Comput. Math. Appl. 2008, 55, 2830–2838.
[CrossRef]
17. Sheng, C.; Wang, Z.; Guo, B. An h p -spectral collocation method for nonlinear Volterra functional integro-differential equations
with delays. Appl. Numer. Math. 2016, 105, 1–24. [CrossRef]
18. Abdi, A.; Berrut, J.P.; Hosseini, S.A. The Linear Barycentric Rational Method for a Class of Delay Volterra Integro-Differential
Equations. J. Sci. Comput. 2018, 75, 1757–1775. [CrossRef]
19. Yaghoobi, S.; Moghaddam, B.P.; Ivaz, K. An efficient cubic spline approximation for variable-order fractional differential
equations with time delay. Nonlinear Dyn. 2017, 87, 815–826. [CrossRef]
20. Rahimkhani, P.; Ordokhani, Y.; Babolian, E. A new operational matrix based on Bernoulli wavelets for solving fractional delay
differential equations. Numer. Algorithms 2017, 74, 223–245. [CrossRef]
21. Rakhshan, S.A.; Effati, S. A generalized Legendre—Gauss collocation method for solving nonlinear fractional differential
equations with time varying delays. Appl. Numer. Math. 2019, 146, 342–360. [CrossRef]
22. Zuniga-Aguilar, C.J.; Gomez-Aguilar, J.F.; Escobar-Jimenez, R.F.; Romero-Ugalde, H.M. A novel method to solve variable-order
fractional delay differential equations based in lagrange interpolations. Chaos Solitons Fractals 2019, 126, 266–282. [CrossRef]
23. Xu, X.; Huang, Q. Superconvergence of discontinuous Galerkin methods for nonlinear delay differential equations with vanishing
delay. J. Comput. Appl. Math. 2019, 348, 314–327. [CrossRef]

34
Mathematics 2021, 9, 1856

24. Bellour, A.; Bousselsal, M. A Taylor collocation method for solving delay integral equations. Numer. Algorithms 2014, 65, 843–857.
[CrossRef]
25. Darania, P.; Sotoudehmaram, F. Numerical analysis of a high order method for nonlinear delay integral equations. J. Comput.
Appl. Math. 2020, 374, 112738. [CrossRef]
26. Khasi, M.; Ghoreishi, F.; Hadizadeh, M. Numerical analysis of a high order method for state-dependent delay integral equations.
Numer. Algorithms 2014, 66, 177–201. [CrossRef]
27. Bica, A.M.; Popescu, C. Numerical solutions of the nonlinear fuzzy Hammerstein-Volterra delay integral equations. Inf. Sci.
2013, 223, 236–255. [CrossRef]
28. Mosleh, M.; Otadi, M. Least squares approximation method for the solution of Hammerstein-Volterra delay integral equations.
Appl. Math. Comput. 2015, 258, 105–110. [CrossRef]
29. Zhang, X. A new strategy for the numerical solution of nonlinear Volterra integral equations with vanishing delays. Appl. Math.
Comput. 2020, 365, 124608.
30. Lima, P.; Diogo, T. Numerical solution of a nonuniquely solvable Volterra integral equation using extrapolation. J. Comput. Appl.
Math. 2002, 140, 537–557. [CrossRef]
31. Sidi, A. Practical Extrapolation Methods; Cambridge University Press: Cambridge, UK, 2003.
32. Tao, L.; Jin, H. High Precision Algorithm for Integral Equations; China Science Press: Beijing, China, 2013.
33. Tao, L.; Yong, H. Extrapolation method for solving weakly singular nonlinear Volterra integral equations of the second kind. J.
Math. Anal. Appl. 2006, 324, 225–237. [CrossRef]
34. Navot, J. A Further Extension of the Euler-Maclaurin Summation Formula. J. Math. Phys. 1962, 41, 155–163. [CrossRef]
35. Tao, L.; Yong, H. A generalization of Gronwall inequality and its application to weakly singular Volterra integral equation of the
second kind. J. Math. Anal. Appl. 2003, 282, 56–62. [CrossRef]
36. Qin, Y. Integral and Discrete Inequalities and Their Applications; Springer International Publishing: Berlin/Heidelberg, Germany,
2016.

35
mathematics

Article
An Iterative Algorithm for Approximating the Fixed Point of a
Contractive Affine Operator
María Isabel Berenguer 1,2, *,† and Manuel Ruiz Galán 1,2,†

1 Department of Applied Mathematics, E.T.S. Ingeniería de Edificación, University of Granada,


18071 Granada, Spain; [email protected]
2 Institute of Mathematics (IMAG), University of Granada, 18071 Granada, Spain
* Correspondence: [email protected]
† These authors contributed equally to this work.

Abstract: First of all, in this paper we obtain a perturbed version of the geometric series theorem,
which allows us to present an iterative numerical method to approximate the fixed point of a
contractive affine operator. This result requires some approximations that we obtain using the
projections associated with certain Schauder bases. Next, an algorithm is designed to approximate
the solution of Fredholm’s linear integral equation, and we illustrate the behavior of the method with
some numerical examples.

Keywords: iterative numerical methods; Schauder bases; Fredholm integral equation

MSC: 65R20; 46B15; 45B05

1. Introduction
The idea of iterative numerical methods is, given a complete metric space X (typically
a Banach space) and a contractive operator T : X −→ X, or at least one which guarantees
Citation: Berenguer, M.I.; Ruiz Galán, the convergence of the Picard iterates, to construct a sequence of approximations of the
M. An Iterative Algorithm for fixed point of that operator x0 = T ( x0 ). The calculation of the Picard iterates is not generally
Approximating the Fixed Point of a easy or even feasible, so several methods which allow us to approximate the elements of
Contractive Affine Operator. the Picard sequence have been proposed. Therefore, a part of the Picard-type iterative
Mathematics 2022, 10, 1012. https:// algorithms are focused on determining, for an element x ∈ X, a value close to T ( x ) and in
doi.org/10.3390/math10071012 this way, successively approximating the iterates. The numerical techniques used are very
Academic Editor: Ioannis K. Argyros
diverse, and the resulting algorithms have numerous applications. Proof of all this are the
recent references [1–16].
Received: 27 February 2022 However, our approach here is completely different: given x, instead of approximating
Accepted: 18 March 2022 successively T ( x ), T 2 ( x ), T 3 ( x ), . . . , which necessarily involves an accumulation of errors,
Published: 22 March 2022 in this paper, we approximate directly T n ( x ) by means of the use of suitable Schauder bases,
Publisher’s Note: MDPI stays neutral transforming it into a simple calculation which, for example, does not involve the resolution
with regard to jurisdictional claims in of systems of algebraic equations or the use of any quadrature formulae because simply
published maps and institutional affil- linear combinations of certain values associated with the operator are calculated. What
iations. is more, motivated by its application for the numerical resolution of the linear Fredholm
integral equation, the operator T is considered to be affine and continuous. This affine
and continuous nature means that, instead of using a fixed-point language, we opted for
resorting to an equivalent version using the geometric series theorem, and more specifically,
Copyright: © 2022 by the authors. our first contribution is to obtain a perturbed version of the same which is susceptible
Licensee MDPI, Basel, Switzerland.
to presenting approximations by means of certain Schauder bases related to the operator.
This article is an open access article
Such an approximation will imply a low computational cost as mentioned above. Thus,
distributed under the terms and
we are going to design an iterative-type algorithm which allows the approximation of the
conditions of the Creative Commons
fixed point of a suitable continuous affine operator.
Attribution (CC BY) license (https://
As we have mentioned, the application that we are presenting consists of a numerical
creativecommons.org/licenses/by/
4.0/).
algorithm to solve the linear Fredholm integral equation, which is chosen for its great versatility.

Mathematics 2022, 10, 1012. https://doi.org/10.3390/math10071012 https://www.mdpi.com/journal/mathematics


37
Mathematics 2022, 10, 1012

The structure of this paper is as follows. In Section 2 we establish an analytical–


numerical result, which provides us with an approximation of the fixed point of a suitable
continuous affine operator in a Banach space. To continue, Section 3 interprets the previous
result in terms of an algorithm when a Schauder basis is introduced into the considered
space. Section 4 derives a specific algorithm in the case of the linear Fredholm integral
equation in two distinct contexts. Next, Section 5 shows some illustrative examples of
equations or a classic model of electrostatics (Love’s equation), and finally, Section 6 rounds
up with some conclusions.

2. Approximating Fixed Points of Affine Operators


The following result provides us with an approximation of the fixed point of a suitable
continuous affine operator, as well as an estimation of the error. It addresses a version of
the geometric series theorem, which we can label as perturbed: it presents the possibility
of converting the precise calculations into approximate ones, in exchange for making the
calculations possible.
Before establishing this, we present some standard notation. Given a (real) Banach
space X, L( X ) will denote the Banach space (usual operator norm) of those bounded and
linear operators from X to X. For T ∈ L( X ) and n ∈ N, T n denotes the power operator
n times

T ◦ · · · ◦ T, while T 0 = I, the identity map on X.

Theorem 1. Let X be a Banach space, y ∈ X and L ∈ L( X ) with L < 1, and consider the
continuous affine operator A : X −→ X defined by

Ax := y + Lx, ( x ∈ X ).

Let y0 ∈ X, n ∈ N and L0 , L1 , . . . , Ln ∈ L( X ). Then, the equation Ax = x has a unique solution


x • ∈ X and
   
 n  n
1 − L n +1 L n +1
 •
 ∑ L j y0 − x  ≤ ∑ L j y0 − L j y0 + y0 − y + y .
 j =0  j =0 1− L 1− L

Proof. Let us first observe that, according to the geometric series theorem, there exists a
unique solution x • ∈ X for the equation Ax = x,

x • = ( I − L)−1 y,

which satisfies for any k ∈ N,


 
 k  L k +1
 •
∑ L j
y − x ≤ y .
 j =0  1− L

Therefore,
       
 n   n n   n n   n 
 •  j   j   •
 ∑ L j y0 − x  ≤  ∑ L j y0 − ∑ L y0  +  ∑ L y0 − ∑ L y  +  ∑ L y − x 
j j
 j =0   j =0 j =0
  j =0 j =0
  j =0 

n   n
L n +1
 
≤ ∑ L j y0 − L j y0  + ∑ L j
y0 − y +
1− L
y
j =0 j =0

n    1 − L n +1  L n +1
 j 
= ∑ j 0L y − L y 0 +
1− L
y0 − y +
1− L
y ,
j =0

as announced.

38
Mathematics 2022, 10, 1012

It is worth mentioning that when y0 = y and for all j = 0, 1, . . . , m, we have that


L j = L j , and we recover a well-known algorithm associated with the geometric series
theorem. However, iterative procedures such as this, used to involve difficult and even
impossible calculations from a practical perspective, so the idea behind this theorem is
to choose the operators L0 , L1 , . . . , Ln in such a way that L0 y0 , L1 y0 , Ln y0 are not only
calculable, but also have a low computational cost. In addition, if y0 represents an
approximation of y—normally due to a certain type of error—the previous result shows
how y0 influences the final approximation. Finally, we can obtain an approximation for x •
for some adequate n ∈ N, and for each j = 0, 1, . . . , n, L j y0 is close to L j y0 . More specifically:

Corollary 1. Suppose that X is a Banach space, L ∈ L( X ) and L < 1, y ∈ X, and that


A : X −→ X is the continuous and affine operator A(·) := y + L(·), whose unique fixed point is
denoted by x • ∈ X. Additionally, assume that for some y0 ∈ X, n ∈ N, L0 , L1 , . . . , Ln ∈ L( X )
and ε, ε 0 , ε n > 0, we have that
n
ε
∑ εj < 2,
j =0

j = 0, . . . , n ⇒ L j y0 − L j y0 < ε j , (1)
and that  
1 − L n +1 L n +1 ε
y0 − y + y < . (2)
1− L 1− L 2
Then  
 n 
 •
 ∑ L j y0 − x  < ε.
 j =0 

Obviously, (2) is valid as soon as n is large enough and y0 − y is small. For condition (1),
we present some analytical tools in the next section.

3. Numerical Ideas behind the Algorithm for the Equation y + Lx = x


In view of Corollary 2.2 and under its hypotheses, we can approximate the fixed point
x • of A by a series close to the geometric one:

 n
 y0
 Ly0 L2 y0 ... L n y0  ∑ L j y0 −−−−→
(n→∞)
x•
 j =0
y0 ∈ X   n n .

 L0 y0

L1 y0 L2 y0 ... L n y0  ∑ L j y0 ≈ ∑ L j y0
j =0 j =0

In order to derive
n n
∑ L j y0 ≈ ∑ L j y0
j =0 j =0

an approximation as that given in (1) is required. To this end, a possible tool appears
provided by the Schauder bases, since they give an explicit linear approximation of any
element of a Banach space by means of the associated projections, which is compatible with
the continuity and affinity of the operator. What is more, in the case of classic bases, we
easily obtain approximations of (the linear part of) A and its powers.
Thus, before continuing, we revise some of the basic notions of Schauder bases that
we are going to need in the design of our algorithm. A sequence {e j } j∈N in a Banach space
X is a Schauder basis if all the element x ∈ X can be uniquely represented as

x= ∑ αj ej ,
j =1

39
Mathematics 2022, 10, 1012

for a sequence of real {α j } j∈N . If we define for each j ∈ N the linear operator Pj : X −→ X,
known as the j-th projection associated with the basis, as

j
Pj x := ∑ αk ek ,
k =0

for such an x, it is easy to prove, as a consequence of the Baire lemma, that it is a continuous
operator and, in view of the representation of x in terms of the elements of the basis,

lim Pj x − x = 0.
j→∞

With the aid of a Schauder basis, we can approximate Lx with L( Pj x ) which, on


occasion, is easy to calculate. To summarize all of this, we focus on a type of affine equation,
linear Fredholm integral equations, although this is the objective of the following section.

4. Algorithm to Approximate the Solution of a Linear Fredholm Integral Equation


In the rest of this paper, we focus our efforts on realizing everything that we explained
thus far in order to address the study of a specific problem, the numerical resolution of a
linear Fredholm integral equation, in two distinct settings.
Let X = C [ a, b] or X = L p [ a, b], (1 < p < ∞), k ∈ C [ a, b]2 or k ∈ L∞ [ a, b]2 , respectively,
and y ∈ X. Then we consider the corresponding linear Fredhlom integral equation
 b
x (t) = y(t) + k(t, s) x (s)ds, (3)
a

where x ∈ X is the unknown function. In view of the previous results, we consider the
continuous and linear operator L : X −→ X defined at each y0 ∈ X as
 b
Ly0 := k (·, s)y0 (s)ds.
a

Then, given j ∈ N,
 b  b 
L j y0 = ··· k (·, t1 )k(t1 , t2 ) · · · k (t j−1 , t j )y0 (t j )dt j · · · dt1 .
a a

From now on, in both cases (X = C [ a, b] or X = L p [ a, b]), we assume that

k (b − a) < 1,

since such a condition is sufficient for the validity of L < 1 and it is very easy to check.
(d)
Furthermore, for each d ∈ N, we fix a Schauder basis {e j } j∈N in C [ a, b]d (if X = C [ a, b]) or
(d)
in L p [ a, b]d (if X = L p [ a, b]) and we denote the projections in this basis as { Pj } j∈N .
With all of this, we are now ready to define the approximate operators L j : for each
x ∈ X and j ∈ N, we take

Φ j ( x )(t, t1 , . . . , t j ) := k(·, t1 )k (t1 , t2 ) · · · k(t j−1 , t j )y0 (t j )

and fixed on r j ∈ N, thus L j : X −→ X is given as


 b  b 
( j +1)
L j y0 : = ··· Pr j (Φ j (y0 )(·, t1 , . . . , t j )dt j · · · dt1 . (4)
a a

Now we can apply the corollary 1 since without going any further, each r j is big
enough, L j x − L j x < ε j .

40
Mathematics 2022, 10, 1012

Corollary 2. For any ε > 0 and y0 ∈ X, there are natural numbers n and r0 , . . . , rn in such a way
that if x • is the unique solution to the linear Fredholm integral Equation (3), then
 
 n 
 •
 ∑ L j y0 − x  < ε,
 j =0 

where L0 = I and for each j ≥ 1, the operator L j is defined by (4).

Thus, we have established the following (Algorithm 1)

Algorithm 1: Algorithm for approximating the solution of the linear Fredholm


integral equation.
(d)
Choose y0 , k, n, ε, ε 0 , . . . , ε n , r0 , . . . , rn ∈ N, and {ei }i∈N , d = 1, . . . , n + 1;
L0 ← I;
j ← 1;  
 n 
 •
while  ∑ L j y0 − x  ≥ ε and j ≤ n
 j =0 
Φ j (y0 )(t, t1 , . . . , t j ) ← k(·, t1 )k(t1 , t2 ) · · · k(t j−1 , t j )y0 (t j );
 b  b 
( j +1)
L j y0 ← ··· Pr j (Φ j (y0 )(·, t1 , . . . , t j )dt j · · · dt1 ;
a a
j ← j + 1; end (while)
n
sol− approx ← ∑ L j y0 .
j =0

Observe that
sol− approx − x • < ε
(d)
and that for an appropriate choice of the bases {e j } j∈N , the calculations are immediate, as
justified below.
Returning to the considered spaces in order to study the linear Fredholm integral
equation, X = C [ a, b] or X = L p [ a, b], we remember how it is possible to tensorially
(d)
construct bases {e j } j∈N in X = C [ a, b]d or X = L p [ a, b]d , respectively, from a basis
(1)
{e j } j∈N in the aforementioned spaces.
Specifically, given d ∈ N, d ≥ 2, we consider in Nd the square ordering introduced
in [17] in a inductive form: for d ≥ 2, (1, 1), (1, 2), (2, 2), (2, 1), (1, 3), (2, 3), (3, 3), (3, 2), . . . ,
and given the ordering o1 , o2 ,. . . of Nd−1 , the order in Nd is (o1 , 1), (o1 , 2), (o2 , 2), (o2 , 1),
(o1 , 3), (o2 , 3), (o3 , 3), . . . . Graphically,

( o1 , 1) / ( o1 , 2) ( o1 , 3) ( o1 , 4)

  
( o2 , 1) o ( o2 , 2) ( o2 , 3) ( o2 , 4)

 
( o3 , 1) o ( o3 , 2) o ( o3 , 3) ( o3 , 4)


······ o ( o4 , 3) o ( o4 , 4)

Thus, we establish a bijection τ : N −→ Nd , that for each j ∈ N a d-upla is assigned in


the form
τ ( j ) = ( α1 , . . . , α d )

41
Mathematics 2022, 10, 1012

and for such a j, we define

(d)
e j ( t 1 , . . . , t d ) : = e α1 ( t 1 ) · · · e α d ( t d ), ((t1 , . . . , td ) ∈ [ a, b]d ). (5)

The usual Schauder basis in C [ a, b] is the Faber–Schauder system, and in L p [ a, b] is the


Haar system [18]. More specifically, and assuming without loss of generality that a = 0 and
b = 1, for the Faber–Schauder system, we start from the nodes {t j } j∈N , which are the points
(1)
of [ a, b] arranged dyadically, and the basis functions {e j } j∈N are continuous piecewise
linear functions, the so-called hat functions, satisfying for each j ∈ N

(1)
e j (t j ) = 1

and
(1)
1 ≤ k < n ⇒ ek (t j ) = 0.
On the other hand, if A is a non-empty subset of [0, 1] and δA : [0, 1] −→ R is the
function defined in each 0 ≤ t ≤ 1 as
!
1, if t ∈ A
δA (t) :=
0, if t ∈
/A

and ϕ : [0, 1] −→ R is the function such that in each 0 ≤ t ≤ 1

ϕ(t) := δ[0,0.5) (t) − δ[0.5,1] (t),

then the Haar system is given by


(1)
e1 : = 1
and for j ≥ 2, written uniquely as j = 2k + r + 1, with k = 0, 1, . . . and r = 0, 1, . . . , 2k − 1,

(1)
e j (·) := ϕ(2k (·) − r ).

In both cases, the tensorial sequences defined as (5) constitute Schauder bases in
their respective spaces, C [ a, b]d and L p [ a, b]d [17,19]. However, what really makes these
bases useful when they are used in our Algorithm 1 is precisely that the calculation of
(d)
the approximate operators L j is very easy, since the basis functions e j are of separate
variables and each factor is immediately integrable. Let us mention that these Schauder
bases allow us to preserve the linearity of the convergence that it is guaranteed by the series
geometric theorem.

5. Numerical Examples
We now show the numerical results obtained in several specific examples. Beforehand,
let us mention that the reordering of a finite number of Schauder basis elements produces
another new Schauder basis, which could be interesting from a computational point of
view. Thus, for each r ∈ N, we reordered the bases of C [ a, b]d and L p [ a, b]d so that the r d
first elements correspond to (α1 , α2 , . . . , αd ) being 1 ≤ αi ≤ r. For these reordered bases,
(d) (d)
we maintain the same previous notation, {e j } j∈Nd for the basis and { Pj } j∈N for the
sequence of projections. Furthermore, given n, r ∈ N, we write
n
x (n,r) := ∑ L j y0 ,
j =0

j + 1 times
  
where the indices r j involved in the definition of L j are given by τ (r j ) = ( r, . . . , r ).

42
Mathematics 2022, 10, 1012

In each example, we consider y0 = y since another choice of y0 is rather more


theoretical, and since as we have indicated previously, it addresses the function y when
some kind of error is produced in this function. Calculations were obtained by means of
the Mathematica 12 software.

Example 1. We consider the equation of the Example 1 in [10]:


 1
30πt − sin(πt) 1
x (t) = + t cos(πts2 ) x (s) ds
15 15 0

whose solution is x • (t) = 2πt.


The errors obtained with our method are comparable to the order of those obtained in the
reference taking m = 4 and p = 2, as shown in Table 1. The advantage in our case is that it is not
necessary to start with an approximate solution “close enough” to the exact solution and it is not
necessary either solve any system of linear equations.

Table 1. x • − x (n,r) for Example 1 using the usual basis in C [0, 1].

n r=9 r = 17 r = 33
1 0.00435474 0.00216401 0.00162257
2 0.00300495 0.000773636 0.000210816
3 0.00298439 0.000752736 0.00018924
4 0.00298394 0.000752272 0.000188771

Example 2. The following equation is also extracted from the same reference (Example 2, [10]):
 1
1
x ( t ) = t2 − t + 1 + ets x (s) ds.
4 0

As in the referenced paper, since the solution of this equation is not known, we consider the operator
F : C [0, 1] → C [0, 1] given by
 1
1
F ( x )(t) = x (t) − t2 + t − 1 − ets x (s)ds
4 0

and we show F ( x (n,r) ) for different values of n and r in Table 2.


The errors obtained are similar to those reported in Table 2 of [10] but with the same advantage
mentioned above.

Table 2. F ( x (n,r) ) for Example 2 using the usual basis in C [0, 1].

n r=9 r = 17 r = 33
1 0.122201 0.123111 0.123339
2 0.0399626 0.0413395 0.0416839
3 0.0120328 0.0136133 0.0140084
4 0.00258589 0.00422851 0.0046442

Example 3. The following equation,


 1
2t2 − 1 2 t 1
x (t) = + e ( t − 1) + t3 ets x (s)ds,
3 3 3 0

is taken from [20], Example 3. Its solution is x • (t) = t2 − 1. See Table 3 for the error generated by
Algorithm 1.

43
Mathematics 2022, 10, 1012

Table 3. x • − x (n,r) for Example 3 using the usual basis in C [0, 1].

n r=9 r = 17 r = 33
1 0.0627248 0.0593289 0.0584785
2 0.0133747 0.0104771 0.00975651
3 0.00503508 0.00242016 0.00156699
4 0.00362812 0.00109348 0.000463489

Example 4. This is a standard test problem, and it arises in electrostatics (see [21]) where it is
called Love’s equation.

δ 1 x (s)
x (t) = y(t) + ds.
π 0 δ2 + ( t − s )2
1
We consider δ = −1 and y(t) = 1 + (arctg(1 − t) + arctg(t)) as in Example 3.2 of [22]. In this
π
case, the exact solution is x • (t) = 1.
The errors—see Tables 4 nad 5—are similar to those obtained by the Haar wavelet method and
rationalized Haar functions method (see Table 1 in [22]), although their computation requires to
solve some high-order systems of linear equations.

Table 4. x • − x (n,r) for Example 4 using the usual basis in C [0, 1].

n r=9 r = 17 r = 33
1 0.0819571 0.0825347 0.0826789
2 0.0241024 0.0234044 0.023227
3 0.0059927 0.00634808 0.00645712
4 0.00299427 0.00211501 0.0018917

Table 5. x • − x (n,r) for Example 4 using the usual basis in L2 [0, 1].

n r=8 r = 16 r = 32
1 0.0785242 0.0785241 0.0785217
2 0.0235324 0.0235323 0.0235264
3 0.0101296 0.0101295 0.0101140
4 0.0083851 0.0083850 0.0083669

Example 5. Now considering Example 2 of [23] which has solution x • (t) = sin(2πt)
 1
x (t) = sin(2πt) + (t2 − t − s2 − s) x (s)ds.
0

We observe that the numerical results obtained with our method (Table 6) significantly improve
those obtained in the reference.

Table 6. x • − x (n,r) for Example 5 using the usual basis in L2 [0, 1].

n r=8 r = 16 r = 32
4 3.17949 × 10−11 4.33093 × 10−11 4.28502 × 10−9

6. Conclusions
In this paper, we present an algorithm for iteratively approximating the fixed point
of a continuous coercive affine operator. Its design is based on a perturbed version of the
classic geometric series theorem, the error control that this provides, and the use of certain
Schauder bases. All of this is illustrated for a wide group of affine problems, the linear
Fredholm integral equations. The low computational cost that our algorithm entails makes
it particularly efficient. All of this is illustrated by several examples. We consider that

44
Mathematics 2022, 10, 1012

future research could be focused on extending the algorithm to solve different types of
integral and even integro-differential equations.

Author Contributions: Conceptualization, M.I.B. and M.R.G.; methodology, M.I.B. and M.R.G.;
software M.I.B. and M.R.G.; validation, M.I.B. and M.R.G.; formal analysis, M.I.B. and M.R.G.;
investigation, M.I.B. and M.R.G.; writing—original draft preparation, M.I.B. and M.R.G.; writing—
review and editing, M.I.B. and M.R.G.; supervision, M.I.B. and M.R.G. All authors have read and
agreed to the published version of the manuscript.
Funding: This research was partially supported by Junta de Andalucía, Project “Convex and
numerical analysis”, reference FQM359, and by the “María de Maeztu” Excellence Unit IMAG,
reference CEX2020-001105-M, funded by MCIN/AEI/10.13039/501100011033/.
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: Not applicable.
Conflicts of Interest: The authors declare no conflict of interest.

References
1. Alguliyev, R.M.; Sukhostat, L.V. Efficient algorithm for big data clustering on single machine. CAAI Trans. Intell. Technol. 2020, 5,
9–14. [CrossRef]
2. Alipour, S.; Mirzaee, F. An iterative algorithm for solving two dimensional nonlinear stochastic integral equations: A combined
successive approximations method with bilinear spline interpolation. Appl. Math. Comput. 2020, 371, 124947. [CrossRef]
3. Asgari, Z.; Toutounian, F.; Babolian, E.; Tohidi, E. LSMR iterative method for solving one- and two-dimensional linear Fredholm
integral equations. Comput. Appl. Math. 2019, 38, 135. [CrossRef]
4. Berenguer, M.I.; Gámez, D. Projected Iterations of Fixed-Point Type to Solve Nonlinear Partial Volterra Integro-Differential
Equations. Bull. Malays. Math. Sci. Soc. 2020, 43, 4431–4442. [CrossRef]
5. Berenguer, M.I.; Garralda-Guillem, A.I.; Ruiz Galán, M. An approximation method for solving systems of Volterra integro-
differential equations. Appl. Numer. Math. 2013, 67, 126–135. [CrossRef]
6. Cominola, A.; Giuliania, M.; Pigab, D.; Castellettia, A.; Rizzoli, A.E. A Hybrid Signature-based Iterative Disaggregation algorithm
for Non-Intrusive Load Monitoring. Appl. Energy 2017, 185, 331–344. [CrossRef]
7. Derviskadic, A.; Romano, P.; Paolone, M. Iterative-Interpolated DFT for Synchrophasor Estimation: A Single Algorithm for P-
and M-Class Compliant PMUs. IEEE Trans. Instrum. Meas. 2018, 67, 547–558. [CrossRef]
8. Ding, F.; Wang, F.; Xua, L.; Wu, M. Decomposition based least squares iterative identification algorithm for multivariate
pseudo-linear ARMA systems using the data filtering. J. Frankl. Inst. 2017, 354, 1321–1339. [CrossRef]
9. Muthuvalu, M.S.; Sulaiman, J. Half-Sweep Arithmetic Mean method with composite trapezoidal scheme for solving linear
Fredholm integral equations. Appl. Math. Comput. 2011, 217, 5442–5448. [CrossRef]
10. Gutiérrez, J.M.; Hernández-Verón, M.A.; Martínez, E. Improved iterative solution of linear fredholm integral equations of second
kind via inverse-free iterative schemes. Mathematics 2020, 8, 1747. [CrossRef]
11. Karimi, S.; Jozi, M. A new iterative method for solving linear Fredholm integral equations using the least squares method. Appl.
Math. Comput. 2015, 250, 744–758. [CrossRef]
12. Qin, X.; Cho, S.Y.; Wang, L. Strong convergence of an iterative algorithm involving nonlinear mappings of nonexpansive and
accretive type. Optimization 2018, 67, 1377–1388. [CrossRef]
13. Rabbani, M.; Das, A.; Hazarika, B.; Arab, R. Existence of solution for two dimensional nonlinear fractional integral equation by
measure of noncompactness and iterative algorithm to solve it. J. Comput. Appl. Math. 2020, 370, 112654. [CrossRef]
14. Ray, S.S.; Sahu, P.K. Numerical Methods for Solving Fredholm Integral Equations of Second Kind. Abstr. Appl. Anal. 2013, 2013,
426916. [CrossRef]
15. Yao, Y.; Postolache, M.; Yao, J.C. An iterative algorithm for solving generalized variational inequalities and fixed points problems.
Mathematics 2019, 7, 61. [CrossRef]
16. Qin, X.; Petruşel, A.; Yao, J.C. CQ iterative algorithms for fixed points of nonexpansive mappings and split feasibility problems in
Hilbert spaces. J. Nonlinear Convex Anal. 2018, 19, 157–165.
17. Gelbaum, B.; de Lamadrid, J.G. Bases of tensor products of Banach spaces. Pac. J. Math. 1961, 11, 1281–1286. [CrossRef]
18. Lindenstrauss, J.; Tzafriri, L. Classical Banach Spaces; Lecture Notes in Mathematics 338; Springer: Berlin/Heidelberg, Germany;
New York, NY, USA, 1973.
19. Pap, E. (Ed.) Handbook of Measure Theory; Elsevier: Amsterdam, The Netherland, 2002; Volume II.
20. Gutiérrez, J.M.; Hernández-Verón, M.A. A Picard-Type Iterative Scheme for Fredholm Integral Equations of the Second Kind.
Mathematics 2021, 9, 83. [CrossRef]
21. Love, E. The electrostatic field of two equal circular conducting disks. Q. J. Mech. Appl. Math. 1949, 2, 420–451. [CrossRef]

45
Mathematics 2022, 10, 1012

22. Amiri, S.; Hajipour, M.; Baleanu, D. On accurate solution of the Fredholm integral equations of the second kind. Appl. Numer.
Math. 2020, 150, 478–490. [CrossRef]
23. Maleknejad, K.; Yousefi, M. Numerical solution of the integral equation of the second kind by using wavelet bases of Hermite
cubic splines. Appl. Math. Comput. 2006, 183, 134–141. [CrossRef]

46
mathematics

Article
Gradient-Based Optimization Algorithm for Solving Sylvester
Matrix Equation
Juan Zhang 1, * and Xiao Luo 2

1 Key Laboratory of Intelligent Computing and Information Processing of Ministry of Education,


Xiangtan University, Xiangtan 411105, China
2 Hunan Key Laboratory for Computation and Simulation in Science and Engineering, Xiangtan University,
Xiangtan 411105, China; [email protected]
* Correspondence: [email protected]; Tel.: +86-131-0722-4973

Abstract: In this paper, we transform the problem of solving the Sylvester matrix equation into an
optimization problem through the Kronecker product primarily. We utilize the adaptive accelerated
proximal gradient and Newton accelerated proximal gradient methods to solve the constrained
non-convex minimization problem. Their convergent properties are analyzed. Finally, we offer
numerical examples to illustrate the effectiveness of the derived algorithms.

Keywords: Sylvester matrix equation; Kronecker product; adaptive accelerated proximal gradient
method; Newton-accelerated proximal gradient method

MSC: 15A24; 65F45

1. Introduction
Citation: Zhang, J.; Luo, X.; Matrix equations are ubiquitous in signal processing [1], control theory [2], and linear
Gradient-Based Optimization systems [3]. Most time-dependent models accounting for the prediction, simulation, and
Algorithm for Solving Sylvester control of real-world phenomena may be represented as linear or nonlinear dynamical
Matrix Equation. Mathematics 2022, systems. Therefore, the relevance of matrix equations within engineering applications
10, 1040. https://doi.org/10.3390/ largely explains the great effort put forth by the scientific community into their numeri-
math10071040 cal solution. Linear matrix equations have an important role in the stability analysis of
Academic Editors: Maria Isabel linear dynamical systems and the theoretical development of the nonlinear system. The
Berenguer and Manuel Ruiz Galán Sylvester matrix equation was first proposed by Sylvester and produced from the research
of relevant fields in applied mathematical cybernetics. It is a famous matrix equation that
Received: 14 February 2022
occurs in linear and generalized eigenvalue problems for the computation of invariant
Accepted: 22 March 2022
subspaces using Riccati equations [4–6]. The Sylvester matrix equation takes part in linear
Published: 24 March 2022
algebra [7–9], image processing [10], model reduction [11], and numerical methods for
Publisher’s Note: MDPI stays neutral differential equations [12,13].
with regard to jurisdictional claims in We consider the Sylvester matrix equation of the form
published maps and institutional affil-
iations. AX + XB = C, (1)

where A ∈ Rm×m , B ∈ Rn×n , C ∈ Rm×n are given matrices, and X ∈ Rm×n is an unknown
matrix to be solved. We discuss a special form of the Sylvester matrix equation, in which A
Copyright: © 2022 by the authors. and B are symmetric positive definite.
Licensee MDPI, Basel, Switzerland. Recently, there has been a lot of discussion on the solution and numerical calculation
This article is an open access article of the Sylvester matrix equation. The standard methods for solving this equation are the
distributed under the terms and Bartels–Stewart method [14] and the Hessenberg–Schur method [15], which are efficient
conditions of the Creative Commons
for small and dense system matrices. When system matrices are small, the block Krylov
Attribution (CC BY) license (https://
subspace methods [16,17] and global Krylov subspace methods [18] are proposed. These
creativecommons.org/licenses/by/
methods use the global Arnoldi process, block Arnoldi process, or nonsymmetric block
4.0/).

Mathematics 2022, 10, 1040. https://doi.org/10.3390/math10071040 https://www.mdpi.com/journal/mathematics


47
Mathematics 2022, 10, 1040

Lanczos process to produce low-dimensional Sylvester matrix equations. More feasible


methods for solving large and sparse problems are iterative methods. When system
matrices are large, there are some effective methods such as the alternating direction implicit
(ADI) method [19], global full orthogonalization method, global generalized minimum
residual method [20], gradient-based iterative method [21], and global Hessenberg and
changing minimal residual with Hessenberg process method [22]. When system matrices
are low-rank, the ADI method [23], block Arnoldi method [17], preconditioned block
Arnoldi method [24], and extended block Arnoldi method [25] and its variants [26,27],
including the global Arnoldi method [28,29] and extended global Arnoldi method [25], are
proposed to obtain the low-rank solution.
The adaptive accelerated proximal gradient (A-APG) method [30] is an efficient nu-
merical method for calculating the steady states of the minimization problem, motivated
by the accelerated proximal gradient (APG) method [31], which has wide applications in
image processing and machine learning. In each iteration, the A-APG method takes the step
size by using a line search initialized with the Barzilai–Borwein (BB) step [32] to accelerate
the numerical speed. Moreover, as the traditional APG method is proposed for the convex
problem and its oscillation phenomenon slows down the convergence, the restart scheme
has been used for speeding up the convergence. For more details, one can refer to [30] and
the references therein.
The main contribution is to study gradient-based optimization methods such as the
A-APG and Newton-APG methods for solving the Sylvester matrix equation through
transforming this equation into an optimization problem by using Kronecker product. The
A-APG and Newton-APG methods are theoretically guaranteed to converge to a global
solution from an arbitrary initial point and achieve high precision. These methods are
especially efficient for large and sparse coefficient matrices.
The rest of this paper is organized as follows. In Section 2, we transform this equation
into an optimization problem by using the Kronecker product. In Section 3, we apply
A-APG and Newton-APG algorithms to solve the optimization problem and compare
them with other methods. In Section 4, we focus on the convergence analysis of the A-
APG method. In Section 5, the computational complexity of these algorithms is analyzed
exhaustively. In Section 6, we offer corresponding numerical examples to illustrate the
effectiveness of the derived methods.
Throughout this paper, let Rn×m be the set of all n × m real matrices. In is the identity
matrix of order n. If A ∈ Rn×n , the symbols A T , A−1 , A and tr ( A) express the transpose,
the inverse, the 2-norm, and the trace of A, respectively. The inner product in matrix space
E is  x, y = tr ( x, y), ∀ x, y ∈ E.

2. The Variant of an Optimization Problem


In this section, we transform the Sylvester equation into an optimization problem. We
recall some definitions and lemmas.

Definition 1. Let Y = (yij ) ∈ Rm×n , Z ∈ R p×q , the Kronecker product of Y and Z be defined by
⎡ ⎤
y11 Z y12 Z ··· y1n Z
⎢ y21 Z y22 Z ··· y2n Z ⎥
⎢ ⎥
Y⊗Z =⎢ . .. .. .. ⎥.
⎣ .. . . . ⎦
ym1 Z ym2 Z · · · ymn Z

Definition 2. If Y ∈ Rm×n , then the straightening operator vec : Rm×n −→ Rmn of Y is

vec(Y ) = (y1T , y2T , . . . , ynT ) T .

48
Mathematics 2022, 10, 1040

Lemma 1. Let Y ∈ Rl ×m , Z ∈ Rm×n , W ∈ Rn×k , then

vec(YZW ) = (W T ⊗ Y )vec( Z ).

From Lemma 1, the Sylvester Equation (1) can be rewritten as

( In ⊗ A + B T ⊗ Im )vec( X ) = vec(C ). (2)

Lemma 2. Let A be a symmetric positive matrix; solving the equation Ax = b is equivalent to


obtaining the minimum of ϕ( x ) = x T Ax − 2b T x.

According to Lemma 2 and Equation (2), define

Ā = ( In ⊗ A + B T ⊗ Im ), x̄ = vec( X ), b̄ = vec(C ).

Therefore, Equation (2) should be Ā x̄ = b̄. Obviously, if A and B are symmetric


positive, then Ā is symmetric positive. The variant of the Sylvester Equation (2) reduces to
the optimization problem:
 
min ϕ( x ) = min x̄ T Ā x̄ − 2b̄ T x̄
 
= min vec( X ) T ( In ⊗ A + B T ⊗ Im )vec( X ) − 2vec( X ) T vec(C )
  (3)
= min vec( X ) T · vec( AX ) + vec( X ) T · vec( XB) − 2vec( X ) T · vec(C )
 
= min tr ( X T AX ) + tr ( X T XB) − 2tr ( X T C ) .

Using the calculation of the matrix differential from [33], we have the following
propositions immediately.

∂tr ( A T X ) ∂tr ( X T A)
Proposition 1. If A = ( aij ) ∈ Rm×n , X = ( xij ) ∈ Rm×n , then ∂X = ∂X = A.

∂tr ( X T AX )
Proposition 2. If A = ( aij ) ∈ Rm×m , X = ( xij ) ∈ Rm×n , then ∂X = AX + A T X.

∂tr ( XX T B)
Proposition 3. If B = (bij ) ∈ Rn×n , X = ( xij ) ∈ Rm×n , then ∂X = XB + XB T .

Using Propositions 2 and 3, the gradient of the objective function (3) is

 ϕ( X ) = AX + XB + A T X + XB T − 2C. (4)

By (4), the Hessian matrix is

2 ϕ ( X ) = A + A T + B + B T . (5)

3. Iterative Methods
In this section, we will introduce the adaptive accelerated proximal gradient (A-APG)
method and the Newton-APG method to solve the Sylvester equation. Moreover, we
compare the A-APG and Newton-APG methods with other existing methods.

3.1. APG Method


The traditional APG method [31] is designed for solving the composite convex problem:

min H ( x ) = g( x ) + f ( x ),
x ∈H

49
Mathematics 2022, 10, 1040

where H is the finite-dimensional Hilbert space equipped with the inner product < ·, · >, g
and f are both continuously convex, and  f has a Lipschitz constant L. Given initializa-
tions x1 = x0 and t0 = 1, the APG method is

tk = ( 4(t − k − 1)2 + 1)/2,
t k −1 − 1
Yk = Xk + ( Xk − Xk −1 ),
tk
Xk +1 = Proxαg (Yk − α f (Yk )),
where α ∈ (0, L] and the mapping Proxαg (·) : Rn → Rn is defined as
! (
1
Proxαg ( x ) = argmin g(y) + y−x 2
.
y 2α

Since our minimization problem is linear, we choose the explicit scheme. The explicit
scheme is a simple but effective approach for the minimization problem. Given an initial
value Y0 and the step αk , the explicit scheme is

Yk+1 = Yk − αk  ϕ(Yk ), (6)

where Yk is the approximation solution. The explicit scheme satisfies the sufficient decrease
property using the gradient descent (GD) method.
Let Xk and Xk−1 be the current and previous states and the extrapolation weight be
wk . Using the explicit method (6), the APG iterative scheme is

wk = k − 2/k + 1,
Yk = (1 + wk ) Xk − wXk−1 , (7)
Yk+1 = Yk − αk  ϕ(Yk ).

Together with the standard backtracking, we adopt the step size αk when the following
condition holds:
ϕ(Yk ) − ϕ(Yk+1 ) ≥ η Yk+1 − Yk 2 , (8)
for some η > 0.
Combining (7) and (8), the APG algorithm is summarized in Algorithm 1.

Algorithm 1 APG algorithm.


Require: X0 , tol, α0 , η > 0, β ∈ (0, 1), and k = 1.
1: while the stop condition is not satisfied do
2: Update Yk via Equation (7);
3: if Equation (8) holds then
4: break
5: else
6: αk = βαk ;
7: Calculate Yk+1 via (7);
8: k = k + 1.

3.2. Restart APG Method


Recently, an efficient and convergent numerical algorithm has been developed for
solving a discretized phase-field model by combining the APG method with the restart tech-
nique [30]. Unlike the APG method, the restart technique involves choosing Xk+1 = Yk+1
whenever the following condition holds:
2
ϕ( Xk ) − ϕ(Yk+1 )  γ Xk − Yk+1 , (9)

50
Mathematics 2022, 10, 1040

for some γ > 0. If the condition is not met, we restart the APG by setting wk = 0.
The restart APG method (RAPG) is summarized in Algorithm 2.

Algorithm 2 RAPG algorithm.


Require: X0 , tol, α0 , η > 0, γ > 0, β ∈ (0, 1), and k = 1.
1: while the stop condition is not satisfied do
2: Calculate Yk+1 by APG Algorithm 1;
3: if Equation (9) holds then
4: Xk+1 = Yk+1 and update ωk+1 ;
5: else
6: Xk+1 = Xk and reset ωk+1 = 0;
7: k = k + 1.

3.3. A-APG Method


In RAPG Algorithm 2, we can adaptively estimate the step size αk by using the line
search technique. Define

s k : = Xk − Xk −1 , gk : =  ϕ ( Xk ) −  ϕ ( Xk −1 ).

We initialize the search step by the Barzilai–Borwein (BB) method, i.e.,

tr (skT sk ) tr ( gkT sk )
αk = or . (10)
tr (skT gk ) tr ( gkT gk )

Therefore, we obtain the A-APG algorithm summarized in Algorithm 3.

Algorithm 3 A-APG algorithm.


Require: X0 , tol, α0 , η > 0, γ > 0, β ∈ (0, 1), and k = 1.
1: while the stop condition is not satisfied do
2: Initialize αk by BB step Equation (10);
3: Update Xk+1 by RAPG Algorithm 2.

3.4. Newton-APG Method


Despite the fast initial convergence speed of the gradient-based methods, the tail
convergence speed becomes slow. Therefore, we use a practical Newton method to solve
the minimization problem. We obtain the initial value from A-APG Algorithm 3, and then
choose the Newton direction as the gradient in the explicit scheme in RAPG Algorithm 2.
Then we have the Newton-APG method shown in Algorithm 4.

Algorithm 4 Newton-APG algorithm.


Require: X0, α0, γ > 0, η > 0, β ∈ (0, 1), , tol and k = 1.
1: Obtain the initial value from A-APG Algorithm 3 by the precision ;
2: while the stop condition is not satisfied do
3: Initialize αk by BB step Equation (10);
4: Update Xk+1 by RAPG Algorithm 2 using Newton direction.

3.5. Gradient Descent (GD) and Line Search (LGD) Methods


Moreover, we show gradient descent (GD) and line search (LGD) methods for compar-
ing with the A-APG and Newton-APG methods. The GD and line search LGD methods are
summarized in Algorithm 5.

51
Mathematics 2022, 10, 1040

Algorithm 5 GD and LGD algorithms.


Require: X0 , tol, α0 , η > 0, β ∈ (0, 1), and k = 1.
1: while the stop condition is not satisfied do
2: if the step size is fixed then
3: Calculate Xk+1 via Xk+1 = Xk − α  ϕ( Xk ) using GD;
4: else
5: Initialize αk by BB step Equation (10);
6: if Equation (8) holds then
7: break
8: else
9: αk = βαk ;
10: Calculate Xk+1 via Xk+1 via Xk+1 = Xk − α  ϕ( Xk ) using LGD;
11: k = k + 1.

3.6. Computational Complexity Analysis


Further, we analyze the computational complexity of each iteration of the derived al-
gorithms.
The computation of APG is mainly controlled by matrix multiplication and addition oper-
ations in three main parts. The iterative scheme needs 4m2 n + 4mn2 + O(mn) computational
complexity. The backtracking linear search needs 14m2 n + 20n2 m + 6n3 + O(mn) + O(n2 )
computational complexity defined by Equation (8). The extrapolation needs O(mn) com-
putational complexity defined by the Equation (7). The total computational complexity is
18m2 n + 24n2 m + 6n3 + O(mn) + O(n2 ) in Algorithm 1.
The computation of RAPG is mainly controlled by matrix multiplication and addi-
tion operations in four main parts. The iterative scheme needs 4m2 n + 4mn2 + O(mn)
computational complexity. The backtracking linear search defined by Equation (8) needs
14m2 n + 20n2 m + 6n3 + O(mn) + O(n2 ) computational complexity. The extrapolation de-
fined by Equation (7) needs O(mn) computational complexity. The restart defined by
Equation (9) needs 4m2 n + 14n2 m + 4n3 + O(mn) + O(n2 ) computational complexity. The
total computational complexity is 22m2 n + 38n2 m + 10n3 + O(mn) + O(n2 ) in Algorithm 2.
The computation of A-APG is mainly controlled by matrix multiplication and ad-
dition operations in four main parts. The iterative scheme needs 4m2 n + 4mn2 + O(mn)
computational complexity. The BB step and the backtracking linear search defined by
Equations (8) and (10) need mn, 4m2 n + 4mn2 + 6mn, 2n2 (2m − 1) + 2n, and 14m2 n +
20n2 m + 6n3 + O(mn) + O(n2 ) computational complexity. The extrapolation defined by
Equation (7) needs O(mn) computational complexity. The restart defined by Equation (9)
needs 4m2 n + 14n2 m + 4n3 + O(mn) + O(n2 ) computational complexity. The total compu-
tational complexity is 26m2 n + 46n2 m + 10n3 + O(mn) + O(n2 ) in Algorithm 3.
The computation of Newton-APG is mainly controlled by matrix multiplication and
addition operations in four main parts, different from the A-APG method. The itera-
tive scheme needs 8n3 + 3n2 + O(n2 ) + O(n3 ) computational complexity. The BB step
and the backtracking linear search defined by Equations (8) and (10) need n2 , 8n3 + 6n2 ,
2n2 (2n − 1) + 2n, and 10n2 (2n − 1) + 8n3 + 3n2 + O(n3 ) + O(n2 ) computational complex-
ity. The extrapolation defined by Equation (7) needs O(n2 ) computational complexity. The
restart defined by Equation (9) needs 5n2 (2n − 1) + n2 + O(n3 ) computational complexity.
The total computational complexity is 50n3 − 10n2 + 2n + O(n2 ) + O(n3 ) in Algorithm 4.
The computation of GD is mainly controlled by matrix multiplication and addition
operations in Equations (4) and (6). It requires mn(2m − 1), mn(2n − 1), mn(2m − 1),
mn(2n − 1) computational complexity to compute AX, XB, A T X, XB T . The total computa-
tional complexity is 4m2 n + 4mn2 + O(mn) in Algorithm 5 using GD.
The computation of LGD is mainly controlled by matrix multiplication and addition
operations in the calculation of s, g defined by Equation (8) and (10), and the calcula-
tion of GD, which require mn, 4m2 n + 4mn2 + 6mn, 2n2 (2m − 1) + 2n, 14m2 n + 20n2 m +

52
Mathematics 2022, 10, 1040

6n3 + O(mn) + O(n2 ), and 4m2 n + 4mn2 + O(mn), respectively. The total computational
complexity is 22m2 n + 32n2 m + 6n3 + O(mn) + O(n2 ) in Algorithm 5 using GD.

4. Convergent Analysis
In this section, we focus on the convergence analysis of A-APG Algorithm 3. The
following proposition is required.

Proposition 4. Let M be a bounded region that contains { ϕ  ϕ( X0 )} in Rn×n , then  ϕ( X )


satisfies the Lipschitz condition in M, i.e., there exists L M > 0 such that

 ϕ( X ) −  ϕ(Y )  L M X − Y f or X, Y ∈ M.

Proof. Using the continuity of  ϕ( X ), note that


   
 2   
 ϕ ( X )  =  ( A + A T ) + ( B + B T ) 

defined by (5) is bounded. Then  ϕ( X ) satisfies the Lipschitz condition in M.

In recent years, the proximal method based on the Bregman distance has been applied
for solving optimization problems. The proximal operator is

1
Proxαϕ (y) := argmin{ ϕ(y) + X − Xk 2
}.
y 2α

Basically, given the current estimation Xk and step size αk > 0, update Xk+1 via

1
Xk+1 = Prox0α ( Xk − αk  ϕ( Xk )) = argmin{ X − ( Xk − α k  ϕ ( Xk ) 2
}. (11)
X 2αk

Thus we obtain
1
(X − ( Xk − αk  ϕ( Xk ))) = 0,
2αk k+1
which implies that
Xk +1 = Xk − α k  ϕ ( Xk ).
This is exactly the explicit scheme in our algorithm.

4.1. Linear Search Is Well-Defined


Using the optimization from Equation (11), it is evident that

1
Xk+1 = argmin{ X − ( Xk − α k  ϕ ( Xk ) 2 }
X 2α k
1
= argmin{ X − Xk 2 +  X − Xk ,  ϕ( Xk )}
X 2α k
1
= argmin{ X − Xk 2 +  X − Xk ,  ϕ( Xk ) + ϕ( Xk )}.
X 2α k

Then we obtain
1 2
ϕ ( Xk )  X − Xk +  Xk+1 − Xk ,  ϕ( Xk ) + ϕ( Xk )
2αk k+1
 2 
1  ϕ ( X ) 
 ϕ ( Xk +1 ) + Xk − Xk +1 2 − Xk − Xk +1 2 (12)
2αk 2
1 L
 ϕ ( Xk +1 ) + ( − M ) Xk − Xk +1 2 ,
2αk 2

53
Mathematics 2022, 10, 1040

where the second inequality follows from Taylor expansion of ϕ( Xk+1 ). By Equation (12), set

1 1
0 < αk < α := min{ , }, (13)
L M + 2η L M + 2γ

the conditions in linear search Equation (8) and non-restart Equation (9) are both satisfied.
Therefore, the backtracking linear search is well-defined.

4.2. Sufficient Decrease Property


In this section, we show the sufficient decrease property of the sequence generated by
A-APG Algorithm 3. If αk satisfies the condition Equation (13), then
2
ϕ( Xk ) − ϕ(Yk+1 )  ρ1 Xk − Yk+1 ,

where ρ1 = min{η, γ} > 0. Since ϕ is a bounded function, then there exists ϕ∗ such that
ϕ( Xk )  ϕ∗ and ϕ( Xk ) → ϕ∗ as k → +∞. This implies

ρ1 ∑ Xk +1 − Xk 2
 ϕ( X0 ) − ϕ∗ < +∞,
k =0

which shows that


lim Xk+1 − Xk = 0.
k→+∞

4.3. Bounded Gradient


Define two sets Ω2 = {k : k = 2} and Ω1 = N \ Ω2 . Let wk = k − 2/k + 1, for any
k ∈ Ω2 , then Xk+1 = Yk+1 when wk = 0. There exists w = k max − 2/k max + 1 ∈ [0, 1) such
that wk  w as k increases. If k ∈ Ω1 , since

1 2
Yk+1 = argmin{ X − (Yk − αk  ϕ(Yk )) },
X 2αk

we have
1
0 =  ϕ(Yk ) + (Y − Yk ).
α k k +1
Thus,
1
 ϕ(Yk ) = (Y − X k + 1 ) .
αk k
Note that Yk = (1 + wk ) Xk − wk Xk−1 , then

1
 ϕ(Yk ) = (1 + w k ) Xk − w k Xk −1 − Xk +1
αk
1
= w ( X − Xk −1 ) + ( Xk − Xk +1 )
αk k k (14)
1
 ( w Xk − Xk −1 + Xk − Xk +1 )
αmin
= c1 ( Xk +1 − Xk + w Xk − Xk −1 ),

where c1 = α 1 > 0.
min
If k ∈ Ω2 , then

1 2
Xk+1 = argmin{ X − ( Xk − αk  ϕ( Xk )) },
X 2αk

which implies that


1
0 =  ϕ ( Xk ) + (X − Xk ).
α k k +1

54
Mathematics 2022, 10, 1040

Thus
1 1
 ϕ ( Xk ) = X − Xk +1  X − Xk +1 = c 1 ( Xk − Xk +1 ), (15)
αk k αmin k

Combining Equations (14) and (15), it follows that

 ϕ ( Xk )  c1 ( Xk +1 − Xk + w Xk − Xk −1 ).

4.4. Subsequence Convergence


As { Xk } ∈ M is compact, there exists a subsequence { Xk j } ⊂ M and X ∗ ∈ M such
that lim Xk j = X ∗ . Then ϕ is bounded, i.e., ϕ( X ) > −∞ and ϕ keeps decreasing. Hence,
j→+∞
there exists ϕ∗ such that lim ϕ( Xk ) = ϕ∗ . Note that
k →+∞

2
ϕ ( Xk ) − ϕ ( Xk +1 )  c0 Xk − Xk +1 , k = 1, 2, . . . (16)

Summation over k yields



c0 ∑ Xk − Xk +1 2
 ϕ( X0 ) − ϕ∗ < +∞.
k =0

Therefore,
lim Xk − Xk+1 = 0.
k →+∞

Due to the property of the gradient, thus


 
 
lim  ϕ( Xk j ) = 0.
j→+∞

Considering the continuity of ϕ and  ϕ, we have

lim ϕ( Xk j ) = ϕ( X ∗ ), lim  ϕ( Xk j ) =  ϕ( X ∗ ) = 0,
j→+∞ j→+∞

which implies that  ϕ( X ∗ ) = 0.

4.5. Sequence Convergence


In this section, the subsequence convergence can be strengthened by using the Kurdyka–
Lojasiewicz property.

Proposition 5. For x ∈ dom ∂ϕ := { x : ∂ϕ( x ) = ∅}, there exists η > 0, an  neighborhood of


x, and ψ ∈ Ψη = {ψ ∈ C [0, η ) ∩ C  (0, η ), where ψ is concave, ψ(0) = 0, ψ > 0 on (0, η )} such
that for all x ∈ Γη ( x, ) : U ∩ { x : ϕ( x ) < ϕ( x ) < ϕ( x ) + η }, we have

ψ ( ϕ( x ) − ϕ( x ))  ϕ( x )  1.

Then we say ϕ( x ) satisfies the Kurdyka–Lojasiewicz property.

Theorem 1. Assume that Propositions 4 and 5 are met. Let { Xk } be the sequence generated by
A-APG Algorithm 3. Then, there exists a point X ∗ ∈ M so that lim Xk = X ∗ and  ϕ( X ∗ ) = 0.
k→+∞

Proof. Let ω ( X0 ) be the set of limiting points of the sequence { Xk }. Based on the bound-
edness of { Xk } and the fact that ω ( X0 ) = ∩q∈ N ∪k>q { Xk }, it follows that ω ( X0 ) is a
non-empty and compact set. In addition, by Equation (16), we know that ϕ( X ) is a con-
stant on ω ( X0 ), denoted by ϕ∗ . If there exists some k0 such that ϕ( Xk0 ) = ϕ∗ , then for
∀k > k0 , we have ϕ( Xk ) = ϕ∗ . Next, we assume that ∀k, ϕ( Xk ) > ϕ∗ . Therefore, for

55
Mathematics 2022, 10, 1040

∀, η > 0, ∃l > 0, for ∀k > l we have dist(ω ( X0 ), Xk )   and ϕ∗ < ϕ( Xk ) < ϕ∗ + η i.e.,
for ∀ X ∗ ∈ ω ( X0 ), X ∈ Γη ( X ∗ , ). Applying Proposition 5, for ∀k > l, we have

ψ ( ϕ( Xk ) − ϕ∗ )  ϕ( Xk )  1.

Then
1
ψ  ( ϕ ( Xk ) − ϕ ∗ )  . (17)
c1 ( Xk − Xk −1 + w Xk −1 − Xk −2 )
By the convexity of ψ, it is obvious that

ψ( ϕ( Xk ) − ϕ∗ ) − ψ( ϕ( Xk+1 ) − ϕ∗ )  ψ ( ϕ( Xk ) − ϕ∗ )( ϕ( Xk ) − ϕ( Xk+1 )). (18)

Define

 p,q = ψ( ϕ( X p ) − ϕ∗ ) − ψ( ϕ( Xq ) − ϕ∗ ), c = (1 + w)c1 /c0 > 0.

Combining with Equations (16)–(18), for ∀k > l, we obtain

c 0 Xk +1 − Xk 2
k,k+1 
c1 ( Xk − Xk −1 + w Xk −1 − Xk −2 )
(19)
Xk +1 − Xk 2
 .
c ( Xk − Xk −1 + Xk −1 − Xk −2 )

Applying the geometric inequality to Equation (19), thus

1
2 Xk +1 − Xk  ( Xk − Xk−1 + Xk−1 − Xk−2 ) + 2ck,k+1 .
2
Therefore, for ∀k > l, summing up the above inequality for i = l + 1, . . . , k, we obtain

k
1 k k
2 ∑ Xi + 1 − Xi 
2 i=∑
( Xi − Xi−1 + Xi−1 − Xi−2 ) + 2c ∑ i,i+1
i = l +1 l +1 i = l +1
k
1
 ∑ Xi + 1 − Xi + X l + 1 − X l + X − Xl −1
2 l
i = l +1
+ 2cl +1,k+1 .

For ∀k > l, ψ  0, it is evident that

k
1
∑ Xi + 1 − Xi  X l + 1 − X l + X − Xl −1 + 2cψ( ϕ( Xl ) − ϕ∗ ),
i = l +1
2 l

which implies that



∑ Xk+1 − Xk < ∞.
k =1

In the end, we have lim Xk = X ∗ .


k →+∞

5. Numerical Results
In this section, we offer two corresponding numerical examples to illustrate the
efficiency of the derived algorithms. All code is written in Python language. Denote
iteration and error by the iteration step and error of the objective function. We take the
matrix order “n” as 128, 1024, 2048, and 4096.

56
Mathematics 2022, 10, 1040

Example 1. Let
⎛ ⎞ ⎛ ⎞
2 −1 1 0.5
⎜ −1 2 −1 ⎟ ⎜0.5 1 0.5 ⎟
⎜ ⎟ ⎜ ⎟
⎜ .. .. .. ⎟ ⎜ .. .. .. ⎟
A1 = ⎜
⎜ . . . ⎟, B1 = ⎜
⎟ ⎜ . . . ⎟

⎜ .. .. ⎟ ⎜ .. .. ⎟
⎝ . . −1 ⎠ ⎝ . . 0.5 ⎠
−1 2 0.5 1

be tridiagonal matrices in the Sylvester Equation (1). Set the matrix C1 as the identity matrix. The
initial step size is 0.01, which is small enough to iterate. The parameters are η1 = 0.25, ω1 = 0.2
taken from (0,1) randomly. Table 1 and Figure 1 show the numerical results of Algorithms 1–5.
It can be seen that the LGD, A-APG, and Newton-APG Algorithms are more efficient than other
methods. Moreover, the iteration step does not increase when the matrix order increases due to the
same initial value. The A-APG method has higher error accuracy compared with other methods.
The Newton-APG method takes more CPU time and fewer iteration steps than the A-APG method.
The Newton method needs to calculate the inverse of the matrix, while it has quadratic convergence.
From Figure 1, the error curves of the LGD, A-APG, and Newton-APG algorithms are hard to
distinguish. We offer another example below.

Table 1. Numerical results for Example 1.

Algorithm n Iteration Error Time(s)


GD 128 356 1.13687 × 10−13 3.30
LGD 128 15 1.26477 × 10−12 0.27
APG 128 374 1.4353 × 10−12 4.31
RAPG 128 69 1.4353 × 10−12 1.45
A-APG 128 19 3.55271 × 10−14 0.38
Newton-APG 128 18 9.47438 × 10−11 0.48
CG 128 19 3.49364 × 10−14 0.42
GD 1024 356 1.02318 × 10−12 806
LGD 1024 15 1.06866 × 10−11 69
APG 1024 374 1.18803 × 10−11 1261
RAPG 1024 69 2.59774 × 10−11 367
A-APG 1024 19 2.84217 × 10−13 113
Newton-APG 1024 18 8.95682 × 10−10 144
CG 1024 19 3.37046 × 10−14 71
GD 2048 356 2.04636 × 10−12 6315
LGD 2048 15 2.13731 × 10−11 569
APG 2048 374 2.38742 × 10−11 9752
RAPG 2048 69 5.20686 × 10−11 2994
A-APG 2048 19 6.82121 × 10−13 926
Newton-APG 2048 18 8.95682 × 10−10 1015
CG 2048 19 3.34616 × 10−14 521
GD 4096 356 4.09273 × 10−12 66,155
LGD 4096 15 4.27463 × 10−11 4199
APG 4096 374 4.77485 × 10−11 71,636
RAPG 4096 69 1.04365 × 10−10 21,596
A-APG 4096 19 1.81899 × 10−12 6829
Newton-APG 4096 18 3.64571 × 10−9 7037
CG 4096 19 3.33322 × 10−14 3553

57
Mathematics 2022, 10, 1040

Figure 1. The error curves when n = 128, 1024, 2048, 4096 for Example 1.

Example 2. Let A2 = A1 A1T , B2 = B1 B1T be positive semi-definite matrices in the Sylvester


Equation (1). Set the matrix C2 as the identity matrix. The initial step size is 0.009. The parameters
are η2 = 0.28, ω2 = 0.25 taken from (0,1) randomly. Table 2 and Figure 2 show the numerical
results of Algorithms 1–5. It can be seen that the LGD, A-APG, and Newton-APG algorithms take
less CPU time compared with other methods. Additionally, we can observe the different error curves
of the LGD, A-APG, and Newton-APG algorithms from Figure 2.

Remark 1. The difference of the iteration step in Examples 1 and 2 emerges due to the given
different initial values. It can be seen that the LGD, A-APG, and Newton-APG algorithms have
fewer iteration steps. Whether the A-APG method or Newton-APG yields fewer iteration steps
varies from problem to problem. From Examples 1 and 2, we observe that the A-APG method has
higher accuracy, although it takes more time and more iteration steps than the LGD method.

Remark 2. Moreover, we compare the performance of our methods with other methods such as the
conjugate gradient method (CG) in Tables 1 and 2. We take the same initial values and set the error
to 1 × 10−14 . From Tables 1 and 2, it can be seen that the LGD and A-APG methods are more
efficient for solving the Sylvester matrix equation when the order n is small. When n is large, the
LGD and A-APG methods nearly have a convergence rate with the CG method.

Figure 2. The error curves when n = 128, 1024, 2048, 4096 for Example 2.

58
Mathematics 2022, 10, 1040

Table 2. Numerical results for Example 2.

Algorithm n Iteration Error Time(s)


GD 128 243 1.63425 × 10−13 2.38
LGD 128 20 2.45137 × 10−12 0.47
APG 128 260 1.58096 × 10−12 4.51
RAPG 128 53 1.90781 × 10−12 1.46
A-APG 128 32 3.55271 × 10−15 0.78
Newton-APG 128 36 2.30926 × 10−13 1.26
CG 128 34 4.13025 × 10−14 0.79
GD 1024 243 1.3074 × 10−12 516
LGD 1024 20 1.89573 × 10−11 95
APG 1024 260 1.25056 × 10−11 835
RAPG 1024 53 1.51772 × 10−11 267
A-APG 1024 32 4.61569 × 10−14 181
Newton-APG 1024 36 4.20641 × 10−12 214
CG 1024 34 4.29936 × 10−14 92
GD 2048 243 2.6148 × 10−12 4129
LGD 2048 20 3.78577 × 10−11 814
APG 2048 260 2.48974 × 10−11 6507
RAPG 2048 53 3.03544 × 10−11 2193
A-APG 2048 32 2.27374 × 10−13 1622
Newton-APG 2048 36 8.52651 × 10−12 2125
CG 2048 34 4.22694 × 10−14 797
GD 4096 243 5.22959 × 10−12 29,859
LGD 4096 20 7.54881 × 10−11 6023
APG 4096 260 4.97948 × 10−11 48,238
RAPG 4096 53 6.07088 × 10−11 16,482
A-APG 4096 32 2.27374 × 10−13 12,896
Newton-APG 4096 36 7.95808 × 10−12 14,901
CG 4096 34 4.18275 × 10−14 5337

6. Conclusions
In this paper, we have introduced the A-APG and Newton-APG methods for solving
the Sylvester matrix equation. The key idea is to change the Sylvester matrix equation to
an optimization problem by using the Kronecker product. Moreover, we have analyzed the
computation complexity and proved the convergence of the A-APG method. Convergence
results and preliminary numerical examples have shown that the schemes are promising in
solving the Sylvester matrix equation.

Author Contributions: J.Z. (methodology, review, and editing); X.L. (software, visualization, data
curation). All authors have read and agreed to the published version of the manuscript.
Funding: The work was supported in part by the National Natural Science Foundation of China
(12171412, 11771370), Natural Science Foundation for Distinguished Young Scholars of Hunan
Province (2021JJ10037), Hunan Youth Science and Technology Innovation Talents Project (2021RC3110),
the Key Project of the Education Department of Hunan Province (19A500, 21A0116).
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: Not applicable.
Conflicts of Interest: The authors declare no conflict of interest.

59
Mathematics 2022, 10, 1040

References
1. Dooren, P.M.V. Structured Linear Algebra Problems in Digital Signal Processing; Springer: Berlin/Heidelberg, Germany, 1991.
2. Gajic, Z.; Qureshi, M.T.J. Lyapunov Matrix Equation in System Stability and Control; Courier Corporation: Chicago, IL, USA, 2008.
3. Corless, M.J.; Frazho, A. Linear Systems and Control: An Operator Perspective; CRC Press: Boca Raton, FL, USA, 2003.
4. Stewart, G.W.; Sun, J. Matrix Perturbation Theory; Academic Press: London, UK, 1990.
5. Simoncini, V.; Sadkane, M. Arnoldi-Riccati method for large eigenvalue problems. BIT Numer. Math. 1996, 36, 579–594.
6. Demmel, J.W. Three methods for refining estimates of invariant subspaces. Computing 1987, 38, 43–57.
7. Chen, T.W.; Francis, B.A. Optimal Sampled-Data Control Systems; Springer: London, UK, 1995.
8. Datta, B. Numerical Methods for Linear Control Systems; Elsevier Inc.: Amsterdam, The Netherlands, 2004.
9. Lord, N. Matrix computations. Math. Gaz. 1999, 83, 556–557.
10. Zhao, X.L.; Wang, F.; Huang, T.Z. Deblurring and sparse unmixing for hyperspectral images. IEEE Trans. Geosci. Remote Sens.
2013, 51, 4045–4058.
11. Obinata, G.; Anderson, B.D.O. Model Reduction for Control System Design; Springer Science & Business Media: London, UK, 2001.
12. Bouhamidi, A.; Jbilou, K. A note on the numerical approximate solutions for generalized Sylvester matrix equations with
applications. Appl. Math. Comput. 2008, 206, 687–694.
13. Bai, Z.Z.; Benzi, M.; Chen, F. Modified HSS iteration methods for a class of complex symmetric linear systems. Computing 2010,
87, 93–111.
14. Bartels, R.H.; Stewart, G.W. Solution of the matrix equation AX + XB = C. Commun. ACM 1972, 15, 820–826.
15. Golub, G.H. A Hessenberg-Schur Method for the Problem AX + XB = C; Cornell University: Ithaca, NY, USA, 1978.
16. Robbé, M.; Sadkane, M. A convergence analysis of GMRES and FOM methods for Sylvester equations. Numer. Algorithms 2002,
30, 71–89.
17. Guennouni, A.E.; Jbilou, K.; Riquet, A.J. Block Krylov subspace methods for solving large Sylvester equations. Numer. Algorithms
2002, 29, 75–96.
18. Salkuyeh, D.K.; Toutounian, F. New approaches for solving large Sylvester equations. Appl. Math. Comput. 2005, 173, 9–18.
19. Wachspress, E.L. Iterative solution of the Lyapunov matrix equation. Appl. Math. Lett. 1988, 1, 87–90.
20. Jbilou, K.; Messaoudi, A.; Sadok, H. Global FOM and GMRES algorithms for matrix equations. Appl. Numer. Math. 1999,
31, 49–63.
21. Feng, D.; Chen, T. Gradient Based Iterative Algorithms for Solving a Class of Matrix Equations. IEEE Trans. Autom. Control 2005,
50, 1216–1221.
22. Heyouni, M.; Movahed, F.S.; Tajaddini, A. On global Hessenberg based methods for solving Sylvester matrix equations. Comput.
Math. Appl. 2019, 77, 77–92.
23. Benner, P.; Kürschner, P. Computing real low-rank solutions of Sylvester equations by the factored ADI method. Comput. Math.
Appl. 2014, 67, 1656–1672.
24. Bouhamidi, A.; Hached, M.; Heyouni, M.J.; Bilou, K. A preconditioned block Arnoldi method for large Sylvester matrix equations.
Numer. Linear Algebra Appl. 2013, 20, 208–219.
25. Heyouni, M. Extended Arnoldi methods for large low-rank Sylvester matrix equations. Appl. Numer. Math. 2010, 60, 1171–1182.
26. Agoujil, S.; Bentbib, A.H.; Jbilou, K.; Sadek, E.M. A minimal residual norm method for large-scale Sylvester matrix equations.
Electron. Trans. Numer. Anal. Etna 2014, 43, 45–59.
27. Abdaoui, I.; Elbouyahyaoui, L.; Heyouni, M. An alternative extended block Arnoldi method for solving low-rank Sylvester
equations. Comput. Math. Appl. 2019, 78, 2817–2830.
28. Jbilou, K. Low rank approximate solutions to large Sylvester matrix equations. Appl. Math. Comput. 2005, 177, 365–376.
29. Liang, B.; Lin, Y.Q.; Wei, Y.M. A new projection method for solving large Sylvester equations. Appl. Numer. Math. 2006,
57, 521–532.
30. Jiang, K.; Si, W.; Chen, C.; Bao, C. Efficient numerical methods for computing the stationary states of phase-field crystal models.
SIAM J. Sci. Comput. 2020, 42, B1350–B1377.
31. Beck, A.; Teboulle, M. A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems. SIAM J. Imaging Sci. 2009,
2, 183–202.
32. Bao, C.; Barbastathis, G.; Ji, H.; Shen, Z.; Zhang, Z. Coherence retrieval using trace regularization. SIAM J. Imaging Sci. 2018,
11, 679–706.
33. Magnus, J.R.; Neudecker, H. Matrix Differential Calculus; John Willey & Sons: Hoboken, NJ, USA, 1999.

60
mathematics

Article
Generalized Three-Step Numerical Methods for Solving
Equations in Banach Spaces
Michael I. Argyros 1 , Ioannis K. Argyros 2, *, Samundra Regmi 3, * and Santhosh George 4

1 Department of Computer Science, University of Oklahoma, Norman, OK 73019, USA;


[email protected]
2 Department of Mathematical Sciences, Cameron University, Lawton, OK 73505, USA
3 Department of Mathematics, University of Houston, Houston, TX 77204, USA
4 Department of Mathematical and Computational Sciences, National Institute of Technology Karnataka,
Mangaluru 575 025, India; [email protected]
* Correspondence: [email protected] (I.K.A.); [email protected] (S.R.)

Abstract: In this article, we propose a new methodology to construct and study generalized three-step
numerical methods for solving nonlinear equations in Banach spaces. These methods are very general
and include other methods already in the literature as special cases. The convergence analysis of the
specialized methods is been given by assuming the existence of high-order derivatives which are
not shown in these methods. Therefore, these constraints limit the applicability of the methods to
equations involving operators that are sufficiently many times differentiable although the methods
may converge. Moreover, the convergence is shown under a different set of conditions. Motivated by
the optimization considerations and the above concerns, we present a unified convergence analysis
for the generalized numerical methods relying on conditions involving only the operators appearing
in the method. This is the novelty of the article. Special cases and examples are presented to conclude
this article.

Citation: Argyros, M.I.; Argyros, I.K.; Keywords: generalized three-step numerical method; convergence; Banach space
Regmi, S.; George, D. Generalized
Three-Step Numerical Methods for MSC: 49M15; 47H17; 65J15; 65G99; 47H17; 41A25; 49M15
Solving Equations in Banach Spaces.
Mathematics 2022, 10, 2621. https://
doi.org/10.3390/math10152621
1. Introduction
Academic Editors: Maria Isabel
Berenguer and Manuel Ruiz Galán A plethora of applications from diverse disciplines of computational sciences are
converted to nonlinear equations such as
Received: 8 July 2022
Accepted: 26 July 2022 F(x) = 0 (1)
Published: 27 July 2022
using modeling (mathematical) [1–4]. The nonlinear operator F is defined on an open and
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
convex subset Ω of a Banach space X with values in X. The solution of the equation is
published maps and institutional affil-
denoted by x∗ . Numerical methods are mainly used to find x∗ . This is the case since the
iations. analytic form of the solution x∗ is obtained in special cases.
Researchers, as well as practitioners, have proposed numerous numerical methods
under a different set of convergence conditions using high-order derivatives, which are not
present in the methods.
Copyright: 
c 2022 by the authors. Let us consider an example.
Licensee MDPI, Basel, Switzerland.
This article is an open access article Example 1. Define the function F on X = [−0.5, 1.5] by
distributed under the terms and
!
conditions of the Creative Commons t3 ln t2 + t5 − t4 , t = 0
F (t) =
Attribution (CC BY) license (https:// 0, t = 0
creativecommons.org/licenses/by/
4.0/).

Mathematics 2022, 10, 2621. https://doi.org/10.3390/math10152621 https://www.mdpi.com/journal/mathematics


61
Mathematics 2022, 10, 2621

Clearly, the point t∗ = 1 solves the equation F (t) = 0. It follows that

F  (t) = 6 ln t2 + 60t2 − 24t + 22.

Then, the function F does not have a bounded third derivative in X.


Hence, many high convergence methods (although they may converge) cannot apply
to show convergence. In order to address these concerns, we propose a unified approach
for dealing with the convergence of these numerical methods that take into account only
the operators appearing on them. Hence, the usage of these methods becomes possible and
under weaker conditions.
Let x0 ∈ Ω be a starting point. Define the generalized numerical method ∀n = 0, 1, 2, . . .
by

yn = an = a( xn )
zn = bn = b ( x n , y n ) (2)
x n +1 = c n = c ( x n , y n , z n ),

where a : Ω −→ X, b : Ω × Ω −→ X and c : Ω × Ω × Ω −→ X are given operators chosen


so that limn−→∞ xn = x∗ .
The specialization of (2) is

yn = xn + αn F ( xn )
zn = u n + β n F ( x n ) + γn F ( y n ) (3)
x n +1 = vn + δn F ( xn ) + n F (yn ) + θn F (zn ),

where un = xn or un = yn , vn = xn or vn = yn or vn = zn , and αn , β n , γn , δn , n , θn are linear


operators on Ω, Ω × Ω and Ω × Ω × Ω, with values in X, respectively. By choosing some
of the linear operators equal to the O linear operators in (3), we obtain the methods studied
in [5]. Moreover, if X = Rk , then we obtain the methods studied in [6,7]. In particular, the
methods in [5] are of the special form
−1
yn = xn − O1,n F ( xn )
−1
zn = yn − O2,n F (yn ) (4)
−1
x n +1 = zn − O3,n F ( z n ),

yn = xn − sF  ( xn )−1 F ( xn )
zn = xn − O4,n F ( xn ) (5)
x n +1 = zn − O5,n F (zn ),

where they, as the methods in [7,8], are of the form

yn = x n − F  ( x n ) −1 F ( x n )
zn = yn − O6,n F  ( xn )−1 F (yn ) (6)
x n +1 = zn − O7,n F  ( xn )−1 F (zn ),

where s ∈ R is a given parameter, and Ok,n , k = 1, 2, . . . , 7 are linear operators acting


between Ω and X. In particular, operators must have a special form to obtain the fourth,
seventh or eighth order of convergence.
Further specifications of operators “O ” lead to well-studied methods, a few of which
are listed below (other choices can be found in [6,7,9,10]):
Newton method (second order) [1,4,11,12]:

y n = x n − F  ( x n ) −1 F ( x n ). (7)

62
Mathematics 2022, 10, 2621

Jarrat method (second order) [13]:

2
y n = x n − F  ( x n ) −1 F ( x n ). (8)
3
Traub-type method (fifth order) [14]:

yn = x n − F  ( x n ) −1 F ( x n )
zn = x n − F  ( x n ) −1 F ( y n ) (9)
x n +1 = x n − F  ( x n ) −1 F ( z n ).

Homeir method (third order) [15]:


1
yn = x n − F  ( x n ) −1 F ( x n )
2
x n +1 = y n − F  ( x n ) −1 F ( y n ). (10)

Cordero–Torregrosa (third Order) [2]:

yn = x n − F  ( x n ) −1 F ( x n )
 
xn + yn
x n +1 = xn − 6 F  ( xn ) + 4F  ( ) F  ( y n ) −1 F ( x n ). (11)
2
or

yn = x n − F  ( x n ) −1 F ( x n ) (12)
/ 0
3xn + yn xn + yn xn + 3yn −1
x n +1 = xn − 2 2F  ( ) − F ( ) + 2F  ( ) F ( x n ).
4 2 4

Noor–Wasseem method (third order) [3]:

yn = x n − F  ( x n ) −1 F ( x n )
/ 0 −1
2xn + yn
x n +1 = xn − 4 3F  ( ) + F  (yn ) F ( x n ). (13)
3

Xiao–Yin method (third order) [16]:

yn = x n − F  ( x n ) −1 F ( x n )
21 2
x n +1 = xn − (3F  (yn ) − F  ( xn ))−1 + F  ( xn )−1 F ( xn ). (14)
3
Corder–Torregrosa method (fifth order) [2]:

2
yn = x n − F  ( x n ) −1 F ( x n )
3
1
zn = xn − (3F  (yn ) − F  ( xn ))−1 (3F  (yn ) + F  ( xn )) F  ( xn )−1 F ( xn ) (15)
2
1 1
x n +1 = zn − ( F  (yn ) + F  ( xn ))−1 F (zn ).
2 2
or

yn = x n − F  ( x n ) −1 F ( x n )
zn = xn − 2( F  (yn ) + F  ( xn ))−1 F ( xn ) (16)
x n +1 = z n − F  ( y n ) −1 F ( z n ).

63
Mathematics 2022, 10, 2621

Sharma–Arora method (fifth order) [17,18]:

yn = x n − F  ( x n ) −1 F ( x n )
x n +1 = xn − (2F  (yn )−1 − F  ( xn )−1 ) F ( xn ). (17)

Xiao–Yin method (fifth order) [16]:


2
yn = x n − F  ( x n ) −1 F ( x n )
3
1
zn = xn − (3F  (yn )−1 + F  ( xn )−1 ) F ( xn ) (18)
4
11 2
x n +1 = xn − (3F  (yn ) − F  ( xn ))−1 F ( xn ).
3
Traub-type method (second order) [14]:

yn = x n − [ w n , x n ; F ] −1 F ( x n )
wn = xn + dF ( xn ), (19)

where [., .; F ] : Ω × Ω −→ L( X, X ) is a divided difference of order one.


Moccari–Lofti method (fourth order) [19]:

yn = x n − [ x n , w n ; F ] −1 F ( x n )
x n +1 = yn − ([yn , wn ; F ] + [yn , xn ; F ] − [ xn , wn ; F ])−1 F (yn ). (20)

Wang–Zang method (seventh order) [8,16,20]:

yn = x n − [ w n , x n ; F ] −1 F ( x n )
zn = M8 ( xn , yn ) (21)
x n +1 = zn − ([zn , xn ; F ] + [zn , yn ; F ] − [yn , xn ; F ])−1 F (zn ),

where M8 is any fourth-order Steffensen-type iteration method.


Sharma–Arora method (seventh order) [17]:

yn = x n − [ w n , x n ; F ] −1 F ( x n )
zn = yn − (3I − [wn , xn ; F ]( [yn , xn ; F ] + [yn , wn ; F ])) (22)
[ w n , x n ; F ] −1 ) F ( y n )
x n +1 = zn − [zn , yn ; F ]−1 ([wn , xn ; F ]
+[yn , xn ; F ] − [zn , xn ; F ])[wn , xn ; F ]−1 F (zn ).

The local, as well as the semi-local, convergence for methods (4) and (5), were pre-
sented in [17], respectively, using hypotheses relating only to the operators on these meth-
ods. However, the local convergence analysis of method (6) requires the usage of derivatives
or divided differences of higher than two orders, which do not appear in method (6). These
high-order derivatives restrict the applicability of method (6) to equations whose operator
F has high-order derivatives, although method (6) may converge (see Example 1).
Similar restrictions exist for the convergence of the aforementioned methods of order
three or above.
It is also worth noticing that the fifth convergence order method by Sharma [18]

yn = x n − F  ( x n ) −1 F ( x n )
zn = yn − 5F  ( xn )−1 F (yn ) (23)
1
x n +1 = yn − [9F  ( xn )−1 F (yn ) + F  ( xn )−1 F (zn )]
5

64
Mathematics 2022, 10, 2621

cannot be handled with the analyses given previously [5–7] for method (4), method (5), or
method (6).
Based on all of the above, clearly, it is important to study the convergence of method (2)
and its specialization method (3) with the approach employed for method (4) or (5). This
way, the resulting unified convergence criteria can apply to their specialized methods listed
or not listed previously. Hence, this is the motivation as well as the novelty of the article.
There are two important types of convergence: the semi-local and the local. The
semi-local uses information involving the initial point to provide criteria, assuring the
convergence of the numerical method, while the local one is based on the information
about the solution to find the radii of the convergence balls.
The local convergence results are vital, although the solution is unknown in general
since the convergence order of the numerical method can be found. This kind of result also
demonstrates the degree of difficulty in selecting starting points. There are cases when the
radius of convergence of the numerical method can be determined without the knowledge
of the solution.
As an example, let X = R. Suppose function F satisfies an autonomous differen-
tial [5,21] equation of the form

H ( F (t)) = F  (t),

where H is a continuous function. Notice that H ( F (t∗ )) = F  (t∗ ) or F  (t∗ ) = H (0). In the
case of F (t) = et − 1, we can choose H (t) = t + 1 (see also the numerical section).
Moreover, the local results can apply to projection numerical methods, such as
Arnoldi’s, the generalized minimum residual numerical method (GMRES), the generalized
conjugate numerical method (GCS) for combined Newton/finite projection numerical
methods, and in relation to the mesh independence principle to develop the cheapest and
most efficient mesh refinement techniques [1,5,11,21].
In this article, we introduce a majorant sequence and use our idea of recurrent functions
to extend the applicability of the numerical method (2). Our analysis includes error bounds
and results on the uniqueness of x∗ based on computable Lipschitz constants not given
before in [5,13,21–24] and in other similar studies using the Taylor series. This idea is very
general. Hence, it applies also to other numerical methods [10,14,22,25].
The convergence analysis of method (2) and method (3) is given in Section 2. Moreover,
the special choices of operators appear in the method in Sections 3 and 4. Concluding
remarks, open problems, and future work complete this article.

2. Convergence Analysis of Method


The local is followed by the semi-local convergence analysis. Let S = [0, ∞) and
S0 = [0, ρ0 ) for some ρ0 > 0. Consider functions h1 : S0 −→ R, h2 : S0 × S0 −→ R and
h3 : S0 × S0 × S0 −→ R be continuous and nondecreasing in each variable.
Suppose that equations
hi (t) − 1 = 0, i = 1, 2, 3 (24)
have the smallest solutions, ρi ∈ S − {0}. The parameter ρ defined by

ρ = min{ρi } (25)

shall be shown to be a radius of convergence for method (2). Let S1 = [0, ρ). It follows by
the definition of radius ρ that for all t ∈ S1

0 ≤ hi (t) < 1. (26)

The notation U ( x, ς) denotes an open ball with center x ∈ X and of radius ς > 0. By
U [ x, ς], we denote the closure of U ( x, ς).
The following conditions are used in the local convergence analysis of the method (2).
Suppose the following:

65
Mathematics 2022, 10, 2621

(H1) Equation F ( x ) = 0 has a solution x∗ ∈ Ω.


(H2) a( x ) − x∗ ≤ h1 ( x − x∗ ) x − x∗ ,

b( x, y) − x∗ ≤ h2 ( x − x∗ , y − x∗ ) x − x∗

and
c( x, y, z) − x∗ ≤ h3 ( x − x∗ , y − x∗ , z − x∗ ) x − x∗
for all x, y, z ∈ Ω0 = Ω ∩ U ( x∗ , ρ0 ).
(H3) Equations (24) have smallest solutions ρi ∈ S0 − {0};
(H4) U [ x∗ , ρ] ⊂ Ω, where the radius ρ is given by Formula (25).
Next, the main local convergence analysis is presented for method (2).

Theorem 1. Suppose that the conditions (H1)–(H4) hold and x0 ∈ U ( x∗ , r ) − { x∗ }. Then, the
sequence { xn } generated by method (2) is well defined and converges to x∗ . Moreover, the following
estimates hold ∀ n = 0, 1, 2, . . .

y n − x ∗ ≤ h1 ( x n − x ∗ ) x n − x ∗ ≤ x n − x ∗ < ρ (27)

z n − x ∗ ≤ h2 ( x n − x ∗ , y n − x ∗ ) x n − x ∗ ≤ x n − x ∗ (28)
and

x n +1 − x ∗ ≤ h3 ( x n − x ∗ , y n − x ∗ , z n − x ∗ ) x n − x ∗ ≤ x n − x ∗ . (29)

Proof. Let x0 ∈ U ( x∗ , ρ0 ). Then, it follows from the first condition in (H1) the definition of
ρ, (26) (for i = 1) and the first substep of method (2) for n = 0 that

y0 − x∗ ≤ h1 ( x0 − x∗ ) x0 − x∗ ≤ x0 − x∗ < ρ, (30)

showing estimate (27) for n = 0 and the iterate y0 ∈ U ( x∗ , ρ). Similarly,

z0 − x ∗ ≤ h2 ( x0 − x ∗ , y0 − x ∗ ) x0 − x ∗
≤ h2 ( x0 − x ∗ , y0 − x ∗ )
≤ h2 ( x0 − x ∗ , x0 − x ∗ ) x0 − x ∗ ≤ x0 − x ∗ (31)

and

x1 − x ∗ ≤ h3 ( x0 − x ∗ , y0 − x ∗ , z0 − x ∗ ) x0 − x ∗
≤ h3 ( x0 − x ∗ , x0 − x ∗ , x0 − x ∗ ) x0 − x ∗
≤ x0 − x ∗ ,

showing estimates (28), (29), respectively and the iterates z0 , x1 ∈ U ( x∗ , ρ). By simply
replacing x0 , y0 , z0 , x1 by xk , yk , zk , xk+1 in the preceding calculations, the induction for
estimates (27)–(29) is terminated. Then, from the estimate

xk+1 − x∗ ≤ d xk − x∗ < ρ,

where
d = h3 ( x0 − x∗ , x0 − x∗ , x0 − x∗ ) ∈ [0, 1) (32)
we conclude xk+1 ∈ U [ x∗ , ρ] and limk−→∞ xk = x∗ .

Remark 1. It follows from the proof of Theorem 1 that y, z can be chosen in particular as yn = a( xn )
and zn = b( xn , yn ). Thus, the condition (H2) should hold for all x, a( x ), b( x, y) ∈ Ω0 and not
x, y, z ∈ Ω0 . Clearly, in this case, the resulting functions hi are at least as tight as the functions hi ,
leading to an at least as large radius of convergence ρ̄ as ρ (see the numerical section).

66
Mathematics 2022, 10, 2621

Concerning the semi-local convergence of method (2), let us introduce scalar sequences
{tn }, {sn } and {un } defined for t0 = 0, s0 = η ≥ 0 and the rest of the iterates, depending
on operators a, b, c and F (see how in the next section). These sequences shall be shown to
be majorizing for method (2). However, first, a convergence result for these sequence is
needed.

Lemma 1. Suppose that ∀ n = 0, 1, 2, . . .

t n ≤ s n ≤ u n ≤ t n +1 (33)

and
tn ≤ λ (34)
for some λ ≥ 0. Then, the sequence {tn } is convergent to its unique least upper bound t∗ ∈ [0, λ].

Proof. It follows from conditions (33) and (34) that sequence {tn } is nondecreasing and
bounded from above by λ, and as such, it converges to t∗ .

Theorem 2. Suppose the following:


(H5) Iterates { xn }, {yn }, {zn } generated by method (2) exist, belong in U ( x0 , t∗ ) and satisfy the
conditions of Lemma 1 for all n = 0, 1, 2, . . .
(H6) a( xn ) − xn ≤ sn − tn ,

b( xn , yn ) − yn ≤ un − sn

and
c ( x n , y n , z n ) − z n ≤ t n +1 − u n
for all n = 0, 1, 2, . . . and
(H7) U [ x0 , t∗ ] ⊂ Ω.
Then, there exists x∗ ∈ U [ x0 , t∗ ] such that limn−→∞ xn = x∗ .

Proof. It follows by condition (H5) that sequence {tn } is complete as convergent. Thus,
by condition (H6), sequence { xn } is also complete in a Banach space X, and as such, it
converges to some x∗ ∈ U [ x0 , t∗ ] (since U [ x0 , t∗ ] is a closed set).

Remark 2. (i) Additional conditions are needed to show F ( x∗ ) = 0. The same is true for the results
on the uniqueness of the solution.
(ii) The limit point t∗ is not given in the closed form. So, it can be replaced by λ in Theorem 2.

3. Special Cases I
The iterates of method (3) are assumed to exist, and operator F has a divided difference
of order one.
Local Convergence
Three possibilities are presented for the local cases based on different estimates for the
determination of the functions hi . It follows by method (3) that
(P1) yn − x∗ = xn − x∗ + αn F ( xn ) = ( I + αn [ xn , x∗ ; F ])( xn − x∗ ),

zn − x∗ = ( I + γn [yn , x∗ ; F ])(yn − x∗ ) + β n [ xn , x∗ ; F ]( xn − x∗ )
= [( I + γn [yn , x∗ ; F ])( I + αn [ xn , x∗ ; F ]) + β n [ xn , x∗ ; F ]]( xn − x∗ )

67
Mathematics 2022, 10, 2621

and

x n +1 − x ∗ = ( I + θn [zn , x∗ ; F ])(zn − x∗ ) + δn [ xn , x∗ ; F ]( xn − x∗ )
+n [yn , x∗ ; F ](yn − x∗ )
= [( I + θn [zn , x∗ ; F ])( I + γn [yn , x∗ ; F ])( I + β n [ xn , x∗ ; F ])
+δn [ xn , x∗ ; F ] + n [yn , x∗ ; F ]( I + αn [ xn , x∗ ; F ])]( xn − x∗ )

Hence, the functions hi are selected to satisfy ∀ xn , yn , zn ∈ Ω

I + α n [ x n , x ∗ ; F ] ≤ h1 ( x n − x ∗ ),

( I + γn [yn , x∗ ; F ])( I + αn [ xn , x∗ ; F ]) + β n [ xn , x∗ ; F ] ≤ h2 ( xn − x∗ , yn − x∗ )

( I + θn [zn , x∗ ; F ])( I + γn [yn , x∗ ; F ])( I + β n [ xn , x∗ ; F ])


+δn [ xn , x∗ ; F ] + n [yn , x∗ ; F ]( I + αn [ xn , x∗ ; F ])
≤ h3 ( x n − x ∗ , y n − x ∗ , z n − x ∗ ).

A practical non-discrete choice for the function h1 is given by

I + α( x )[ x, x∗ ; F ] ≤ h1 ( x − x∗ ) ∀ x ∈ Ω.

Another choice is given by

h1 ( t ) = sup I + α( x )[ x, x∗ ; F ] .
x ∈Ω, x − x∗ ≤t

The choices of functions h2 and h3 can follow similarly.


(P2) Let Mi : Ω −→ X be a linear operator. By Mni we denote Mi ( xn ) ∀n = 0, 1, 2, . . . .
Then, it follows from method (3)

yn − x∗ = xn − x∗ − Mn1 F ( xn ) + (αn + Mn ) F ( xn )
= ( I − Mn2 [ xn , x∗ ; F ]) + (αn + Mn2 )[ xn , x∗ ; F ])( xn − x∗ ),
zn − x∗ = (( I − Mn2 [yn , x∗ ; F ]) + (γn + Mn2 )[yn , x∗ ; F ])(yn − x∗ )

and
xn+1 − x∗ = (( I − Mn3 [zn , x∗ ; F ]) + (θn + Mn3 )[zn , x∗ ; F ])(zn − x∗ ).
Thus, the functions hi must satisfy

I + α n ≤ h1 ( x n − x ∗ ),

( I + γn )( I + αn ) ≤ h2 ( xn − x∗ , yn − x∗ )
and

xn+1 − x∗ ≤ ( I + θn )( I + γn )( I + αn ) ≤ h3 ( xn − x∗ , yn − x∗ , zn − x∗ ).

Clearly, the function h1 can be chosen again as in case (P1). The functions h2 and h3
can be defined similarly.
(P3) Assume ∃ function ϕ0 : [0, ∞) −→ R continuous and non-decreasing such that

F  ( x∗ )−1 ( F  ( x ) − F  ( x∗ )) ≤ ϕ0 ( x − x∗ ) ∀ x ∈ Ω.

Then, we can write


 1
F ( xn ) = F ( xn ) − F ( x∗ ) = F  ( x∗ + θ ( xn − x∗ ))dθ ( xn − x∗ )
0

68
Mathematics 2022, 10, 2621

leading to
 1
F  ( x ∗ ) −1 F ( x n ) ≤ ϕ0 (θ xn − x∗ )dθ xn − x∗ .
0
Then, by method (3) we obtain, in turn, that

yn − x∗ = [ I + α n F  ( x ∗ ) F  ( x ∗ ) −1
 1 
× F  ( x∗ + θ ( xn − x∗ ))dθ − F  ( x∗ ) + F  ( x∗ ) ]( xn − x∗ ),
0

so, the function h1 must satisfy


 1
I + αn F  ( x∗ + θ ( xn − x∗ ))dθ ≤ h1 ( xn − x∗ )
0

or  1
h1 ( t ) = sup I + α( x ) F  ( x∗ + θ ( xn − x∗ ))dθ
x − x∗ ≤t, x ∈Ω 0

or  1
I + α n F  ( x ∗ ) (1 + ϕ0 (θ xn − x∗ )dθ ) ≤ h1 ( xn − x∗ )
0
or  1
h1 ( t ) = sup I + α ( x ) F  ( x ∗ ) (1 + ϕ0 (θ xn − x∗ )dθ ).
x − x∗ ≤t, x ∈Ω 0

Similarly, for the other two steps, we obtain in the last choice
 1
zn − x∗ ≤ I + γn F  ( x ∗ ) ( 1 + ϕ0 (θ yn − x∗ )dθ ) yn − x∗
0
 1
+ β n F  ( x ∗ ) (1 + ϕ0 (θ xn − x∗ )dθ ) xn − x∗
0

and
 1
x n +1 − x ∗ ≤ I + θ n F  ( x ∗ ) (1 + ϕ0 (θ zn − x∗ )dθ ) zn − x∗
0
 1
+ δn F  ( x∗ ) (1 + ϕ0 (θ xn − x∗ )dθ ) xn − x∗
0
 1
+ n F  ( x ∗ ) (1 + ϕ0 (θ yn − x∗ )dθ ) yn − x∗ .
0

Thus, the function h2 satisfies


 1
I + γn F  ( x ∗ ) ( 1 + ϕ0 (θ yn − x∗ )dθ ) yn − x∗
0
 1
+ β n F  ( x ∗ ) (1 + ϕ0 (θ xn − x∗ )dθ )
0
≤ h2 ( x n − x ∗ , y n − x ∗ )

or

h2 (s, t) = sup [ I + γ( x ) F  ( x∗ )
x − x∗ ≤s, y− x∗ ≤t
 1
×(1 + ϕ0 (θt)dθ )t)
0
 1
+ β ( x ) F  ( x ∗ ) (1 + ϕ0 (θs)dθ )].
0

69
Mathematics 2022, 10, 2621

Finally, concerning the choice of the function h3 , by the third substep of method (3)
 1
x n +1 − x ∗ ≤ I + θ n F  ( x ∗ ) (1 + ϕ0 (θ zn − x∗ )dθ ) zn − x∗
0
 1
+ δn F  ( x∗ ) (1 + ϕ0 (θ xn − x∗ )dθ ) xn − x∗
0
 1
+ n F  ( x ∗ ) (1 + ϕ0 (θ yn − x∗ )dθ ) yn − x∗ ,
0

so the function h3 must satisfy


 1
I + θ n F  ( x ∗ ) (1 + ϕ0 (θ yn − x∗ )dθ )h2 ( xn − x∗ , yn − x∗ )
0
 1
+ δn F  ( x∗ ) (1 + ϕ0 (θ xn − x∗ )dθ )
0
 1
+ n F  ( x ∗ ) (1 + ϕ0 (θ yn − x∗ )dθ )h1 ( xn − x∗ )
0
≤ h3 ( x n − x ∗ , y n − x ∗ , z n − x ∗ )

or

h( x, s, t, u) = sup μ( x, s, t, u),
x − x∗ ≤s, y− x∗ ≤t, z− x∗ ≤u

where

μ( x, s, t, u) = I + θ ( x ) F  ( x∗ )
 1
×(1 + ϕ0 (θu)dθ )h2 (t, s)
0
 1
+ δ ( x ) F  ( x ∗ ) (1 + ϕ0 (θs)dθ )
0
 1
+  ( x ) F  ( x ∗ ) (1 + ϕ0 ((θt)dθ )h1 (s)].
0

The functions h2 and h3 can also be defined with the other two choices as those of
function h1 given previously.
Semi-local Convergence
Concerning this case, we can have instead of the conditions of Theorem 2 (see (H6))
but for method (3)
αn F ( xn ) ≤ sn − tn ,
β n F ( x n ) + γn F ( y n ) ≤ u n − s n
and
δn F ( xn ) + n F (yn ) + θn F (zn ) ≤ tn+1 − un ∀n = 0, 1, 2, . . . .
Notice that under these choices,

yn − xn ≤ sn − tn

zn − yn ≤ un − sn
and
x n +1 − z n ≤ t n +1 − u n .
Then, the conclusions of Theorem 2 hold for method (3). Even more specialized choices
of linear operators appearing on these methods as well as function hi can be found in the
Introduction, the next section, or in [1,2,11,21] and the references therein.

70
Mathematics 2022, 10, 2621

4. Special Cases II
The section contains even more specialized cases of method (2) and method (3). In
particular, we study the local and semi-local convergence first of method (22) and second
of method (20). Notice that to obtain method (22), we set in method (3)

αn = − F  ( xn )−1 , un = yn , β n = O, γn = −5F  ( xn )−1 ,


9 1
vn = yn , δn = O, n = − F  ( xn )−1 and θn = − F  ( xn ). (35)
5 5
Moreover, for method (20), we let

αn = −[ xn , wn ; F ]−1 , un = yn , β n = O, zn = xn+1 ,
γn = ([yn , wn ; F ] + [yn , xn ; F ] − [ xn , wn ; F ])−1 , δn = n = θn = O (36)

and vn = zn .

5. Local Convergence of Method


The local convergence analysis of method (23) utilizes some functions parameters. Let
S = [0, ∞).
Suppose the following:
(i) ∃ function w0 : S −→ R continuous and non-decreasing such that equation

w0 ( t ) − 1 = 0

has a smallest solution ρ0 ∈ S − {0}. Let S0 = [0, ρ0 ).


(ii) ∃ function w : S0 −→ R continuous and non-decreasing such that equation

h1 ( t ) − 1 = 0

has a smallest solution ρ1 ∈ S0 − {0}, where the function h1 : S0 −→ R defined by


1
w((1 − θ )t)dθ
h1 ( t ) = 0
.
1 − w0 ( t )

(iii) Equation
w0 ( h 1 ( t ) t ) − 1 = 0
has a smallest solution ρ̄1 ∈ S0 − {0}. Let

ρ̄¯ 0 = min{ρ0 , ρ̄1 }

and S̃1 = [0, ρ̄¯ 0 ).


(iv) Equation
h2 ( t ) − 1 = 0
has a smallest solution ρ2 ∈ S̃1 − {0}, where the function h2 : S̃1 −→ R is defined as
3 1
w((1 − θ )h1 (t)t)dθ
h2 ( t ) = 0
1 − w0 ( h 1 ( t ) t )
1
w((1 + h1 (t))t)(1 + 0 w0 (θh1 (t)t)dθ )
+
(1 − w0 (t))(1 − w0 (h1 (t)t))
1 4
4(1 + 0 w0 (θh1 (t)t)dθ
+ h1 ( t ).
1 − w0 ( t )

71
Mathematics 2022, 10, 2621

(v) Equation
h3 ( t ) − 1 = 0
has a smallest solution ρ3 ∈ S̃1 − {0}, where the function h3 : S̃1 −→ R is defined by
1
1 9(1 + w0 (θh1 (t)t)dθ )h1 (t)
h3 ( t ) = h1 ( t ) + [ 0
5 1 − w0 ( t )
 1
(1 + w0 (θh2 (t)t)dθ )h2 (t)].
0

The parameter ρ defined by

ρ = min{ρ j } j = 1, 2, 3 (37)
is proven to be a radius of convergence for method (2) in Theorem 3. Let S1 = [0, ρ). Then,
it follows by these definitions that ∀ t ∈ S2

0 ≤ w0 ( t ) < 1 (38)

0 ≤ w0 ( h 1 ( t ) t ) < 1 (39)
and
0 ≤ hi (t) < 1. (40)
The conditions required are as follows:
(C1) Equation F ( x ) = 0 has a simple solution x∗ ∈ Ω.
(C2) F  ( x∗ )−1 ( F  ( x ) − F  ( x∗ )) ≤ w0 ( x − x∗ ) ∀ x ∈ Ω.
Set Ω1 = U ( x∗ , ρ0 ) ∩ Ω.
(C3) F  ( x∗ )−1 ( F  (y) − F  ( x )) ≤ w( y − x ) ∀ x, y ∈ Ω1
and
(C4) U [ x0 , ρ] ⊂ Ω.
Next, the main local convergence result follows for method (23).

Theorem 3. Suppose that conditions (C1)–(C4) hold and x0 ∈ U ( x∗ , ρ) − { x∗ }. Then, the


sequence { xn } generated by method (23) is well defined in U ( x∗ , ρ), remains in U ( x∗ , ρ) ∀n =
0, 1, 2, . . . and is convergent to x∗ . Moreover, the following assertions hold:

yn − x∗ ≤ h1 ( xn − x∗ ) xn − x∗ ≤ xn − x∗ < ρ, (41)

z n − x ∗ ≤ h2 ( x n − x ∗ ) x n − x ∗ ≤ x n − x ∗ , (42)
and
x n +1 − x ∗ ≤ h3 ( x n − x ∗ ) x n − x ∗ ≤ x n − x ∗ , (43)
where functions hi are defined previously and the radius ρ is given by Formula (37).

Proof. Let u ∈ U ( x∗ , ρ) − { x∗ }. By using conditions (C1), (C2) and (37), we have that

F  ( x∗ )−1 ( F  (u) − F  ( x∗ )) ≤ w0 ( x0 − x∗ ) ≤ w0 (r ) < 1. (44)

It follows by (44) and the Banach lemma on invertible operators [11,15] that F  (u)−1 ∈
L( X, X ) and
1
F  ( u ) −1 F  ( x ∗ ) ≤ . (45)
1 − w0 ( x 0 − x ∗ )
If u = x0 , then the iterate y0 is well defined by the first substep of method (23) and we
can write

72
Mathematics 2022, 10, 2621

y0 − x ∗ = x0 − x ∗ − F  ( x0 ) −1 F ( x0 )
 1
= F  ( x0 ) −1 ( F  ( x∗ + θ ( x0 − x∗ ))dθ − F  ( x0 ))( x0 − x∗ ). (46)
0

In view of (C1)–(C3), (45) (for u = x0 ), (40) (for i = 1) and (46), we obtain in turn that
1
w((1 − θ ) x0 − x∗ )dθ x0 − x∗
y0 − x ∗ ≤ 0
1 − w0 ( x 0 − x ∗ )
≤ h1 ( x0 − x∗ ) x0 − x∗ < x0 − x∗ < ρ. (47)

Thus, the iterate y0 ∈ U ( x∗ , r ) and (41) holds for n = 0. The iterate z0 is well defined
by the second substep of method (23), so we can write

z0 − x ∗ = y0 − x0 − 5F  ( x0 )−1 F (y0 )
= y0 − x ∗ − F  ( y0 ) −1 F ( y0 )
+ F  (y0 )−1 ( F ( x0 ) − F  (y0 )) F  ( x0 )−1 F (y0 )
−4F  ( x0 )−1 F (y0 ). (48)

Notice that linear operator F  (y0 )−1 exists by (45) (for u = y0 ). It follows by (37), (40)
(for j = 1), (C3), (45) (for u = x0 , y0 ), in turn that
3 1
w((1 − θ ) y0 − x∗ )dθ
z0 − x ∗ ≤ 0
1 − w0 ( y 0 − x ∗ )
1
w( y0 − x0 )(1 + 0 w0 (θ y0 − x∗ )dθ )
+
(1 − w0 ( x0 − x∗ ))(1 − w0 ( y0 − x∗ ))
1 4
4(1 + 0 w0 (θ y0 − x∗ )dθ
+ y0 − x ∗
1 − w0 ( x 0 − x ∗ )
≤ h2 ( x0 − x ∗ ) x0 − x ∗ ≤ x0 − x ∗ . (49)

Thus, the iterate z0 ∈ U ( x∗ , ρ) and (42) holds for n = 0, where we also used (C1) and
(C2) to obtain the estimate
 1
F  ( x ∗ ) −1 F ( y0 ) = F  ( x ∗ ) −1 [ F  ( x∗ + θ (y0 − x∗ ))dθ − F  ( x∗ )
0
+ F  ( x∗ )](y0 − x∗ )
 1
≤ (1 + w0 (θ y0 − x∗ )dθ ) y0 − x∗ .
0

Moreover, the iterate x1 is well defined by the third substep of method (23), so we
can have
1
x1 − x∗ = y0 − x∗ − F  ( x0 )−1 (9F (y0 ) + F (z0 )),
5
leading to
 1
1 9(1 + w0 (θ y0 − x∗ )dθ ) y0 − x∗
x1 − x ∗ ≤ y0 − x ∗ + 0
5 1 − w0 ( y 0 − x ∗ )
 1 
+ (1 + w0 ( θ z 0 − x ∗ )dθ ) z0 − x∗
0
≤ h3 ( x0 − x∗ ) x0 − x∗ ≤ x0 − x∗ < ρ. (50)

Therefore, the iterate x1 ∈ U ( x∗ , ρ) and (43) holds for n = 0.

73
Mathematics 2022, 10, 2621

Switch x0 , y0 , z0 , x1 by xm , ym , zm , xm+1 ∀m = 0, 1, 2 . . . in the preceding calculations to


complete the induction for the estimates (41)–(43). Then, by the estimate

xm+1 − x∗ ≤ d xm − x∗ < ρ, (51)

where d = h3 ( x0 − x∗ ) ∈ [0, 1), we obtain that xm+1 ∈ U ( x∗ , ρ) and limm−→∞ xm =


x∗ .

The uniqueness of the solution result for method (23) follows.

Proposition 1. Suppose the following:


(i) Equation F ( x ) = 0 has a simple solution x∗ ∈ U ( x∗ , r ) ⊂ Ω for some r > 0.
(ii) Condition (C2) holds.
(iii) There exists r1 ≥ r such that
 1
w0 (θr1 )dθ < 1. (52)
0

Set Ω2 = U [ x∗ , r1 ] ∩ Ω. Then, the only solution of equation F ( x ) = 0 in the set Ω2 is x∗ .


1
Proof. Let y∗ ∈ D2 be such that F (y∗ ) = 0. Define the linear operator J = 0 h( x∗ + θ (y∗ −
x∗ ))dθ. It then follows by (ii) and (52) that
 1
h( x∗ )−1 ( J − F  ( x∗ )) ≤ w0 (θ y∗ − x∗ )dθ
0
 1
≤ w0 (θr1 )dθ < 1.
0

Hence, we deduce x∗ = y∗ by the invertibility of J and the estimate J ( x∗ − y∗ ) =


F ( x∗ ) − F (y∗ ) = 0.

Remark 3. Under all conditions of Theorem 3, we can set ρ = r.

Example 2. Consider the motion system

F1 (v1 ) = ev1 , F2 (v2 ) = (e − 1)v2 + 1, F3 (v3 ) = 1

with F1 (0) = F2 (0) = F3 (0) = 0. Let F = ( F1 , F2 , F3 )tr . Let X = R3 , Ω = U [0, 1], x∗ =


(0, 0, 0)tr . Let function F on Ω for v = (v1 , v2 , v3 )tr given as

e−1 2
F (v) = (ev1 − 1, v + v2 , v3 )tr .
2 2
Using this definition, we obtain the derivative as
⎡ ⎤
e v1 0 0

F (v) = ⎣ 0 ( e − 1) v2 + 1 0 ⎦.
0 0 1

Hence, F  ( x∗ ) = I. Let v ∈ R3 with v = (v1 , v2 , v3 )tr . Moreover, the nor for N ∈ R3 × R3


is
3
N = max ∑
1≤ j ≤3 i =1
n j,i .

Conditions (C1)–(C3) are verified for w0 (t) = (e − 1)t and w(t) = 2(1 + e−1 ) t.
1
Then, the
radii are

ρ1 = 0.3030, ρ2 = 0.1033 = ρ and ρ3 = 0.1461.

74
Mathematics 2022, 10, 2621

Example 3. If X = C [0, 1] is equipped with the max-norm, Ω = U [0, 1], consider G : Ω −→ E1


given as
 1
G (λ)( x ) = ϕ( x ) − 6 xτλ(τ )3 dτ. (53)
0
We obtain
 1
G  (λ(ξ ))( x ) = ξ ( x ) − 18 xτλ(τ )2 ξ (τ )dτ, for each ξ ∈ D.
0

Clearly, x∗ = 0 and the conditions (C1)–(C3) hold for w0 (t) = 9t and w(t) = 18t. Then, the
radii are
ρ1 = 0.0556, ρ2 = 0.0089 = ρ and ρ3 = 0.0206.

6. Semi-Local Convergence of Method


As in the local case, we use some functions and parameters for the method (23).
Suppose:
There exists function v0 : S −→ R that is continuous and non-decreasing such
that equation
v0 ( t ) − 1 = 0
has a smallest solution τ0 ∈ S − {0}. Consider function v : S0 −→ R to be continuous and
non-decreasing. Define the scalar sequences for η ≥ 0 and ∀ n = 0, 1, 2, . . . by

t0 = 0, s0 = η
1
5 v(θ (sn − tn ))dθ (sn − tn )
un = sn + 0 ,
1 − v0 ( t n )
 1
1
t n +1 = un + [(1 + v0 (un + θ (un − sn ))dθ (un − sn ) (54)
1 − v0 ( t n ) 0
 1
+3 v(θ (sn − tn ))dθ (sn − tn )]
0
 1
1
s n +1 = t n +1 + [ v(θ (tn+1 − tn ))dθ (tn+1 − tn )
1 − v0 ( t n +1 ) 0
 1
+(1 + v0 (θtn )dθ (tn+1 − sn )].
0

This sequence is proven to be majorizing for method (23) in Theorem 4. However, first,
we provide a general convergence result for sequence (54).

Lemma 2. Suppose that ∀ n = 0, 1, 2, . . .

v0 ( t n ) < 1 (55)

and there exists τ ∈ [0, τ0 ) such that


tn ≤ τ. (56)
Then, sequence {tn } converges to some t∗ ∈ [0, τ ].

Proof. It follows by (54)–(56) that sequence {tn } is non-decreasing and bounded from
above by τ. Hence, it converges to its unique least upper bound t∗ .

Next, the operator F is related to the scalar functions.


Suppose the following:
(h1) There exists x0 ∈ Ω, η ≥ 0 such that F  ( x0 )−1 L( B2 , B1 ) and F  ( x0 )−1 F ( x0 ) ≤ η.
(h2) F  ( x0 )−1 ( F  ( x ) − F  ( x0 )) ≤ v0 ( x − x0 ) for all x ∈ Ω.
Set Ω3 = Ω ∩ U ( x0 , τ0 ).
(h3) F  ( x0 )−1 ( F  (y) − F  ( x )) ≤ v( y − x ) for all x, y ∈ Ω3 .

75
Mathematics 2022, 10, 2621

(h4) Conditions of Lemma 2 hold.


and
(h5) U [ x0 , t∗ ] ⊂ Ω.
We present the semi-local convergence result for the method (23).

Theorem 4. Suppose that conditions (h1)–(h5) hold. Then, sequence { xn } given by method (23) is
well defined, remains in U [ x0 , t∗ ] and converges to a solution x∗ ∈ U [ x0 , t∗ ] of equation F ( x ) = 0.
Moreover, the following assertions hold:

yn − xn ≤ sn − tn , (57)

zn − yn ≤ un − sn (58)
and
x n +1 − z n ≤ t n +1 − u n . (59)

Proof. Mathematical induction is utilized to show estimates (57)–(59). Using (h1) and
method (23) for n = 0

y0 − x0 = F  ( x0 ) −1 F ( x0 ) ≤ η = s0 − t0 ≤ t ∗ .

Thus, the iterate y0 ∈ U [ x0 , t∗ ] and (57) holds for n = 0.


Let u ∈ U [ x0 , t∗ ]. Then, as in Theorem 3, we get

1
F  ( u ) −1 F  ( x0 ) ≤ . (60)
1 − v0 ( u − x0

Hence, if we set u = x0 , iterates y0 , z0 and x1 are well defined by method (23) for n = 0.
Suppose iterates xk , yk , zk , xk+1 also exist for all integer values k smaller than n. Then, we
have the estimates

zn − yn = 5 F  ( x n ) −1 F ( y n )
1
5 0 v(θ yn − xn )dθ yn − xn

1 − v0 ( x n − x0 )
1
5 0 v(θ sn − tn ))dθ (sn − tn )
≤ = un − sn ,
1 − v0 ( t n )
1 
x n +1 − z n = F ( xn )−1 ( F (yn ) − F (zn )) + 3F  ( xn )−1 F (yn )
5

1 1 1
≤ [(1 + v0 ( zn − x0 + θ zn − yn )dθ ) yn − xn
1 − v0 ( x n − x0 ) 5 0
 1
+3 v(θ yn − xn dθ yn − xn ]
0
≤ t n +1 − u n

and

y n +1 − x n +1 = F  ( x n +1 ) −1 F ( x n +1 )
≤ F  ( x n +1 ) −1 F  ( x0 ) F  ( x0 ) −1 F ( x n +1 )
 1
1
≤ [ v(θ xn+1 − xn )dθ xn+1 − xn
1 − v0 ( x n +1 − x0 ) 0
 1
+(1 + v0 (θ xn − x0 )dθ ) xn+1 − yn ]
0
≤ s n +1 − t n +1 ,

76
Mathematics 2022, 10, 2621

where we also used

F (yn ) = F (yn ) − F ( xn ) − F  ( xn )(yn − xn )


 1
= [ F  ( xn + θ (yn − xn ))dθ − F  ( xn )](yn − xn ),
0

so  1
F  ( x0 ) −1 F ( y n ) ≤ v(θ yn − xn )dθ yn − xn
0
and

F ( x n +1 ) = F ( xn+1 ) − F ( xn ) − F  ( xn )(yn − xn )
− F  ( xn )( xn+1 − xn ) + F  ( xn )( xn+1 − xn )
= F ( xn+1 ) − F ( xn ) − F  ( xn )( xn+1 − xn ) + F  ( xn )( xn+1 − yn ),

so
 1
F  ( x0 ) −1 F ( x n +1 ) ≤ v(θ xn+1 − xn )dθ xn+1 − xn
0
+(1 + v0 ( xn − x0 )) xn+1 − yn
 1
≤ v(θ (tn+1 − tn ))dθ (tn+1 − tn )
0
+(1 + v0 (tn ))(tn+1 − sn ), (61)
z n − x0 ≤ z n − y n + y n − x0
≤ u n − s n + s n − t0 ≤ t ∗

and

x n +1 − x0 ≤ x n +1 − z n + z n − x0
≤ t n +1 − u n + u n − t0 ≤ t ∗ .

Hence, sequence {tn } is majorizing for method (2) and iterates { xn }, {yn }, {zn } belong
in U [ x0 , t∗ ]. The sequence { xn } is complete in Banach space X and as such, it converges to
some x∗ ∈ U [ x0 , t∗ ]. By using the continuity of F and letting n −→ ∞ in (61), we deduce
F ( x∗ ) = 0.

Proposition 2. Suppose:
(i) There exists a solution x∗ ∈ U ( x0 , ρ2 ) of equation F ( x ) = 0 for some ρ2 > 0.
(ii) Condition (h2) holds.
(iii) There exists ρ3 ≥ ρ2 such that
 1
v0 ((1 − θ )ρ2 + θρ3 )dθ < 1. (62)
0

Set Ω4 = Ω ∩ U [ x0 , ρ3 ]. Then, x∗ is the only solution of equation F ( x ) = 0 in the region Ω4 .


1
Proof. Let y∗ ∈ Ω4 with F (y∗ ) = 0. Define the linear operator Q = 0 F  ( x∗ + θ (y∗ −
x∗ ))dθ. Then, by (h2) and (62), we obtain in turn that
 1
F  ( x0 )−1 ( Q − F  ( x0 )) ≤ v0 ((1 − θ ) x0 − y∗ + θ x0 − x∗ )dθ
0
 1
≤ v0 ((1 − θ )ρ2 + θρ3 )dρ < 1.
0

Thus, x∗ = y∗ .

77
Mathematics 2022, 10, 2621

The next two examples show how to choose the functions v0 , v, and the parameter η.

Example 4. Set X = R. Let us consider a scalar function F defined on the set Ω = U [ x0 , 1 − μ]


for μ ∈ (0, 1) by
F ( x ) = x3 − μ.
1− μ
Choose x0 = 1. Then, the conditions (h1)–(h3) are verified for η = 3 , v0 ( t ) = (3 − μ ) t
and v(t) = 2(1 + 3−1 μ )t.

Example 5. Consider X = C [0, 1] and Ω = U [0, 1]. Then the problem [5]

Ξ(0) = 0, Ξ(1) = 1,

Ξ = −Ξ − ιΞ2
is also given as integral equation of the form
 1
Ξ ( q2 ) = q2 + Θ(q2 , q1 )(Ξ3 (q1 ) + ιΞ2 (q1 ))dq1
0

where ι is a constant and Θ(q2 , q1 ) is the Green’s function


!
q1 (1 − q2 ), q1 ≤ q2
Θ ( q2 , q1 ) =
q2 (1 − q1 ), q2 < q1 .

Consider F : Ω −→ X as
 1
[ F ( x )](q2 ) = x (q2 ) − q2 − Θ(q2 , q1 )( x3 (q1 ) + ιx2 (q1 ))dq1 .
0

Choose Ξ0 (q2 ) = q2 and Ω = U (Ξ0 , 0 ). Then, clearly U (Ξ0 , 0 ) ⊂ U (0, 0 + 1), since
Ξ0 = 1. If 2ι < 5. Then, conditions (C1)–(C3) are satisfied for

2ι + 3ρ0 + 6 ι + 6ρ0 + 3
w0 ( t ) = t, w(t) = t.
8 4
Hence, w0 (t) ≤ w(t).

7. Local Convergence of Method


The local analysis is using on certain parameters and real functions. Let L0 , L and α be
positive parameters. Set T1 = [0, (2+1α) L ] provided that (2 + α) L0 < 1.
0
Define the function h1 : T1 −→ R by

(1 + α) Lt
h1 ( t ) = .
1 − (2 + α ) L0 t

Notice that parameter ρ

1
ρ=
(1 + α ) L + (2 + α ) L0

is the only solution of equation


h1 ( t ) − 1 = 0
in the set T1 .
Define the parameter ρ0 by

1
ρ0 = .
(2 + α)( L0 + L)

78
Mathematics 2022, 10, 2621

Notice that ρ0 < ρ. Set T0 = [0, ρ0 ].


Define the function h2 : T0 −→ R by

(2 + 2α + h1 (t)) Lh1 (t)t


h2 ( t ) = .
1 − (2 + α)( L0 + L)t

The equation
h2 ( t ) − 1 = 0
has a smallest solution ρ ∈ T0 − {0} by the intermediate value theorem, since h2 (0) − 1 = −1
and h2 (t) −→ ∞ as y −→ ρ0− . It shall be shown that R is a radius of convergence for
method (20). It follows by these definitions that ∀t ∈ T0

0 ≤ ( L0 + L)(2 + α)t < 1 (63)


0 ≤ h1 ( t ) < 1 (64)
and
0 ≤ h2 (t) < 1. (65)
The following conditions are used:
(C1) There exists a solution x∗ ∈ Ω of equation F ( x ) = 0 such that F  ( x∗ )−1 ∈ L( X, X ).
(C2) There exist positive parameters L0 and α such that ∀v, z ∈ Ω

F  ( x∗ )−1 ([v, z; F ] − F  ( x∗ )) ≤ L0 ( v − x∗ + z − x∗ )

and
F ( x ) ≤ α x − x∗ .
Set Ω1 = U ( x∗ , ρ) ∩ Ω.
(C3) There exists a positive constant L > 0 such that ∀ x, y, v, z ∈ Ω1

F  ( x∗ )−1 ([ x, y; F ] − [v, z; F ]) ≤ L( x − v + y − z )

and
(C4) U [ x0 , ρ] ⊂ Ω.
Next, the local convergence of method (20) is presented using the preceding terminol-
ogy and conditions.

Theorem 5. Under conditions (C1)–(C4), further suppose that x0 ∈ U ( x∗ , ρ). Then, the sequence
{ xn } generated by method (20) is well defined in U ( x∗ , ρ), stays in U ( x∗ , ρ) ∀n = 0, 1, 2, . . . and
is convergent to x∗ so that

y n − x ∗ ≤ h1 ( x n − x ∗ ) x n − x ∗ ≤ x n − x ∗ < Ω (66)

and
x n +1 − x ∗ ≤ h2 ( x n − x ∗ ) x n − x ∗ ≤ x n − x ∗ , (67)
where the functions h1 , h2 and the radius ρ are defined previously.

Proof. It follows by method (20), (C1), (C2) and x0 ∈ U ( x∗ , ρ) in turn that

F  ( x∗ )−1 ( A0 − F  ( x∗ )) = F  ( x∗ )−1 ([ x0 , x0 + F ( x0 ); F ] − F  ( x∗ ))
≤ L0 (2 x0 − x ∗ + F ( x0 ) − F ( x ∗ ) )
≤ L0 (2 + α ) x0 − x ∗
< L0 (2 + α)ρ. (68)

It follows by (68) and the Banach lemma on invertible operators [24] that A0−1 ∈
L( X, X ) and

79
Mathematics 2022, 10, 2621

1
A0−1 F  ( x∗ ) ≤ . (69)
1 − (2 + α ) L0 x0 − x ∗
Hence, the iterate y0 exists by the first substep of method (20) for n = 0. It follows
from the first substep of method (20), (C2) and (C3), that

y0 − x ∗ ≤ x0 − x∗ − A0−1 F ( x0 )
A0−1 F  ( x∗ ) F  ( x∗ )−1 ( A0 − ( F ( x0 ) − F ( x∗ )))( x − 0 − x∗ )
≤ A0−1 F  ( x∗ ) F  ( x∗ )−1 ( A0 − ( F ( x0 ) − F ( x∗ ))) x0 − x ∗
L( x0 − x∗ + F ( x0 ) − F ( x∗ ))
≤ (70)
1 − L0 (2 + α ) x0 − x ∗
≤ h1 ( x0 − x∗ ) x0 − x∗ ≤ x0 − x∗ < ρ.

Thus, the iterate y0 ∈ U ( x∗ , ρ) and (66) holds for n = 0. Similarly, by the second
substep of method (20), we have

F  ( x∗ )−1 ( B0 − F  ( x∗ )) = F  ( x∗ )−1 ([y0 , w0 ; F ]


−[y0 , x0 ; F ] − [ x0 , w0 ; F ] − [ x∗ , x∗ ; F ])
≤ L y 0 − w0 + L 0 ( y 0 − x ∗ + w0 − x ∗ )
≤ L ( y 0 − x ∗ + w0 − x ∗ ) + L 0 ( y 0 − x ∗ + w0 − x ∗ )
L + L0
≤ ( L + L0 )(2 + α)ρ ≤ = 1. (71)
L + L0

Hence, B0−1 ∈ L( X, X ) and

1
B0−1 F  ( x∗ ) ≤ . (72)
1 − ( L + L0 )(2 + α) x0 − x∗

Thus, the iterate x1 exists by the second sub-step of method (20). Then, as in (70) we
obtain in turn that

x1 − x ∗ ≤ y0 − x∗ − B0−1 F (y0 )
≤ B0−1 F  ( x∗ ) F  ( x∗ )−1 ( B0 − ( F (y0 ) − F ( x∗ ))) y0 − x ∗
F ( x ∗ )−1 ([y 0 , w0 ; F ] + [ y 0 , x 0 ; F ] − [ x 0 , w0 ; F ] − [ y 0 , x ∗ : F ])

1 − ( L + L0 )(2 + α) x0 − x∗
y0 − x ∗
L(2 + 2α + h2 ( x0 − x∗ )) x0 − x∗
≤ h1 ( x0 − x ∗ )
1 − ( L + L0 )(2 + α) x0 − x∗
x0 − x ∗ (73)
≤ h2 ( x0 − x∗ ) x0 − x∗ ≤ x0 − x∗ < ρ.

Therefore, the iterate x1 ∈ U ( x∗ , ρ) and (67) holds for n = 0.


Simply replace x0 , y0 , x1 by xm , ym , xm+1 ∀m = 0, 1, 2 . . . in the preceding calculations
to complete the induction for (66) and (67). It then follows from the estimate

xm+1 − x∗ ≤ μ xm − x∗ < ρ, (74)

where, μ = h2 ( x0 − x∗ ) ∈ [0, 1) leading to xm+1 ∈ U ( x∗ , ρ) and limm−→∞ xm = x∗ .

Concerning the uniqueness of the solution x∗ (not given in [9]), we provide the result.

80
Mathematics 2022, 10, 2621

Proposition 3. Suppose:
(i) The point x∗ is a simple solution x∗ ∈ U ( x∗ , r ) ⊂ Ω for some r > 0 of equation F ( x ) = 0.
(ii) There exists positive parameter L1 such that ∀y ∈ Ω

F  ( x∗ )−1 ([ x∗ , y; F ] − F  ( x∗ )) ≤ L1 y − x∗ (75)

(iii) There exists r1 ≥ r such that


L1 r1 < 1. (76)
Set Ω2 = U [ x∗ , r1 ] ∩ Ω. Then, x∗ is the only solution of equation F ( x ) = 0 in the set Ω2 .

Proof. Set P = [ x∗ , y∗ ; F ] for some y∗ ∈ D2 with F (y∗ ) = 0. It follows by (i), (75) and (76)
that

F  ( x∗ )−1 ( P − F  ( x∗ )) ≤ L1 y∗ − x∗ ) < 1.
Thus, we conclude x∗ = y∗ by the invertability of P and identity P( x∗ − y∗ ) =
F ( x∗ ) − F (y∗ ) = 0.

Remark 4. (i) Notice that not all conditions of Theorem 5 are used in Proposition 3. If they were,
then we can set r1 = ρ.
(ii) By the definition of set Ω1 we have
Ω1 ⊂ Ω. (77)
Therefore, the parameter
L ≤ L2 , (78)
where L2 is the corresponding Lipschitz constant in [1,3,9,19] appearing in the condition ∀x, y, z ∈ Ω

F  ( x∗ )−1 ([ x, y; F ] − [v, z; F ]) ≤ L2 ( x − v + y − z ). (79)

Thus, the radius of convergence R0 in [1,7,8,20] uses L2 instead of L. That is by (78)

R0 ≤ ρ. (80)

Examples where (77), (78) and (80) are strict can be found in [2,5,11–13,15,21–24].

8. Majorizing Sequences for Method


Let K0 , K, be given positive parameters and δ ∈ [0, 1), K0 ≤ K, η ≥ 0, and T = [0, 1).
Consider recurrent polynomials defined on the interval T for n = 1, 2, . . . by

(1)
f n (t) = Kt2n η + Kt2n−1 η + 2K0 (1 + t + . . . + t2n+1 )η
+K0 (t2n+1 + 2t2n )t2n+1 η + δ − 1,
(2)
f n (t) = Kt2n+1 η + K (t2n+1 + 2t2n )t2n η
+2K0 (1 + t + . . . + t2n+2 )η + δ − 1,
(1)
gn ( t ) = Kt3 + Kt2 − Kt − K + 2K0 (t3 + t4 )
+K0 (t2n+3 + 2tn+2 )t4 η − K0 (t2n+1 + 2t2n )t2 η,
(2)
gn ( t ) = Kt3 + K (t3 + 2t2 )t2n+2 η
+2K0 (t3 + t4 ) − Kt − K (t + 2)t2n η,
(1) (1) (1)
h n +1 ( t ) = gn +1 ( t ) − gn ( t ),
(2) (2) (2)
h n +1 ( t ) = gn +1 ( t ) − gn ( t ),

81
Mathematics 2022, 10, 2621

and polynomials

(1)
g∞ (t) = g1 (t) = Kt3 + Kt2 − Kt − K + 2K0 (t3 + t4 ),

(2)
g∞ (t) = g2 (t) = Kt3 + 2K0 (t3 + t4 ) − Kt = g3 (t)t
and
g(t) = (t − 1)2 (t5 + 4t4 + 6t3 + 6t2 + 5t + 2).
Then, the following auxiliary result connecting these polynomials can be shown.

Lemma 3. The following assertions hold:

(1) (1) (1)


f n+1 (t) = f n (t) + gn (t)t2n−1 η, (81)

(2) (2) (2)


f n+1 (t) = f n (t) + gn (t)t2n η, (82)
(1)
h n +1 ( t ) = g(t)K0 t2n+2 η, (83)
(2)
hn+1 (t) = g(t)Kt2n η, (84)
polynomials g1 and g2 have smallest zeros in the interval T − {0} denoted by ξ 1 and α2 , respectively,

(1)
hn+1 (t) ≥ 0 ∀ t ∈ [0, ξ 1 ) (85)

and
(2)
hn+1 (t) ≥ 0 ∀ t ∈ [0, ξ 2 ). (86)
Moreover, define functions on the interval T by

(1) (1)
g∞ (t) = lim gn (t) (87)
n−→∞

and
(2) (2)
g∞ (t) = lim gn (t). (88)
n−→∞

Then,
(1)
g∞ (t) = g1 (t) ∀ t ∈ [0, α1 ), (89)
(2)
g∞ (t) = g2 (t) ∀ t ∈ [0, α2 ), (90)
(1) (1)
f n +1 ( t ) ≤ f n (t) + g1 (t)t2n−1 η ∀ t ∈ [0, ξ 1 ), (91)
(2) (2)
f n+1 (t) ≤ f n (t) + g2 (t)t2n η ∀ t ∈ [0, ξ 2 ), (92)
(1) (1)
f n +1 ( ξ 1 ) ≤ f n ( ξ 1 ), (93)
and
(2) (2)
f n +1 ( ξ 2 ) ≤ f n ( ξ 2 ). (94)

Proof. Assertions (81)–(84) hold by the definition of these functions and basic algebra. By
the intermediate value theorem polynomials g1 and g3 have zeros in the interval T − {0},
since g1 (0) = −K, g1 (1) = 4K0 , g2 (0) = −K and g2 (1) = 4K0 . Then, assertions (85) and
(86) follow by the definition of these polynomials and zeros ξ 1 and ξ 2 . Next, assertions (91)
and (94) also follow from (87), (88) and the definition of these polynomials.

82
Mathematics 2022, 10, 2621

The preceding result is connected to the scalar sequence defined ∀ n = 0, 1, 2, . . . by


t0 = 0, s0 = η,

K (η + δ)η
t1 = s0 + ,
1 − K0 (2η + δ)
K (tn+1 − tn + sn − tn )(tn+1 − sn )
s n +1 = t n +1 + (95)
1 − K0 (2tn+1 + γn + δ)
K (sn+1 − tn+1 + γn )(sn+1 − tn+1 )
t n +2 = s n +1 + ,
1 − K0 (2sn+1 + δ)

where γn = K (tn+1 − tn + sn − tn )(tn+1 − sn ), δ ≥ γ0 .


K (s −t +γ ) K (t +s )
Moreover, define parameters ξ 1 = 1−K1 (2s1 +0δ) , ξ 2 = 1−K (2t1 +γ0 +δ) and a = max{ξ 1 , ξ 2 },
0 1 0 1 0
Then, the first convergence result for sequence {tn } follows.

Lemma 4. Suppose
Kη ≤ 1, 0 < ξ 1 , 0 < ξ 2 , a < ξ < 1, (96)
(1)
f 1 (ξ 1 ) ≤0 (97)
and
(1)
f 2 (ξ 2 ) ≤ 0. (98)
η
Then, scalar sequence {tn } is non-decreasing, bounded from above by t∗∗ = 1− ξ ,
and con-
verges to its unique least upper bound t∗ ∈ [0, t∗∗ ]. Moreover, the following error bounds hold

0 < tn+1 − sn ≤ ξ (sn − tn ) ≤ ξ 2n+1 η, (99)

0 < sn − tn ≤ ξ (tn − sn−1 ) ≤ ξ 2n η (100)


and
γn+1 ≤ γn ≤ γ0 . (101)

Proof. Assertions (99)–(101) hold if we show using induction that

K ( t n +1 − t n + s n − t n )
0< ≤ ξ1, (102)
1 − K0 (2tn+1 + γn + δ)

K ( s n + 1 − t n + 1 + γn )
0< ≤ ξ2, (103)
1 − K0 (2sn+1 + δ)
and
t n ≤ s n ≤ t n +1 . (104)
By the definition of t1 , we obtain

t1 1 − Kη
= > 1,
s0 1 − K0 (2η + δ)

so s 0 < t 1 , and (103) holds for n = 0. Suppose assertions (101)–(103) hold for each
m = 0, 1, 2, 3, . . . , n. By (99) and (100) we have

sm ≤ tm + ξ 2m η ≤ sm−1 + ξ 2m−1 η + ξ 2m η
≤ η + ξη + . . . + ξ 2m η
1 − ξ 2m+1
= η ≤ t∗∗ (105)
1−ξ

and

83
Mathematics 2022, 10, 2621

t m +1 ≤ sm + ξ 2m+1 η ≤ tm + ξ 2m+1 η + ξ 2m η
≤ η + ξη + . . . + ξ 2m+1 η
1 − ξ 2m+2
= η ≤ t∗∗ . (106)
1−ξ

By the induction hypotheses sequences {tm }, {sm } are increasing. Evidently, estimate
(101) holds if

1 − ξ 2m+2
Kξ 2m+1 η + Kξ 2m η + 2K0 ξ η
1−ξ
+K0 ξδ + ξγm K0 − ξ ≤ 0

or
(1)
f m (t) ≤ 0 at t = ξ 1 , (107)
where γm ≤ K (ξ 2m+1 + 2ξ 2m )ξ 2m+1 η 2 .
By (91), (93), and (98) estimate (107) holds.
Similarly, assertion (103) holds if

Kξ 2m+2 η + K2 (ξ 2m+1 η + 2ξ 2m η )ξ 2m+1 η


+2ξK0 (1 + ξ + . . . + ξ 2m+2 )η + δξ − ξ ≤ 0

or
(2)
f m (t) ≤ 0 at t = ξ 2 . (108)
By (92) and (94), assertion (108) holds. Hence, (100) and (103) also hold. Notice that
γn can be written as γn = K ( En + En1 ) En2 , where En = tn+1 − tn > 0, En1 = sn − tn , and
En2 = tn+1 − sn > 0. Hence, we get

En+1 − En = tn+2 − 2tn+1 + tn ≤ ξ 2n (ξ 2 − 1)(ξ + 1)η < 0,

En1 +1 − En1 = sn+1 − tn+1 − (sn − tn ) ≤ ξ 2n (ξ 2 − 1)η < 0,


and
En2 +1 − En2 = tn+2 − sn+1 − (tn+1 − sn ) ≤ ξ 2n+1 (ξ 2 − 1)η < 0,
so
γn+1 ≤ γn ≤ γ0 .
It follows that sequence {tn } is non-decreasing, bounded from above by t∗∗ . Thus, it
converges to t∗ .

Next, a second convergence result for sequence (95) is presented but the sufficient
criteria are weaker but more difficult to verify than those of Lemma 4.

Lemma 5. Suppose
K0 δ < 1, (109)
K0 (2tn+1 + γn + δ) < 1, (110)
and
K0 (2sn+1 + δ) < 1 (111)
1 − K0 δ
hold. Then, sequence {tn } is increasing and bounded from above by t1∗∗ = 2K0 , so it converges to
its unique least upper bound t1∗ ∈ [0, t1∗∗ ].

Proof. It follows from the definition of sequence (95), and conditions (109)–(111).

9. Semi-Local Convergence of Method


The conditions (C) shall be used in the semi-local convergence analysis of method (20).

84
Mathematics 2022, 10, 2621

Suppose
(C1) There exist x0 ∈ Ω, η ≥ 0, δ ∈ [0, 1) such that A0−1 ∈ L( X, X ), A0−1 F ( x0 ) ≤ η, and
F ( x0 ) ≤ δ.
(C2) There exists K0 > 0 such that for all u, v ∈ Ω

A0−1 ([u, v; F ] − A0 ) ≤ K0 ( u − x0 + v − w0 ).

Set Ω0 = U ( x0 , 1− K0 δ
2K0 ) ∩ Ω for K0 δ < 1.
(C3) There exists K > 0 such that for all u, v, ū, v̄ ∈ Ω0

A0−1 ([u, v; F ] − [ū, v̄; F ]) ≤ K ( u − ū + v − v̄ ).


!
t∗ + γ0 or t∗∗ , if conditions of Lemma 4 hold
(C4) U [ x0 , ρ + δ] ⊂ Ω, where ρ =
t1∗ + γ0 or t1∗∗ , if conditions of Lemma 5 hold.

Remark 5. The results in [19] are given in the non-affine form. The benefits of using affine invariant
results over non-affine are well-known [1,5,11,21]. In particular, they assumed A0−1 ≤ β and
(C3)’ [ x, y; F ] − [ x̄, ȳ; F ] ≤ K̄ ( x − x̄ + y − ȳ ) holds for all x, y, x̄ ȳ ∈ Ω. By the definition
of the set Ω0 , we get

Ω0 ⊂ Ω, (112)
so
K0 ≤ βK̄ (113)
and
K ≤ βK̄. (114)
Hence, K can replace βK̄ in the results in [19]. Notice also that using (C3)’ they estimated

1
Bn−+1 1 A0 ≤ (115)
1 − βK̄ (2s̄n+1 + δ)

and
1
A0−1 ( An+1 − A0 ) ≤ , (116)
1 − βK̄ (t̄n+1 − t̄0 ) + γ̄n + δ)
where {t̄n }, {s̄n } are defined for n = 0, 1, 2, . . . by t̄0 = 0, s̄0 = η,

βK̄ (η + δ)η
t̄1 = s̄0 + ,
1 − βK̄ (2s̄0 + δ)
βγ̄
s̄n+1 = t̄n+1 + (117)
1 − βK̄ (2t̄n+1 + γ̄n + δ)
βK̄ (s̄n+1 − t̄n+1 + γ̄n )(s̄n+1 − t̄n+1 )
t̄n+2 = s̄n+1 + ,
1 − βK̄ (2s̄n+1 + δ)

where γ̄n = K̄ (t̄n+1 − t̄n + s̄n − t̄n )(t̄n+1 − s̄n ), δ ≥ γ̄0 . But using the weaker condition (C2) we
obtain respectively,
1
Bn−+1 1 A0 ≤ (118)
1 − K0 (2sn+1 + δ)
and
1
A0−1 ( An+1 − A0 ) ≤ (119)
1 − K0 ( t n + 1 − t 0 + γn + δ )
which are tighter estimates than (115) and (116), respectively. Hence, K0 , K can replace βK̄, β, K̄
and (118), (119) can replace (115), (116), respectively, in the proof of Theorem 3 in [19]. Examples
where (112)–(114) are strict can be found in [1,5,11,21]. Simple induction shows that

85
Mathematics 2022, 10, 2621

0 < sn − tn ≤ s̄n − t̄n (120)

0 < tn+1 − sn ≤ t̄n+1 − s̄n (121)


and
t∗ ≤ t̄∗ = lim t̄n . (122)
n−→∞

These estimates justify the claims made at the introduction of this work along the same lines.
The local results in [19] can also be extended using our technique.

Next, we present the semi-local convergence result for the method (20).

Theorem 6. Suppose that conditions (C) hold. Then, iteration { xn } generated by method (20)
exists in U [ x0 , t∗ ], remains in U [ x0 , t∗ ] and limn−→∞ xn = x∗ ∈ U [ x0 , t∗ ] with F ( x∗ ) = 0, so
that
xn − x∗ ≤ t∗ − tn .

Proof. It follows from the comment above Theorem 6.

Next, we present the uniqueness of the solution result, where conditions (C) are not
necessarily utilized.

Proposition 4. Suppose the following:


(i) There exists a simple solution x∗ ∈ U ( x0 , r ) ⊂ Ω for some r > 0.
(ii) Condition (C2) holds
and
(iii) There exists r ∗ ≥ r such that K0 (r + r ∗ + δ) < 1.
1 − K0 ( δ + r )
Set Ω1 = U ( x0 , K0 ) ∩ Ω. Then, the element x∗ is the only solution of equation
F ( x ) = 0 in the region Ω1 .

Proof. Let z∗ ∈ Ω1 with F (z∗ ) = 0. Define Q = [ x∗ , z∗ ; F ]. Then, in view of (ii) and (iii),

A0−1 ( Q − A0 ) ≤ K0 ( x∗ − x0 + z∗ − w0 ≤ K0 (r + r ∗ + δ) < 1.

Therefore, we conclude z∗ = x∗ is a consequence of the invertibility of Q and the


identity Q( x∗ − z∗ ) = F ( x∗ ) − F (z∗ ) = 0.

Remark 6. (i) Notice that r can be chosen to be t∗ .


(ii) The results can be extended further as follows. Replace
(C3) A0−1 ([u, v; F ] − [ū, v̄; F ]) ≤ K̃ ( u − ū + v − v̄ ), ∀ u, ū ∈ Ω0 , v = u − A(u)−1 F (u)
and v̄ = A(ū)−1 F (ū). Then, we have
(iii) K̃ ≤ K.
1−K0 (δ+γ0 )
Another way is if we define the set Ω2 = U ( x1 , 2K0 − η ) provided that K0 (δ + γ0 ) <
1. Moreover, suppose Ω2 ⊂ Ω. Then, we have Ω2 ⊂ Ω0 if condition (C3) on Ω2 , say, with
constant K̃0 . Then, we have that
K̃0 ≤ K
also holds. Hence, tighter K̃ or K̃0 can replace K in Theorem 6.

10. Conclusions
The convergence analysis is developed for generalized three-step numerical methods.
The advantages of the new approach include weaker convergence criteria and a uniform
set of conditions utilizing information on these methods in contrast to earlier works on
special cases of these methods, where the existence of high-order derivatives is assumed to
prove convergence. The methodology is very general and does not depend on the methods.

86
Mathematics 2022, 10, 2621

That is why it can be applied to multi-step and other numerical methods that shall be the
topic of future work.
The weak point of this methodology is the observation that the computation of the
majorant functions “h” at this generality is hard in general. Notice that this is not the
case for the special cases of method (2) or method (3) given below them (see, for example,
Examples 4 and 5). As far as we know, there is no other methodology that can be compared
to the one introduced in this article to handle the semi-local or the local convergence of
method (2) or method (3) at this generality.

Author Contributions: Conceptualization, M.I.A., I.K.A., S.R. and S.G.; methodology, M.I.A., I.K.A.,
S.R. and S.G.; software, M.I.A., I.K.A., S.R. and S.G.; validation, M.I.A., I.K.A., S.R. and S.G.; formal
analysis, M.I.A., I.K.A., S.R. and S.G.; investigation, M.I.A., I.K.A., S.R. and S.G.; resources, M.I.A.,
I.K.A., S.R. and S.G.; data curation, M.I.A., I.K.A., S.R. and S.G.; writing—original draft preparation,
M.I.A., I.K.A., S.R. and S.G.; writing—review and editing, M.I.A., I.K.A., S.R. and S.G.; visualization,
M.I.A., I.K.A., S.R. and S.G.; supervision, M.I.A., I.K.A., S.R. and S.G.; project administration, M.I.A.,
I.K.A., S.R. and S.G.; funding acquisition, M.I.A., I.K.A., S.R. and S.G. All authors have read and
agreed to the published version of the manuscript.
Funding: This research received no external funding.
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: Not applicable.
Conflicts of Interest: The authors declare no conflict of interest.

References
1. Appell, J.; DePascale, E.; Lysenko, J.V.; Zabrejko, P.P. New results on Newton-Kantorovich approximations with applications to
nonlinear integral equations. Numer. Funct. Anal. Optim. 1997, 18, 1–17. [CrossRef]
2. Ezquerro, J.A.; Hernandez, M.A. Newton’s Method: An Updated Approach of Kantorovich’s Theory; Birkhäuser: Cham Switzer-
land, 2018.
3. Proinov, P.D. New general convergence theory for iterative processes and its applications to Newton-Kantorovich type theorems.
J. Complex. 2010, 26, 3–42. [CrossRef]
4. Regmi, S.; Argyros, I.K.; George, S.; Argyros, C. Numerical Processes for Approximating Solutions of Nonlinear Equations.
Axioms 2022, 11, 307. [CrossRef]
5. Argyros, I.K. The Theory and Applications of Iteration Methods, 2nd ed.; Engineering Series; CRC Press: Boca Raton, FL, USA; Taylor
and Francis Group: Abingdon, UK, 2022.
6. Zhanlav, K.H.; Otgondorj, K.H.; Sauul, L. A unified approach to the construction of higher-order derivative-free iterative methods
for solving systems of nonlinear equations. Int. J. Comput. Math. 2021.
7. Zhanlav, T.; Chun, C.; Otgondorj, K.H.; Ulziibayar, V. High order iterations for systems of nonlinear equations. Int. J. Comput.
Math. 2020, 97, 1704–1724. [CrossRef]
8. Wang, X. An Ostrowski-type method with memory using a novel self-accelerating parameters. J. Comput. Appl. Math. 2018, 330,
710–720. [CrossRef]
9. Moccari, M.; Lofti, T. On a two-step optimal Steffensen-type method: Relaxed local and semi-local convergence analysis and
dynamical stability. J. Math. Anal. Appl. 2018, 468, 240–269. [CrossRef]
10. Shakhno, S.M.; Gnatyshyn, O.P. On an iterative Method of order 1.839. . . for solving nonlinear least squares problems. Appl. Math.
Comput. 2005, 161, 253–264.
11. Argyros, I.K. Unified Convergence Criteria for Iterative Banach Space Valued Methods with Applications. Mathematics 2021,
9, 1942. [CrossRef]
12. Potra, F.-A.; Pták, V. Nondiscrete Induction and Iterative Processes; Pitman Publishing: Boston, MA, USA, 1984.
13. Cordero, A.; Torregrosa, J.R. Variants of Newton’s method using fifth-order quadrature formulas. Appl. Math. Comput. 2007, 190,
686–698. [CrossRef]
14. Traub, J.F. Iterative Methods for the Solution of Equations; Prentice Hall: Hoboken, NJ, USA, 1964.
15. Kantorovich, L.V.; Akilov, G.P. Functional Analysis; Pergamon Press: Oxford, UK, 1982.
16. Xiao, X.; Yin, H. Achieving higher order of convergence for solving systems of nonlinear equations. Appl. Math. Comput. 2017,
311, 251–261. [CrossRef]
17. Sharma, J.R.; Arora, H. Efficient derivative-free numerical methods for solving systems of nonlinear equations. Comput. Appl.
Math. 2016, 35, 269–284. [CrossRef]

87
Mathematics 2022, 10, 2621

18. Sharma, J.R.; Guha, R.K. Simple yet efficient Newton-like method for systems of nonlinear equations. Calcolo 2016, 53, 451–473.
[CrossRef]
19. Noor, M.A.; Waseem, M. Some iterative methods for solving a system of nonlinear equations. Comput. Math. Appl. 2009, 57,
101–106. [CrossRef]
20. Wang, X.; Zhang, T. A family of Steffensen type methods with seventh-order convergence. Numer. Algor. 2013, 62, 429–444.
[CrossRef]
21. Argyros, I.K.; Magréñan, A.A. A Contemporary Study of Iterative Methods; Elsevier: Amsterdam, The Netherlands; Academic Press:
New York, NY, USA, 2018.
22. Grau-Sanchez, M.; Grau, A.; Noguera, M. Ostrowski type methods for solving system of nonlinear equations. Appl. Math. Comput.
2011, 218, 2377–2385. [CrossRef]
23. Homeier, H.H.H. A modified Newton method with cubic convergence: The multivariate case. J. Comput. Appl. Math. 2004, 169,
161–169. [CrossRef]
24. Kou, J.; Wang, X.; Li, Y. Some eight order root finding three-step methods. Commun. Nonlinear Sci. Numer. Simul. 2010, 15, 536–544.
[CrossRef]
25. Verma, R. New Trends in Fractional Programming; Nova Science Publisher: New York, NY, USA, 2019.

88
mathematics

Article
A Methodology for Obtaining the Different Convergence
Orders of Numerical Method under Weaker Conditions
Ioannis K. Argyros 1 , Samundra Regmi 2 , Stepan Shakhno 3, * and Halyna Yarmola 4

1 Department of Mathematical Sciences, Cameron University, Lawton, OK 73505, USA


2 Department of Mathematics, University of Houston, Houston, TX 77204, USA
3 Department of Theory of Optimal Processes, Ivan Franko National University of Lviv, Universytetska Str. 1,
79000 Lviv, Ukraine
4 Department of Computational Mathematics, Ivan Franko National University of Lviv, Universytetska Str. 1,
79000 Lviv, Ukraine
* Correspondence: [email protected]

Abstract: A process for solving an algebraic equation was presented by Newton in 1669 and later
by Raphson in 1690. This technique is called Newton’s method or Newton–Raphson method and is
even today a popular technique for solving nonlinear equations in abstract spaces. The objective of
this article is to update developments in the convergence of this method. In particular, it is shown
that the Kantorovich theory for solving nonlinear equations using Newton’s method can be replaced
by a finer one with no additional and even weaker conditions. Moreover, the convergence order two
is proven under these conditions. Furthermore, the new ratio of convergence is at least as small. The
same methodology can be used to extend the applicability of other numerical methods. Numerical
experiments complement this study.

Keywords: nonlinear equation; criterion; integral equation; convergence


Citation: Argyros, I.K.; Regmi, S.;
Shakhno, S.; Yarmola, H. A
MSC: 49M15; 47H17; 65G99; 65H10; 65N12; 58C15
Methodology for Obtaining the
Different Convergence Orders of
Numerical Method under Weaker
Conditions. Mathematics 2022, 10,
2931. https://doi.org/10.3390/
1. Introduction
math10162931 Given Banach spaces U , V . Let L(U , V ) stand for the space of all continuous linear oper-
ators mapping U into V . Consider differentiable as per Fréchet operator
Academic Editors: Maria Isabel
L : D ⊆ U −→ V and its corresponding nonlinear equation
Berenguer and Manuel Ruiz Galán

Received: 22 July 2022 L( x ) = 0, (1)


Accepted: 10 August 2022
Published: 14 August 2022 with D denoting a nonempty open set. The task of determining a solution x ∗ ∈ D is very
challenging but important, since applications from numerous computational disciplines are
Publisher’s Note: MDPI stays neutral
brought in form (1) [1,2]. The analytic form of x ∗ is rarely attainable. That is why mainly
numerical methods are used generating approximations to solution x ∗ . Most of them are
with regard to jurisdictional claims in
published maps and institutional affil-
based on Newton’s method [3–7]. Moreover, authors developed efficient high-order and
iations.
multi-step algorithms with derivative [8–13] and divided differences [14–18].
Among these processes the most widely used is Newton’s and its variants. In particu-
lar, Newton’s Method (NM) is developed as
Copyright: © 2022 by the authors.
Licensee MDPI, Basel, Switzerland. x0 ∈ D, xn+1 = xn − L ( xn )−1 L( xn ) ∀ n = 0, 1, 2, . . . . (2)
This article is an open access article
There exists a plethora of results related to the study of NM [3,5–7,19–21]. These
distributed under the terms and
papers are based on the theory inaugurated by Kantorovich and its variants [21]. Basically,
conditions of the Creative Commons
the conditions (K) are used in non-affine or affine invariant form. Suppose (K1) ∃ point
Attribution (CC BY) license (https://
x0 ∈ D and parameter s ≥ 0 : L ( x0 )−1 ∈ L(V , U ), and
creativecommons.org/licenses/by/
4.0/).

Mathematics 2022, 10, 2931. https://doi.org/10.3390/math10162931 https://www.mdpi.com/journal/mathematics


89
Mathematics 2022, 10, 2931

L ( x0 )−1 L( x0 ) ≤ s,
(K2) ∃ parameter M1 > 0 : Lipschitz condition

L ( x0 )−1 (L (w1 ) − L (w2 )) ≤ M1 w1 − w2


holds ∀w1 ∈ D and w2 ∈ D,
(K3)

1
s≤
2M1
and
(K4) B[ x0 , ρ] ⊂ D, where parameter ρ > 0 is given later. √
1 − 2M1 s1−
Denote B[ x0 , r ] := { x ∈ D : x − x0 ≤ r } for r > 0. Set ρ = r1 = .
M1
There are many variants of Kantorovich’s convergence result for NM. One of these
results follows [4,7,20].

Theorem 1. Under conditions (K) for ρ = r1 ; NM is contained in B( x0 , r1 ), convergent to a


solution x ∗ ∈ B[ x0 , r1 ] of Equation (1), and

x n +1 − x n ≤ u n +1 − u n .
1 1
Moreover, the convergence is linear if s = and quadratic if s < . Further-
2M1 2M1
solution is unique B[ x0 , r1 ] in the first case and in B( x0 , r2 ) in the second case where
more, the √
1 + 1 − 2M1 s
r2 = and scalar sequence {un } is given as
M1
M1 (un − un−1 )2
u0 = 0, u1 = s, un+1 = un + .
2(1 − M1 un )

A plethora of studies have used conditions (K) [3–5,19,21–23].

Example 1. Consider the cubic polynomial

c( x ) = x3 − a
1
for D = B( x0 , 1 − a) and parameter a ∈ (0, ). Select initial point x0 = 1. Conditions (K) give
2
1−a
s= and M1 = 2(2 − a). It follows that estimate
3
1−a 1
>
3 4(2 − a )
1
holds ∀ a ∈ (0, ). That is condition (K3) is not satisfied. Therefore convergence is not assured by
2
this theorem. However, NM may converge. Hence, clearly, there is a need to improve the results
based on the conditions K.

By looking at the crucial sufficient condition (K3) for the convergence, (K4) and the
majorizing sequence given by Kantorovich in the preceding Theorem 1 one sees that if
the Lipschitz constants M1 is replaced by a smaller one, say L > 0, than the convergence
domain will be extended, the error distances xn+1 − xn , xn − x ∗ will be tighter and the
location of the solution more accurate. This replacement will also lead to fewer Newton
iterates to reach a certain predecided accuracy (see the numerical Section). That is why with
the new methodology, a new domain is obtained inside D that also contains the Newton

90
Mathematics 2022, 10, 2931

iterates. However, then, L can replace M1 in Theorem 1 to obtain the aforementioned


extensions and benefits.
In this paper several avenues are presented for achieving this goal. The idea is to
replace Lipschitz parameter M1 by smaller ones.
(K5) Consider the center Lipschitz condition

L ( x0 )−1 (L (w1 ) − L ( x0 )) ≤ M0 w1 − x0 ∀w1 ∈ D,


1
the set D0 = B[ x0 , ] ∩ D and the Lipschitz-2 condition
M0
(K6)

L ( x0 )−1 (L (w1 ) − L (w2 )) ≤ M w1 − w2 ∀w1 , w2 ∈ D0 .


These Lipschitz parameters are related as

M0 ≤ M1 , (3)

M ≤ M1 (4)
since
D0 ⊂ D. (5)
Notice also since parameters M0 and M are specializations of parameter M1 , M1 = M1 ( D ),
M0 = M0 ( D ), but M = M( D0 ). Therefore, no additional work is required to find M0 and
M0
M (see also [22,23]). Moreover the ratio can be very small (arbitrarily). Indeed,
M

Example 2. Define scalar function

F (t) = b0 t + b1 + b2 sin eb3 t ,


for t0 = 0, where b j , j = 0, 1, 2, 3 are real parameters. It follows by this definition that for b3
M0 M0
sufficiently large and b2 sufficiently small, can be small (arbitrarily), i.e., −→ 0.
M1 M1

Then, clearly there can be a significant extension if parameters M1 and M0 or M and


M0 can be replace M1 in condition (K3). Looking at this direction the following replace-
ments are presented in a series of papers [19,22,23], respectively
1
(N2): s≤ ,
q2
1
(N3): s≤ ,
q3
and
1
(N4): s≤ ,
q4
1 
where q1 = 2M1 , q2 = M1 + M0 , q3 = (4M0 + M1 + M12 + 8M1 M0 ) and
 4
1 √
q4 = (4M0 + M12 + 8M0 M1 + M1 M0 ). These items are related as follows:
4
q4 ≤ q3 ≤ q2 ≤ q1 ,
( N2) ⇒ ( N3) ⇒ ( N4),
M0
and as relation −→ 0,
M1
q2 1 q 1 q
−→ , 3 −→ , 4 −→ 0
q1 2 q2 4 q3
and

91
Mathematics 2022, 10, 2931

q4
−→ 0.
q2
Preceding items indicate the times (at most) one is improving the other. These are the
extensions given in this aforementioned references. However, it turns out that parameter L
can replace M1 in these papers (see Section 3). Denote by Ñ, q̃ the corresponding items. It
follows

q̃1 M q̃ q̃
= −→ 0, 2 −→ 0, 3 −→ 0
q1 M1 q2 q3
M0 M
for −→ 0 and −→ 0. Hence, the new results also extend the ones in the afore-
M1 M1
mentioned references. Other extensions involve tighter majorizing sequences for NM (see
Section 2) and improved uniqueness report for solution x ∗ (Section 3). The applications
appear in Section 4 followed by conclusions in Section 5.

2. Majorizations
Let K0 , M0 , K, M be given positive parameters and s be a positive variable. The real
K ( t1 − t0 )2
sequence {tn } defined for t0 = 0, t1 = s, t2 = t1 + and ∀n = 0, 1, 2, . . . by
2 ( 1 − K0 t 1 )
M ( t n +1 − t n )2
t n +2 = t n +1 + (6)
2(1 − M0 tn+1 )
plays an important role in the study of NM, we adopted the notation tn (s) = tn
∀n = 1, 2, . . . . That is why some convergence results for it are listed in what follows
next in this study.

Lemma 1. Suppose conditions

1
K0 t1 < 1 and tn+1 < (7)
M0
hold ∀ n = 1, 2, . . . . Then, the following assertions hold

1
t n < t n +1 < (8)
M0
1
and ∃ t∗ ∈ [s, ] such that lim tn = t∗ .
M0 n→∞

Proof. The definition of sequence {tn } and the condition (7) implies (8). Moreover, increas-
1
ing sequence {tn } has as an upper bound. Hence, it is convergent to its (unique) least
M0
upper bound t∗ .

Next, stronger convergence criteria are presented. However, these criteria are easier to
verify than conditions of Lemma 1. Define parameter δ by

2M
δ= 5 . (9)
M+ M2 + 8M0 M
This parameter plays a role in the following results.
Case: K0 = M0 and K = M.
Part (i) of the next auxiliary result relates to the Lemma in [19].

Lemma 2. Suppose condition

1
s≤ (10)
2M2

92
Mathematics 2022, 10, 2931

holds, where

1 5
M2 = ( M + 4M0 + M2 + 8M0 M). (11)
4
Then, the following assertions hold
(i) Estimates
t n +1 − t n ≤ δ ( t n − t n −1 ) (12)
1 − δ n +1 s
tn < s< (13)
1−δ 1−δ
hold. Moreover, conclusions of Lemma 1 are true for sequence {tn }. The sequence, {tn } converges
s
linearly to t∗ ∈ (0, ]. Furthermore, if for some μ > 0
1−δ
μ
s< . (14)
(1 + μ) M2
Then, the following assertions hold
(ii)
M
t n +1 − t n ≤ (1 + μ)(tn − tn−1 )2 (15)
2
and
1 n
t n +1 − t n ≤ (αs)2 , (16)
α
M
where α = (1 + μ) and the conclusions of Lemma 1 for sequence {tn } are true. The sequence,
2
{tn } converges quadratically to t∗ .

Proof. (i) It is given in [19].


(ii) Notice that condition (14) implies (11) by the choice of parameter μ. Assertion (15)
holds if estimate

M M
0< ≤ (1 + μ ) (17)
2(1 − M0 tn+1 ) 2
μ
is true. This estimate is true for n = 1, since it is equivalent to M0 s ≤ . But this is
1+μ
μM0 μ
true by M0 ≤ 2M2 , condition (11) and inequality ≤ . Then, in view of
(1 + μ)2M2 1+μ
estimate (13), estimate (17) certainly holds provided that

(1 + μ) M0 (1 + δ + . . . + δn+1 )s − μ ≤ 0. (18)
This estimate motivates the introduction of recurrent polynomials pn which are defined
by
pn (t) = (1 + μ) M0 (1 + t + . . . + tn+1 )s − μ, (19)
∀t ∈ [0, 1). In view of polynomial pn assertion (18) holds if

pn (t) ≤ 0 at t = δ. (20)

The polynomials pn are connected:

pn+1 (t) − pn (t) = (1 + μ) M0 tn+2 s > 0,


so
pn (t) < pn+1 (t) ∀ t ∈ [0, 1). (21)

93
Mathematics 2022, 10, 2931

Define function p∞ : [0, 1) −→ R by

p∞ (t) = lim pn (t). (22)


n−→∞

It follows by definitions (19) and (20) that

(1 + μ) M0 s
p∞ (t) = − μ. (23)
1−t
Hence, assertion (20) holds if

p∞ (t) ≤ 0 at t = δ, (24)

or equivalently
5
μ M2 + 8M0 M − M
M0 s ≤ 5 ,
1 + μ M2 + 8M0 M + M
which can be rewritten as condition (14). Therefore, the induction for assertion (17) is
completed. That is assertion (15) holds by the definition of sequence {tn } and estimate (15).
It follows that

α ( t n +1 − t n ) ≤ α2 (tn − tn−1 ) = (α(tn − tn−1 ))2 ,


≤ α2 (α(tn−1 − tn−2 ))2
2
≤ α2 α2 ( t n −1 − t n −2 )2
3
≤ α 2 α2 α2 ( t n −2 − t n −3 )2
..
.

so
n −1
≤ α1+2+2 +...+2
2 n
t n +1 − t n s2
1 n
= (αs)2 .
α
μ 2
Notice also that Mμ < 4M2 , then < , so αs < μ.
(1 + μ) M1 M (1 + μ )

Remark 1. (1) The technique of recurrent polynomials in part (i) is used: to produce convergence
condition (11) and a closed form upper bound on sequence {tn } (see estimate (13)) other
1
than and t∗ (which is not given in closed form). This way we also established the linear
M0
convergence of sequence {tn }. By considering condition (14) but being able to use estimate
(13) we establish the quadratic convergence of sequence {tn } in part (ii) of Lemma 2.
(2) If μ = 1, then (14) is the strict version of condition (10).
(3) Sequence {tn } is tighter than the Kantorovich sequence {un } since M0 ≤ M1 and M ≤ M1 .
2M s
Concerning the ration of convergence αs this is also smaller than r = √ 1
(1 − 1 − 2M1 s)2
given in the Kantorovich Theorem [19]. Indeed, by these definitions αs < r provided that
4M
μ ∈ (0, μ1 ), where μ1 = √ 1 − 1. Notice that
M (1 + 1 − 2M1 s)2
5 4M1
(1 + 1 − 2M1 s)2 < (1 + 1)2 = 4 ≤ ,
M
so μ1 > 0.
Part (i) of the next auxiliary result relates to a Lemma in [19]. The case M0 = M has been
studied in the introduction. So, in the next Lemma we assume M0 = M in part (ii).

94
Mathematics 2022, 10, 2931

Lemma 3. Suppose condition


1
s≤ (25)
2M3
holds, where
 5
1
M3 = (4M0 + M0 M + 8M02 + M0 M ).
8
Then, the following assertions hold
(i)
δn−1 M0 s2
t n +1 − t n ≤ δ ( t n − t n −1 ) ≤ (26)
2(1 − M0 s)
and

1 − δ n +1 t − t1
t n +2 ≤ s + (t2 − t1 ) < t∗∗ = s + 2 s, ∀n = 1, 2, . . . . (27)
1−δ 1−δ
Moreover, conclusions of Lemma 1 are true for sequence {tn }. The sequence {tn } converges
linearly to t∗ ∈ (0, t∗∗ ]. Define parameters h0 by
 √
2( M0 M + 8M02 + M0 M ) h0
h0 =  √ , M̄3 = ,
M( M M + 8M + M M + 4M )
2 2
0 0 0 0

μ
γ = 1 + μ, β = , d = 2(1 − δ )
1+μ
and

M0
μ= .
2M3 − M0
(ii) Suppose
M0
M0 < M ≤ (28)
θ
and (25) hold, where θ ≈ 0.6478 is the smallest solution of scalar equation 2z4 + z − 1 = 0.
Then, the conclusions of Lemma 2 also hold for sequence {tn }. The sequence converges
quadratically to t∗ ∈ (0, t∗∗ ].
(iii) Suppose

1 1
M≥ M0 and s < (29)
θ 2 M̄3
hold. Then, the conclusions of Lemma 2 are true for sequence {tn }. The sequence {tn } con-
verges quadratically to t∗ ∈ (0, t∗∗ ].

(iv) M0 > M and (25) hold. Then, M̄3 ≤ M3 and the conclusions of Lemma 2 are true for
sequence {tn }. The sequence {tn } converges quadratically to t∗ ∈ (0, t∗∗ ].

Proof. (i) It is given in Lemma 2.1 in [23].


(ii) As in Lemma 2 but using estimate (27) instead of (13) to show

M Mγ
≤ .
2(1 − M0 tn+1 ) 2
It suffices
 
1 − δn
γM0 s + ( t2 − t1 ) + 1 − γ ≤ 0
1−δ

95
Mathematics 2022, 10, 2931

or

pn (t) ≤ 0 at t = δ, (30)
where

pn (t) = γM0 (1 + t + . . . + tn−1 )(t2 − t1 ) + γM0 s + 1 − γ.


Notice that

pn+1 (t) − pn (t) = γM0 tn (t2 − t1 ) > 0.


Define function p∞ : [0, 1) −→ R by

p∞ (t) = lim pn (t).


n−→∞

It follows that

γM0 (t2 − t1 )
p∞ (t) = + γM0 s + 1 − γ.
1−t
So, (30) holds provided that

p∞ (t) ≤ 0 at t = δ. (31)

By the definition of parameters γ, d, β and for M0 s = x, (31) holds if

x2
+x ≤ β
2(1 − x )(1 − δ)
or

( d − 1) x 2 + (1 + β ) x − β ≤ 0
or
5
1+β− (1 − β)2 + 4βd
x≤
2(1 − d )
or
5
1+β− (1 − β)2 + 4βd
s≤ . (32)
2(1 − d )
1
Claim. The right hand side of assertion (31) equals . Indeed, this is true if
M2

2M0 (1 − d)
1 + β − (1 − β)2 + 4βd =
M2
or 
2M0 (1 − d)
1+β− = (1 − β)2 + 4βd
2M3
or by squaring both sides

4M02 (1 − d)2 4M0 (1 − d) 4βM0 (1 − d)


1 + β2 + + 2β − − = 1 + β2 − 2β + 4βd
4M32 2M3 2M3

or  
M0 (1 − d) M (1 − d )  M0 
β 1− −d = 0 1−
2M3 2M3 2M3
or

96
Mathematics 2022, 10, 2931

 M0   M0  M
β 1− (1 − d ) = 1 − (1 − d ) 0
2M3 2M3 2M3
or

M0
β=
2M3
or

μ M0
=
1+μ 2M3
or

M0
μ= ,
2M3 − M0
which is true. Notice also that
5 
1
2M3 − M0 =
(4M0 + M0 M + M0 M + 8M02 )
4

1 5
= ( M0 M + M0 M + 8M02 ) > 0
4
√ 
M0 M + M0 M + 8M02 − 4M0 √
and 2M3 − 2M0 > 0, since 2M3 − M0 = , M0 < M0 M
 4
and 3M0 < M0 M + 8M02 (by condition (25)). Thus, μ ∈ (0, 1). It remains to show

M
α= (1 + μ ) s < 1
2
or by the choice of μ and M2
 
M2 M0
1+ s<1
2 2M3 − M0
or
1
s< . (33)
2 M̄3
Claim. M̄3 ≤ M3 . By the definition of parameters M2 and M̄3 it must be shown that
√  √ 
M ( M0 M + M0 M + 8M02 + 4M0 M0 M + M0 M + 8M02 + 4M0
√  ≤
2( M0 M + M0 M + 8M02 ) 4

M0
or if for y =
M


2− y≤ y + 8y2 . (34)
√ √
By (28) 2 − y > 0, so estimate (34) holds if 2y2 + y − 1 ≥ 0 or

2z4 + z − 1 ≥ 0 for z = y.
However, the last inequality holds by (28). The claimed is justified. So, estimate (33)
holds by (25) and this claim.
(iii) It follows from the proof in part (ii). However, this time M2 ≤ M̄2 follows from (29).
Notice also that according to part (ii) condition (25) implies (29). Moreover, according
to part (iii) condition (29) implies (25).

97
Mathematics 2022, 10, 2931

(iv) As in case (ii) estimate (34) must be satisfied. If M0 ≥ 4M, then the estimate (34)

holds, since 2 − y ≤ 0. If M < M0 < 4M then again M0 > θ M, so estimate (34) or
equivalently 2z + z − 1 > 0 holds.
2

Comments similar to Remark 1 can follow for Lemma 3.

Case. Parameters K0 and K are not equal to M0 . Comments similar to Remark 1 can
follow for Lemma 3.
It is convenient to define parameter δ0 by

K ( t2 − t1 )
δ0 =
2 ( 1 − K0 t 2 )
and the quadratic polynomial ϕ by

ϕ(t) = ( MK + 2δM0 (K − 2K0 ))t2 + 4δ( M0 + K0 )t − 4δ.


The discriminant  of polynomial q can be written as

 = 16δ(δ( M0 − K0 ))2 + ( M + 2δM0 )K > 0.


1
It follows that the root given by the quadratic formula can be written as
h1
1 2
= 5 .
2h1 δ( M0 + K0 ) + (δ( M0 + K0 ))2 + δ( MK + 2δM0 )(K − 2K0 )
1
Denote by the unique positive zero of equation
h2
M0 (K − 2K0 )t2 + 2M0 t − 1 = 0.
This root can be written as

1 1
= 5 .
2h2 M0 + M2 + M0 (K − 2K0 )
Define parameter M4 by
! (
1 1 1
= min , . (35)
M4 h1 h2
Part (i) of the next auxiliary result relates to Lemma 2.1 in [22].

Lemma 4. Suppose
1
s≤ (36)
2M4
holds, where parameter M4 is given by Formula (35). Then, the following assertions hold
(i) Estimates

Ks2
tn+2 − tn+1 ≤ δ0 δn−1 ,
2 ( 1 − K0 s )
and
   
1 − δn δ
tn+2 ≤ s + 1 + δ0 (t2 − t1 ) ≤ t̄ = s + 1 + 0 ( t2 − t1 ).
1−δ 1−δ
Moreover, conclusions of Lemma 2 are true for sequence {tn }. The sequence {tn } converges
linearly to t∗ ∈ (0, t̄].

98
Mathematics 2022, 10, 2931

(ii) Suppose
 
δ0 (t2 − t1 )
M0 +s ≤ β, (37)
1−δ
2
s< (38)
(1 + μ ) M
and (36) hold for some μ > 0. Then, the conclusions of Lemma 3 are true for sequence {tn }.
The sequence {tn } converges quadratically to t∗ ∈ (0, t̄].

Proof. (i) It is given in Lemma 2.1 in [22].


(ii) Define polynomial pn by

pn (t) = γM0 δ0 (1 + t + . . . + tn−1 )(t2 − t − 1) + γM0 s + 1 − γ.


By this definition it follows

pn+1 (t) − pn (t) = γM0 δ0 (t2 − t1 )tn > 0.


As in the proof of Lemma 3 (ii), estimate

M M
≤ γ
2(1 − M0 tn+1 ) 2
holds provided that

pn (t) ≤ 0 at t = δ. (39)
Define function p∞ : [0, 1) −→ R by

p∞ (t) = lim pn (t).


n−→∞

It follows by the definition of function p∞ and polynomial pn that

γM0 δ0 (t2 − t1 )
p∞ (t) = + γM0 s − γ.
1−t
Hence, estimate (39) holds provided that

p∞ (t) ≤ 0 at t = δ.
However, this assertion holds, since μ ∈ (0, 1). Moreover, the definition of α and
condition (38) of the Lemma 4 imply

M
αs = (1 + μ ).
2
Hence, the sequence {tn } converges quadratically to t∗ .

Remark 2. Conditions (36)–(38) can be condensed and a specific choice for μ can be given as
1 1 
follows: Define function f : 0, −→ R by
K0
 
δ0 (t)(t2 (t) − t1 (t))
f (t) = 1 − M0 +t .
1−δ
It follows by this definition

1 −
f (0) = 1 > 0, f (t) −→ −∞ as t −→ .
K0

99
Mathematics 2022, 10, 2931

 1 
Denote by μ2 the smallest solution of equation f (t) = 0 in 0, . Then, by choosing
K0
μ = μ2 conditions (37) holds as equality. Then, if follows that if we solve the first condition in (37)
for “s", then conditions (36)–(38) can be condensed as
! (
1 2
s ≤ s1 min , . (40)
M4 (2 + μ2 ) M

2
If s1 = , then condition (40) should hold as a strict inequality to show quadratic
(2 + μ2 ) M
convergence.

3. Semi-Local Convergence
Sequence {tn } given by (6) was shown to be majorizing for { xn } and tighter than
{un } under conditions of Lemmas in [19,22,23], respectively. These Lemmas correspond
to part (i) of Lemma 1, Lemma 3 and Lemma 4, respectively. However, by asking the
initial approximation s to be bounded above by a slightly larger bound the quadratic order
of convergence is recovered. Hence, the preceding Lemmas can replace the order ones,
respectively in the semi-local proofs for NM in these references. The parameter K0 and K
are connected to x0 and L as follows
(K7) ∃ parameter K0 > 0 such that for x1 = x0 − L ( x0 )−1 L( x0 )

L ( x0 )−1 (L ( x1 ) − L ( x0 )) ≤ K0 x1 − x0 ,
(K8) ∃ parameter K such that ∀ξ ∈ [0, 1], ∀ x, y ∈ D0 ,
 1 
 
 L 
( x ) −1
(L 
( x + ξ ( y − x )) − L 
( x )) dξ  ≤ K y−x .
 0 0  2
Note that K0 ≤ M0 and K ≤ M. The convergence criteria in Lemmas 1, 3 and 4 do not
necessarily imply each other in each case. That is why we do not only rely on Lemma 4 to
show the semi-local convergence of NM. Consider the following three sets of conditions:
(A1): (K1), (K4), (K5), (K6) and conditions of Lemma 1 hold for ρ = t∗ , or
(A2): (K1), (K4) (K5), (K6), conditions of Lemma 2 hold with ρ = t∗ , or
(A3): (K1), (K4) (K5), (K6), conditions of Lemma 3 hold with ρ = t∗ , or
(A4): (K1), (K4) (K5), (K6), conditions of Lemma 4 hold with ρ = t∗ .
The upper bounds of the limit point given in the Lemmas and in closed form can
replace ρ in condition (K4). The proof are omitted in the presentation of the semi-local
convergence of NM since the proof is given in the aforementioned references [19,20,22,23]
with the exception of quadratic convergence given in part (ii) of the presented Lemmas.

Theorem 2. Suppose any of conditions Ai, i = 1, 2, 3, 4 hold. Then, sequence { xn } generated by


NM is well defined in B[ x0 , ρ], remains in B[ x0 , ρ] ∀n = 0, 1, 2, . . . and converges to a solution
x ∗ ∈ B[ x0 , ρ] of equation L( x ) = 0. Moreover, the following assertion hold ∀n = 0, 1, 2, . . .

x n +1 − x n ≤ t n +1 − t n
and
x ∗ − xn ≤ t∗ − tn .

The convergence ball is given next. Notice, however that we do not use all conditions Ai.

Proposition 1. Suppose: there exists a solution x ∗ ∈ B( x0 , ρ0 ) of equation L( x ) = 0 for some


ρ0 > 0; condition (K5) holds and ∃ ρ1 ≥ ρ0 such that

M0
(ρ0 + ρ1 ) < 1. (41)
2

100
Mathematics 2022, 10, 2931

Set D1 = D ∩ B[ x0 , ρ1 ]. Then, the only solution of equation L( x ) = 0 in the set D1 is x ∗ .

Proof. Let x∗ ∈ D1 be a solution of equation L( x ) = 0. Define linear operator


1
J = 0 L ( x ∗ + τ ( x∗ − x ∗ ))dτ. Then, using (K5) and (41)
 1
L ( x0 )−1 (L ( x0 ) − J ) ≤ M0 ((1 − τ ) x0 − x ∗ + τ x0 − x∗ )dτ
0
M0
≤ (ρ0 + ρ1 ) < 1. (42)
2
Therefore, x ∗ = x∗ is implied by the invertability of J and

J ( x ∗ − x∗ ) = L( x ∗ ) − L( x∗ ) = 0.
If conditions of Theorem 2 hold, set ρ0 = ρ.

4. Numerical Experiments
Two experiments are presented in this Section.

1−a
Example 3. Recall Example 1 (with L( x ) = c( x )). Then, the parameters are s = ,
1 23
a+5 1
K0 = , M0 = 3 − a, M1 = 2(2 − a). It also follows D0 = B(1, 1 − a) ∩ B 1, =
3 M0
1 1 2  1 
B 1, , so K = M = 2 1 + . Denote by Ti , i = 1, 2, 3, 4 the set of values a for
M0 3−a
which conditions (K3), ( N2) − N4) are satisfied. Then, by solving these inequalities for a :
T1 = ∅, T2 = [0.4648, 0.5), T3 = [0.4503, 0.5), and T4 = [0.4272, 0.5), respectively.
1
The domain can be further extended. Choose a = 0.4, then, = 0.3846. The following
M0
Table 1 shows, that the conditions of Lemma 1, since K0 t < 1 and M0 tn+1 < 1 ∀ n = 1, 2, . . ..

Table 1. Sequence (6) for Example 1.

n 1 2 3 4 5 6 7 8
tn 0.2000 0.2865 0.3272 0.3425 0.3455 0.3456 0.3456 0.3456

Example 4. Let U = V = IR3 , D = B( x0 , 0.5) and


 T
L( x ) = e x1 − 1, x23 + x2 , x3 .
The equation L( x ) = 0 has the solution x ∗ = (0, 0, 0) T and L ( x ) = diag(e x1 , 3x22 + 1, 1).
Let x0 = (0.1, 0.1, 0.1) T . Then s = L ( x0 )−1 L( x0 ) ∞ ≈ 0.1569,
! (
e0.6 3(0.6 + 0.1)
M0 = max , ≈ 2.7183,
e0.1 1.03
! 0.6 (
e 3(0.6 + 0.6)
M1 = max 0.1 , ≈ 3.49513.
e 1.03
1 1
It also follows that ≈ 0.3679, D0 = D ∩ B[ x0 , ] = B[0.1, 0.3679] and
M0 M0
! p (
e 1 3( p2 + 0.1)
K0 = max 0.1 , ≈ 2.3819,
e 1.03
! p (
e 1 6p
M = K = max 0.1 , 1 ≈ 2.7255,
e 1.03
1
where p1 = 0.1 + ≈ 0.4679, p2 ≈ 0.0019.
M0

101
Mathematics 2022, 10, 2931

Notice that M0 < M1 and M < M1 . The Kantorovich convergence condition (K3) is not
fulfilled, since 2M1 s ≈ 1.0968 > 1. Hence, convergence of converge NM is not assured by the
Kantorovich criterion. However, the new conditions (N2)–(N4) are fulfilled, since q2 s ≈ 0.9749 < 1,
q3 s ≈ 0.9320 < 1, q4 s ≈ 0.8723 < 1.
The following Table 2 shows, that the conditions of Lemma 1 are fulfilled, since K0 t < 1 and
M0 tn+1 < 1 ∀n = 1, 2, . . ..

Table 2. Sequence (6) for Example 4.

n 1 2 3 4 5 6
tn 0.1569 0.2154 0.2266 0.2271 0.2271 0.2271

Example 5. Let U = V = C [0, 1] be the domain of continuous real functions defined on the
interval [0, 1]. Set D = B[ x0 , 3], and define operator L on D as
 1
L(v)(v1 ) = v(v1 ) − y(v1 ) − N (v1 , t)v3 (t)dt, v ∈ C [0, 1], v1 ∈ [0, 1], (43)
0

where y is given in C [0, 1], and N is a kernel given by Green’s function as


!
(1 − v1 )t, t ≤ v1
N ( v1 , t ) = (44)
v1 (1 − t), v1 ≤ t.
By applying this definition the derivative of L is
 1
[L (v)(z)](v1 ) = z(v1 ) − 3 N (v1 , t)v2 (t)z(t)dt (45)
0

z ∈ C [0, 1], v1 ∈ [0, 1]. Pick x0 (v1 ) = y(v1 ) = 1. The norm-max is used. It then follows from
(43)–(45) that L ( x0 )−1 ∈ L( B2 , B1 ),

I − L ( x0 ) < 0.375, L ( x0 )−1 ≤ 1.6,


s = 0.2, M0 = 2.4, M1 = 3.6,
and D0 = B( x0 , 3) ∩ B[ x0 , 0.4167] = B[ x0 , 0.4167], so M = 1.5. Notice that M0 < M1 and
M < M1 . Choose K0 = K = M0 . The Kantorovich convergence condition (K3) is not fulfilled,
since 2M1 s = 1.44 > 1. Hence, convergence of converge NM is not assured by the Kantorovich
criterion. However, new condition (36) is fulfilled, since 2M4 s = 0.6 < 1.

Example 6. Let U = V = IR, D = (−1, 1) and

L( x ) = e x + 2x − 1.
 e x0 + 2x − 1 
 
The equation L( x ) = 0 has the solution x ∗ = 0. The parameters are s = 
0
,
e x0 + 2
M0 = M1 = e, K0 = K = M = e 0 e and
x + 1

1 1 12 1 1 12
D0 = (−1, 1) ∩ x0 − , x0 + = x0 − , x0 + .
e e e e
Let us choose x0 = 0.15. Then, s ≈ 0.1461. Conditions (K3) and (N2) are fulfilled. The
majorizing sequences {tn } (6) and {un } from Theorem 1 are:

{tn } = {0, 0.1461, 0.1698, 0.1707, 0.1707, 0.1707, 0.1707},

{un } = {0, 0.1461, 0.1942, 0.2008, 0.2009, 0.2009, 0.2009, 0.2009}.


In Table 3, there are error bounds. Notice that the new error bounds are tighter, than the ones
in Theorem 1.

102
Mathematics 2022, 10, 2931

Table 3. Results for x0 = 0.15 for Example 6.

n | x n +1 − x n | | tn +1 − tn | | u n +1 − u n |
0 1.4607 × 10−1 1.4607 × 10−1 1.4607 × 10−1
1 3.9321 × 10−3 2.3721 × 10−2 4.8092 × 10−2
2 2.5837 × 10−6 8.7693 × 10−4 6.6568 × 10−3
3 1.1126 × 10−12 1.2039 × 10−6 1.3262 × 10−4
4 0 2.2688× 10−12 5.2681× 10−8

Let us choose x0 = 0.2. Then, s ≈ 0.1929. In this case condition (K3) is not held, but (N2)
holds. The majorizing sequence {tn } (6) is:

{tn } = {0, 0.1929, 0.2427, 0.2491, 0.2492, 0.2492, 0.2492, 0.2492 }.

Table 4 shows the error bounds from Theorem 2.

Table 4. Results for x0 = 0.2 for Example 6.

n | x n +1 − x n | | tn +1 − tn |
0 1.929 × 10−1 1.929 × 10−1
1 7.0934 × 10−3 4.9769 × 10−2
2 8.4258 × 10−6 6.4204 × 10−3
3 1.1832 × 10−11 1.1263 × 10−4
4 0 3.4690 × 10−8

5. Conclusions
We developed a comparison between results on the semi-local convergence of NM.
There exists an extensive literature on the convergence analysis of NM. Most convergence
results are based on recurrent relations, where the Lipschitz conditions are given in affine
or non-affine invariant forms.The new methodology uses recurrent functions. The idea
is to construct a domain included in the one used before which also contains the Newton
iterates. That is important, since the new results do not require additional conditions. This
way the new sufficient convergence conditions are weaker in the Lipschitz case, since they
rely on smaller constants. Other benefits include tighter error bounds and more precise
uniqueness of the solution results. The new constants are special cases of earlier ones. The
methodology is very general making it suitable to extend the usage of other numerical
methods under Hölder or more generalized majorant conditions. This will be the topic of
our future work.

Author Contributions: Conceptualization I.K.A.; Methodology I.K.A.; Investigation S.R., I.K.A., S.S.
and H.Y. All authors have read and agreed to the published version of the manuscript.
Funding: This research received no external funding.
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: Not applicable.
Conflicts of Interest: The authors declare no conflict of interest.

References
1. Appell, J.; DePascale, E.; Lysenko, J.V.; Zabrejko, P.P. New results on Newton-Kantorovich approximations with applications to
nonlinear integral equations. Numer. Funct. Anal. Optim. 1997, 18, 1–17. [CrossRef]
2. Traub, J.F. Iterative Methods for the Solution of Equations; Prentice Hall: Hoboken, NJ, USA, 1964.
3. Ezquerro, J.A.; Hernández-Verón, M.A. Newton’s Method: An Updated Approach of Kantorovich’s Theory. Frontiers in Mathematics;
Birkhäuser/Springer: Cham, Switzerland, 2017.
4. Kantorovich, L.V.; Akilov, G.P. Functional Analysis; Pergamon Press: Oxford, UK, 1982.

103
Mathematics 2022, 10, 2931

5. Potra, F.A.; Pták, V. Nondiscrete induction and iterative processes. In Research Notes in Mathematics; Pitman (Advanced Publishing
Program): Boston, MA, USA, 1984; Volume 103.
6. Verma, R. New Trends in Fractional Programming; Nova Science Publisher: New York, NY, USA, 2019.
7. Yamamoto, T. Historical developments in convergence analysis for Newton’s and Newton-like methods. J. Comput. Appl. Math.
2000, 124, 1–23. [CrossRef]
8. Zhanlav, T.; Chun, C.; Otgondorj, K.H.; Ulziibayar, V. High order iterations for systems of nonlinear equations. Int. J. Comput.
Math. 2020, 97, 1704–1724. [CrossRef]
9. Sharma, J.R.; Guha, R.K. Simple yet efficient Newton-like method for systems of nonlinear equations. Calcolo 2016, 53, 451–473.
[CrossRef]
10. Grau-Sanchez, M.; Grau, A.; Noguera, M. Ostrowski type methods for solving system of nonlinear equations. Appl. Math. Comput.
2011, 218, 2377–2385. [CrossRef]
11. Homeier, H.H.H. A modified Newton method with cubic convergence: The multivariate case. J. Comput. Appl. Math. 2004, 169,
161–169. [CrossRef]
12. Kou, J.; Wang, X.; Li, Y. Some eight order root finding three-step methods. Commun. Nonlinear Sci. Numer. Simul. 2010, 15, 536–544.
[CrossRef]
13. Nashed, M.Z.; Chen, X. Convergence of Newton-like methods for singular operator equations using outer inverses. Numer. Math.
1993, 66, 235–257. [CrossRef]
14. Wang, X. An Ostrowski-type method with memory using a novel self-accelerating parameters. J. Comput. Appl. Math. 2018, 330,
710–720. [CrossRef]
15. Moccari, M.; Lofti, T. On a two-step optimal Steffensen-type method: Relaxed local and semi-local convergence analysis and
dynamical stability. J. Math. Anal. Appl. 2018, 468, 240–269. [CrossRef]
16. Sharma, J.R.; Arora, H. Efficient derivative-free numerical methods for solving systems of nonlinear equations. Comput. Appl.
Math. 2016, 35, 269–284. [CrossRef]
17. Noor, M.A.; Waseem, M. Some iterative methods for solving a system of nonlinear equations. Comput. Math. Appl. 2009, 57,
101–106. [CrossRef]
18. Shakhno, S.M. On a two-step iterative process under generalized Lipschitz conditions for first-order divided differences. J. Math.
Sci. 2010, 168, 576–584. [CrossRef]
19. Argyros, I.K. On the Newton-Kantorovich hypothesis for solving equations. J. Comput. Math. 2004, 169, 315–332. [CrossRef]
20. Argyros, I.K. Unified Convergence Criteria for Iterative Banach Space Valued Methods with Applications. Mathematics 2021, 9,
1942. [CrossRef]
21. Proinov, P.D. New general convergence theory for iterative processes and its applications to Newton-Kantorovich type theorems.
J. Complex. 2010, 26, 3–42. [CrossRef]
22. Argyros, I.K.; Hilout, S. On an improved convergence analysis of Newton’s scheme. Appl. Math. Comput. 2013, 225, 372–386.
23. Argyros, I.K.; Hilout, S. Weaker conditions for the convergence of Newton’s scheme. J. Complex. 2012, 28, 364–387. [CrossRef]

104
mathematics

Article
Constructing a Class of Frozen Jacobian Multi-Step Iterative
Solvers for Systems of Nonlinear Equations
R. H. Al-Obaidi and M. T. Darvishi ∗

Department of Mathematics, Faculty of Science, Razi University, Kermanshah 67149, Iran


* Correspondence: [email protected]

Abstract: In this paper, in order to solve systems of nonlinear equations, a new class of frozen Jacobian
multi-step iterative methods is presented. Our proposed algorithms are characterized by a highly
convergent order and an excellent efficiency index. The theoretical analysis is presented in detail.
Finally, numerical experiments are presented for showing the performance of the proposed methods,
when compared with known algorithms taken from the literature.

Keywords: iterative method; frozen Jacobian multi-step iterative method; system of nonlinear
equations; high-order convergence

MSC: 65Hxx

1. Introduction
Approximating a locally unique solution α of the nonlinear system

F (x) = 0 (1)
Citation: Al-Obaidi, R.H.; Darvishi,
has many applications in engineering and mathematics [1–4]. In (1), we have n equations
M.T. Constructing a Class of Frozen
with n variables. In fact, F is a vector-valued function with n variables. Several problems
Jacobian Multi-Step Iterative Solvers
arising from the different areas in natural and applied sciences take the form of systems of
for Systems of Nonlinear Equations.
Mathematics 2022, 10, 2952. https://
nonlinear Equation (1) that need to be solved, where F (x) = ( f 1 (x), f 2 (x), · · · , f n (x)) such
doi.org/10.3390/math10162952
that for all k = 1, 2, · · · , n, f k is a scalar nonlinear function. Additionally, there are many
real life problems for which, in the process of finding their solutions, one needs to solve
Academic Editors: Maria Isabel a system of nonlinear equations, see for example [5–9]. It is known that finding an exact
Berenguer and Manuel Ruiz Galán
solution αt = (α1 , α2 , · · · , αn ) of the nonlinear system (1) is not an easy task, especially
Received: 1 July 2022 when the equation contains terms consisting of logarithms, trigonometric and exponential
Accepted: 8 August 2022 functions, or a combination of transcendental terms. Hence, in general, one cannot find
Published: 16 August 2022 the solution of Equation (1) analytically, therefore, we have to use iterative methods. Any
iterative method starts from one approximation and constructs a sequence such that it
Publisher’s Note: MDPI stays neutral
converges to the solution of the Equation (1) (for more details, see [10]).
with regard to jurisdictional claims in
The most commonly used iterative method to solve (1) is the classical Newton method,
published maps and institutional affil-
given by
iations.
x ( k +1) = x ( k ) − J F ( x ( k ) ) −1 F ( x ( k ) ),

where JF (x) (or F  (x)) is the Jacobian matrix of function F, and x(k) is the k-th approxima-
tion of the root of (1) with the initial guess x(0) . It is well known
√ that Newton’s method is
Copyright: © 2022 by the authors.
a quadratic convergence method with the efficiency index 2 [11]. The third and higher-
Licensee MDPI, Basel, Switzerland.
This article is an open access article
order methods such as the Halley and Chebyshev methods [12] have little practical value
distributed under the terms and
because of the evaluation of the second Frechèt-derivative. However, third and higher-
conditions of the Creative Commons order multi-step methods can be good substitutes because they require the evaluation of
Attribution (CC BY) license (https:// the function and its first derivative at different points.
creativecommons.org/licenses/by/ In the recent decades, many authors tried to design iterative procedures with better effi-
4.0/). ciency and higher order of convergence than the Newton scheme, see, for example, ref. [13–24]

Mathematics 2022, 10, 2952. https://doi.org/10.3390/math10162952 https://www.mdpi.com/journal/mathematics


105
Mathematics 2022, 10, 2952

and references therein. However, the accuracy of solutions is highly dependent on the
efficiency of the utilized algorithm. Furthermore, at each step of any iterative method,
we must find the exact solution of an obtained linear system which is expensive in actual
applications, especially when the system size n is very large. However, the proposed
higher-order iterative methods are futile unless they have high-order convergence. There-
fore, the important aim in developing any new algorithm is to achieve high convergence
order with requiring as small as possible the evaluations of functions, derivatives and
matrix inversions. Thus, here, we focus on the technique of the frozen Jacobian multi-step
iterative algorithms. It is shown that this idea is computationally attractive and economical
for constructing iterative solvers because the inversion of the Jacobian matrix (regarding
LU-decomposition) is performed once. Many researchers have reduced the computational
cost of these algorithms by frozen Jacobian multi-step iterative techniques [25–28].
In this work, we construct a new class of frozen Jacobian multi-step iterative methods
for solving the nonlinear systems of equations. This is a high-order convergent algorithm
with an excellent efficiency index. The theoretical analysis is presented completely. Further,
by solving some nonlinear systems, the ability of the methods is compared with some
known algorithms.
The rest of this paper is organized as follows. In the following section, we present
our new methods with obtaining of their order of convergence. Additionally, their com-
putational efficiency are discussed in general. Some numerical examples are considered
in Sections 3 and 4 to show the asymptotic behavior of these methods. Finally, a brief
concluding remark is presented in Section 5.

2. Constructing New Methods


In this section, two high-order frozen Jacobian multi-step iterative methods to solve
systems of nonlinear equations are presented. These come by increasing the convergence in
Newton’s method and simultaneously decreasing its computational costs. The framework
of these Frozen Jacobian multi-step iterative Algorithms (FJA) can be described as

⎪ y0 = initial guess
⎧ ⎪


⎪ No. of steps = m > 1, ⎪

⎪ y1 = y0 − J F ( y0 ) −1 F ( y0 )

⎪ ⎪

⎨ Order of convergence = m + 1, ⎨ for i = 1 : m − 1
Function evaluations = m, FJA : Œi = JF (y0 )−1 ( F (yi ) + F (yi−1 )) (2)

⎪ ⎪


⎪ Jacobian evaluations = 1, ⎪
⎪ y i +1 = y i −1 − Œ i
⎩ ⎪

No. of LU decomposition = 1; ⎪
⎪ end

y0 = y m .

In (2), for an m-step method (m > 1), one needs m function evaluations and only
one Jacobian evaluation. Further, the number of LU decompositions is one. The order
of convergence for such FJA method is m + 1. In the right-hand side column of (2), the
algorithm is briefy described.
In the following subsections, by choosing two different values for m, a third- and a
fourth-order frozen Jacobian multi-step iterative algorithm are presented.

2.1. The Third-Order FJA


First, we investigate case m = 2, that is,

y ( k ) = x ( k ) − J F ( x ( k ) ) −1 F ( x ( k ) ),
(3)
x(k+1) = x(k) − JF (x(k) )−1 ( F (y(k) ) + F (x(k) )),

we denote this by M3 .

106
Mathematics 2022, 10, 2952

2.1.1. Convergence Analysis


In this part, we prove that the order of convergence of method (3) is three. First, we
need to definition of the Frechèt derivative.

Definition 1 ([29]). Let F be an operator which maps a Banach space X into a Banach space Y. If
there exists a bounded linear operator T from X into Y such that

F (x + y) − F (x) − T (y)
lim = 0,
y →0 y

then F is said to be Frechèt differentiable and F  (x0 ) = T (x0 ).


For more details on the Frechèt differentiability and Frechèt derivative, we refer the interested
readers to a review article by Emmanuel [30] and references therein.

Theorem 1. Let F : I ⊆ Rn → Rn be a Frechèt differentiable function at each point of an open


convex neighborhood I of α, the solution of system F (x) = 0. Suppose that JF (x(k) ) is continuous
and nonsingular in α, then, the sequence {x(k) }(k0) obtained using the iterative method (3)
converges to α and its rate of convergence is three.

Proof. Suppose that En = x(n) − α, using Taylor’s expansion [31], we obtain

1  1 1
F (x(n) ) = F (α) + F  (α) En + F (α) En2 + F  (α) En3 + F  (α) En4 + . . .
2! 3! 4!
as α is the root of F so F (α) = 0. As a matter of fact, one may yield the following equations
of F (x(n) ) and F  (x(n) ) in a neighborhood of α by using Taylor’s series expansions [32],
/ 0
F (x(n) ) = F  (α) En + C2 En2 + C3 En3 + C4 En4 + C5 En5 + O|| En6 || , (4)
/ 0
F  (x(n) ) = F  (α) I + 2C2 En + 3C3 En2 + 4C4 En3 + 5C5 En4 + 6C6 En5 + O|| En6 || , (5)

[ F  (α)]−1 F (n) (α)


wherein Cn = n! and I is the identity matrix whose order is the same as the
order of the Jacobian matrix. Note that iCi Eni−1 ∈ L(Rn ). Using (4) and (5) we obtain

F  (x(n) )−1 F (x(n) ) = En − C2 En2 + (2C22 − 2C3 ) En3 + (−4C23 + 7C2 C3 − 3C4 ) En4
+(−32C25 + 8C24 − 20C22 C3 + 10C2 C4 + 6C32 − 4C5 ) En5 + O|| En6 ||.

Since y(n) = x(n) − F  (x(n) )−1 F (x(n) ), we find

y(n) = α + C2 En2 + (−2C22 + 2C3 ) En3 + (4C23 − 7C2 C3 + 3C4 ) En4


(6)
+(32C25 − 8C24 + 20C22 C3 − 10C2 C4 − 6C32 + 4C5 ) En5 + O|| En6 ||.

By the definition of error term En , the error term of y(n) as an approximation of α,


that is, y(n) − α is obtained from the second term of the right-hand side of Equation (6).
Similarly, the Taylor’s expansion of the function F (y(n) ) is
/
F (y(n) ) = F  (α) C2 En2 + (−2C22 + 2C3 ) En3 + (5C23 − 7C2 C3 + 3C4 ) En4 +
0 (7)
(32C25 − 12C24 + 24C22 − 10C2 C4 − 6C32 + 4C5 ) En5 + O|| En6 || .

107
Mathematics 2022, 10, 2952

From (4) and (7), we obtain


/
( F (x(n) ) + F (y(n) )) = F  (α) En + 2C2 En2 + (−2C22 + 3C3 ) En3 + (5C23 − 7C2 C3 +
0
4C4 ) En4 + (32C25 − 12C24 + 24C22 − 10C2 C4 − 6C32 + 6C5 ) En5 ] + O|| En6 || .

Thus,

F  (x(n) )−1 ( F (x(n) ) + F (y(n) )) = En − (2C22 ) En3 + (9C23 − 7C2 C3 ) En4


+(−30C24 + 44C22 C3 − 10C2 C4 − 6C32 + C5 ) En5 + O|| En6 ||.

Finally, since

x(n+1) = x(n) − JF (x(n) )−1 ( F (x(n) ) + F (y(n) )),

we have

x(n+1) = α − (2C22 ) En3 − (9C23 − 7C2 C3 ) En4 − (−30C24 + 44C22 C3 − 10C2 C4 +


(8)
· · · − 6C32 + C5 ) En5 + O|| En6 ||.

Clearly, the error Equation (8) shows that the order of convergence of the frozen
Jacobian multi-step iterative method (3) is three. This completes the proof.

2.1.2. The Computational Efficiency


In this section, we compare the computational efficiency of our third-order scheme (3),
denoted as M3 , with some existing third-order methods. We will assess the efficiency
index of our new frozen Jacobian multi-step iterative method in contrast with the existing
methods for systems of nonlinear equations, using two famous efficiency indices. The first
one is the classical efficiency index [33] as
1
IE = p c

where p is the rate of convergence and c stands for the total computational cost per iteration
in terms of the number of functional evaluations, such that c = (rn + mn2 ) where r refers
to the number of function evaluations needed per iteration and m is the number of Jacobian
matrix evaluations needed per iteration.
It is well known that the computation of LU factorization by any of the existing
methods in the literature normally needs 2n3 /3 flops in floating point operations, while
the floating point operations to solve two triangular systems needs 2n2 flops.
The second criterion is the flops-like efficiency index (FLEI) which was defined by
Montazeri et al. [34] as
1
FLEI = p c
where p is the order of convergence of the method, c denotes the total computational
cost per loop in terms of the number of functional evaluations, as well as the cost of LU
factorization for solving two triangular systems (based on the flops).
As the first comparison, we compare M3 with the third-order method given by
Darvishi [35], which is denoted as M3,1

y ( k ) = x ( k ) − J F ( x ( k ) ) −1 F ( x ( k ) ),
x(k+1) = x(k) − 2( JF (x(k) ) + JF (y(k) ))−1 F (x(k) ).

108
Mathematics 2022, 10, 2952

The second iterative method shown by M3,2 is the following third-order method
introduced by Hernández [36]

y(k) = x(k) − 12 JF (x(k) )−1 F (x(k) ),


x(k+1) = x(k) + JF (x(k) )−1 ( JF (y(k) ) − 2JF (x(k) )) × JF (x(k) )−1 F (x(k) ).

Another method is the following third-order iterative method given by Babajee


et al. [37], M3,3 ,

y ( k ) = x ( k ) − J F ( x ( k ) ) −1 F ( x ( k ) ),
x(k+1) = x(k) + 12 JF (x(k) )−1 ( JF (y(k) ) − 3JF (x(k) )) × JF (x(k) )−1 F (x(k) ).

Finally, the following third-order iterative method, M3,4 , ref. [38] is considered

y(k) = x(k) − 23 JF (x(k) )−1 F (x(k) ),


x(k+1) = x(k) − 4( JF (x(k) ) + 3JF (y(k) ))−1 F (x(k) ).

The computational efficiency of our third-order method revealed that our method, M3 ,
is the best one in respect with methods M3,1 , M3,2 , M3,3 and M3,4 , as presented in Table 1,
and Figures 1 and 2.

Table 1. Comparison of efficiency indices between M3 and other third-order methods.

Methods M3 M3,1 M3,2 M3,3 M3,4


No. of steps 2 2 2 2 2
Order of convergence 3 3 3 3 3
Functional evaluations 2n + n2 n + 2n2 n + 2n2 n + 2n2 n + 2n2
31/(2n+n ) 31/(n+2n
2)
31/(n+2n
2)
31/(n+2n
2)
31/(n+2n
2 2)
The classical efficiency index (IE)
No. of LU decompositions 1 2 1 1 2
Cost of LU decompositions 2n3 4n3 2n3 2n3 4n3
3 3 3 3 3
2n3 4n3 5n3 5n3 4n3
3 + 4n 3 + 4n 3 + 2n 3 + 2n 3 + 4n
Cost of linear systems (based on flops) 2 2 2 2 2
3 3 3 3 3
Flops-like efficiency index (FLEI) 1/( 2n3 +5n2 +2n) 1/( 4n3 +6n2 +n) 1/( 5n3 +4n2 +n) 1/( 5n3 +4n2 +n) 1/( 4n3 +6n2 +n)
3 3 3 3 3

Figure 1. The classical efficiency index for methods M3 , M3,1 , M3,2 , M3,3 and M3,4 .

109
Mathematics 2022, 10, 2952

Figure 2. The flops-like efficiency index for methods M3 , M3,1 , M3,2 , M3,3 and M3,4 .

2.2. The Fourth-Order FJA


By setting m = 3 in FJA, the following three-step algorithm is deduced

y ( k ) = x ( k ) − J F ( x ( k ) ) −1 F ( x ( k ) ),
z(k) = x(k) − JF (x(k) )−1 ( F (y(k) ) + F (x(k) )), (9)
x(k+1) = y(k) − JF (x(k) )−1 ( F (z(k) ) + F (y(k) )).

In the following subsections, the order of convergence and efficiency indices are
obtained for the method described in (9).

2.2.1. Convergence Analysis


The frozen Jacobian three-step iterative process (9) has the rate of convergence order
four by using three evaluations of function F and one first-order Frechèt derivative F per
full iterations. To avoid any repetition, we take a sketch of proof on this subject. Similar to
the proof of Theorem 1, by setting z(k) = x(k+1) in (8) we obtain

F (z(k) ) = F  (α)[2C22 En3 + (−9C23 + 7C2 C3 ) En4 + (30C24 − 44C22 C3 +


. . . + 10C2 C4 − C5 ) En5 + O|| En6 ||].

Hence,
/
( F (z(k) ) + F (y(k) )) = F  (α) C2 En2 + 2C3 En3 + (−4C23 + 3C4 ) En4
0 (10)
+(32C25 + 18C24 − 20C22 C3 + 3C5 ) En5 + O|| En6 || .

Therefore, from (5) and (10), we find


/
−1
F  (x(k) ) ( F (z(k) ) + F (y(k) )) = C2 En2 + (−2C22 + 2C3 ) En3 + (−7C2 C3 + . . .
0 (11)
+3C4 ) En4 + (18C24 − 10C2 C4 − 6C32 + 3C5 ) En5 + O|| En6 || .

110
Mathematics 2022, 10, 2952

Since we have x(k+1) = y(k) − JF (x(k) ))−1 ( F (z(k) ) + F (y(k) )) from (6) and (11), the
following result is obtained

x(k+1) = α + (4C23 ) En4 + (32C25 − 26C24 + 20C22 C3 + C5 ) En5 + O|| En6 ||. (12)

This completes the proof, since error Equation (12) shows that the order of convergence
of the frozen Jacobian multi-step iterative method (9) is four.

2.2.2. The Computational of Efficiency


Now, we compare the computational efficiency of our fourth-order scheme (9), called
by M4 , with some existing fourth-order methods. The considered methods are: the third-
order method M4,1 given by Sharma et al. [39],

y(k) = 23 x(k) − JF (/x(k) )−1 F (x(k) ), 0


x ( k +1) = x ( k ) − 1
2 − I + 94 JF (y(k) )−1 JF (x(k) ) + 34 JF (x(k) )−1 JF (y(k) )
× J F ( x ( k ) ) −1 F ( x ( k ) ),

the fourth-order iterative method M4,2 given by Darvishi and Barati [40],

y ( k ) = x ( k ) − J F ( x ( k ) ) − 1
F ( x ( k ) ), 
z ( k ) = x ( k ) − J F ( x ( k ) ) −1 F ( y ( k ) ) + F ( x ( k ) ) ,
/ 0 −1
(x(k) + z(k) )
x(k+1) = x(k) − 16 JF (x(k) ) + 23 JF ( 2 ) + 1
6 J F ( z (k) ) F ( x ( k ) ),

the fourth-order iterative method M4,3 given by Soleymani et al. [34,41],

y(k) = 23 x(k) − J/F (x(k) )


−1 F ( x ( k ) ),
0
 2
x(k+1) = x(k) − I − 38 I − JF (y(k) )−1 JF (x(k) ) J F ( x ( k ) ) −1 F ( x ( k ) ),

and the following Jarratt fourth-order method M4,4 [42],

y(k) = 23 x(k) − JF (x(k) )−1 F (x(k) ),


  −1  
x(k+1) = x(k) − 12 3JF (y(k) ) − JF (x(k) ) 3JF (y(k) ) + JF (x(k) )
× J F ( x ( k ) ) −1 F ( x ( k ) ).

The computational efficiency of our fourth-order method showed that our method M4
is better than methods M4,1 , M4,2 , M4,3 and M4,4 as the comparison results are presented in
Table 2, and Figures 3 and 4. As we can see from Table 2, the indices of our method M4 are
better than similar ones in methods M4,1 , M4,2 , M4,3 and M4,4 . Furthermore, Figures 3 and 4
show the superiority of our method in respect with the another schemes.

Table 2. Comparison of efficiency indices between M4 and other fourth-order methods.

Methods M4 M4,1 M4,2 M4,3 M4,4


No. of steps 3 2 3 2 2
Order of convergence 4 4 4 4 4
Functional evaluations 3n + n2 n + 2n2 2n + 3n2 n + 2n2 n + 2n2
41/(3n+n ) 41/(n+2n
2)
41/(2n+3n ) 41/(n+2n
2)
41/(n+2n
2 2 2)
The classical efficiency index (IE)
No. of LU decompositions 1 2 2 2 2
Cost of LU decompositions 2n3 4n3 4n3 4n3 4n3
3 3 3 3 3
2n3 10n3 4n3 7n3 n3
3 + 6n 3 + 2n 3 + 6n 3 + 2n 7 3 + 2n2
Cost of linear systems (based on flops) 2 2 2 2
3 3 3 3 3
Flops-like efficiency index (FLEI) 1/( 2n3 +7n2 +3n) 1/( 10n
3 +4n + n )
2 1/( 4n3 +9n2 +2n) 1/( 7n3 +4n2 +n) 1/( 7n3 +4n2 +n)
4 4 4 4 4

111
Mathematics 2022, 10, 2952

Figure 3. The classical efficiency index for methods M4 , M4,1 , M4,2 , M4,3 and M4,4 .

Figure 4. The Flops-like efficiency index for methods M4 , M4,1 , M4,2 , M4,3 and M4,4 .

3. Numerical Results
In order to check the validity and efficiency of our proposed frozen Jacobian multi-
step iterative methods, three test problems are considered to illustrate convergence and
computation behaviors such as efficiency index and some another indices of the frozen
Jacobian multi-step iterative methods. Numerical computations have been performed using
variable precision arithmetic that uses floating point representation of 100 decimal digits of
mantissa in MATLAB. The computer specifications are: Intel(R) Core(TM) i7-1065G7 CPU
1.30 GHz with 16.00 GB of RAM on Windows 10 pro.

Experiment 1. We begin with the following nonlinear system of n equations [43],

f i (x) = cos( xi ) − 1, i = 1, 2, . . . , n. (13)

The exact zero of F (x) = ( f 1 (x), f 2 (x), . . . , f n (x))t = 0 is (0, 0, . . . , 0)t . To solve (13),
we set the initial guess as (0.78, 0.78, . . . , 0.78)t . The stopping criterion is selected as
|| f (x(k) )|| ≤ 10−3 .

112
Mathematics 2022, 10, 2952

Experiment 2. The next test problem is the following system of nonlinear equations [44],

f i (x) = (1 − xi 2 ) + xi (1 + xi xn−2 xn−1 xn ) − 2, i = 1, 2, . . . , n. (14)

The exact root of F (x) = 0 is (1, 1, . . . , 1)t . To solve (14), the initial guess is taken as
(2, 2, . . . , 2)t . The stopping criterion is selected as || f (x(k) )|| ≤ 10−8 .

Experiment 3. The last test problem is the following nonlinear system [9],

f i (x) = xi 2 xi+1 − 1, i = 1, 2, . . . , n − 1,
(15)
f n (x) = xn 2 x1 − 1,
with the exact solution (1, 1, . . . , 1)t . To solve (15), the initial guess and the stopping criterion
are respectively considered as (3, 3, . . . , 3)t and || f (x(k) )|| ≤ 10−8 .

Table 3 shows the comparison results between our third-order frozen Jacobian two-
step iterative method M3 and some third-order frozen Jacobian iterative methods, namely,
M3,1 , M3,2 , M3,3 and M3,4 . For all test problems, two different values for n are considered,
namely, n = 50, 100. As this table shows, in all cases, our method works better than the
others. Similarly, in Table 4, CPU time and number of iterations are presented for our
fourth-order method, namely, M4 and methods M4,1 , M4,2 , M4,3 and M4,4 . Similar to M3 ,
the CPU time for M4 is less than the CPU time for the other methods. These tables show
superiority of our methods in respect with the other ones. In Tables 3 and 4, it shows the
number of iterations.

Table 3. Comparison results between M3 and other third-order methods.

Experiment 1 Experiment 2 Experiment 3


Methods
n it cpu n it cpu n it cpu
M3 50 4 7.7344 50 5 10.6250 50 5 10.4844
100 5 59.6406 100 5 59.8594 100 5 60.0313
M3,1 50 4 11.0625 50 5 13.8125 50 5 14.1406
100 4 69.4219 100 5 87.3594 100 5 87.4063
M3,2 50 4 18.7188 50 5 24.9375 50 5 21.5469
100 5 157.2344 100 5 143.7344 100 5 146.2656
M3,3 50 4 20.7031 50 5 23.1563 50 5 24.2969
100 5 153.1719 100 5 143.2969 100 5 145.4063
M3,4 50 4 13.1719 50 5 13.2500 50 4 11.0156
100 4 73.2500 100 5 88.2031 100 4 70.2500

Table 4. Comparison results between M4 and other fourth-order methods.

Experiment 1 Experiment 2 Experiment 3


Methods
n it cpu n it cpu n it cpu
M4 50 4 12.2463 50 4 13.3218 50 4 11.5781
100 4 78.1563 100 5 94.9063 100 4 74.2969
M4,1 50 4 23.6875 50 4 21.9531 50 4 21.7969
100 4 151.9844 100 4 144.7656 100 4 140.8438
M4,2 50 3 15.3906 50 4 18.9531 50 4 18.6875
100 4 121.6563 100 4 122.7344 100 4 118.5781
M4,3 50 3 12.2188 50 4 17.8750 50 4 15.2656
100 4 97.5469 100 4 99.0469 100 4 97.1250
M4,4 50 3 16.4688 50 4 21.7344 50 4 20.7188
100 3 109.1719 100 4 152.0156 100 4 140.2969

113
Mathematics 2022, 10, 2952

4. Another Comparison
In the previous parts, we presented some comparison results between our methods
M3 and M4 with some another frozen Jacobian multi-step iterative methods from third- and
fourth-order methods. In this section, we compare our presented methods with three other
methods which are fourth- and fifth-order ones. As Tables 5 and 6 and Figures 5 and 6
show, our methods are also better than these methods.

First. The fourth-order method given by Qasim et al. [25], M A ,

J F ( x ( k ) ) θ1 = F ( x ( k ) ),
y ( k ) = x ( k ) − θ1 ,
J F ( x ( k ) ) θ2 = F ( y ( k ) ),
JF ( x ( k ) ) θ3 = JF ( y ( k ) ) θ2 ,
x(k+1) = y(k) − 2θ2 + θ3 .

Second. The fourth-order Newton-like method by Amat et al. [26], MB ,

y ( k ) = x ( k ) − J F ( x ( k ) ) −1 F ( x ( k ) ),
z ( k ) = y ( k ) − J F ( x ( k ) ) −1 F ( y ( k ) ),
x ( k +1) = z ( k ) − J F ( x ( k ) ) −1 F ( z ( k ) ).

Third. The fifth-order iterative method by Ahmad et al. [28], MC ,

JF ( x ( k ) ) θ1 = F ( x ( k ) ),
y ( k ) = x ( k ) − θ1 ,
J F ( x ( k ) ) θ2 = F ( y ( k ) ),
z(k) = y(k) − 3θ2 ,
JF ( x ( k ) ) θ3 = JF ( z ( k ) ) θ2 ,
JF ( x ( k ) ) θ4 = JF ( z ( k ) ) θ3 ,
x(k+1) = y(k) − 74 θ2 + 12 θ3 + 14 θ4 .

Figure 5. The classical efficiency index for M3 , M4 , M A , MB and MC .

114
Mathematics 2022, 10, 2952

Figure 6. The Flops-like efficiency index for M3 , M4 , M A , MB and MC .

Table 5. Numerical results for comparing of M3 and M4 with M A , MB and MC .

Methods M3 M4 MA MB MC
No. of steps 2 3 2 3 3
Order of convergence 3 4 4 4 5
Functional evaluations 2n + n2 3n + n2 2n + 2n2 3n + n2 2n + 2n2
31/(2n+n ) 41/(3n+n ) 41/(2n+2n ) 41/(3n+n ) 51/(2n+2n )
2 2 2 2 2
The classical efficiency index (IE)
No. of LU decompositions 1 1 1 1 1
Cost of LU decompositions 2n3 2n3 2n3 2n3 2n3
3 3 3 3 3
2n3 2n3 5n3 2n3 5n3
3 + 4n 3 + 6n 3 + 4n 3 + 6n 3 + 4n
Cost of linear systems (based on flops) 2 2 2 2 2
3 3 3 3 3
Flops-like efficiency index (FLEI) 1/( 3 +5n2 +2n)
2n 1/( 3 +7n2 +3n)
2n 1/( 3 +6n2 +2n)
5n 1/( 3 +7n2 +3n)
2n 1/( 3 +6n2 +2n)
5n
3 4 4 4 5

The comparison results of computational efficiency between our methods M3 and


M4 with selected methods M A , MB and MC are presented in Table 5. Additionally,
Figures 5 and 6 show the graphical comparisons between these methods. Finally, Table 6
shows CPU time and number of iterations to solve our test problems by methods M3 ,
M4 , M A , MB and MC . These numerical and graphical reports show the quality of our
algorithms.

Table 6. Comparison results between M3 , M4 , M A , MB and MC .

Experiment 1 Experiment 2 Experiment 3


Methods
n it CPU n it CPU n it CPU
M3 50 4 7.7344 50 5 10.6250 50 5 10.4844
100 5 59.6406 100 5 59.8594 100 5 60.0313
M4 50 4 12.2463 50 4 13.3218 50 4 11.5781
100 4 78.1563 100 5 94.9063 100 4 74.2969
MA 50 6 23.1875 50 7 25.0625 50 6 25.4063
100 6 139.5625 100 7 173.8125 100 6 150.8594
MB 50 4 15.2509 50 4 12.1563 50 4 12.9219
100 4 76.1406 100 5 91.1719 100 4 71.6406
MC 50 4 23.4688 50 4 23.4854 50 4 22.1531
100 4 139.9844 100 4 185.1406 100 4 138.4063

115
Mathematics 2022, 10, 2952

5. Conclusions
In this article, two new frozen Jacobian two- and three-step iterative methods to solve
systems of nonlinear equations are presented. For the first method, we proved that the order
of convergence is three, while for the second one, a fourth-order convergence is proved.
By solving three different examples, one may see our methods work as well. Further, the
CPU time of our methods is less than some selected frozen Jacobian multi-step iterative
methods in the literature. Moreover, other indices of our methods such as number of steps,
functional evaluations, the classical efficiency index, and so on, are better than these indices
for other methods. This class of the frozen Jacobian multi-step iterative methods can be a
pattern for new research on the frozen Jacobian iterative algorithms.

Author Contributions: Investigation, R.H.A.-O. and M.T.D.; Project administration, M.T.D.; Re-
sources, R.H.A.-O.; Supervision, M.T.D.; Writing—original draft, M.T.D. All authors have read and
agreed to the published version of the manuscript.
Funding: This research received no external funding.
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: Not applicable.
Acknowledgments: The authors would like to thank the editor of the journal and three anonymous
reviewers for their generous time in providing detailed comments and suggestions that helped us to
improve the paper.
Conflicts of Interest: The authors declare no conflict of interest.

References
1. Fay, T.H.; Graham, S.D. Coupled spring equations. Int. J. Math. Educ. Sci. Technol. 2003, 34, 65–79. [CrossRef]
2. Petzold, L. Automatic selection of methods for solving stiff and non stiff systems of ordinary differential equations. SIAM J. Sci.
Stat. Comput. 1983, 4, 136–148. [CrossRef]
3. Ehle, B.L. High order A-stable methods for the numerical solution of systems of D.E.’s. BIT Numer. Math. 1968, 8, 276–278.
[CrossRef]
4. Wambecq, A. Rational Runge-Kutta methods for solving systems of ordinary differential equations. Computing 1978, 20, 333–342.
[CrossRef]
5. Liang, H.; Liu, M.; Song, M. Extinction and permanence of the numerical solution of a two-prey one-predator system with
impulsive effect. Int. J. Comput. Math. 2011, 88, 1305–1325. [CrossRef]
6. Harko, T.; Lobo, F.S.N.; Mak, M.K. Exact analytical solutions of the Susceptible-Infected-Recovered (SIR) epidemic model and of
the SIR model with equal death and birth rates. Appl. Math. Comput. 2014, 236, 184–194. [CrossRef]
7. Zhao, J.; Wang, L.; Han, Z. Stability analysis of two new SIRs models with two viruses. Int. J. Comput. Math. 2018, 95, 2026–2035.
[CrossRef]
8. Kröger, M.; Schlickeiser, R. Analytical solution of the SIR-model for the temporal evolution of epidemics, Part A: Time-independent
reproduction factor. J. Phys. A Math. Theor. 2020, 53, 505601. [CrossRef]
9. Ullah, M.Z.; Behl, R.; Argyros, I.K. Some high-order iterative methods for nonlinear models originating from real life problems.
Mathematics 2020, 8, 1249. [CrossRef]
10. Argyros, I.K. Concerning the “terra incognita” between convergence regions of two Newton methods. Nonlinear Anal. 2005, 62,
179–194. [CrossRef]
11. Drexler, M. Newton Method as a Global Solver for Non-Linear Problems. Ph.D Thesis, University of Oxford, Oxford, UK, 1997.
12. Guti’errez, J.M.; Hern’andez, M.A. A family of Chebyshev-Halley type methods in Banach spaces. Bull. Aust. Math. Soc. 1997, 55,
113–130. [CrossRef]
13. Cordero, A.; Jordán, C.; Sanabria, E.; Torregrosa, J.R. A new class of iterative processes for solving nonlinear systems by using one
divided differences operator. Mathematics 2019, 7, 776. [CrossRef]
14. Stefanov, S.M. Numerical solution of systems of non linear equations defined by convex functions. J. Interdiscip. Math. 2022, 25,
951–962. [CrossRef]
15. Lee, M.Y.; Kim, Y.I.K. Development of a family of Jarratt-like sixth-order iterative methods for solving nonlinear systems with
their basins of attraction. Algorithms 2020, 13, 303. [CrossRef]
16. Cordero, A.; Jordán, C.; Sanabria-Codesal, E.; Torregrosa, J.R. Design, convergence and stability of a fourth-order class of iterative
methods for solving nonlinear vectorial problems. Fractal Fract. 2021, 5, 125. [CrossRef]

116
Mathematics 2022, 10, 2952

17. Amiri, A.; Cordero, A.; Darvishi, M.T.; Torregrosa, J.R. A fast algorithm to solve systems of nonlinear equations. J. Comput. Appl.
Math. 2019, 354, 242–258. [CrossRef]
18. Argyros, I.K.; Sharma, D.; Argyros, C.I.; Parhi, S.K.; Sunanda, S.K. A family of fifth and sixth convergence order methods for
nonlinear models. Symmetry 2021, 13, 715. [CrossRef]
19. Singh, A. An efficient fifth-order Steffensen-type method for solving systems of nonlinear equations. Int. J. Comput. Sci. Math.
2018, 9, 501–514. [CrossRef]
20. Ullah, M.Z.; Serra-Capizzano, S.; Ahmad, F. An efficient multi-step iterative method for computing the numerical solution of
systems of nonlinear equations associated with ODEs. Appl. Math. Comput. 2015, 250, 249–259. [CrossRef]
21. Pacurar, M. Approximating common fixed points of Pres̆ic-Kannan type operators by a multi-step iterative method. An. St. Univ.
Ovidius Constanta 2009, 17, 153–168.
22. Rafiq, A.; Rafiullah, M. Some multi-step iterative methods for solving nonlinear equations. Comput. Math. Appl. 2009, 58,
1589–1597. [CrossRef]
23. Aremu, K.O.; Izuchukwu, C.; Ogwo, G.N.; Mewomo, O.T. Multi-step iterative algorithm for minimization and fixed point
problems in p-uniformly convex metric spaces. J. Ind. Manag. Optim. 2021, 17, 2161. [CrossRef]
24. Soleymani, F.; Lotfi, T.; Bakhtiari, P. A multi-step class of iterative methods for nonlinear systems. Optim. Lett. 2014, 8, 1001–1015.
[CrossRef]
25. Qasim, U.; Ali, Z.; Ahmad, F.; Serra-Capizzano, S.; Ullah, M.Z.; Asma, M. Constructing frozen Jacobian iterative methods for
solving systems of nonlinear equations, associated with ODEs and PDEs using the homotopy method. Algorithms 2016, 9, 18.
[CrossRef]
26. Amat, S.; Busquier, S.; Grau, À.; Grau-Sánchez, M. Maximum efficiency for a family of Newton-like methods with frozen
derivatives and some applications. Appl. Math. Comput. 2013, 219, 7954–7963. [CrossRef]
27. Kouser, S.; Rehman, S.U.; Ahmad, F.; Serra-Capizzano, S.; Ullah, M.Z.; Alshomrani, A.S.; Aljahdali, H.M.; Ahmad, S.; Ahmad, S.
Generalized Newton multi-step iterative methods GMNp,m for solving systems of nonlinear equations. Int. J. Comput. Math.
2018, 95, 881–897. [CrossRef]
28. Ahmad, F.; Tohidi, E.; Ullah, M.Z.; Carrasco, J.A. Higher order multi-step Jarratt-like method for solving systems of nonlinear
equations: Application to PDEs and ODEs. Comput. Math. Appl. 2015, 70, 624–636. [CrossRef]
29. Kaplan, W.; Kaplan, W.A. Ordinary Differential Equations; Addison-Wesley Publishing Company: Boston, MA, USA, 1958.
30. Emmanuel, E.C. On the Frechet derivatives with applications to the inverse function theorem of ordinary differential equations.
Aian J. Math. Sci. 2020, 4, 1–10.
31. Ortega, J.M.; Rheinboldt, W.C.; Werner, C. Iterative Solution of Nonlinear Equations in Several Variables; Society for Industrial and
Applied Mathematics: Philadelphia, PA, USA, 2000.
32. Behl, R.; Bhalla, S.; Magre nán, A.A.; Kumar, S. An efficient high order iterative scheme for large nonlinear systems with dynamics.
J. Comput. Appl. Math. 2022, 404, 113249. [CrossRef]
33. Ostrowski, A.M. Solutions of the Equations and Systems of Equations; Prentice-Hall: England Cliffs, NJ, USA; New York, NY,
USA, 1964.
34. Montazeri, H.; Soleymani, F.; Shateyi, S.; Motsa, S.S. On a new method for computing the numerical solution of systems of
nonlinear equations. J. Appl. Math. 2012, 2012, 751975. [CrossRef]
35. Darvishi, M.T. A two-step high order Newton-like method for solving systems of nonlinear equations. Int. J. Pure Appl. Math.
2009, 57, 543–555.
36. Hernández, M.A. Second-derivative-free variant of the Chebyshev method for nonlinear equations. J. Optim. Theory Appl. 2000, 3,
501–515. [CrossRef]
37. Babajee, D.K.R.; Dauhoo, M.Z.; Darvishi, M.T.; Karami, A.; Barati, A. Analysis of two Chebyshev-like third order methods free
from second derivatives for solving systems of nonlinear equations. J. Comput. Appl. Math. 2010, 233, 2002–2012. [CrossRef]
38. Noor, M.S.; Waseem, M. Some iterative methods for solving a system of nonlinear equations. Comput. Math. Appl. 2009, 57,
101–106. [CrossRef]
39. Sharma, J.R.; Guha, R.K.; Sharma, R. An efficient fourth order weighted-Newton method for systems of nonlinear equations.
Numer. Algorithms 2013, 62, 307–323. [CrossRef]
40. Darvishi, M.T.; Barati, A. A fourth-order method from quadrature formulae to solve systems of nonlinear equations. Appl. Math.
Comput. 2007, 188, 257–261. [CrossRef]
41. Soleymani, F.,Regarding the accuracy of optimal eighth-order methods. Math. Comput. Modell. 2011, 53, 1351–1357. [CrossRef]
42. Cordero, A.; Hueso, J.L.; Martínez, E.; Torregrosa, J.R. A modified Newton-Jarratt’s composition. Numer. Algorithms 2010, 55,
87–99. [CrossRef]
43. Shin, B.C.; Darvishi, M.T.; Kim, C.H.,A comparison of the Newton–Krylov method with high order Newton-like methods to solve
nonlinear systems. Appl. Math. Comput. 2010, 217, 3190–3198. [CrossRef]
44. Waziri, M.Y.; Aisha, H.; Gambo, A.I. On performance analysis of diagonal variants of Newton’s method for large-scale systems of
nonlinear equations. Int. J. Comput. Appl. 2011, 975, 8887.

117
mathematics

Article
Approximation of the Fixed Point of the Product of Two
Operators in Banach Algebras with Applications to Some
Functional Equations
Khaled Ben Amara 1 , Maria Isabel Berenguer 2,3, * and Aref Jeribi 1

1 Department of Mathematics, Faculty of Sciences of Sfax, University of Sfax, Sfax 3000, Tunisia
2 Department of Applied Mathematics, E.T.S. de Ingeniería de Edificación, University of Granada,
18071 Granada, Spain
3 Institute of Mathematics (IMAG), University of Granada, 18071 Granada, Spain
* Correspondence: [email protected]

Abstract: Making use of the Boyd-Wong fixed point theorem, we establish a new existence and
uniqueness result and an approximation process of the fixed point for the product of two nonlinear
operators in Banach algebras. This provides an adequate tool for deriving the existence and
uniqueness of solutions of two interesting type of nonlinear functional equations in Banach algebras,
as well as for developing an approximation method of their solutions. In addition, to illustrate the
applicability of our results we give some numerical examples.

Keywords: Banach algebras; fixed point theory; functional equations; Schauder bases

MSC: 65J15; 47J25; 46B15; 32A65

Citation: Ben Amara, K; Berenguer,


M.I.; Jeribi, A. Approximation of the
Fixed Point of the Product of Two 1. Introduction
Operators in Banach Algebras with
Many phenomena in physics, chemistry, mechanics, electricity, and so as, can be
Applications to Some Functional
Equations. Mathematics 2022, 10, 4179.
formulated by using the following nonlinear differential equations with nonlocal initial
https://doi.org/10.3390/
condition of the form:
⎧  
math10224179

⎪ d x (t)
⎨ = g(t, x (t)), t ∈ J := [0, ρ],
Academic Editor: Salvador dt f (t, x (t))
(1)
Romaguera ⎪


x (0) = μ ( x ),
Received: 19 September 2022
Accepted: 3 November 2022
where ρ > 0 is a real constant, f : J × R → R \ {0}, g : J × R → R are supposed to
Published: 9 November 2022
be D -Lipschitzian with respect to the second variable, and the operator μ : C ( J ) → R
Publisher’s Note: MDPI stays neutral represents the nonlocal initial condition, see [1,2]. Here, C ( J ) is the space of all continuous
with regard to jurisdictional claims in functions from J into R endowed with the norm · ∞ = supt∈ J x (t) .
published maps and institutional affil- The nonlocal condition x (0) = μ( x ) can be more descriptive in physics with better
iations. effect than the classical initial condition x (0) = x0 , (see, e.g., [2–5]). In the last case, i.e.,
x (0) = x0 , the problem (1) has been studied by Dhage [6] and O’Regan [7]. Therefore it is of
interest to discuss and to approximate the solution of (1) with a nonlocal initial condition.
Similarly another class of nonlinear equations is used frequently to describe many
Copyright: © 2022 by the authors.
phenomena in different fields of applied sciences such as physics, control theory, chemistry,
Licensee MDPI, Basel, Switzerland.
biology, and so forth (see [8–11]). This class is generated by the nonlinear integral equations
This article is an open access article
of the form:
distributed under the terms and
/  η (t) 0
conditions of the Creative Commons
Attribution (CC BY) license (https:// x (t) = f (t, x (σ (t))) · q(t) + K (t, s, x (τ (s)))ds , t ∈ J := [0, ρ], (2)
0
creativecommons.org/licenses/by/
4.0/).

Mathematics 2022, 10, 4179. https://doi.org/10.3390/math10224179 https://www.mdpi.com/journal/mathematics


119
Mathematics 2022, 10, 4179

where ρ > 0 is a real constant, σ, τ, η : J → J and q : J → R are supposed to be continuous,


and the functions f : J × R → R, K : J × J × R → R are supposed to be D -Lipschizian with
respect to the second and the third variable, respectively.
Both, (1) and (2), can be interpreted as fixed point problems in which the equation
involved is a nonlinear hybrid equation on a Banach algebra E of the type

x = A ( x ) · B ( x ), (3)

where A and B are nonlinear operators map a nonempty closed convex subset Ω ⊂ E
into E.
A hybrid fixed point result to (3) was proved by Dhage in [12] and since then, several
extensions and generalizations of this result have been achieved. See [13–15] and the
references therein. These results can be used to achieves the existence of solutions. Although
the explicit calculation of the fixed point is difficult in most cases, the previous cited results
are regarded as one of the most powerful tools to give an approximation of the fixed
point by a computational method and to develop numerical methods that allow us to
approximate the solution of these equations.
In Banach spaces, several works deals with developing numerical techniques in
order to approximate the solutions of integral and integro–differential equations, by using
different methods such as the Chebyshev polynomial [16], the secant-like methods [17],
using Schauder’s basis [18,19], the parameterization method [20], the wavelet methods [21],
a collocation method in combination with operational matrices of Berstein polynomials [22],
the contraction principle and a suitable quadrature formula [23], the variational iteration
method [24], etc.
Since the Banach algebras represents a practical framework for several equations such
as (1) and (2), and in general (3), the purposes of this paper are twofold. Firstly, to present,
under suitable conditions, a method to approximate the fixed point of a hybrid equation of
type (3), by means of the product and composition of operators defined in a Banach algebra.
Secondly, to set forth and apply the proposed method to obtain an approximation of the
solutions of (1) and (2).
The structure of this work is as follows: in Section 2 we present some definitions and
auxiliary results; in Section 3 we derive an approximation method for the fixed point of the
hybrid Equation (3); in Sections 4 and 5, we apply our results to prove the existence and the
uniqueness of solution of (1) and (2), we give an approximation method for these solutions
and moreover, we establish some numerical examples to illustrate the applicability of our
results. Finally, some conclusions are quoted in Section 6.

2. Analytical Tools
In this section, we provide some concepts and results that we will need in the following
sections. The first analytical tool to be used comes from the theory of the fixed point. Let X
be a Banach space with norm · and the zero element θ. We denote by B( x, r ) the closed
ball centered at x with radius r. We write Br to denote B(θ, r ). For any bounded subset Ω of
X, the symbol Ω denotes the norm of a set Ω, i.e., Ω = sup{ x , x ∈ Ω}.
Let us introduce the concept of D -Lipschitzian mappings which will be used in
the sequel.

Definition 1. Let X be a Banach space. A mapping A : X −→ X is said to be D -Lipschitzian if

Ax − Ay ≤ φ( x − y ) ∀ x, y ∈ X

with φ : R+ −→ R+ a continuous nondecreasing function such that φ(0) = 0. The mapping φ is


called the D -function associate to A. When φ(r ) < r for r > 0, the mapping A is called a nonlinear
contraction on X.

120
Mathematics 2022, 10, 4179

The class of D -Lipschitzian mappings on X contains the class of Lipschitzian mapping


on X, indeed if φ(r ) = α r, for some α > 0, then A is called Lipschitzian mapping with
Lipschitz constant α or an α-Lipschitzian mapping. When 0 ≤ α < 1, we say that A is
a contraction.
The Banach fixed point theorem ensures that every contraction operator A on a
complete metric space X has a unique fixed point x̃ ∈ X, and, for every x0 ∈ X, the
sequence { An ( x0 )}n∈N converges to x̃. Much attention has been paid to Banach principle
and it was generalized in different works (we quote, for instance, [25,26]). In [25], Boyd
and Wong established the following result.

Theorem 1. Let ( X, d) be a complete metric space, and let A : X → X be a mapping satisfying

d( A( x ), A(y)) ≤ ϕ(d( x, y)), ∀ x, y ∈ X

where ϕ : [0, ∞) → [0, ∞) is a continuous function such that ϕ(r ) < r if r > 0. Then A has a
unique fixed point x̃ ∈ X and for any x0 ∈ X, the sequence { An ( x0 )}n∈N converges to x̃.

On the other hand, Schauder bases will constitute the second essential tool. We recall
that a Schauder basis in a Banach space E is a sequence {en }n∈N ⊂ E such that for every
x ∈ E, there is a unique sequence { an }n∈N ⊂ R such that

x= ∑ an en .
n ≥1

This notion produces the concept of the sequence of projections Pn : E → E, defined by the
formula  
n
Pn ∑ a k ek = ∑ ak ek ,
k ≥1 k =1

and the sequence of coordinate functionals en∗ ∈ E∗ defined as


 
en∗ ∑ a k ek = an .
k ≥1

Moreover, in view of the Baire category Theorem [27], that for all n ≥ 1, en∗ and Pn are
continuous. This yields, in particular, that

lim Pn ( x ) − x = 0.
n→∞

3. Existence, Uniqueness and Approximation of a Fixed Point of the Product of Two


Operators in Banach Algebras
Based on the Boyd-Wong Theorem, we establish the following fixed point result for
the product of two nonlinear operators in Banach algebras.

Theorem 2. Let X be a nonempty closed convex subset of a Banach algebra E. Let A, B : X → E


be two operators satisfying the following conditions:
(i ) A and B are D -lipschitzian with D -functions ϕ and ψ respectively,
(ii ) A( X ) and B( X ) are bounded,
(iii ) A( x ) · B( x ) ∈ X, for all x ∈ X.
Then, if A( X ) ψ(r ) + B( X ) ϕ(r ) < r when r > 0, there is a unique point x̃ ∈ X such that
A( x̃ ) · B( x̃ ) = x̃. In addition, for each x0 ∈ X, the sequence {( A · B)n ( x0 )}n∈N converges to x̃.

Proof. Let x, y ∈ X. we have

A( x ) · B( x ) − A(y) · B(y) ≤ A( x ) · ( B( x ) − B(y)) + ( A( x ) − A(y)) · B(y) ≤

121
Mathematics 2022, 10, 4179

A( x ) B ( x ) − B ( y ) + B ( y ) A ( x ) − A ( y ) ≤ A ( X ) ψ ( x − y ) + B ( X ) ϕ ( x − y ).
This implies that A · B defines a nonlinear contraction with D -function

φ(r ) = A( X ) ψ(r ) + B( X ) ϕ(r ), r > 0.

Applying the cited Boyd-Wong’s fixed point Theorem, we obtain the desired result.

Boyd-Wong’s fixed point Theorem expresses the fixed point of A · B as the limit of the
sequence {( A · B)n ( x0 )}n∈N with x0 ∈ X. If it is possible explicitly compute ( A · B)n ( x0 ),
then for each n, the expression ( A · B)n ( x0 ) would be an approximation of the fixed point.
But in the practice, this explicit calculation use to be not possible. For that, our aim is to
propose another approximation of the fixed point which simple to calculate. We will need
the following lemma.

Lemma 1. Let X be a nonempty closed convex subset of a Banach algebra E. Let A, B : X → E be


two D -Lipschitzian operators with D -functions ϕ and ψ, respectively, and A · B maps X into X.
Moreover, suppose that
φ(r ) < r, r > 0.
Let x̃ be the unique fixed point of A · B and x0 ∈ X. Let ε > 0, m ∈ N, and T0 , T1 , . . . , Tm : E → E,
with T0 ≡ I, I being the identity operator on E, such that
ε
x̃ − ( A · B)m ( x0 ) ≤ (4)
2
and

m −1  
∑ φm− p ( A · B) ◦ Tp−1 ◦ . . . ◦ T1 ( x0 ) − Tp ◦ . . . ◦ T1 ( x0 ) +
p =1
ε
( A · B) ◦ Tm−1 ◦ . . . ◦ T1 ( x0 ) − Tm ◦ . . . ◦ T1 ( x0 ) ≤ . (5)
2
Then,
x̃ − Tm ◦ . . . ◦ T1 ( x0 ) ≤ ε.

Proof. Arguing as in the proof of Theorem 2, it follows that A · B is a nonlinear contraction


with D -function φ, and by induction argument, it is easy to show that

( A · B)n ( x ) − ( A · B)n (y) ≤ φn ( x − y ), x, y ∈ X. (6)

By using the triangular inequality, we have

( A · B)m ( x0 ) − Tm ◦ . . . ◦ T1 ( x0 ) ≤
 
 
( A · B)m−1 ◦ ( A · B)( x0 ) − ( A · B)m−1 ◦ T1 ( x0 )
 
 
+ ( A · B)m−2 ◦ ( A · B) ◦ T1 ( x0 ) − ( A · B)m−2 ◦ T2 ◦ T1 ( x0 ) + · · · +
+ ( A · B) ◦ ( A · B) ◦ Tm−2 ◦ . . . ◦ T1 ( x0 ) − ( A · B) ◦ Tm−1 ◦ . . . ◦ T1 ( x0 )
+ ( A · B) ◦ Tm−1 ◦ . . . ◦ T1 ( x0 ) − Tm ◦ . . . ◦ T1 ( x0 ) .

Taking into account (6), we obtain

( A · B)m ( x0 ) − Tm ◦ . . . ◦ T1 ( x0 ) ≤
m −1  
∑ φm− p ( A · B) ◦ Tp−1 ◦ . . . ◦ T1 ( x0 ) − Tp ◦ . . . ◦ T1 ( x0 )
p =1

+ ( A · B) ◦ Tm−1 ◦ . . . ◦ T1 ( x0 ) − Tm ◦ . . . ◦ T1 ( x0 ) .

122
Mathematics 2022, 10, 4179

This implies, by using the Triangular inequality again, that

x̃ − Tm ◦ . . . ◦ T1 ( x0 ) ≤
m −1  
∑ φm− p ( A · B) ◦ Tp−1 ◦ . . . ◦ T1 ( x0 ) − Tp ◦ . . . ◦ T1 ( x0 )
p =1

+ ( A · B) ◦ Tm−1 ◦ . . . ◦ T1 ( x0 ) − Tm ◦ . . . ◦ T1 ( x0 ) + x̃ − ( A · B)m ( x0 ) ≤ ε. (7)

Taking into account the above lemma, observe that, under the previous hypotheses,

x ∗ = Tm ◦ . . . ◦ T1 ( x0 ) ≈ x̃

In order to get the approximation x ∗ = Tm ◦ . . . ◦ T1 ( x0 ) of the fixed point x̃, it is evident


that, given ε > 0, by Theorem 2, condition (4) is satisfied for m sufficiently large. So, we are
interested in building T1 , T2 , . . ., Tm satisfying (5), i. e. with the idea that

( A · B)m ( x0 ) ≈ Tm ◦ . . . ◦ T1 ( x0 ).

Schauder bases are the tool we will use next to build such operators. Concretely, for the
case of problems (1) and (2), which can be written as a fixed point problem x = A( x ) · B( x ),
where B is given by an integral operator, we will choice to approximate only the power
terms of the operator B which is difficult to compute in general, unlike operator A which is
easy to calculate and does not need to approximate their power terms. For this reason, we
specifically propose the following scheme, in which we will construct S1 , S2 ,· · · , Sm :

x0

( A · B)( x0 ) ≈ T1 ( x0 ) = A( x0 ) · S1 ( x0 )
↓ ↓
( A · B )2 ( x0 ) ≈ T2 ◦ T1 x0 = ( A · S2 ) ◦ T1 ( x0 )
.. .. ..
. . .

.. .. ..
. . .
↓ ↓
( A · B ) m ( x0 ) ≈ Tm ◦ . . . ◦ T1 ( x0 ) = ( A · Sm ) ◦ Tm−1 ◦ . . . ◦ T1 ( x0 ) ≈ x̃

Remark 1. The above scheme is constructed as follows. In the first term, we approximate B( x0 )
by S1 ( x0 ), then we obtain T1 ( x0 ) := A( x0 ) · S1 ( x0 ) as an approximation of the first term of the
Picard iterate, A( x0 ) · B( x0 ). In the second term of our scheme, we approximate the second term
of the Picard iterate, ( A · B)2 ( x0 ) = A(( A · B)( x0 )) · B(( A · B)( x0 )). So we obtain the second
term of our scheme by combining the first term T1 ( x0 ), with an approximation of the operator
B, which denoted by S2 , and consequently we obtain a second term of our scheme T2 ◦ T1 ( x0 ) =
( A · S2 )( T1 ( x0 )) which approximate ( A · B)2 ( x0 ).

4. Nonlinear Differential Equations with Nonlocal Initial Condition


In this section we focus our attention in the nonlinear differential equation with
nonlocal initial condition (1). This equation will be studied when the mappings f , g :
J × R → R are such that:
(i) The partial mappings t → f (t, x ), t → g(t, x ) are continuous and the mapping
μ : C ( J ) → R is Lμ -Lipschitzian.

123
Mathematics 2022, 10, 4179

(ii) There exist r > 0, α, γ : J → R two continuous functions and ϕ, ψ : R+ −→ R+ two


nondecreasing, continuous functions such that

| f (t, x ) − f (t, y)| ≤ α(t) ϕ(| x − y|), t ∈ J, and x, y ∈ R with | x |, |y| ≤ r,

and

| g(t, x ) − g(t, y)| ≤ γ(t)ψ(| x − y|), t ∈ J and x, y ∈ R with | x |, |y| ≤ r.

(iii) There is a constant δ > 0 such that supx∈R,| x|≤r | f (0, x )|−1 ≤ δ.
Throughout this section, Ω will denote the closed ball Br of C ( J ), where r is defined in
the above assumption (ii ). Observe that Ω is a non-empty, closed, convex and bounded
subset of C ( J ).

4.1. Existence and Uniqueness of Solutions


In this subsection, we prove the existence and the uniqueness of a solution to the
functional differential problem (1).

Theorem 3. Assume that the assumptions (i ), (ii ) and (iii ) hold. If

M A MB ≤ r and
 
M A δLμ t + M A δ2 |α(0)| Mμ + MB α ∞ ϕ(t) + M A γ(·) L1 ψ ( t ) < t, ∀t > 0,

where M A = α ∞ ϕ(r ) + f (·, 0) ∞ , MB = δMμ + γ ∞ ρψ(r ) + ρ g(·, 0) ∞ and


Mμ = Lμ r + |μ(0)| , then the nonlinear differential problem (1) has a unique solution in Ω.

Proof. Notice that the problem of the existence of a solution to (1) can be formulated in the
following fixed point problem x = A( x ) · B( x ), where A, B are given for x ∈ C ( J ) by

( A( x ))(t) = f (t, x (t))


/  t 0 (8)
1
( B( x ))(t) = μ( x ) + g(s, x (s))ds , t ∈ J.
f (0, x (0)) 0

Let x ∈ Ω and t, t ∈ J. Since f is D -lipschitzian with respect to the second variable and is
continuous with respect to the first variable, then by using the inequality

| f (t, x (t)) − f (t , x (t ))| ≤ | f (t, x (t)) − f (t , x (t))| + | f (t , x (t)) − f (t , x (t ))|,

we can show that A maps Ω into C ( J ).


Now, let us claim that B maps Ω into C ( J ). In fact, let x ∈ Ω and t, t ∈ J be arbitrary.
Taking into account that t → g(t, x ) is a continuous mapping, it follows from assumption
(ii ) that
 t
|( B( x ))(t) − ( B( x ))(t )| ≤ | g(s, x (s)) − g(s, 0)|ds + (t − t ) g(·, 0) ∞ ≤
t

(t − t )( γ ∞ ψ (r ) + g(·, 0) ∞ ).

This proves the claim. Our strategy is to apply Theorem 2 to show the existence and the
uniqueness of a fixed point for the product A · B in Ω which in turn is a continuous solution
for problem (1).
For this purpose, we will claim, first, that A and B are D -lipschitzian mappings on Ω.
The claim regarding A is clear in view of assumption (ii ), that is A is D -lipschitzian with
D -function Φ such that
Φ(t) = α ∞ ϕ(t), t ∈ J.

124
Mathematics 2022, 10, 4179

We corroborate now the claim for B. Let x, y ∈ Ω, and let t ∈ J. By using our assumptions,
we obtain

|( B( x ))(t) − ( B(y))(t)| =
  t 
 1 1 
 μ ( x ) − μ ( y ) + g ( s, x ( s )) − g ( s, y ( s )) ds ≤
 f (0, x (0)) f (0, y(0)) 0

Lμ |α(0)|  
x−y + Lμ r + |μ(0)| ϕ( x − y )+
| f (0, x (0))| | f (0, x (0)) f (0, y(0))|
 t
|γ(s)|ψ(| x (s) − y(s)|)ds ≤
0
 
δLμ x − y + δ |α(0)| Lμ r + |μ(0)| ϕ( x − y ) + γ(·)
2
L1 ψ ( x − y ).

Taking the supremum over t, we obtain that B is D -lipschitzian with D -function Ψ such
that  
Ψ(t) = δLμ t + δ2 |α(0)| Lμ r + |μ(0)| ϕ(t) + γ(·) L1 ψ(t), t ∈ J.
On the other hand, bearing in mind assumption (i ), by using the above discussion we can
see that A(Ω) and B(Ω) are bounded with bounds M A and MB respectively. Taking into
account the estimate M A MB ≤ r, we obtain that A · B maps Ω into Ω.
Since
   t
 1 
|( B( x ))(t)| ≤  μ( x ) + | g(s, x (s))|ds
f (0, x (0)) 0
 t  t
≤ δ(|μ( x ) − μ(0)| + |μ(0)|) + | g(s, x (s)) − g(s, 0)|ds + | g(s, 0)|ds
0 0
 t  t
≤ δ( Lμ x + |μ(0)|) + |γ(s)|ψ(| x (s)|)ds + | g(s, 0)|ds,
0 0

and using the fact that |γ(s)|ψ(| x (s)|) ≤ γ ∞ ψ( x )≤ γ ∞ ψ (r ), we have that

B( x ) ≤ δ( Lμ x + |μ(0)|) + ρ γ ∞ ψ (r ) + ρ g(·, 0) ∞ = MB .

On the other hand, A( x ) ≤ M A since

|( A( x ))(t)| = | f (t, x (t))| ≤ | f (t, x (t)) − f (t, 0)| + | f (t, 0)| ≤


|α(t)| ϕ(| x (t)|) + | f (t, 0)| ≤ α ∞ ϕ(r ) + f (·, 0) ∞ = MA .

Taking into account that

( A · B)( x ) − ( A · B)(y) ≤ A( x ) B( x ) − B(y) + B(y) A( x ) − A(y) ,

we can notice that A · B is a nonlinear contraction with D -function


Θ(·) := M A Ψ(·) + MB Φ(·), i.e.,

( A · B)( x ) − ( A · B)(y) ≤ Θ( x − y ), x, y ∈ Ω. (9)

Now, applying Theorem 2, we infer that (1) has one and only one solution x̃ in Ω, and
for each x0 ∈ Ω we have

lim ( A · B)n ( x0 ) = x̃. (10)


n→∞

125
Mathematics 2022, 10, 4179

In what follows we will assume that the hypotheses of the Theorem 3 are satisfied.

4.2. Numerical Method to Approximate the Solution


In this subsection we find a numerical approximation of the solution to the nonlinear
Equation (1) using a Schauder basis {en }n≥1 in C ( J ) and the sequence of associated
projections { Pn }n≥1 . Let p ∈ N and n p ∈ N. We consider

S p : C( J ) −→ C ( J )
x −→ S p ( x )

defined as  t
1
S p ( x )(t) = μ( x ) + Pn p (U0 ( x ))(s)ds,
f (0, x (0)) 0

where U0 : C ( J ) −→ C ( J ) is given by U0 ( x )(s) = g(s, x (s)).


Now consider the operator Tp : C ( J ) −→ C ( J ) such that for each x ∈ C ( J ), Tp ( x ) is
defined by

Tp ( x )(t) = A( x )(t)S p ( x )(t), t ∈ J, (11)

with A : C ( J ) −→ C ( J ), A( x )(t) = f (t, x (t)).

Remark 2. For p ≥ 1 and any n p ∈ N that we use for defining Tp , the operator Tp maps Ω into
Ω, since just keep in mind that for x ∈ Ω, we have
   t 
   1 
 Tp ( x )(t) =  A( x )(t) μ( x ) + Pn p (U0 ( x ))(s)ds  ≤
 f (0, x (0)) 0
  t  
 
| f (t, x (t))| δ|μ( x )| + Pn p (U0 ( x ))(s)ds ,
0

and proceeding as in the above subsection and using the fact that Pn p is a bounded linear operator
on C ( J ), we get 1  2
 
 Tp ( x )(t) ≤ M A δ|μ( x )| + ρ 
Pn p (U0 ( x )) ≤
3 4
M A δ( Lμ r + |μ(0)|) + ρ sup | g(s, x (s))| ≤ M A MB < r.
s∈ J

In particular, for m ≥ 1, the operator Tm ◦ . . . ◦ T1 maps Ω into Ω.

Our goal is to prove that we can chose n1 , n2 , . . . ∈ N in order that T1 , T2 , . . ., which


are defined above, can be used to approximate the solution of (1).

Theorem 4. Let x̃ be the unique solution to the nonlinear problem (1). Let x0 ∈ Ω and ε > 0, then
there exist m ∈ N and ni ∈ N to construct Ti for i = 1, . . . , m, in such a way that

x̃ − Tm ◦ . . . ◦ T1 ( x0 ) ≤ ε.

Proof. Let x0 ∈ Ω and ε > 0. By using (10), there is m ∈ N such that

( A · B)m ( x0 ) − x̃ ≤ ε/2.

For that m, and for p ∈ {1, . . . , m}, we define U p : C ( J ) → C ( J ) by

U p ( x )(s) := g(s, Tp ◦ . . . ◦ T1 ( x )(s)), s ∈ J, x ∈ C ( J )

126
Mathematics 2022, 10, 4179

and A p : C ( J ) → C ( J ) by
 
A p ( x )(s) := f s, Tp ◦ . . . ◦ T1 ( x )(s) , s ∈ J, x ∈ C ( J ).

According to inequality (9), in view of (5) of Lemma 1, it suffices to show that

m −1  
∑ Θm− p ( A · B) ◦ Tp−1 ◦ . . . ◦ T1 ( x0 ) − Tp ◦ . . . ◦ T1 ( x0 ) +
p =1

( A · B) ◦ Tm−1 ◦ . . . ◦ T1 ( x0 ) − Tm ◦ . . . ◦ T1 ( x0 ) ≤ ε/2.

In view of (11), we have


 
( A · B) ◦ Tp−1 ◦ . . . ◦ T1 ( x0 )(t) − Tp ◦ Tp−1 ◦ . . . ◦ T1 ( x0 )(t) =
 
( A · B) ◦ Tp−1 ◦ . . . ◦ T1 ( x0 )(t) − ( A · S p ) ◦ Tp−1 ◦ . . . ◦ T1 ( x0 )(t) =
  
 A p−1 ( x0 )(t) B ◦ Tp−1 ◦ . . . ◦ T1 ( x0 )(t) − S p ◦ Tp−1 ◦ . . . ◦ T1 ( x0 )(t) .
 
Taking into account Remark 2, we infer that  A p−1 ( x ) is bounded, and consequently
we get  
( A · B) ◦ Tp−1 ◦ . . . ◦ T1 ( x0 )(t) − Tp ◦ Tp−1 ◦ . . . ◦ T1 ( x0 )(t) =
  t  t 
   
 A p−1 ( x0 )(t) g s, T ◦ . . . ◦ T ( x )( s ) ds − P ( U ( x ))( s ) ds ≤
 0
p −1 1 0 np
0
p −1 0 
   t   

 A p−1 ( x0 )(t)  Pn p (U p−1 ( x0 )) − U p−1 ( x0 ) (s) ds ≤
0
 


ρ A p−1 ( x0 ) Pn p (U p−1 )( x0 ) − U p−1 ( x0 ).

Taking the supremum over t, we get


 
( A · B) ◦ Tp−1 ◦ . . . ◦ T1 ( x0 ) − Tp ◦ Tp−1 ◦ . . . ◦ T1 ( x0 ) ≤
 
 
ρM A Pn p (U p−1 )( x0 ) − U p−1 ( x0 ).

Since Θ is a nondecreasing continuous mapping, and taking into account the convergence
of the projection operators associated to the Schauder basis, for all 1 ≤ p ≤ m we obtain
  
 
Θm− p ρM A Pn p (U p−1 ( x0 )) − U p−1 ( x0 ) ≤ ε/2m,

for n p sufficiently large. Consequently, we consider those n1 , . . . , nm ∈ N for defining T1 ,


T2 , . . . , Tm respectively, and we obtain

m −1  
∑ Θm− p ( A · B) ◦ Tp−1 ◦ . . . ◦ T1 ( x0 ) − Tp ◦ . . . ◦ T1 ( x0 ) +
p =1

( A · B) ◦ Tm−1 ◦ . . . ◦ T1 ( x0 ) − Tm ◦ . . . ◦ T1 ( x0 ) ≤
m −1   
 
∑ Θm− p ρM A Pn p (U p−1 ( x0 )) − U p−1 ( x0 ) + ρM A Pnm (Um−1 ( x0 )) − Um−1 ( x0 ) ≤ ε/2.
p =1

Now apply Lemma 1, in order to get x̃ − Tm ◦ . . . ◦ T1 ( x0 ) < ε.

4.3. Numerical Experiments


This subsection is devoted to providing some examples and their numerical results
to illustrate the theorems of the above sections. We will consider J = [0, 1] and the
classical Faber-Schauder system in C ( J ) where the nodes are the naturally ordered dyadic

127
Mathematics 2022, 10, 4179

numbers (see Table 1 in [18] and [28,29] for details). In following examples, we will denote
x ∗ = Tm ◦ . . . ◦ T1 ( x0 ) with m = 4 and n1 = · · · = nm = l with l = 9 or l = 33.

Example 1. Consider the nonlinear differential equation with a nonlocal initial condition
⎧  

⎪ d x (t)

⎪ = ae− x(t) , t ∈ J,

⎨ dt f (t, x (t))
  (12)



⎪ 3

⎩ x (0) = b sup | x (t)| + ,
t∈ J 4

b
where 0 < a < 1/ log(2) and f (t, x ) = .
1 + ae−b t
Let us define the mappings g : J × R → R and μ : C ( J ) → R by

g(t, x ) = ae− x , t ∈ J, x ∈ R

and  
μ(u) = b sup |u(t)| + 3/4 , u ∈ C ( J ).
t∈ J

Let R be small enough such that a(log(2) + R) < 1. Let x, y ∈ [− R, R], by an elementary
calculus we can show that the functions f and g satisfy the condition (ii), with α(t) = ϕ(t) = 0,
γ(t) = ae R (1 − e−t ), and ψ(t) = t.
On the other hand, we have that μ is Lipschizian with a Lipschiz constant Lμ = b, and

1
sup [ f (0, x )]−1 ≤ δ = .
x,| x |≤ R b

Applying Theorem 3, we obtain that (12) has a unique solution in BR = { x ∈ C ( J ); x ≤ R} with


R = 3/4, when a is small enough. In fact thesolution is x̃ (t) = b. We apply the numerical method

for a = 0.1, b = 14 and the initial x0 (t) = 14 bt + 1 . Table 1 collects the obtained results.
√ 
Table 1. Numerical results for (12) with initial x0 (t) = 1
4 bt + 1 .

t x̃(t ) x∗ (t ) with l = 9 x∗ (t ) with l = 33


0.1 0.25 0.2526360625738145 0.2506238401703868
0.2 0.25 0.2512245431325148 0.2506151528771704
0.3 0.25 0.2510208953229317 0.2506066551064274
0.4 0.25 0.2510087458298449 0.2505983412941664
0.5 0.25 0.2509968386936278 0.2505902060799007
0.6 0.25 0.2509851672563384 0.2505822442972077
0.7 0.25 0.2509737250885047 0.2505744509661791
0.8 0.25 0.2509625059364119 0.2505668212861210
0.9 0.25 0.2509515037642987 0.2505593506272617
1 0.25 0.2509407127451644 0.2505520345235613
x ∗ − x̃ ∞ 2.86369 × 10−3 1.0862 × 10−3

128
Mathematics 2022, 10, 4179

Example 2. Consider the nonlinear differential equation with a nonlocal initial condition
⎧  
⎪ d x (t)

⎪ = a( x (t))2 , t ∈ J,
⎨ dt f (t, x (t))
(13)



⎩ x (0) = 1/(4b) sup | x (t)|2 ,
t∈ J

b ( t + 1)
where a, b are positive constants such that ab2 < 3 and f (t, x ) = .
ab2
1+ 3 ( x /b
3 3 − 1)
Let us define the mappings g : J × R → R and μ : C ( J ) → R by

g(t, x ) = ax2 , t ∈ J, x ∈ R and μ(u) = 1/(4b) sup |u(t)|2 , u ∈ C ( J ).


t∈ J

Let R > 0 such that 2b ≤ R and 3b ( b


+ R3 ) < 1. Let x, y ∈ [− R, R]. By an elementary calculus
a 3

a ( t + 1) R2
we can show that f and g satisfy the condition (ii) with α(t) =  2 , γ(t) = 2aR,
1 − 3b
a
( R3 + b3 )
and ϕ(t) = ψ(t) = t.
On the other hand, we have that
R
|μ(u) − μ(v)| ≤ u−v .
2b

Consequently, μ is Lipschizian with a Lipschiz constant Lμ = R


2b . It is easy to prove that

sup [ f (0, x )]−1 ≤ δ = aR3 /(3b2 ) + 1/b.


x ∈R,| x |≤ R

Now, applying Theorem 3, in order to obtain that (13), with a is small enough, has a unique solution
in BR with R = 1/2. We can check that the solution is x̃ (t) = b(t + 1). Table 2 shows the numerical
results of the proposed method for a = 0.05, b = 1/4 and x0 (t) = 12 t.

Table 2. Numerical results for (13) with initial x0 (t) = 12 t.

t x̃(t ) x∗ (t ) with l = 9 x∗ (t ) with l = 33


0.1 0.275 0.2715154513364088 0.2714532970472882
0.2 0.3 0.2961167353030552 0.2961332465465061
0.3 0.325 0.3207837845940706 0.3208140511167786
0.4 0.35 0.3454635279153586 0.3454958547548318
0.5 0.375 0.3701445199310059 0.3701788114857308
0.6 0.40 0.3948268789541488 0.3948630864085328
0.7 0.425 0.4195107187398104 0.4195488540144761
0.8 0.45 0.4441962543294659 0.4442362958308083
0.9 0.475 0.4688837174935067 0.4689256009587782
1 0.5 0.4935733558651244 0.4936169655580174
x ∗ − x̃ ∞ 6.42664 × 10−3 6.38303 × 10−3

5. Nonlinear Integral Equations


This section deals with the nonlinear integral Equation (2). More precisely, we prove
the existence and the uniqueness of a solution to Equation (2) under the hypothesis that the
mappings f : J × R → R and K : J × J × R → R are such that:
(i) The partial mappings t → f (t, x ) and (t, s) → K (t, s, x ) are continuous.

129
Mathematics 2022, 10, 4179

(ii) There exist r > 0, γ : J × J → R, α : J → R two continuous functions and


ϕ, ψ : R+ −→ R+ two nondecreasing continuous functions such that

| f (t, x ) − f (t, y)| ≤ α(t) ϕ(| x − y|), t ∈ J, and x, y ∈ R with | x |, |y| ≤ r,

and

|K (t, s, x ) − K (t, s, y)| ≤ γ(t, s)ψ(| x − y|), t, s ∈ J and x, y ∈ R with | x |, |y| ≤ r.

Throughout this section, Ω will denote the closed ball Br of C ( J ), where r is defined in
the above assumption (ii ).

5.1. Existence and Uniqueness of Solutions


To allow the abstract formulation of Equation (2), we define the following operators
on C ( J ) by

( Ax )(t) = f (t, x (σ(t))),


/  η (t)
0 (14)
( Bx )(t) = q(t) + K (t, s, x (τ (s)))ds , t ∈ J.
0

First, we will establish the following result which shows the existence and uniqueness
of a solution.

Theorem 5. Assume that the assumptions (i ) and (ii ) hold. If

M A MB ≤ r and M A ρ γ ∞ ψ(t) + MB α ∞ ϕ(t) < t, ∀t > 0,

where

MA = α ∞ ϕ (r ) + f (·, θ ) ∞ and MB = q(·) ∞ + ρ( K (·, ·, 0) ∞ + γ ∞ ψ (r )),

then the nonlinear integral Equation (2) has a unique solution in Ω.

Proof. By using similar arguments to those in the above section, we can show that A
and B define D -lipschitzian mappings from Ω into C ( J ), with D -functions α ∞ ϕ and
ρ γ ∞ ψ, respectively. Also it is easy to see that A(Ω) and B(Ω) are bounded with bounds,
respectively, M A and MB . Taking into account our assumptions, we deduce that A · B maps
Ω into Ω.
Notice that A · B defines a nonlinear contraction with D -function

Θ(t) := ρ γ ∞ M A ψ(t) + α ∞ M B ϕ ( t ), t ≥ 0, i.e.,

( A · B)( x ) − ( A · B)(y) ≤ Θ( x − y ), x, y ∈ Ω. (15)

Now, an application of Theorem 2 yields that (2) has one and only one solution x̃ in Ω, and
for each x0 ∈ Ω we have

lim ( A · B)n ( x0 ) = x̃. (16)


n→∞

130
Mathematics 2022, 10, 4179

5.2. A Numerical Method to Approximate the Solution


Now we consider a Schauder basis {en }n≥1 in C ( J × J ) and the sequence of associated
projections { Pn }n≥1 . Let p ∈ N, n p ∈ N and consider


⎪ S p : C( J ) −→ C ( J )

 η (t)


⎩ x −→ S p ( x )(t) = q(t) + Pn p (U0 ( x ))(t, s)ds,
0

where U0 : C ( J ) −→ C ( J × J ) is defined as U0 ( x )(t, s) = K (t, s, x (τ (s))). Also, we consider


the operator Tp : C ( J ) −→ C ( J ), which assigns for all x ∈ C ( J ) the valued Tp ( x ) ∈ C ( J )
such that
Tp ( x )(t) = A( x )(t)Sn p ( x )(t), t ∈ J,
where A : C ( J ) −→ C ( J ) is defined as A( x )(t) = f (t, x (σ(t))).

Remark 3. Since for p ≥ 1,


   η (t) 
   
 Tp ( x )(t) =  A( x )(t) q(t) + P ( U ( x ))( t, s ) ds ≤
 0
n p 0 
  η (t)   
 
| f (t, x (σ(t)))| |q(t)| + Pn p (U0 ( x ))(t, s)ds ,
0

proceeding essentially as in the above section and using the fact that Pn p is a bounded linear operator
on C ( J × J ), we get

    
 Tp ( x )(t) ≤ M A |q(t)| + ρ 
Pn p (U0 ( x )) ≤
 
MA q ∞ + ρ sup | K ( t, s, x ( τ ( s )))| ≤ M A MB .
t,s∈ J

Accordingly, under the hypotheses of the Theorem 5, the mapping Tp maps Ω into Ω. In particular,
for m ≥ 1, the operator Tm ◦ . . . ◦ T1 maps Ω into Ω.

Analogously as we did in the previous section, the following result allow us to justify
it is possible to choose n1 , n2 , . . . in order that T1 , T2 , . . . can be used to approximate the
unique solution to Equation (2).

Theorem 6. Let x̃ be the unique solution to the nonlinear Equation (2). Let x0 ∈ Ω and ε > 0,
then there exists m ∈ N and ni ∈ N to construct Ti for i = 1, . . . , m, such that

x̃ − Tm ◦ . . . ◦ T1 ( x0 ) ≤ ε.

Proof. Let ε > 0, by using (16), there is m ∈ N such that

( A · B)m ( x0 ) − x̃ ≤ ε/2.

For that m, and for p ∈ {1, . . . , m}, we define U p : C ( J ) → C ( J × J ) by

U p ( x )(t, s) := K (t, s, Tp ◦ . . . ◦ T1 ( x )(s)), t, s ∈ J, x ∈ C ( J )

and A p : C ( J ) → C ( J ) by
 
A p ( x )(s) := f s, Tp ◦ . . . ◦ T1 ( x )(s) , s ∈ J, x ∈ C ( J ).

131
Mathematics 2022, 10, 4179

Proceeding essentially, as in the Theorem 4, and taking into account (15) together with
Remark 3 the desired thesis can be proved.

5.3. Numerical Experiments


This section is devoted to give some numerical examples to illustrate the previous
results using the usual Schauder basis in C ([0, 1]2 ) with the well know square ordering
(see Table 1 in [18] and [28,29]). In each example, we will denote x ∗ = Tm ◦ . . . ◦ T1 ( x0 ) for
m = 4 and n1 = · · · = nm = l 2 with l = 9 or l = 33.

Example 3. Consider the nonlinear integral equation


/  t 0
b b2  
x ( t ) = a ( t + 1) − ( t + 1)3 − 1 + ( x (s))2 ds , t ∈ J. (17)
a 3 0

Now we consider the mappings q : J → J, f : J × R → R and K : J × J × R → R such


2 
that q(t) = b/a − b3 (t + 1)3 − 1 , f (t, x ) = a(t + 1) and K (t, s, x ) = x2 . Let R > 0 and let
x, y ∈ [− R, R]. We have that

|K (t, s, x ) − K (t, s, y)| ≤ γ(t, s)ψ(| x − y|),

where γ(t, s) = 2R, and ψ(t) = t. An application of Theorem 5, yields that (17) has a unique
solution in BR , with R = 3. In fact the solution is x̃ (t) = b(t + 1).
Using the proposed method with a = 0.1, b = 0.1 and x0 (t) = t2 , we obtain Table 3.

Table 3. Numerical results for the (17).

t x̃(t ) x∗ (t ) with l = 9 x∗ (t ) with l = 33


0.1 0.11 0.1099446333333333 0.1099595576568532
0.2 0.12 0.1198179180577049 0.1199472782251611
0.3 0.13 0.1297511699020331 0.1299327014013851
0.4 0.14 0.1396866403161547 0.1399156114644378
0.5 0.15 0.1496116012197044 0.1498957849652041
0.6 0.16 0.1595251486759711 0.1598729913214837
0.7 0.17 0.1694262809122463 0.1698469898893412
0.8 0.18 0.1793140741901599 0.1798175262525480
0.9 0.19 0.1891875688779072 0.1897843325246908
1 0.2 0.1990457618518603 0.1997471266515799
x ∗ − x̃ ∞ 9.544238 × 10−4 2.52873 × 10−4

Example 4. Consider the nonlinear differential equation


 / t 1
 t 0
x (t) = ae− x(t) + b −
+ log(cos(1 − c)t) + tan((1 − c) x (s))ds . (18)
ae + b 1 − c
t
0

Similarly to that above, (18) can be written as a fixed point problem with the same notations in (14).
Let R > 0 and let x, y ∈ [− R, R]. By an elementary calculus we can show that the functions f and
g satisfy the condition (ii), with α(t) = ae R , γ(t) = (1 + tan2 (1 − c) R), and ϕ(t) = (1 − e−t )
and ψ(t) = tan(1 − c)t.
Apply Theorem 5, (18), with a small enough and c = 1 − a, has a unique solution in
BR with R = 3, in fact the solution is x̃ (t) = t. We obtain the results given in Table 4 for
a = 0.01, b = 1, R = 3, and x0 (t) = sin(t).

132
Mathematics 2022, 10, 4179

Table 4. Numerical results for (18) with initial x0 (t) = sin(t).

t x̃(t ) x∗ (t ) with l = 9 x∗ (t ) with l = 33


0.1 0.1 0.0999495927525812 0.0999734131829520
0.2 0.2 0.1998269806205324 0.1999419676240642
0.3 0.3 0.2997014781005956 0.2999105694862292
0.4 0.4 0.3995761128223367 0.3998792008487213
0.5 0.5 0.4994508163308592 0.4998478468962116
0.6 0.6 0.5993255387084228 0.5998164954408373
0.7 0.7 0.6992002390137386 0.6997851365136741
0.8 0.8 0.7990748839377436 0.7997537620153589
0.9 0.9 0.8989494465775325 0.8997223654190059
1 1 0.9988239054111422 0.9996909415162489
x ∗ − x̃ ∞ 1.17609 × 10−3 3.09058 × 10−4

Example 5. Consider the problem (2) with


/  t  0 −1
t
f (t, x ) = at (b + t)2 + 1 − e−(t+1)( as+1) ds ,
( t + 1) 0

 x +1
(19)
K (t, s, x ) = e−(t+1)u du,
0

q(t) = ( b + t )2 .

Let 0 < R < 1 and let x, y ∈ [− R, R]. By an elementary calculus, we can show that f and g
 2t
satisfy the condition (ii ), with α(t) = ϕ(t) = 0, ψ(t) = 0 e−s ds, and γ(t, s) = t+1 1 e(t+1)( R−1) .
Taking a = 0.1, b = 1, and applying Theorem 5, the problem has a unique solution in
BR = { x ∈ C ([0, 1]); x ≤ R}, in fact the solution is x̃ (t) = at. We obtain the results given in
Table 5.

Table 5. Numerical results for (19) with initial x0 (t) = 1/2cos(10πt).

t x̃(t ) x∗ (t ) with l = 9 x∗ (t ) with l = 33


0.1 0.01 0.0098078897681979 0.0098501736202539
0.2 0.02 0.0191334693414161 0.0197640067592651
0.3 0.03 0.0288588703908235 0.0297138485291223
0.4 0.04 0.0387456185368957 0.0396854768250511
0.5 0.05 0.0486866179763731 0.0496708731179798
0.6 0.06 0.0586657967463166 0.0596654694199951
0.7 0.06 0.0686685394448633 0.0696660302996126
0.8 0.08 0.0786865051341015 0.0796705375310556
0.9 0.09 0.0887140587924687 0.0896776281114000
1 0.09 0.0987473453913395 0.0996863636633998
x ∗ − x̃ ∞ 1.33705 × 10−3 3.34982 × 10−4

6. Conclusions
In this paper we have presented a numerical method, based on the use of Schauder’s
bases, to solve hybrid nonlinear equations in Banach algebras. To do this, we have used

133
Mathematics 2022, 10, 4179

Boyd-Wong’s theorem to establish the existence and uniqueness of a fixed point for the
product of two nonlinear operators in Banach algebra (Theorem 2). The method is applied
to a wide class of nonlinear hybrid equations such as the ones we have illustrated by means
of several numerical examples.
The possibility of applying this process or a similar idea to other types of hybrid
equations or systems of such equations is open and we hope to discuss this in the near future.

Author Contributions: Conceptualization, K.B.A. and M.I.B.; methodology, K.B.A., M.I.B. and A.J.;
software, K.B.A. and M.I.B.; validation, K.B.A. and M.I.B.; formal analysis, K.B.A., M.I.B. and A.J.;
investigation, K.B.A. and M.I.B.; writing—original draft preparation, K.B.A. and M.I.B.; writing—review
and editing, K.B.A. and M.I.B.; supervision, K.B.A., M.I.B. and A.J. All authors have read and agreed to
the published version of the manuscript.
Funding: The research of Aref Jeribi and Khaled Ben Amara has been partially supported by the
University of Sfax (Tunisia). The research of María Isabel Berenguer has been partially supported by
Junta de Andalucía (Spain), Project Convex and numerical analysis, reference FQM359, and by the María
de Maeztu Excellence Unit IMAG, reference CEX2020-001105-M, funded by MCIN/AEI/10.13039/
501100011033/.
Data Availability Statement: Not applicable.
Acknowledgments: This work was partially carried out during the first author’s visit to the
Department of Applied Mathematics, University of Granada. The authors wish to thank the
anonymous referees for their useful comments. They also acknowledge the financial support of the
University of Sfax (Tunisia), the Consejería de Conocimiento, Investigación y Universidad, Junta de
Andalucía (Spain) and the María de Maeztu Excellence Unit IMAG (Spain).
Conflicts of Interest: The authors declare no conflict of interest.

References
1. Deimling, K. Nonlinear Functional Analysis; Springer: Berlin/Heidelberg, Germany, 1985; ISBN 3-540-13928-1.
2. Djebali, S.; Sahnoun, Z. Nonlinear alternatives of Schauder and Krasnosel’skii types with applications to Hammerstein integral
equations in L1 -spaces. J. Differ. Equ. 2010, 249, 2061–2075. [CrossRef]
3. Byszewski, L. Theorems about the existence and uniqueness of solutions of a semilinear evolution nonlocal Cauchy problem. J.
Math. Anal. Appl. 1991, 162, 494–505. [CrossRef]
4. Byszewski, L. Existence and Uniqueness of Mild and Classical Solutions of Semilinear Functional-Differential Evolution Nonlocal Cauchy
Problem; Selected Problems of Mathematics; Cracow University of Technology: Krakow, Poland, 1995
5. Deng, K. Exponential decay of solutions of semilinear parabolic equations with nonlocal initial conditions. J. Math. Anal. Appl.
1993, 179, 630–637. [CrossRef]
6. Dhage, B.C. Multi-valued mappings and fixed points I. Nonlinear Funct. Anal. Appl. 2005, 10, 359–378. [CrossRef]
7. O’Regan, D. New fixed point results for 1-set contractive set-valued maps. Comput. Math. Appl. 1998, 35, 27–34. [CrossRef]
8. Ben Amar, A.; Chouayekh, S.; Jeribi, A. Fixed point theory in a new class of Banach algebras and application. Afr. Mat. 2013, 24,
705–724. [CrossRef]
9. Dhage, B.C. On some nonlinear alternatives of Leray-Schauder type and functional integral equations. Arch. Math. (Brno) 2006,
42, 11–23.
10. Jeribi, A.; Kaddachi, N.; Krichen, B. Existence results for a system of nonlinear integral equations in Banach algebras under weak
topology. Fixed Point Theory 2017, 18, 247–267. [CrossRef]
11. Jeribi, A.; Krichen, B. Nonlinear functional analysis in Banach spaces and Banach algebras: Fixed point theory under weak
topology for nonlinear operators and block operator matrices with applications. In Monographs and Research Notes in Mathematics;
CRC Press/Taylor and Francis: Boca Raton, FL, USA, 2015.
12. Dhage, B.C. On some variants of Schauder’s fixed point principle and applications to nonlinear integral equations. J. Math. Phy.
Sci. 1988, 25, 603–611.
13. Cichon, M.; Metwali, M.M.A. On a fixed point theorem for the product of operators. J. Fixed Point Theory Appl. 2016, 18, 753–770.
[CrossRef]
14. Dhage, B.C. On a fixed point theorem in Banach algebras with aplications. Appl. Math. Lett. 2005, 18, 273–280. [CrossRef]
15. Dhage, B.C. A hybrid fixed point theorem in Banach algebras with applications. Commun. Appl. Nonlinear Anal. 2006, 13, 71–84.
16. Akyüz-Daşcioǧlu, A.; Sezer, M. Chebyshev polynomial solutions of systems of higher-order linear Fredholm-Volterra integro-
differential equations. J. Frankl. Inst. 2005, 342, 688–701. [CrossRef]
17. Argyros, I.K.; Ezquerro, J.A.; Hernández, M.A.; Hilout, S.; Romero, N.; Velasco, A.I. Expanding the applicability of secant like
methods for solving nonlinear equations. Carphatian J. Math. 2015, 31, 11–30. [CrossRef]

134
Mathematics 2022, 10, 4179

18. Berenguer, M.I.; Gámez, D.; López Linares, A.J. Fixed point techniques and Schauder bases to approximate the solution of the
first order nonlinear mixed Fredholm-Volterra integro-differential equation. J. Comput. Appl. Math. 2013, 252, 52–61. [CrossRef]
19. Berenguer, M.I.; Gámez, D. A computational method for solving a class of two dimensional Volterra integral equations. J. Comput.
Appl. Math. 2017, 318, 403–410. [CrossRef]
20. Dzhumabaev, D.S. On one approach to solve the linear boundary value problems for Fredholm integro differential equations.
J. Comput. Appl. Math. 2016, 294, 342–357. [CrossRef]
21. Heydari, M.H.; Hooshmandasl, M.R.; Mohammadi, F.; Cattani, C. Wavelets method for solving systems of nonlinear singular
fractional Volterra integro-differential equations. Commun. Nonlinear Sci. 2014, 19, 37–48. [CrossRef]
22. Maleknejad, K.; Basirat, B.; Hashemizadeh, E. A Berstein operational matrix approach for solving a system of high order linear
Volterra-Fredholm integro-differential equations. Math. Comput. Model. 2012, 55, 1363–1372. [CrossRef]
23. Micula, S. On some iterative numerical methods for a Volterra functional integral equation of the second kind. J. Fixed Point
Theory Appl. 2017, 19, 1815–1824. [CrossRef]
24. Saberi-Nadjafi, J.; Tamamgar, M. The variational iteration method: A highly promising method for solving the system of
integro-differential equations. Comput. Math. Appl. 2008, 56, 346–351. [CrossRef]
25. Boyd, D.W.; Wong, J.S.W. On nonlinear contractions. Proc. Am. Math. Soc. 1969, 20, 458–464. [CrossRef]
26. Pata, V. A fixed point theorem in metric spaces. J. Fixed Point Theory Appl. 2011, 10, 299–305. [CrossRef]
27. Brezis, H. Functional Analysis, Sobolev Spaces and Partial Differential Equations; Universitext; Springer: New York, NY, USA, 2011;
ISBN 978-0-387-70913-0.
28. Gelbaum, B.R.; Gil de Lamadrid, J. Bases of tensor products of Banach spaces. Pac. J. Math. 1961, 11, 1281–1286. [CrossRef]
29. Semadeni, Z. Product Schauder bases and approximation with nodes in spaces of continuous functions. Bull. Acad. Polon. Sci.
1963, 11, 387–391.

135
mathematics

Article
Finding an Efficient Computational Solution for the Bates
Partial Integro-Differential Equation Utilizing the
RBF-FD Scheme
Gholamreza Farahmand 1 , Taher Lotfi 1, *, Malik Zaka Ullah 2 and Stanford Shateyi 3, *

1 Department of Mathematics, Hamedan Branch, Islamic Azad University, Hamedan, Iran;


[email protected]
2 Mathematical Modeling and Applied Computation (MMAC) Research Group, Department of Mathematics,
King Abdulaziz University, Jeddah 21589, Saudi Arabia; [email protected]
3 Department of Mathematics and Applied Mathematics, School of Mathematical and Natural Sciences,
University of Venda, P. Bag X5050, Thohoyandou 0950, South Africa
* Correspondence: lotfi@iauh.ac.ir (T.L.); [email protected] (S.S.)

Abstract: This paper proposes a computational solver via the localized radial basis function finite
difference (RBF-FD) scheme and the use of graded meshes for solving the time-dependent Bates
partial integro-differential equation (PIDE) arising in computational finance. In order to avoid facing
a large system of discretization systems, we employ graded meshes along both of the spatial variables,
which results in constructing a set of ordinary differential equations (ODEs) of lower sizes. Moreover,
an explicit time integrator is used because it can bypass the need to solve the large discretized linear
systems in each time level. The stability of the numerical method is discussed in detail based on the
eigenvalues of the system matrix. Finally, numerical tests revealed the accuracy and reliability of the
presented solver.

Keywords: PIDE; stochastic volatility; semi-discretiztion; RBF-FD; Bates model

MSC: 65M22; 91G60; 91B25


Citation: Farahmand, G.; Lotfi, T.;
Ullah, M.Z.; Shateyi, S. Finding an
Efficient Computational Solution for
the Bates Partial Integro-Differential
Equation Utilizing the RBF-FD
1. Introductory Notes
Scheme. Mathematics 2023, 11, 1123. The Bates model for option pricing considers that the underlying asset St , the volatility
https://doi.org/10.3390/ Vt , the riskless constant r and Nt as the Poisson process satisfy the following system of
math11051123 stochastic differential equations (SDEs) [1]:
Academic Editors: Maria Isabel √
dSt = Vt St dWt1 + (−λξ − q + r )St dt + ( − 1)St dNt ,
Berenguer and Manuel Ruiz Galán
√ (1)
Received: 18 January 2023 dVt = σ Vt dWt2 + κ (−Vt + θ )dt,
Revised: 3 February 2023
Accepted: 21 February 2023 wherein Wt2 and Wt1 are standard Brownian motions having dWt1 dWt2 = ρdt. Here κ is the
Published: 23 February 2023 reversion’s rate of the variance Vt , λ is the Poisson process intensity, ξ is the mean jump, q
is the dividend,  is the jump size, while θ is the mean level and σ stands for the volatility
fixed value.
Financial derivatives such as European call or put options play pioneer roles in the risk
Copyright: © 2023 by the authors.
management of some portfolios and their pricing as efficiently as possible is of importance.
Licensee MDPI, Basel, Switzerland.
On the other hand for the financial derivative price, since analytical relations are available
This article is an open access article
only in limited settings, one is in need for the construction and the application of fast
distributed under the terms and
and stable numerical solvers. More concretely, starting from the initial time zero, we
conditions of the Creative Commons
Attribution (CC BY) license (https://
must numerically solve a second-order high-dimensional time-dependent partial integro-
creativecommons.org/licenses/by/
differential equation (PIDE) or a partial differential equation (PDE) and then compute the
4.0/). present value of the financial derivative [2–4].

Mathematics 2023, 11, 1123. https://doi.org/10.3390/math11051123 https://www.mdpi.com/journal/mathematics


137
Mathematics 2023, 11, 1123

The Heston model, which could be considered as a generalization of the Black–Scholes


model [5], can be extended further if one follows the consideration of Bates [1,6] by impos-
ing the jump component into the modeling. In fact, in the stochastic volatility jump (SVJ)
model, the price of an option is computed by solving a time-dependent 2D PIDE [7,8]. It is
requisite to recall some related models [9,10] discussing stochastic volatility for PDEs in
control theory and AI.
The Bates PIDE based on the price function u( x, y, τ ) for European options is expressed
by the following [11]:

∂u( x, y, τ ) 1 ∂2 u( x, y, τ ) 1 2 ∂2 u( x, y, τ )
= yx2 + σ y
∂τ 2 ∂x2 2 ∂y2
∂2 u( x, y, τ )
+ ρσyx
∂x∂y (2)
∂u( x, y, τ ) ∂u( x, y, τ )
+ (−λξ − q + r ) x + κ (θ − y)
∂x ∂y
 ∞
− (λ + r )u( x, y, τ ) + λ u( x, y, τ )b()d = Au( x, y, τ ),
0

wherein T is the time to maturity and τ = T − t is a time transformation to have forward in


time PIDE formulation, unlike the original Bates PIDE, which is backward in time. Besides,
both the differential and integral parts of the PIDE problem have been encapsulated in the
operator A, that is to say, we also can write

Au( x, y, τ ) = A D u( x, y, τ ) + λA I u( x, y, τ ), (3)

in which A D and A I stand for the differential and integral1 portions of2 the PIDE prob-
(ln()−γ)2
lem. The probability density function is b() = √ 1 exp − 2σ̂2 , where it reads
∞ 2π σ̂

0 b (  ) d = 1. Here, σ̂ and γ are the standard deviation



and the mean, respectively, which

are positive constants. Additionally, we have ξ = exp γ + 12 σ̂2 − 1.
The so-called payoff which is the the initial condition for the PIDE problem in call-type
option pricing can be expressed as [12]:

u( x, y, 0) = (0, x − K )+ , (4)

wherein K is the strike price. The payoff for a put option could be written similarly. The
point is that the initial condition is written only on x and does not rely on the second
independent variable of the PIDE, i.e., y.
The side conditions for x and y could be given as follows [12]:

u( x, y, τ )  0, x → 0, (5)
u( x, y, τ )  xmax exp (−qτ ) − K exp (−rτ ), x → xmax , (6)
∂u( x, y, τ )
 0, y → +ymax . (7)
∂y

Note that for the case when y = 0, the PIDE (2) is degenerate and no boundaries should
be incorporated while xmax and ymax are large constants. Similarly for the put option, the
boundary conditions are described by the following:

u( x, y, τ )  K exp (−rτ ) − xmax exp (−qτ ), x → 0, (8)


u( x, y, τ )  0, x → xmax , (9)
∂u( x, y, τ )
 0, y → ymax . (10)
∂y

138
Mathematics 2023, 11, 1123

The Bates PIDE (2) is given on ( x, y, τ ) ∈ [0, +∞) × [0, +∞) ×(0, T ]. To solve our high-
dimensional linear PDE, we must truncate the unbounded domain while quite delicately
ignoring the error caused by imposing the boundary conditions. This can be pursued
as follows:
Ω = [0, xmax ] × [0, ymax ], (11)
wherein xmax , ymax are fixed values. The values for xmax , and ymax should be considered
large enough to be able to neglect the effect of imposing artificial boundary conditions
or imposing the boundary conditions for truncated domains. Some choices are Ω =
[0, 4K ] × [0, 1] or Ω = [0, 3K ] × [0, 1].
Assume that { xi }im=1 is a mesh of nodes for x. The hyperbolic stretching of nodes [13]
can be expressed as follows (1 ≤ i ≤ m):

xi = c sinh( β i ) + K, (12)

wherein c > 0 stands for a fixed value that controls the density around x = K and m 3.
In implementations, one can employ c as in [14], i.e., c = K/5. This puts a focus around
the strike price, in which the initial condition of the PIDE has nonsmoothness. Moreover,
{ β i }im=0 stands for the uniform points given by the following:
 
K
β i = (i − 1)Δβ + sinh−1 − , 1 ≤ i ≤ m, (13)
c
    
wherein Δβ = (m − 1)−1 sinh−1 S−c K − sinh−1 −cK .
Also, if {y j }nj=1 is a partition for y, then this stretching strategy can be expressed by
the following:
y j = sinh(ς j )ν, 1 ≤ j ≤ n, (14)
wherein ν > 0 is a fixed value that controls the density around y = 0 and n 3. Basically,
we use ν = K/500 [14]. Additionally, the ς j are equidistant nodes provided by ς j =
 
−1 K
(Δς)( j − 1), and for any 1 ≤ j ≤ n we have Δς = n− 1
1 sinh ν .
Numerical solution methods generally utilize the discretization means to realize the
approximate calculation. When the computational domain/interval is partitioned more
finely, the calculated result is closer to the theoretical solution. Indeed, the time required for
the calculation increases. For high-dimensional PIDE problems with kink behavior at the
initial conditions, sometimes special solvers such as high-order sparse numerical methods
are necessary, see, e.g., [15]. Noting finite difference (FD) methods are discussed in [16,17].
In this paper, the main aim is to propose a novel computational method for resolving
(2) via the radial basis function generated finite difference (RBF-FD) methodology [18].
This is mostly because (2) is a (1+2)D problem with variable coefficients, in which there
is one cross derivatives. Hence, the computational solvers should be constructed for this
aim with much attention. In fact, the motivation of this work lies in the fact that literature
lacks the application of efficient RBF-FD methodology which result in fast and sparse
procedures for solving the Bates PIDE model. Hence, such an application and investigation
on the theoretical stability issues will help price option under stochastic volatility in equity
markets.
The RBF-FD formulations in this paper, see, e.g., [19], are written so they can be
applied to graded meshes in which there is a clear concentration on the hot zone. The
procedure taken here is to employ tensor grids and then time discretize the semi-discretized
constructed problem. We note that the present work is related to the pioneering works
in [20–22]. Meanwhile, these works motivate us to propose a new variant of the RBF-FD
scheme for the Bates PIDE problem that competes with these efficient works.
In this paper, after reviewing the well-resulted maps for generating graded meshes
along spatial variables with a clear focus around the hot area, the rest of this article is
unfolded as follows. The RBF-FD formulas associated with the GMQ RBF are given in

139
Mathematics 2023, 11, 1123

Section 2. Then, the semi-discretization of the two-dimensional PDE (2) is described in


Section 3. Then in Section 4, an explicitly quadratically convergent method is taken into
account. In fact, an explicit time integrator is used because it can avoid to the need to solve
the large discretized linear systems in each time level. It is shown that the proposed solver
is fast and conditionally stable. The numerical pieces of evidence are given in Section 5,
which overwhelmingly uphold the theoretical discussions of the paper. Concluding notes
are provided in Section 6.

2. RBF-FD: The Weights


Generally speaking, for computing the weights αi in the methodology of the RBF-FD,
one must consider L as a linear operator and at x = x p , for the node locations xi , the
following is written down [20]:
⎡ ⎤⎡ ⎤ ⎡ ⎤
Λ1 ( x 1 ) Λ1 ( x 2 ) ··· Λ1 ( x m ) α1 LΛ1 ( x )| x= x p
⎢ Λ2 ( x 1 ) Λ2 ( x 2 ) ··· Λ2 ( x m ) ⎥ ⎢ ⎥ ⎢ LΛ2 ( x )| x= x p ⎥
⎢ ⎥ ⎢ α2 ⎥ ⎢ ⎥
⎢ .. .. .. ⎥⎢ .. ⎥ = ⎢
⎢ .. ⎥,
⎥ (15)
⎣ . . . ⎦ ⎣ . ⎦ ⎣ . ⎦
Λm ( x1 ) Λm ( x2 ) ··· Λm ( x m ) αm LΛm ( x )| x= x p

where the underlined x shows a vector quantity in the dimension d and 1 ≤ k ≤ m for some
set of test functions Λ( x ). It is noted that the extension of RBF-FD methodology for solving
computational finance models was revived by the works of Soleymani and co-authors, see
for instance [23,24].
Now, we consider the famous generalized multiquadric RBF (GMQ RBF) as follows
([25] Chapter 4):

Λ(ri ) = ( p2 + ri2 )l , i = 1, 2, . . . , m, (16)

where l is a suitable parameter, the parameter of shape is p and ri = y − yi shows the


Euclidean distance.
It is now focused on computing the weights for the GMQ RBF (in the 1D case without
loss of generality). So, we consider a graded mesh including three points along the first
spatial variable. For finding the weights of the RBF-FD methodology, by taking into account
L as an operator, we could write down [26]:

ψ
L[Λ(y j )]  ∑ α i Λ ( y i ), j = 1, 2, . . . , ψ. (17)
i =1

This gives us ψ unknowns for ψ equations while the solutions will be αi . For computing
the 1st derivative, three graded nodes are considered (ψ = 3) as comes next: {yi − h, yi , yi +
wh}, w > 0, h > 0, and find (17) as follows:

g  ( y i )  α i −1 g ( y i −1 ) + α i g ( y i ) + α i +1 g ( y i +1 ). (18)

Noting that we assume that the function g is smooth sufficiently. In estimating the 1st
derivative of a function, the analytical weighting coefficients associated to this RBF can be
given as follows [22]:
 
ω p2 (9 − 6l ) − h2 (l − 1)(4(l − 5)ω − 10l + 29)
α i −1 = , (19)
3p2 h(2l − 3)(ω + 1)
 2 
(ω − 1) p (6l − 9) + 4h2 (l − 5)(l − 1)ω
αi = , (20)
3p2 h(2l − 3)ω
p2 (6l − 9) − h2 (l − 1)ω (2l (5ω − 2) − 29ω + 20)
α i +1 = . (21)
3p2 h(2l − 3)ω (ω + 1)

140
Mathematics 2023, 11, 1123

Similarly, in estimating the function’s second derivative, we can obtain

i +1
g (yi )  ∑ Θ j g ( y j ), (22)
j = i −1

along with the following weighting coefficients:


  
2 p2 (6l − 9) − h2 (l − 1) 4(l − 5)ω 2 + (34 − 8l )ω + 10l − 29
Θ i −1 = , (23)
3p2 h2 (2l − 3)(ω + 1)
 2  
2 p (9 − 6l ) + h2 (l − 1) 4(l − 5)ω 2 + (25 − 2l )ω + 4(l − 5)
Θi = , (24)
3p2 h2 (2l − 3)ω
 2 
2 p (6l − 9) − h2 (l − 1)(2l (ω (5ω − 4) + 2) + ω (34 − 29ω ) − 20)
Θ i +1 = . (25)
3p2 h2 (2l − 3)ω (ω + 1)

Also noting that the given RBF-FD formulations are valid for the interior nodes and
at boundary points, similar formulations must be constructed. We give the derivation for
the independent variable y and it would be similar for the other cases. The formulations
(19)–(21) and (23)–(24) are useful for the rows two to the row before the last one, while for
the 1st and the last rows of the derivative matrices (30) and (31), the weighting coefficients
could not be valid on boundaries and sided estimations should be incorporated. Hence, by
the work [21] on the stencil {y1 , y2 , y3 }, we have:
 
g  ( y1 ) = g [ y2 , y1 ] − g [ y3 , y2 ] + g [ y3 , y1 ] + O ( y2 − y1 )2 , (26)

and
 
g  ( y m ) = − g [ y m −1 , y m −2 ] + g [ y m −2 , y m ] + g [ y m , y m −1 ] + O ( y m −1 − y m )2 , (27)

wherein g[l, p] = ( g(l ) − g( p))/(l − p). In a similar manner, for the four nodes {{y1 , g(y1 )},
{y2 , g(y2 )}, {y3 , g(y3 )}, {y4 , g(y4 )}}, we can obtain

2(δy1,2 + δy1,3 + δy1,4 ) 2(δy3,1 + δy4,1 )


g (y1 ) = g ( y1 ) + g ( y2 )
δy1,2 δy1,3 δy1,4 δy1,2 δy2,3 δy2,4
  (28)
2(δy2,1 + δy4,1 ) 2(δy2,1 + δy3,1 )
+ g ( y3 ) + g ( y4 ) + O h2 ,
δy1,3 δy3,2 δy3,4 δy1,4 δy4,2 δy4,3

where δyl,q = yl − yq , h is the maximum space width for the considered stencil nodes.
Similarly, we have:

2(δym−3,m + δym−2,m + δym−1,m ) 2(δym−3,m + δym−2,m )


g (ym ) = g(ym ) + g ( y m −1 )
δym−3,m δym,m−2 δym,m−1 δym−3,m−1 δym−1,m−2 δym−1,m
2(δym−3,m + δym−1,m ) 2(δym−2,m + δym−1,m ) (29)
+ g ( y m −2 ) + g ( y m −3 )
δym−3,m−2 δym−2,m−1 δym−2,m δym−2,m−3 δym−1,m−3 δym,m−3
 
+ O h2 .

3. A New Solution Method


Let us use the well-known procedure of method of lines (MOL) for semi discretization
of the time-dependent problem [27,28] and convert the PIDE problem into a set of linear
ordinary differential equations (ODEs). Hence, the following derivative matrices for the

141
Mathematics 2023, 11, 1123

1st and 2nd derivatives of the function in approximating the PDE problem (2) via semi-
discretization are considered on non-uniform stencils given in Section 2 as comes next:


⎪ αi,j using (19) i − j = 1,

αi,j using (20) i − j = 0,
Mx = (30)
⎪ αi,j using (21)


j − i = 1,
0 otherwise,

and ⎧

⎪ Θi,j using (23) i − j = 1,

Θi,j using (24) i − j = 0,
Mxx = (31)
⎪ Θi,j using (25)
⎪ j − i = 1,

0 otherwise.
Consider the N × N unit matrix I = Ix ⊗ Iy , while N = m × n, Ix and Iy are unit
matrices of appropriate sizes. The MOL can be resulted in the following coefficient matrix
for the 1 + 2 dimensional PIDE:
1 1
B = Y X 2 ( Mxx ⊗ In ) + σ2 Y ( Im ⊗ Myy ) + ρσY X Mx,y
2 2 (32)
+ (−λξ − q + r )X ( Mx ⊗ In ) + κ (θ IN − Y )( Im ⊗ My ) − (−λ + r ) IN ,

where ⊗ stands for the Kronecker product. The square matrices Mx , My , Mz , Mxx , and
Myy , are constructed by the associated weights similarly. Additionally, the sparse diagonal
matrices Y and X are written as:

Y = Ix ⊗ diag(y1 , y2 , · · · , yn ), (33)

X = diag( x1 , x2 , · · · , xm ) ⊗ Iy . (34)
Here the weights corresponding the cross derivative term in the structure of the PIDE (2)
can be obtained by employing the Kronecker product as follows:

Mx,y = Mx ⊗ My . (35)

Now it is possible to find the following system of ODEs for pricing (2):

u (τ ) = Bu(τ ). (36)

Now, note that we can use the work of [22,29] to discretize the integral part as follows. By a
linear interpolation for u( x, y, τ ) among the adaptive numerical grid nodes, the nonlocal
integral given in (2) can be solved using
 ∞
A I (u) = u( x, y, τ )b()d. (37)
0

Employing z = x, one can transform (37) into the integral below:
 ∞  z  1 
A I (u) = u(z, y, τ )b dz. (38)
0 x x

Using a linear interpolation for (38), we can find the following:

m −1
Ai ( u )  ∑ Qi,l , (39)
l =1

142
Mathematics 2023, 11, 1123

for every node xi , i = 2, . . . , m − 1, wherein


 x     
l +1 x l +1 − z z − xl z 1
Qi,l = u( xl , y, τ ) + u( xl +1 , y, τ ) b dz, (40)
xl Δxl Δxl xi xi

wherein Δxl = xl +1 − xl is the graded step size. Hence, we have

⎛ ⎛ ⎛   ⎞
  − ln xxli + γ + σ̂2
1 ⎝ σ̂2 ⎝ ⎝ ⎠
Qi,l = exp γ + −erf √
2Δxl 2 2σ̂
⎛   ⎞⎞ ⎛ ⎛  ⎞ (41)
x
− ln lx+i 1 + γ + σ̂2 γ − ln xxl
+erf⎝ √ ⎠⎠ xi (u( xl , y, τ ) − u( xl +1 , y, τ )) + ⎝erf⎝ √ ⎠
i

2σ̂ 2σ̂

⎛   ⎞⎞ ⎞⎞
x l +1
γ − ln xi
−erf⎝ √ ⎠⎠( xl +1 u( xl , y, τ ) − xl u( xl +1 , y, τ ))⎠⎠,
2σ̂

wherein erf(·) stands for the Gaussian distribution.


So, (36) turns into
u (τ ) = B̄u(τ ), (42)
where B̄ is the system matrix after imposing the integral part. Finally, after considering the
boundary conditions, a set of ODEs can be attained as follows:

u (τ ) = F (τ, u(τ )) = B̄u(τ ) + b, (43)

wherein b consists of the boundary conditions.

4. The Time-Stepping Solver


Time stepping schemes must be used to solve (43). Although very recently some
optimal time stepping solvers have been proposed in literature [30–32] for solving system
of ODEs, here we focus on a basic but efficient one. Now it is considered that uι as an
approximation to u(τι ), then we could derive our final (explicit) time-integrator method.
Select k + 1 uniform temporal nodes and 0 ≤ ι ≤ k, τι+1 = τι + ζ, ζ = Tk > 0 with u0 = (4),
then the second-order RK solver (RK2) also known as the mid-point explicit method is
given by [33] (p. 95):
uι+1 = uι + ψ2 + O(ζ 3 ), (44)
where  
1 1
ψ2 = ζ F τι + ζ, uι + ψ1 , ψ1 = ζ F (τι , uι ). (45)
2 2
The approach (44) is useful because its explicit procedure helps programming in lower
computational time than many of its competitors from the RK methods. This is a moti-
vation of choosing (44) and not other higher order members of the RK family since their
computational cost per time level increases. Anyhow, the investigation for finding the best
time-stepping solver from the RK family of an optimal order for our specific PIDE problem
remains an open question which could be focused on forthcoming works. Now the most
important thing is to investigate that under what conditions this stability can be kept.

Theorem 1. Let us assume that (43) satisfies the Lipschitz condition, then we have a conditional
time-stable iteration process using (44) for solving (43).

143
Mathematics 2023, 11, 1123

Proof. Considering the time-stepping method (44) on the set of ODEs (43) gives:
 
(ζ B̄)0 (ζ B̄)1 (ζ B̄)2
u ι +1 = + + uι . (46)
1 1 2

The explicit method (46) is clearly time-stable if the matrix eigenvalues of


 
(ζ B̄)2
I + ζ B̄ + (47)
2

have modulus less than or equal to one. Viewing (46) as an iterative map, it would be clear
(ζ B̄ )2
that the eigenvalues of this matrix are 1 + ζ B̄i + 2i , where B̄i are the eigenvalues of
matrix B̄. Thus, for i = 1, 2, . . . , m, the A-stability is simplified to
 
 2
1 + ζ B̄i + (ζ B̄i )  ≤ 1. (48)
 2 

Therefore, our proposed method is time-stable if the time step size ζ reads as (48). The
stability function in (48) shows a conditional stable behavior for (44). Using (48) along with
ζ > 0 we have the following:

2
0<ζ≤ , (49)
Re(λmax ( B̄))

where Re(·) is the real part and λmax (·) is the largest eigenvalue (in the absolute value
sense). Note that we also obtain

− ξ i ≤ Im( B̄i ) ≤ ξ i , (50)

while   1/2
1/2
Re( B̄i )(ζRe( B̄i ) + 2) Re( B̄i )(ζRe( B̄i ) + 2)
ξi = 2 − − . (51)
ζ3 ζ

These inequalities on the real and the imaginary parts of the eigenvalues will determine
the conditional time stability bounds of the proposed solver when pricing (2). This ends
the proof.

To discuss about the advantages of the proposed approach, we briefly express that our
solver has now been expressed all in matrix notations as in (43) which is a system of linear
ODEs. When it couples by the ODE solver (44) with the stability condition (50), it solves (2)
and the stability relied only on the largest eigenvalue of the system matrix.

5. Numerical Aspects
The goal here is to resolve (2) for at-the-money options, i.e., the value of u at x0 = K
and also y0 = 0.04 and K = 100$. The comparing methods are given below:
• The 2nd-order FD scheme with equidistant stencils for space and the explicit 1st order
Euler’s scheme denoted by FDM,
• The method of scalable algebraic multigrid discussed in [34] and shown by AFFT.
• The scheme recently proposed by Soleymani et al. in [21] based on efficient non-
uniform procedure denoted by SM.
• The presented solver in Sections 2–4 shown via RBF-FD-PM in this section.
Noting that all the programs have been written carefully under similar conditions in
Mathematica 13 [35,36]. Here, the whole CPU time (for constructing the meshes, the
derivative matrices, the set of ODEs and employing the time-stepping method) is in second.

144
Mathematics 2023, 11, 1123

We use more number of nodes along x than y, since its computational domain is larger. The
criterion given below is used for computing the errors
 
ε i,j,ι = uapprox ( xi , y j , τι ) − uref ( x, y, τ ), (52)

where uapprox and uref are the approximate and exact solutions. uref is selected from the
already well-known literature [14,34].
It is remarked that one efficient way to compute the shape parameter is to calculate it
adaptively via the number of discretization points, the numerical domain as well as the
structure of the PIDE problem. Hence, here we use (1 ≤ i ≤ m − 1):

p = 4 max{Δxi }, (53)

where Δxi are the increments along the variable mesh. We can write and use (53) similarly
for the other variable. Throughout the tables of this paper, a E-b stands for the scientific
notation a × 10−b .

Example 1 ([14]). Let us investigate the computational results for the call option of (2) using the
following settings:

ρ = −0.9, r = 0.025, λ = 0, σ = 0.3, κ = 1.5, θ = 0.04, q = 0, T = 1. (54)

The reference price, which is obtained by the FFT approach [14], is 8.894869 at the
point ( x0 , y0 ) = (100, 0.04). The numerical truncated domain is Ω = [0, 3K ] × [0, 1] and
ψ = 1.5. Economically speaking, the values for the variance (for domain truncating) that
are larger than one are not significant. The results in this case are provided in Table 1, which
shows the superiority of the proposed solver RBF-FD-PM.

Example 2 ([34]). Let us investigate the computational results of a European put option for (2)
using the following settings:

γ = −0.5, σ̂ = 0.4, ρ = −0.5, λ = 0.2, σ = 0.25,

r = 0.03, T = 0.5, κ = 2.0, θ = 0.04, q = 0. (55)

Table 1. Numerical results for Example 1.

Solver m n N k+1 u ε Time


FDM
20 20 400 401 8.700 1.94E-1 0.87
40 25 1000 2001 8.597 2.97E-1 2.33
40 40 1600 2001 8.673 2.20E-1 4.92
65 45 2925 4001 8.860 3.39E-2 14.07
80 50 4000 10,001 8.874 2.03E-2 31.09
AFFT
10 10 100 251 8.346 5.48E-1 0.41
15 15 225 251 8.698 1.96E-1 0.54
25 20 500 401 8.860 3.47E-2 0.56
30 30 900 601 8.870 2.43E-2 1.49
50 30 1500 2001 8.885 9.62E-3 4.46
80 30 2400 5001 8.890 4.32E-3 11.49

145
Mathematics 2023, 11, 1123

Table 1. Cont.

Solver m n N k+1 u ε Time


SM
10 10 100 251 8.388 5.06E-1 0.55
15 15 225 251 8.746 1.47E-1 0.81
25 20 500 401 8.877 1.71E-2 1.83
30 30 900 601 8.888 6.09E-3 3.64
80 30 2400 2501 8.894 8.19E-4 27.19
RBF-FD-PM
10 10 100 251 8.389 5.05E-1 0.52
15 15 225 251 5.753 1.41E-1 0.80
25 20 500 401 8.876 1.88E-2 1.69
30 30 900 601 8.889 5.56E-3 3.29
80 30 2400 2501 8.894 6.69E-4 25.74

The reference prices for specific locations of the domain are 11.302917 at (90, 0.04, 0.5),
6.589881 at (100, 0.04, 0.5) and 4.191455 at (110, 0.04, 0.5) using [34]. The convergence results
are provided in Tables 2 and 3 and confirm the superiority of the proposed solver with
ψ = 2 in this paper.
The FDM solver is back-of-the-envelope accounting because it is clear that the uniform
grids for the PIDE problem are not a fast calculation to obtain highly accurate prices. To
check the stability and positivity of the numerical solution for RBF-FD-PM, the numerical
solution for Example 2 is plotted in Figure 1, which shows the stable behavior of RBF-FD-
PM using m = 16, n = 8 and k = 1001.

Figure 1. Numerical solution of Example 2 using the RBF-FD-PM solver when τ = 0 on the left and
τ = 0.5 on the right. Green points show the location of the graded discretization points on the red
curve, which is the numerical solution.

The reason for providing Figure 1 is twofold. We must first reveal that the numerical
solution obtained by RBF-FD-PM using some m and n is stable and does not have oscil-
lations. This is important since the PIDE model has a mixed derivative term, which can
lead to oscillations in the numerical solution as long as a careless numerical method is
employed. Second, we must reveal how the graded meshes (the green points in Figure 1)
located on the numerical solution are obtained by employing an automatic interpolation
on the obtained solutions.
An inquiry might arise by analyzing the results in Tables 1 and 2. tt is not easy to
find out the advantages of the proposed approach since the numerical values are given for
different values of the parameters m, n, N and k + 1. In fact, larger time step sizes (lower
k) are taken for SM and RBF-FD-PM since their ODE solver, i.e., (44), has a larger stability

146
Mathematics 2023, 11, 1123

region, and the overall solvers must be compared by fixing an accuracy for the errors and
then checking the computational times.

Table 2. Numerical results of the different solvers in Example 2.

Solver m n N k+1 ε at x = 90 ε at x = 100 ε at x = 110 Time


FDM
8 5 40 26 4.54E0 3.10E0 3.56E-1 0.37
16 8 128 51 1.23E0 1.31E-1 1.02E0 0.52
32 16 512 101 1.34E-1 2.77E-1 2.26E-1 1.49
32 32 1024 201 1.25E-1 2.02E-1 2.23E-1 2.93
64 32 2048 501 5.35E-3 5.85E-2 1.87E-2 9.47
SM
8 5 32 51 2.57E-1 7.48E-1 7.13E-1 0.55
16 8 128 101 7.31E-2 2.51E-1 3.89E-2 0.81
32 16 512 501 2.18E-3 6.02E-3 5.64E-3 2.40
64 32 2048 1001 1.79E-3 6.24E-4 1.39E-3 16.71
RBF-FD-PM
8 5 32 51 2.49E-1 6.89E-1 7.11E-1 0.50
16 8 128 101 6.25E-2 2.39E-1 3.84E-2 0.76
32 16 512 501 2.07E-3 5.44E-3 5.04E-3 2.11
64 32 2048 1001 1.08E-3 5.81E-4 1.04E-3 15.34

To also show how the instability may ruin the numerical pricing using the stability
bound (49), we provide the numerical results of solving (2) by the RBF-FD-PM using m = 16
and n = 8, but with k = 25 uniform discretization nodes along time in Figure 2. This shows
that all the involved solvers have some limitations, but the proposed solver sounds more
efficient than others.

Figure 2. The instable numerical solution of Example 2 at τ = 0.5 using k = 25 nodes.

However, due to nonsmoothness at the strike price in the initial condition, it might be
useful to employ a time-stepping solver that works on graded meshes over time with more
focus at the beginning of the starting time, i.e., zero (the solution near the initial time point
has a weak singularity). One such method is the Rannacher time-marching method [37].
Although such an application will help our solver a lot, we will try to focus on this in
forthcoming related works.

147
Mathematics 2023, 11, 1123

Table 3. Numerical results of the AFFT solver in Example 2.

Solver m n ε at x = 90 ε at x = 100 ε at x = 110


AFFT 17 9 1.08E0 1.57E0 1.96E-1
33 17 2.80E-2 5.20E-1 1.38E-1
65 33 4.78E-3 1.25E-1 2.84E-2
129 65 7.38E-3 3.09E-2 5.25E-3

6. Concluding Remarks
PIDEs arise in the mathematical modeling of many processes in different fields of
engineering and finance. This paper has presented an approximate solution of the linear
Bates PIDE with clear application in financial option pricing using a local integral term. The
solution method was considered on graded meshes at which there is a clear concentration of
the discretization nodes on the financially important are of the problem. Then, an RBF-FD
solver using semi-discretization via sparse arrays have been constructed for solving the
Bates PIDE. The numerical results were furnished and supported the theoretical discussions.
These results have been provided in Tables 1 and 2 which implicitly state that the proposed
approach can compete the most efficient solver (SM) for the same purpose. Additionally, the
prospects for future research can be focused on how to obtain RBF-FD weights on stencils
having five/six adjacent nodes on graded meshes or employing the Rannacher time-
marching method in order to obtain higher accuracies for solving the PIDE problem (2).

Author Contributions: Conceptualization, T.L. and G.F.; formal analysis, G.F.; T.L. and S.S.; funding
acquisition, M.Z.U. and T.L.; investigation, T.L., M.Z.U. and S.S.; methodology, G.F. and S.S.; supervi-
sion, T.L.; validation, M.Z.U. and S.S.; writing—original draft, G.F., M.Z.U. and S.S.; writing—review
and editing, T.L., M.Z.U. and S.S. All authors have read and agreed to the published version of the
manuscript.
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: For the data-availability statement, we state that data sharing is not
applicable to this article as no new data were created in this study.
Acknowledgments: The third author states that: The Deanship of Scientific Research (DSR) at
King Abdulaziz University (KAU), Jeddah, Saudi Arabia, has funded this project, under grant no.
(KEP-MSc: 65-130-1443). The authors are very much thankful to three anonymous referees for their
suggestions, which helped to improve this paper.
Conflicts of Interest: The authors declare that they have no known competing financial interests or
personal relationships that could have appeared to influence the work reported in this paper.

References
1. Bates, D. Jumps and stochastic volatility: The exchange rate processes implicit in Deutsche mark options. Rev. Fin. Stud. 1996, 9,
69–107. [CrossRef]
2. Itkin, A. Pricing Derivatives Under Lévy Models: Modern Finite-Difference and Pseudo–Differential Operators Approach, Birkhäuser Basel;
Springer: Berlin/Heidelberg, Germany, 2017.
3. Kim, K.-H.; Sin, M.-G. Efficient hedging in general Black-Scholes model. Finan. Math. Appl. 2014, 3, 1–9.
4. Ghanadian, A.; Lotfi, T. Approximate solution of nonlinear Black-Scholes equation via a fully discretized fourth-order method.
Aims Math. 2020, 5, 879–893. [CrossRef]
5. Heston, S. A closed-form solution for options with stochastic volatility with applications to bond and currency options. Rev.
Finan. Stud. 1993, 6, 327–343. [CrossRef]
6. Chang, Y.; Wang, Y.; Zhang, S. Option pricing under double Heston jump-diffusion model with approximative fractional stochastic
volatility. Mathematics 2021, 9, 126. [CrossRef]
7. Gómez-Valle, L.; Martínez-Rodríguez, J. Including jumps in the stochastic valuation of freight derivatives. Mathematics 2021, 9,
154. [CrossRef]
8. Liu, J.; Yan, J. Convergence rate of the high-order finite difference method for option pricing in a Markov regime-switching
jump-diffusion model. Fractal Fract. 2022, 6, 409. [CrossRef]

148
Mathematics 2023, 11, 1123

9. Chen, K.-S.; Huang, Y.-C. Detecting jump risk and jump-diffusion model for Bitcoin options pricing and hedging. Mathematics
2021, 9, 2567. [CrossRef]
10. Hellmuth, K.; Klingenberg, C. Computing Black Scholes with uncertain volatility-a machine learning approach. Mathematics 2022,
10, 489. [CrossRef]
11. Ballestra, L.; Sgarra, C. The evaluation of American options in a stochastic volatility model with jumps: An efficient finite element
approach. Comput. Math. Appl. 2010, 60, 1571–1590. [CrossRef]
12. Ballestra, L.; Cecere, L. A fast numerical method to price American options under the Bates model. Comput. Math. Appl. 2016, 72,
1305–1319. [CrossRef]
13. Kluge, T. Pricing Derivatives in Stochastic Volatility Models Using the Finite Difference Method. Ph.D. Thesis, TU Chemnitz,
Chemnitz, Germany, 2002.
14. in ’t Hout, K.J.; Foulon, S. ADI finite difference schemes for option pricing in the Heston model with correlation. Int. J. Numer.
Anal. Model. 2010, 7, 303–320.
15. Balajewicz, M.; Toivanen, J. Reduced order models for pricing European and American optionsunder stochastic volatility and
jump-diffusion models. J. Comput. Sci. 2017, 20, 198–204. [CrossRef]
16. Duffy, D. Finite Difference Methods in Financial Engineering: A Partial Differential Equation Approach; Wiley: England, UK, 2006.
17. Düring, B.; Fournié, M. High-order compact finite difference scheme for option pricing in stochastic volatility models. J. Comput.
Appl. Math. 2012, 236, 4462–4473. [CrossRef]
18. Soleymani, F.; Ullah, M. A multiquadric RBF-FD scheme for simulating the financial HHW equation utilizing exponential
integrator. Calcolo 2018, 55, 51. [CrossRef]
19. Milovanović, S.; von Sydow, L. A high order method for pricing of financial derivatives using radial basis function generated
finite differences. Math. Comput. Simul. 2020, 174, 205–217. [CrossRef]
20. Milovanović, S.; von Sydow, L. Radial basis function generated finite differences for option pricing problems. Comput. Math. Appl.
2018, 75, 1462–1481. [CrossRef]
21. Soleymani, F.; Barfeie, M. Pricing options under stochastic volatility jump model: Astable adaptive scheme. Appl. Numer. Math.
2019, 145, 69–89. [CrossRef]
22. Soleymani, F.; Zhu, S. RBF-FD solution for a financial partial-integro differential equation utilizing the generalized multiquadric
function. Comput. Math. Appl. 2021, 82, 161–178. [CrossRef]
23. Soleymani, F.; Itkin, A. Pricing foreign exchange options under stochastic volatility and interest rates using an RBF-FD method. J.
Comput. Sci. 2019, 37, 101028. [CrossRef]
24. Itkin, A.; Soleymani, F. Four-factor model of quanto CDS with jumps-at-default and stochastic recovery. J. Comput. Sci. 2021, 54,
101434. [CrossRef]
25. Fasshauer, G. Meshfree Approximation Methods with MATLAB; World Scientific Publishing Co.: Singapore, 2007.
26. Bayona, V.; Moscoso, M.; Carretero, M.; Kindelan, M. RBF-FD formulas and convergence properties. J. Comput. Phys. 2010, 229,
8281–8295. [CrossRef]
27. Meyer, G. The Time-Discrete Method of Lines for Options and Bonds, A PDE Approach; World Scientific Publishing: Hackensack, NJ,
USA, 2015.
28. Knapp, R. A method of lines framework in Mathematica. J. Numer. Anal. Indust. Appl. Math. 2008, 3, 43–59.
29. Salmi, S.; Toivanen, J. An iterative method for pricing American options under jump-diffusion models. Appl. Numer. Math. 2011,
61, 821–831. [CrossRef]
30. Michel, V.; Thomann, D. TVD-MOOD schemes based on implicit-explicit time integration. Appl. Math. Comput. 2022, 433, 127397.
31. Shymanskyi, V.; Protsyk, Y. Simulation of the heat conduction process in the Claydite-Block construction with taking into account
the fractal structure of the material. In Proceedings of the 2018 IEEE 13th International Scientific and Technical Conference on
Computer Sciences and Information Technologies (CSIT), Lviv, Ukraine, 11–14 September 2018.
32. Sayfidinov, O.; Bognár, G.; Kovács, E. Solution of the 1D KPZ equation by explicit methods. Symmetry 2022, 14, 699. [CrossRef]
33. Butcher, J. Numerical Methods for Ordinary Differential Equations, 2nd ed.; Wiley: England, UK, 2008.
34. Salmi, S.; Toivanen, J.; von Sydow, L. An IMEX-scheme for pricing options under stochastic volatility models with jumps. SIAM J.
Sci. Comput. 2014, 36, 817–834. [CrossRef]
35. Mangano, S. Mathematica Cookbook; O’Reilly Media: Newton, MA, USA, 2010.
36. Wellin, P.; Gaylord, R.; Kamin, S. An Introduction to Programming with Mathematica; Cambridge University Press: Cambridge,
UK, 2005.
37. Giles, M.; Carter, R. Convergence analysis of Crank-Nicolson and Rannacher time-marching. J. Comput. Financ. 2006, 9, 89–112.
[CrossRef]

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual
author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to
people or property resulting from any ideas, methods, instructions or products referred to in the content.

149
mathematics

Article
Hybrid Newton–Sperm Swarm Optimization Algorithm for
Nonlinear Systems
Obadah Said Solaiman 1 , Rami Sihwail 2, *, Hisham Shehadeh 3 , Ishak Hashim 1,4 and Kamal Alieyan 2

1 Department of Mathematical Sciences, Faculty of Science & Technology, Universiti Kebangsaan Malaysia,
Bangi 43600, Selangor, Malaysia; [email protected] (O.S.S.); [email protected] (I.H.)
2 Department of Cyber Security, Faculty of Computer Science & Informatics, Amman Arab University,
Amman 11953, Jordan; [email protected]
3 Department of Computer Information System, Faculty of Computer Science & Informatics, Amman Arab
University, Amman 11953, Jordan; [email protected]
4 Nonlinear Dynamics Research Center (NDRC), Ajman University, Ajman P.O. Box 346, United Arab Emirates
* Correspondence: [email protected]; Tel.: +962-7-8814-2270

Abstract: Several problems have been solved by nonlinear equation systems (NESs), including
real-life issues in chemistry and neurophysiology. However, the accuracy of solutions is highly
dependent on the efficiency of the algorithm used. In this paper, a Modified Sperm Swarm Optimiza-
tion Algorithm called MSSO is introduced to solve NESs. MSSO combines Newton’s second-order
iterative method with the Sperm Swarm Optimization Algorithm (SSO). Through this combination,
MSSO’s search mechanism is improved, its convergence rate is accelerated, local optima are avoided,
and more accurate solutions are provided. The method overcomes several drawbacks of Newton’s
method, such as the initial points’ selection, falling into the trap of local optima, and divergence.
In this study, MSSO was evaluated using eight NES benchmarks that are commonly used in the
literature, three of which are from real-life applications. Furthermore, MSSO was compared with
several well-known optimization algorithms, including the original SSO, Harris Hawk Optimization
(HHO), Butterfly Optimization Algorithm (BOA), Ant Lion Optimizer (ALO), Particle Swarm Opti-
mization (PSO), and Equilibrium Optimization (EO). According to the results, MSSO outperformed
Citation: Said Solaiman, O.; Sihwail, the compared algorithms across all selected benchmark systems in four aspects: stability, fitness
R.; Shehadeh, H.; Hashim, I.; Alieyan, values, best solutions, and convergence speed.
K. Hybrid Newton–Sperm Swarm
Optimization Algorithm for
Keywords: nonlinear systems; Newton’s method; iterative methods; sperm swarm optimization
Nonlinear Systems. Mathematics 2023,
algorithm; optimization algorithm
11, 1473. https://doi.org/10.3390/
math11061473
MSC: 65D99; 65H10; 65K10
Academic Editors: Maria
Isabel Berenguer and Manuel Ruiz
Galán
1. Introduction
Received: 6 February 2023
Revised: 11 March 2023 Many issues in the natural and applied sciences are represented by systems of non-
Accepted: 14 March 2023 linear equations F ( X ) = 0 that require solving, where F ( X ) = ( f 1 , f 2 , . . . , f n ) such that f i
Published: 17 March 2023 is nonlinear for all i = 1, 2, . . . , n. It is well known that determining the precise solution
α = (α1 , α2 , . . . , αn )t to the nonlinear system F ( X ) = 0 is a difficult undertaking, especially
when the equation comprises terms made up of logarithmic, exponential, trigonometric,
or a mix of any transcendental terms. Thus, finding approximate solutions to this type of
Copyright: © 2023 by the authors.
problem has emerged as a need. The iterative methods, including Newton’s method, are
Licensee MDPI, Basel, Switzerland.
some of the most famous methods for finding approximate solutions to nonlinear equation
This article is an open access article
systems (NESs) [1]. Alternatively, optimization algorithms have been applied in attempts
distributed under the terms and
to extract the root solution of nonlinear systems.
conditions of the Creative Commons
In the last ten years, various optimization algorithms have been developed. Those
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
methods can be divided into four primary categories: human-based methods, swarm-based
4.0/).

Mathematics 2023, 11, 1473. https://doi.org/10.3390/math11061473 https://www.mdpi.com/journal/mathematics


151
Mathematics 2023, 11, 1473

methods, physical-based methods, and evolutionary-based methods [2]. Human percep-


tion, attitude, or lifestyle influence human-based methods. Examples of these methods
are the “Harmony Search Algorithm (HSA)” [3] and the “Fireworks Algorithm (FA)” [4].
Swarm-based methods mimic the behavior of swarms or animals to reproduce or survive.
Examples of this algorithm are “Sperm Swarm Optimization (SSO)” [5–8], “Harris Hawks
Optimization (HHO)” [9], “The Ant Lion Optimizer (ALO)” [10], and “Butterfly Opti-
mization Algorithm (BOA)” [11]. Some representative swarm intelligence optimization
methods and applications have also been proposed; see for example, [12]. Physical-based
methods are inspired by both physical theories and the universe’s rules. An example of
these algorithms is the “Gravitational Search Algorithm (GSA)” [2], and “Equilibrium
Optimizer (EO)” [13]. Evolutionary-based methods are inspired by the Darwinian theory of
evolution. An example of this method is the “Genetics Algorithm (GA)” [14]. Finally, some
advanced optimization methods with applications from the real-life have been proposed,
for example [15,16].
The primary objectives of these methods are to yield the optimal solution and a
higher convergence rate. Meta-heuristic optimization should be based on exploration
and exploitation concepts to achieve global optimum solutions. The exploitation concept
indicates the ability of a method to converge to the optimal potential solution. In contrast,
exploration refers to the power of algorithms to search the entire space of a problem domain.
Therefore, the main goal of meta-heuristic methods is to balance the two concepts.
However, different meta-heuristic methods have been developed to find solutions to
various real-life tasks. The use of optimization algorithms for solving NESs is significant
and critical. Various optimization algorithms are used in the solution of nonlinear systems.
The following may be summarized:
By improving the performance of optimization algorithms, researchers have been
able to target more accurate solutions. For example, Zhou and Li [17] provided a unified
solution to nonlinear equations using a modified CSA version. FA was modified by
Ariyaratne et al. [18], who made it possible to make the root approximation simultaneously
with continuity, differentiation, and initial assumptions. Ren et al. [19] proposed another
variation by combining GA with harmonic and symmetric individuals. Chang [20] also
revised the GA to estimate better parameters for NESs.
Furthermore, complex systems were handled by Grosan and Abraham [21] by putting
them in the form of multi-objective optimization problems. Jaberipour et al. [22] addressed
NESs using a modified PSO method; the modification aims to overcome the core PSO’s
drawbacks, such as delayed convergence and trapping at local minimums. Further, NESs
have been addressed by Mo and Liu [23], who added the “Conjugate Direction Method
(CDM)” into the PSO algorithm. The algorithm’s efficiency for solving high-dimensional
problems and overcoming local minima was increased by using CDM [24].
Several research methods involved combining two population-based algorithms
(PBAs) to achieve more precise results in nonlinear modeling systems. These combinations
produce hybrid algorithms that inherit the benefits of both techniques while reducing
their downsides [25]. Hybrid ABC [26], hybrid ABC and PSO [27], hybrid FA [28], hybrid
GA [29], hybrid KHA [30], hybrid PSO [31], and many others [32–36] are some examples of
hybridizing PBAs.
NESs have often been solved using optimization techniques, either using a “Single
Optimization Algorithm (SOA)” or a hybrid algorithm that combines two optimization
procedures. Only a few researchers have attempted to combine the iterative method and
an optimization approach. Karr et al. [37] presented a hybrid method combining Newton’s
method and GA for obtaining solutions for nonlinear testbed problems. After using GA
to identify the most efficient starting solution, Newton’s approach was utilized. To solve
systems of nonlinear models, a hybrid algorithm described by Luo et al. [38] can be utilized;
the combination includes GA, Powell algorithm, and Newton’s method. Luo et al. [39]
have provided a method for solving NESs by integrating chaos and quasi-Newton tech-
niques. Most of the previous research has concentrated on a specific topic or issue rather

152
Mathematics 2023, 11, 1473

than attempting to examine NESs. In a relatively recent study, Sihwail et al. [40] developed
a hybrid algorithm known as NHHO to solve arbitrary NESs of equations that combine
Harris Hawks’ optimization method and Newton’s method. Very recently, Sihwail et al. [41]
proposed a new algorithm for solving NESs of equations in which Jarratt’s iterative ap-
proach and the Butterfly optimization algorithm were combined to create the new scheme
known as JBOA.
A hybrid algorithm can leverage the benefits of one method while overcoming the
drawbacks of the other. However, most hybrid methods face problems with premature
convergence due to the technique used in the original algorithms [42]. As a result, choosing
a dependable combination of algorithms to produce an efficient hybrid algorithm is a
crucial step.
One of the more recent swarm-based methods is Sperm Swarm Optimization (SSO),
which is based on the mobility of flocks of sperm to fertilize an ovum. There are various
benefits of SSO, which can be listed as follows [2,5,6]:
• The capability of exploitation of SSO is very robust.
• Several kinds of research have validated its simplicity, efficiency, and ability to con-
verge to the optimal solution.
• Its theory can be applied to a wide range of problems in the areas of engineering
and science.
• Its mathematical formulation is easy to implement, understand, and utilize.
However, most NESs simulate different data science and engineering problems that
have more than one solution. Hence, it is difficult to give accurate solutions to these prob-
lems. Like other optimization algorithms, SSO may fall into a local minimum (solution)
instead of the optimal solution. As a result, we developed a hybrid approach that incor-
porates Newton’s iterative scheme with the SSO algorithm to mitigate the drawback. It is
worth mentioning that Newton’s method is the first known iterative scheme for solving
nonlinear equations using the successive approximation technique. According to Newton’s
method, the correct digits nearly double each time a step is performed, referred to as the
second order of convergence.
Newton’s method is highly dependent on choosing the correct initial point. To achieve
good convergence toward the root, the starting point, like other iterative approaches, must
be close enough to the root. The scheme may converge slowly or diverge if the initial point
is incorrect. Consequently, Newton’s method can only perform limited local searches in
some cases.
For the reasons outlined above, a hybrid SSO algorithm (MSSO) has been proposed to
solve NESs, where Newton’s method is applied to improve the search technique and SSO
is used to enhance the selection of initial solutions and make global search more efficient.
It is not the concern of this study to demonstrate that hybridizing the SSO and New-
ton’s methods performs better than other optimization algorithms such as PSO or genetic
algorithms. However, this work aims to highlight the benefits of hybridizing an opti-
mization algorithm with an iterative method. This is to enhance the iterative method’s
accuracy in solving nonlinear systems and reduce its complexity. Further, it is also able to
overcome several drawbacks of Newton’s method, such as initial point selection, trapping
in local optima, and divergence problems. Moreover, hybridization in MSSO is beneficial
in finding better roots for the selected NSEs. Optimization algorithms alone are unlikely to
provide precise solutions compared to iterative methods such as Newton’s method and
Jarratt’s method.
The proposed modification improves the initial solution distribution in the search
space domain. Moreover, compared to the random distribution used by the original
technique, Newton’s approach improves the computational accuracy of SSO and accelerates
its convergence rate. Hence, this research paper aims to improve the accuracy of NES
solutions. The following are the main contributions of this paper:

153
Mathematics 2023, 11, 1473

1. We present a Modified Newton–Sperm Swarm Optimization Algorithm (MSSO) that


combines Newton’s method and SSO to enhance its search mechanism and speed up
its convergence rate.
2. The proposed MSSO method is intended to solve nonlinear systems of different orders.
3. Different optimization techniques were compared with MSSO, including the original
SSO, PSO, ALO, BOA, HHO, and EO. The comparison was made based on multiple
metrics, such as accuracy, fitness value, stability, and convergence speed.
The rest of the paper is organized as follows: Section 2 discusses SSO algorithms and
Newton’s iterative method. Section 3 describes the proposed MSSO. Section 4 describes the
experiments on the benchmark systems and their results. Further discussion of the findings
is provided in Section 5. Finally, Section 6 presents the study’s conclusion.

2. Background
2.1. Standard Sperm Swarm Optimization (SSO) Algorithm
SSO is a newly created swarm-based technique proposed by Shehadeh et al. [2,5,6]
that draws inspiration from the actions of a group of sperm as they fertilize an ovum.
In the process of fertilization, a single sperm navigates a path against overwhelming
odds to merge with an egg (ova). In general, there are 130 million sperm involved in the
insemination process. Eventually, one of these sperm will fertilize the ovum. Based on
Shehadeh et al. [6], the procedure of fertilization can be summarized as follows:
A male’s reproductive system releases the sperm into the cervix, where the fertilization
process starts. Each sperm is given a random location inside the cervix to begin the
fertilization process as part of this task. Further, every sperm has two velocities on the
Cartesian plane. The initial velocity value of sperm denotes this velocity. The procedure of
fertilization is demonstrated in Figure 1.

Figure 1. The procedure of fertilization [6].

From this point, every sperm in the swarm is ready to swim until it reaches the outer
surface of the ovum. Scientists found that the sperm float on the surface as a flock or swarm,
moving from the zone of low temperature to the area of high temperature. Moreover,
they observed that the ovum triggers a chemical to pull the swarm; this is known as a
chemotactic process. According to researchers, these cells also beat at the same frequency
as the tail movements through the grouping. The ovum and its location in the fallopian

154
Mathematics 2023, 11, 1473

tubes are illustrated in Figure 1. Based on Shehadeh et al. [6], this velocity is denoted by
the personal best velocity of the sperm.
Usually, in a typical scenario, one sperm can fertilize an ovum. Based on that,
Shehadeh et al. [2,5–8] calls this sperm the winner. The winner and the flock of sperm
are illustrated in Figure 2.

Figure 2. A flock of sperm and the winner [2].

The best answer is found and obtained using this strategy, which makes use of a
group of sperm (potential solutions) floating over the whole search area. Concurrently, the
possible solutions will consider the most suitable sperm in their path, who will be the victor
(the sperm that is closest to the egg). Alternatively, the flock will consider on the winner’s
position and the position of its prior best solution. Thus, every sperm enhances its initial
zone across the optimum area by taking into consideration its current velocity, current
location, and the location of both the global’s best solution (the winner) and the sperm’s
best solution. Mathematically speaking, in SSO, the flock updated their sites according to
the following formula:
x i +1 ( t ) = x i ( t ) + v i ( t ) (1)
where
• vi is the velocity of potential solution i at iteration t;
• xi is the current position of possible solution i at iteration t;
Three velocities can be used to calculate the sperm’s velocity: the initial velocity of a
potential solution, the personal best solution, and the global best solution.
First is the initial velocity of sperm, which takes a random value based on the velocity
dumping parameter and the pH value of the initial location. The model can be calculated
by applying the following formula:

Initial_Velocity = D · Vi (t) · Log10 ( pH_Rand1 ) (2)

Second is a personal best location for the potential solution, adjusted in memory based
on the prior location until it is closest to the optimal value. However, this velocity can be
changed based on the pH and temperature values. The following formula may be used to
calculate this model:
 
Current _Best _Solution = Log10 ( pH_Rand2 ) · Log10 ( Temp_Rand1 ) · xsbesti []− xi [] (3)

Third, the global best solution is simulated by the winner, which is denoted by the
closest sperm to the ovum. The mathematical model of the winning velocity of the potential
solution Vi(t) can be represented in Equation (4). The flock of sperm and the value of the
winner are depicted in Figure 2.

155
Mathematics 2023, 11, 1473

 
Global_Best_Solution(the_winner ) = Log10 ( pH_Rand3 ) · Log10 ( Temp_Rand2 ) · xsgbesti []− xi [] (4)

The symbols of the prior equations are as follows:


• vi is the velocity of potential solution i at iteration t;
• D is the velocity damping factor and is a random parameter with a range of 0 to 1;
• pH_Rand1 , pH_Rand2 , and pH_Rand3 are the reached site pH values, which are random
parameters that take values between 7 to 14;
• Temp_Rand1 and Temp_Rand2 are values of the site temperature, which are random
parameters that take values between 35.1 to 38.5;
• xi is the current position of potential solution i at iteration t;
• xsbest is the personal best location of potential solution i at iteration t;
• xsgbest is the global best location of the flock.
Based on the equations mentioned above, the total velocity rule Vi (t) can be formal-
ized based on velocity initial value, personal best solution, and global best solution as
follows [2,5–8]:
 
Vi (t) = Log10 ( pH_Rand1 ).Vi + Log10 ( pH_Rand2 ).Log10 ( Temp_Rand
 1 ).) · xsbesti − xi ( t ) +
(5)
Log10 ( pH_Rand3 ).Log10 ( Temp_Rand2 ). xsbesti − xi (t)

Based on the theory of SSO, both pH and temperature affect the velocity rule. The pH
changes depending on the woman’s attitude, whether depressed or happy, and on the food
consumed. The value of the pH parameter falls in a range between seven and fourteen.
Alternatively, the temperature ranges from 35.1 to 38.5 ◦ C according to blood pressure
circulation in the reproductive system [7].
Further, SSO is a swarm-based method that simulates the metaphor of natural fer-
tilization. SSO, however, has a few disadvantages in terms of efficiency. Applied to a
broad search domain, SSO is prone to getting trapped in local optima [2], which is one
of its main drawbacks. Therefore, improvements are needed to enhance the method’s
exploration process.

2.2. Newton’s Method


An iterative technique is a technique (method) for finding an approximate solution by
making successive approximations. Iterative approaches usually cannot deliver accurate
answers. Accordingly, researchers generally select a tolerance level to distinguish between
approximate and exact answers for the solutions obtained through iterative approaches.
Newton’s method, also known as the Newton–Raphson method, was proposed by Isaac
Newton and is the most widely used iterative method. The procedure of Newton’s scheme
is described by
−1
Xn+1 = Xn − F  ( Xn ).F ( Xn ), (6)
where F ( X ) is the nonlinear system of equations, and F  ( Xn ) represents the “Jacobian of
F ( X )”. Newton’s second-order convergence method may be easily applied to various
nonlinear algebraic problems [1]. As a result, mathematical tools such as Mathematica
and MATLAB provide built-in routines for finding nonlinear equations’ roots based on
Newton’s scheme.
In Newton’s method, many studies and refinements have been performed to improve
approximation solutions to nonlinear problems as well as the order of convergence, which
impact the speed at which the desired solution can be reached; see, for example, [43–47]
and their references.

3. Modified Sperm Swarm Optimization (MSSO)


SSO is a powerful optimization technique that can address various issues. No algo-
rithm, however, is suitable for tackling all problems, according to the “No Free Lunch

156
Mathematics 2023, 11, 1473

(NFL)” theorem [48]. By using Newton’s method, the proposed MSSO outperforms the
original SSO in terms of solving nonlinear equation systems. In MSSO, Newton’s methods
are used as a local search to enhance the search process, as shown in Figure 3.

Figure 3. The framework of the proposed MSSO.

When Newton’s method is applied to the sperm position, at each iteration, the fitness
value of the potential solution is compared to the fitness of the location calculated by
Newton’s scheme. The newly computed location by Newton’s method is shown in Figure 3
as ( Xn+1 ).
In each iteration, MSSO employs both the SSO algorithm and Newton’s method. The
SSO first determines the most optimal sperm location among the twenty initial locations as
an optimal candidate location. The optimal candidate location is then fed into Newton’s
method. In other words, the output from SSO is considered a potential solution or a tem-
porary solution. The obtained solution is then treated as an input for Newton’s method.
Newton’s method as an iterative method calculates the next candidate solution based on
Equation (6). Newton’s method’s ability to find a better candidate is very high since it is a
second-order convergence method. However, in order to avoid a local optimal solution,
the candidate solution obtained from Newton’s method (Xn+1 ) is compared to the solution
calculated by SSO (Xsperm ). Thus, the location with the lowest fitness value determines
the potential solution to the problem. The next iteration is then performed based on the
current most promising solution. Algorithm 1 shows the pseudocode for the suggested

157
Mathematics 2023, 11, 1473

MSSO algorithm.

Algorithm 1. Modified Sperm Swarm Optimization (MSSO).


Begin
Step 1: Initialize potential solutions.
Step 2: for i = 1: size of flock do
Step 3: apply the fitness for potential solution.
if obtained fitness > best solution of the potential solution then
give the current value as the best solution of the potential solution.
end if
end for
Step 4: depends on the winner, give the value of winner.
Step 5: for i =1: size of flock do
Perform Equation (5)
Perform Equation (1).
end for
Step 6: Calculate Newton’s location Xn+1 using Equation (6)
Calculate the fitness of Xn+1 and Xsperm using Equation (7)
if fitness (Xn+1 ) < fitness ( Xsperm )
Xsperm = X n+1
end if
Step 7: while final iterations is not reached go to Step 2.
End.

The initialization, exploitation, and exploration phases of the SSO method are shown
in the algorithm. The alterations specified in the red box are implemented at the end of
each iteration. We compare Newton’s location with the sperm’s optimal location based on
their fitness values and select the one that has the best fitness value.

Computational Complexity
The complexity of the new MSSO’s can be obtained by adding up the SSO’s complexity
and Newton’s method’s complexity. At first glance, Newton’s technique is overly com-
plicated compared to optimization methods. At each iteration, one has to solve a N × N
system of linear models, which is time-consuming because every Jacobian calculation
requires n2 scalar function evaluations. As a result, combining Newton’s approach with
any optimization process is likely to make it more complicated.
On the other hand, combining SSO with Newton’s technique did not significantly
increase processing time. However, the MSSO can overcome Newton’s method limitations,
including selecting the starting points and divergence difficulties. As a result, the MSSO is
superior at solving nonlinear equation systems.
The MSSO’s time complexity is influenced by the initial phase, the process of updat-
ing the position of the sperm, and the use of Newton’s scheme. The complexity of the
initialization process is O(S), where S is the total number of sperm. The updating process,
which includes determining the optimal solution and updating sperm positions, has a
complexity equal to O(I × S) + O(I × S × M), where I and M represent the maximum
number of iterations and the complexity of the tested benchmark equation respectively.
Furthermore, Newton’s scheme complexity is calculated as O(I × T), where T is the compu-
tation time. Consequently, the proposed MSSO has an overall computational complexity of
O(S × (I + IM + 1) + IT).
Every improvement certainly has a cost. The principal objective of the proposed
hybrid algorithm is to enhance the fitness value and the convergence speed of the existing
algorithms. However, as a result of adding one algorithm to another, the complexity and
the time cost of the hybrid algorithm are increased compared to the original algorithm.
Eventually, a tradeoff between the merits and disadvantages should be considered while
using any algorithm.

158
Mathematics 2023, 11, 1473

4. Numerical Tests
Eight nonlinear systems of several orders were selected as indicators to clarify the
efficiency and capability of the new hybrid MSSO scheme. Comparisons between MSSO
and the other six well-known optimization algorithms have been performed. Those op-
timization algorithms are the original SSO [2], HHO [9], PSO [49], ALO [10], BOA [11],
and EO [13]. For consistency, all selected systems used in the comparisons are arbitrary
problems that are common in the literature, for instance, [19,21,40,44,50–53].
The comparison between the optimization algorithms is based on the fitness value of
each algorithm in each benchmark. A solution with less fitness value is more accurate than
a solution with a higher fitness value. Hence, the most effective optimization algorithm is
the one that solves with the least fitness value. The fitness function used in the comparison
is the Euclidean norm, also called the square norm or norm-2. Using the Euclidean norm,
we can determine the distance from the origin, which is expressed as follows:

Fitness = F ( x ) 2 = f 12 + f 22 + . . . + f n2 , (7)

Similar settings have been used in all benchmarks to guarantee a fair comparison of
all selected algorithms. The parameter values of all optimization algorithms have been
fine-tuned to improve the performance of the algorithms. The best solution was chosen
by every optimization method 30 times. Search agents (population size) have been set to
20 and the maximum iteration to 50. Furthermore, the best solution with the least fitness
value is chosen if there is more than one solution for a particular benchmark. In the end,
for lack of space, answers are shortened to 11 decimal places.
Calculations were conducted using MATLAB software version R2020a with the default
variable precision of 16 digits. This was on an Intel Core i5 processor running at 2.2 GHz
and 8 GB of RAM under the Microsoft Windows 8 operating system.
Problem 1: Let us consider the first problem to be the following nonlinear system of
two equations: !
x1 + 1 − ex2 = 0,
F1 ( X ) =
x1 + cos( x2 ) − 2 = 0,
For this system, the precise solution is given by α = {1.3401918575555883401 . . . ,
0.8502329164169513268 . . .}t . After running the algorithms 30 times, MSSO significantly
surpassed all other optimization algorithms in the comparison. Table 1 shows that the
proposed hybrid MSSO algorithm has attained the best solution with the least fitness value
equaling zero. This means that the solution obtained by MSSO is an exact solution for the
given system.

Table 1. Comparison of different optimization algorithms for Problem 1.

MSSO HHO [9] PSO [49] ALO [10] BOA [11] EO [13] SSO [2]
x1 1.34019185756 1.34020535556 1.34019185727 1.34019196042 1.34359319240 1.34019194567 1.34502836805
x2 0.85023291642 0.85023195766 0.85023291632 0.85023300025 0.85138606082 0.85023289034 0.85355356706
Fitness 0 2.1212 × 10−5 2.2401 × 10−10 1.0147 × 10−7 2.6296 × 10−3 1.8396 × 10−7 3.7618 × 10−3

Problem 2: The second benchmark is the system of two nonlinear equations given by:
!
2 − e x1 + tan−1 x2 = 0,
F2 ( X ) =
tan−1 x12 + x22 − 5 = 0,

Here, the exact zero for the system in this problem is given by α = ( 1.1290650391602 . . . ,
1.9300808629035 . . .)t . As shown in Table 2, it is evident that MSSO achieved the exact
solution of this system with a fitness value of zero. It also outperformed all other algorithms
with a substantial difference, especially in comparison with SSO, BOA, and HHO.

159
Mathematics 2023, 11, 1473

Table 2. Comparison of different optimization algorithms for Problem 2.

MSSO HHO [9] PSO [49] ALO [10] BOA [11] EO [13] SSO [2]
x1 1.12906503916 1.12903302177 1.12906503916 1.12906515112 1.12588512395 1.12906504185 1.14402014766
x2 1.93008086290 1.93011297982 1.93008086290 1.93008085965 1.93375637741 1.93008086329 1.92067058635
Fitness 0 0.000117763 8.01 × 10−15 4.22 × 10−7 0.012716315 1.1201 × 10−8 0.048651092

Problem 3: The third system of nonlinear equations is given by:



⎨ cos( x2 ) − sin( x1 ) = 0,
F3 ( X ) = x3 x1 − x12 = 0,
⎩ x1
e − x32 = 0.

This NES of three equations has the exact solution α = {0.9095694945200448838 . . . ,


0.6612268322748517354 . . ., 1.575834143906999036 . . .}t . According to Table 3, the proposed
MSSO achieved a zero fitness value. The superiority of MSSO is evident in this example,
with a significant difference between MSSO and all other compared optimization algorithms.

Table 3. Comparison of different optimization algorithms for Problem 3.

MSSO HHO [9] PSO [49] ALO [10] BOA [11] EO [13] SSO [2]
x1 0.90956949452 0.90449212115 0.89176809239 0.85453639710 0.83212389642 0.90775456824 1.03817572093
x2 0.66122683227 0.66642798414 0.67275154835 0.69673611158 0.69808559231 0.66254037960 0.56914672488
x3 1.57583414391 1.57229467736 1.56169705842 1.53258611089 1.52262989677 1.57448413869 1.69602879530
Fitness 0 0.005442108 0.004315295 0.013715754 0.036158224 0.000699083 0.061770954

Problem 4: Consider the following system of three nonlinear equations:



⎨ x2 + x3 − e− x1 = 0,
F4 ( X ) = x + x3 − e− x2 = 0,
⎩ 1
x1 + x2 − e− x3 = 0.

The precise solution of the nonlinear system in this problem is equal to α = (0.351733711249 . . . ,
0.351733711249 . . . , 0.351733711249 . . .)t . The best solution achieved by the compared
schemes for the given system is illustrated in Table 4. The proposed MSSO found a precise
answer, with zero as a fitness value. ALO recorded the second-best solution with a fitness
value of 2.27 × 10−6 , while the rest of the compared algorithms were far from the exact
answer. Again, the proposed MSSO has proved it has an efficient local search mechanism.
Hence, it can achieve more accurate solutions for nonlinear systems.

Table 4. Comparison of different optimization algorithms for Problem 4.

MSSO HHO [9] PSO [49] ALO [10] BOA [11] EO [13] SSO [2]
x1 0.35173371125 0.36165321762 0.35083292352 0.35172698088 0.38459199838 0.35086562122 0.37260511330
x2 0.35173371125 0.35137717774 0.35226253114 0.35173655019 0.33171697596 0.35200965295 0.34576550099
x3 0.35173371125 0.34410796587 0.35213140099 0.35173726704 0.34030291514 0.35226146573 0.33588500543
Fitness 0 0.005300022 0.000379475 2.2674 × 10−6 0.016625262 0.000294859 0.010254721

Problem 5: The next benchmark is the following system of two nonlinear equations:
!
x1 + e x2 − cos( x2 ) = 0,
F5 ( X ) =
3x1 − sin( x1 ) − x2 = 0,

This nonlinear system has the trivial solution α = (0, 0)t . Table 5 illustrates the comparison
between the different optimization algorithms for the given system. Compared with the
other algorithms, the original SSO and HHO achieved excellent results, with fitness values

160
Mathematics 2023, 11, 1473

of 5.36 × 10−15 and 6.92 × 10−14 , respectively. However, MSSO outperformed both of them
and delivered the exact solution for the given system.

Table 5. Comparison of different optimization algorithms for Problem 5.

MSSO HHO [9] PSO [49] ALO [10] BOA [11] EO [13] SSO [2]

x 3.6298689 × 10−22 1.0162783 × 10−14 −2.0631743 × 10−8 −2.0631743 × 10−8 0.00019546601 −1.0265357 × 10−14 −5.8109345 × 10−16
y 7.2597377 × 10−22 -4.1451213 × 10−14 2.4507340 × 10−7 2.4507340 × 10−7 1.1132830 × 10−5 1.0797593 × 10−13 −3.9989603 × 10−15
Fitness 0 6.92 × 10−14 3.64 × 10−7 3.64 × 10−7 4.32 × 10−4 1.61 × 10−13 5.36 × 10−15

Problem 6: The sixth system considered for the comparison is an interval arithmetic
benchmark [53] given by the following system of ten equations:


⎪ x1 − 0.25428722 − 0.18324757x4 x3 x9 = 0,



⎪ x2 − 0.37842197 − 0.16275449x1 x10 x6 = 0,



⎪ x3 − 0.27162577 − 0.16955071x1 x2 x10 = 0,



⎪ x4 − 0.19807914 − 0.15585316x7 x1 x6 = 0,


x5 − 0.44166728 − 0.19950920x7 x6 x3 = 0,
F6 ( X ) =
⎪ x6 − 0.14654113 − 0.18922793x8 x5 x10 = 0,




⎪ x7 − 0.42937161 − 0.21180486x2 x5 x8 = 0,



⎪ x8 − 0.07056438 − 0.17081208x1 x7 x6 = 0,



⎪ x9 − 0.34504906 − 0.19612740x10 x6 x8 = 0,


x10 − 0.42651102 − 0.21466544x4 x8 x1 = 0,
−10 ≤ x1 , x2 , . . . , x10 ≤ 10.

In this benchmark, MSSO has proven its efficiency. Table 6 clearly shows the significant
differences between MSSO and the other compared algorithms. MSSO achieved the best
solution with a fitness value of 5.21 × 10−17 , while all different algorithms achieved
solutions far from the exact answer. When we compare the fitness values of the hybrid
MSSO and the original SSO, we can see how substantial modifications were made to the
local search mechanism of the original SSO to produce the hybrid MSSO.

Table 6. Comparison of different optimization algorithms for Problem 6.

MSSO HHO [9] PSO [49] ALO [10] BOA [11] EO [13] SSO [2]
x1 0.25783339370 0.34365751785 0.25784839865 0.26464526597 0.33136834430 0.25516109743 0.20435054402
x2 0.38109715460 0.33753782972 0.38110810543 0.40023813660 0.38789340931 0.37760106529 0.28412716608
x3 0.27874501735 0.29465973836 0.27883198050 0.30288150337 0.21629745964 0.27543881117 0
x4 0.20066896423 0.25159175619 0.20067772983 0.19561671789 0.11897384735 0.20247039332 4.6624555 × 10−14
x5 0.44525142484 0.29083336278 0.44529373708 0.42832138835 0.43899648474 0.44562023380 0.21484320995
x6 0.14918391997 0.17861978035 0.14916957364 0.13017287705 0.11989963467 0.14456849647 0.04811561607
x7 0.43200969898 0.45287147997 0.43201094116 0.42448051059 0.41892967958 0.43104930617 0.46906778944
x8 0.07340277778 0.12886919949 0.07336337021 0.08657096366 0.00941718057 0.07245346262 0.04141333025
x9 0.34596682688 0.41390929124 0.34597891260 0.35142553752 0.31940825594 0.34552658400 0.44010425014
x10 0.42732627599 0.31843020513 0.42732508540 0.40501764912 0.31956474381 0.42687560151 0.45420039449
Fitness 5.21 × 10−17 0.238337 0.000107027 0.049509462 0.182742367 0.007434684 0.448061654

161
Mathematics 2023, 11, 1473

Problem 7: Consider the model A combustion chemistry problem for a temperature


of 3000 ◦ C [21], which can be described by the following nonlinear system of equations:


⎪ x2 + 2x6 + x9 + 2x10 − 10−5 = 0,

⎪ x + x − 3 × 10−5 = 0,

⎪ 3 8


⎪ x1 + x3 + 2x5 + 2x8 + x9 + x10 − 5 × 10−5 = 0,





⎪ x4 + 2x7 − 10−5 = 0,

0.5140437 × 10−7 x5 − x12 = 0,
F7 ( X ) = −6
⎪ 0.1006932 × 10 x6 − 2x2 = 0,
2


⎪ −15 x − x2 = 0,

⎪ 0.7816278 × 10

⎪ −6
7 4

⎪ 0.1496236 × 10 x8 − x1 x3 = 0,

⎪ −7

⎪ 0.6194411 × 10−14x9 − x1 x2 =


0,
0.2089296 × 10 x10 − x1 x22 = 0,
−10 ≤ x1 , x2 , . . . , x10 ≤ 10.

In Table 7, the comparison for this system shows that MSSO has the least fitness value of
7.09 × 10−21, while PSO and EO have fitness values of 2.85 × 10−9 and 3.45 × 10−8, respectively.

Table 7. Comparison of different optimization algorithms for Problem 7.

MSSO HHO [9] PSO [49] ALO [10] BOA [11] EO [13] SSO [2]

x1 1.8492683 × 10−7 4.8416050 × 10−6 1.9790922 × 10−5 1.0594162 × 10−5 1 × 10−22 1.0078652 × 10−5 1 × 10−22
x2 1.5794030 × 10−7 1 × 10−22 1.3635593 × 10−15 3.3602174 × 10−7 1 × 10−22 3.5661155 × 10−8 1 × 10−22
x3 1.3864372 × 10−5 1.6731599 × 10−5 3.1002047 × 10−5 3.0337169 × 10−5 4.1974292 × 10−6 3.0993197 × 10−5 3.8503515 × 10−6
x4 7.1476236 × 10−11 9.8490309 × 10−6 5.7239289 × 10−10 1.4332843 × 10−8 6.9533835 × 10−6 9.9663562 × 10−6 8.9634004 × 10−7
x5 6.6527288 × 10−21 1 × 10−22 1.3480554 × 10−18 1 × 10−22 1 × 10−22 1.0400080 × 10−22 1 × 10−22
x6 2.4773409 × 10−6 1 × 10−22 4.8969622 × 10−6 2.6156272 × 10−7 2.6521256 × 10−6 1.0305788 × 10−9 2.7036389 × 10−7
x7 4.9999643 × 10−6 1 × 10−22 4.9991846 × 10−6 5.0388691 × 10−6 1 × 10−22 9.0975075 × 10−10 4.1982561 × 10−6
x8 1.7135628 × 10−5 1.1426668 × 10−5 1.1003359 × 10−10 7.7891263 × 10−7 2.1564327 × 10−5 2.1008819 × 10−22 2.2239423 × 10−5
x9 4.7151213 × 10−7 5.6143966 × 10−6 2.0556945 × 10−7 7.7518075 × 10−6 3.3294486 × 10−6 9.9131258 × 10−6 6.2906500 × 10−6
x10 2.2079329 × 10−6 2.4214874 × 10−6 2.7225791 × 10−15 6.7696638 × 10−7 1 × 10−22 1.9269643 × 10−8 1 × 10−22
Fitness 7.09 × 10−21 3.23 × 10−6 2.85 × 10−9 1.73 × 10−7 6.22 × 10−6 3.45 × 10−8 6.91 × 10−6

Problem 8: The last benchmark is an application from neurophysiology [52], described


by the nonlinear system of six equations:


⎪ x12 + x32 − 1 = 0

⎪ x22 + x42 − 1 = 0



x5 x3 3 + x6 x4 3 = 0
F8 ( X ) = −10 ≤ x1 , x2 , . . . , x6 ≤ 10.

⎪ x5 x13 + x6 x2 3 = 0



⎪ x x x + x6 x2 x4 = 0,
2 2
⎩ 5 1 32
x5 x3 x1 + x6 x4 x2 2 = 0

There is more than one exact solution to this system. Table 8 shows that the proposed
MSSO algorithm achieved the most accurate solution with a fitness value of 1.18 × 10−24 ,
and the PSO algorithm achieved second place with a fitness value of 5.26 × 10−7 . In
contrast, the rest of the algorithms recorded answers that differ significantly from the exact
solution. Further, NESs in problems 6–8 prove the flexibility of the proposed hybrid MSSO
as it remains efficient even in a wide interval [−10, 10].

Table 8. Comparison of different optimization algorithms for Problem 8.

MSSO HHO [9] PSO [49] ALO [10] BOA [11] EO [13] SSO [2]
x1 0.68279148724 0.52702319411 0.76960300904 0.28887548289 0.95829879077 0.26693676403 1.00511003439
x2 0.50647432076 0.29250343550 0.66834059064 0.24588295652 0.10377244360 0.73023242916 −0.14156714998
x3 −0.7306132937 0.84391409892 0.63852284443 −0.95725516399 0.20563151204 −0.96364982722 0.12921880541
x4 −0.8622550449 0.96128971140 −0.74385526431 0.96902915055 −0.98879741269 0.68370357562 0.99423873612
x5 3.8805276 × 10−19 −0.01763142313 −5.5341563 × 10−7 0.00262835607 −0.02586929684 −0.00260602535 0.01451788346
x6 −3.013005 × 10−19 −0.00227648751 −2.3175063 × 10−7 0.00282255517 0.01218071672 −0.00190637065 −0.00244565414
Fitness 1.18 × 10−24 0.020764231 5.26 × 10−7 0.001489 0.048684 0.002456 0.032031

162
Mathematics 2023, 11, 1473

The comparison results in all benchmarks confirm the hypothesis that we have men-
tioned in the first section; that is, that the hybridization of two algorithms inherits the
efficient merits of both algorithms (SSO and Newton’s methods). This can be seen by
looking at the comparison results between the MSSO and the original SSO, where the
MSSO has outperformed the original SSO in all selected benchmarks. The reason for this
remarkable performance is the use of Newton’s method as a local search, which strengthens
the hybrid’s capability to avoid the local optimum in Problems 1–5 (where MSSO has
obtained the exact solution), and significantly improves the obtained fitness values in
Problems 6-8. The comparisons indicate that the proposed hybrid algorithm MSSO has
avoided being trapped in the local optima in all problems, compared with the majority of
the other algorithms.

5. Results and Analysis


5.1. Stability and Consistency of MSSO
Table 9 shows the average fitness value of the MSSO and the other algorithms com-
pared in the previous benchmarks. This is when each problem is run 30 times to illustrate
the continuous efficiency and power of the proposed MSSO algorithm.

Table 9. The comparison results of the average 30-run solution for all problems.

MSSO HHO [9] PSO [49] ALO [10] BOA [11] EO [13] SSO [2]
Problem 1 2.2709 × 10−16 0.0022869 2.2927 × 10−6 6.2912 × 10−7 0.039573 9.3817 × 10−5 0.050354
Problem 2 4.7855 × 10−16 0.13913 0.037009 0.11703 0.12076 0.060332 0.27432
Problem 3 1.1842 × 10−16 0.038848 2.1189 × 10−12 7.5657 × 10−6 0.20288 3.3604 × 10−5 0.14209
Problem 4 1.1102 × 10−17 0.052119 0.0055764 0.042144 0.063878 0.015678 0.12467
Problem 5 0 2.706 × 10−9 0.011783 0.058917 0.0035033 0.011783 2.1635E-09
Problem 6 5.2147 × 10−17 0.37777 0.00092349 0.16493 0.36299 0.037007 0.56687
Problem 7 4.7872 × 10−9 4.4292 × 10−5 2.5874 × 10−6 3.4687 × 10−6 2.0904 × 10−5 2.3701 × 10−6 1.1393 × 10−5
Problem 8 0.010581 0.18797 0.01278 2.9582 0.13696 0.014989 0.11305
Mean (F-test) 1 5.375 2.625 4.375 5.5 3.375 5.625
Rank 1 5 2 4 6 3 7

According to Table 9, MSSO has surpassed all other compared algorithms. The average
fitness values of MSSO and the original SSO show a significant difference in all benchmarks.
Consequently, this improvement confirms the flexibility of the hybrid MSSO in seeking the
best solution without being entrapped by local optima. Furthermore, as shown in Table 9,
MSSO outperforms all of the other compared algorithms, particularly for problems 2, 4, 6,
and 8.
Additionally, the algorithm is considered consistent and stable if it maintains consis-
tency over 30 runs. The average of the solutions must, therefore, be the same as or very
close to the best solution in order to achieve consistency. It has been demonstrated in
this study that MSSO consistency has been maintained for all selected problems. More-
over, the average standard deviation achieved by each algorithm is shown in Table 10,
in which smaller values of standard deviation indicate more stability. The hybrid MSSO
demonstrated stable results in most of the selected problems.

Table 10. The average standard deviation for all problems.

MSSO HHO [9] PSO [49] ALO [10] BOA [11] EO [13] SSO [2]
Problem 1 7.7097 × 10−16 0.0035697 4.4735 × 10−6 8.3914 × 10−7 0.046935 0.00012725 0.030655
Problem 2 1.1592 × 10−16 0.062883 0.021589 0.069769 0.058383 0.058383 0.085942
Problem 3 3.4857 × 10−16 0.039902 3.889 × 10−12 8.6567 × 10−6 0.18297 5.8122 × 10−5 0.069194
Problem 4 6.0809 × 10−17 0.031155 0.0043532 0.039841 0.022589 0.045526 0.082401
Problem 5 0 9.7731 × 10−9 0.064539 0.13399 0.0026465 0.064539 4.3991 × 10−9
Problem 6 0 0.076223 0.00068543 0.061743 0.064055 0.038589 0.07177
Problem 7 6.8883 × 10−9 2.0068 × 10−5 4.9339 × 10−6 2.582 × 10−6 1.0476 × 10−5 2.8284 × 10−6 2.3389 × 10−6
Problem 8 0.057522 0.11493 0.0348 12.8391 0.051742 0.016544 0.050028
Mean (F-test) 1.5 5.25 2.875 4.625 4.875 3.75 4.875
Rank 1 7 2 4 5.5 3 5.5

163
Mathematics 2023, 11, 1473

Furthermore, the significance of MSSO improvements was examined using the statis-
tical t-test in Table 11. Improvements were considered significant if the p-value was less
than 0.05; otherwise, they were not. Results show that all algorithms have p-values lower
than 0.05 in all tested problems, except for HHO, which has a single value above 0.05 in
Problem 5. It is evident from this that MSSO has a higher level of reliability than competing
algorithms. Further, MSSO’s solutions are significantly more accurate than those of other
algorithms since the majority of its p-values are close to 0. The results demonstrate that the
MSSO is a robust search method capable of finding precise solutions. Moreover, it is able to
avoid local optimal traps and immature convergence.

Table 11. p-values for the fitness based on the t-test.

HHO [9] PSO [49] ALO [10] SSO [2] EO [13] BOA [11]
Problem 1 7.425 × 10−5 0.0032362 7.305 × 10−5 6.322 × 10−11 0.00053712 0.032655
Problem 2 1.396 × 10−8 0.0078581 0.00018975 7.512 × 10−17 0.0053228 0.002542
Problem 3 2.1404 × 10−8 0.00038876 1.7706 × 10−6 2.171 × 10−13 5.5596 × 10−5 0.000194
Problem 4 6.4406 × 10−10 5.0472 × 10−5 1.2494 × 10−7 6.948 × 10−11 0.013013 6.918 × 10−11
Problem 5 0.32558 9.7628 × 10−8 0.0012288 0.32563 0.001503 4.3991 × 10−6
Problem 6 1.8665 × 10−26 3.7433 × 10−5 1.1361 × 10−16 2.345 × 10−26 2.9061 × 10−10 2.855 × 10−22
Problem 7 3.7355 × 10−12 0.0044676 3.9777 × 10−9 7.247 × 10−21 0.00029067 2.3389 × 10−6
Problem 8 2.3479 × 10−12 0.049502 0.00014768 1.574 × 10−9 2.3043 × 10−6 0.000028

Moreover, one of the criteria that is considered when comparing algorithms is their
speed of convergence. Figure 4 indicates that MSSO enhanced the convergence speed of the
original SSO in all problems. It also shows that MSSO achieves the best solution with much
fewer iterations than the other algorithms. Consequently, the superiority of the proposed
MSSO is confirmed.
It is known that any optimization method has some constraints that slow down the
algorithm from finding the optimum solution or, in some cases, prevent it from finding the
solution. HHO, for instance, probably attains local optima instead of the optimal answer.
SSO quickly falls into a local minimum of systems of nonlinear equations, which consists
of a set of models [2]. PSO has some drawbacks, such as a lack of population variety and
the inability to balance local optima and global optima [54]. The EO method, on the other
hand, does not function well for large-scale situations [55].
The novel hybrid algorithm MSSO’s convergence speed is attributed to combining
Newton’s iterative method as a local search and the SSO algorithm. On the one hand, MSSO
benefits from the originality of Newton’s method, which was developed to find solutions to
nonlinear equation systems. On the other hand, SSO ensures appropriate initial solutions
for Newton’s method by employing search agents. Furthermore, Newton’s method features
a second-order of convergence, which implies that the scheme converges to approximately
two significant digits in each iteration [1]. Thus, the hybridization between Newton’s
method and the SSO algorithm inherits the merits from both sides to produce an efficient
algorithm that can overcome the main disadvantages [56,57].
It is worth noting that the default precision value of the variable in MATLAB was used
for all calculations in this study, which is 16 digits of precision. This precision is timesaving
compared with more significant digits. However, in some situations, this may impact the
outcome. In MATLAB, the function ”vpa” may be used to enhance variable precision.
Thus, increasing the number of digits can improve the accuracy of the findings, but this is
a time-consuming operation. More details and examples of this case can be seen in [40].
In this research, the use of ”vpa” has increased the accuracy of the results in Problem 5,
Problem 7, and Problem 8.

164
Mathematics 2023, 11, 1473

Figure 4. The convergence speed for the eight problems based on an average of 30 runs.

5.2. Comparison between MSSO and Newton’s Method


The effectiveness of MSSO is demonstrated by the correctness of the generated solu-
tions and its ability to avoid local optima compared to Newton’s method. Accordingly,

165
Mathematics 2023, 11, 1473

both strategies were examined for problems 1–4. Tables 12–15 compare the fitness values
achieved by MSSO and Newton’s method using three randomly chosen starting points. We
examined both strategies for comparison purposes at iteration 5, iteration 7, and iteration 10.
In addition, variables of 1200-digit precision in all selected problems were used to clarify
the solutions’ accuracy. If, as noted earlier, the number of digits is increased, the findings
may also improve.

Table 12. A comparison of Newton’s method and MSSO for Problem 1.

Newton Newton Newton


Iteration MSSO
x0 =(0,0) x0 =(1.5,0.5) x0 =(1,1)
5 1.14 × 10−17 1.17 × 10−24 3.01 × 10−35 0
7 2.08 × 10−70 2.26 × 10−98 9.97 × 10−141 0
10 1.20 × 10−562 2.34 × 10−786 3.33 × 10−1125 0

Table 13. A comparison of Newton’s method and MSSO for Problem 2.

Newton Newton Newton


Iteration MSSO
x0 =(1.5,2) x0 =(1,1.5) x0 =(2,2)
5 6.38 × 10−9 Diverge Diverge 0
7 2.61 × 10−37 Diverge Diverge 0
10 1.92 × 10−302 Diverge Diverge 0

Table 14. A comparison of Newton’s method and MSSO for Problem 3.

Newton Newton Newton


Iteration MSSO
x0 =(0,0,0) x0 =(1,0,0.5) x0 =(1,1,1)
5 4.68 × 10−4 Not Applicable 6.15 × 10−8 2.82 × 10−16
7 6.10 × 10−13 Not Applicable 1.67 × 10−28 8.09 × 10−57
10 7.68 × 10−96 Not Applicable 2.35 × 10−220 0

Table 15. A comparison of Newton’s method and MSSO for Problem 4.

Newton Newton Newton


Iteration MSSO
x0 =(0,0,0) x0 =(1,0,0.5) x0 =(1,1,1)
5 Not Applicable 1.72 × 10−17 1.45 × 10−35 1.12 × 10−45
7 Not Applicable 4.77 × 10−68 1.73 × 10−144 3.76 × 10−183
10 Not Applicable 1.85 × 10−538 4.22 × 10−1161 0

MSSO surpassed Newton’s approach in all of the chosen problems, as shown in


Tables 12–15. Newton’s method, like all other iterative methods, is extremely sensitive to
the starting answer x0 . Choosing an incorrect starting point can slow down the convergence
of Newton’s method (see Tables 12 and 14) or cause Newton’s method to diverge (see
Table 13). Further, a singularity in the Jacobian in Newton’s method’s denominator can be
caused by the improper selection of the initial point. The Jacobian’s inverse does not thus
exist. Therefore, it is impossible to utilize Newton’s approach (refer to Tables 14 and 15).
Tables 12–15 show a considerable improvement in MSSO outcomes compared with
Newton’s technique. The primary issue with Newton’s starting point has been addressed
by relying on 20 search agents at the early stages of the hybrid MSSO. This is rather than
picking one point as Newton does. The MSSO selects several random starting points, called
search agents, unlike Newton’s method. MSSO examines each search agent’s fitness value,
then chooses the search agent with the lowest fitness value as an initial guess. Selecting the
starting point in this manner is crucial for improving the accuracy of the answer.
The previous experiments show that the proposed MSSO outperforms Newton’s
method in selected problems. As opposed to Newton’s method, which normally starts

166
Mathematics 2023, 11, 1473

with one initial point, MSSO starts with 20 search agents. The superiority of the MSSO is
demonstrated by the accuracy of its solutions. In addition, the time required to reach the
convergence criteria is less in MSSO. Having 20 random initial solutions clearly requires
more time for Newton’s method. Therefore, this is another reason why hybridizing both
SSO and Newton’s method is better than depending on one of them.
Moreover, the speed of convergence towards the best solution is astonishing. MSSO
can choose the best initial point in a few iterations and move quickly toward the global
optima. Figure 5 shows the convergence speed of problems 1–4 for the first five iterations
on an average of 30 runs.

Figure 5. The convergence speed of problems 1–4 for five iterations based on an average of 30 runs.

To clarify the significant improvements of MSSO over Newton’s iterative method,


a comparison between Newton’s technique and MSSO for Problems 1, 2, 3, and 4 were
performed. Table 16 shows the CPU-time needed for Newton’s technique and MSSO to
attain the stopping criterion (ε ≤ 1 × 10−15 ).

Table 16. Comparing Newton’s method and MSSO in terms of average time (in seconds).

Newton MSSO
Problem
Initial Guess Iteration ε ≤ 1 × 10 −15 ε ≤ 1 × 10 −15
Problem 1 {0,0} 5 1.389 0.283
Problem 2 {1.5,1.5} 5 1.871 0.179
Problem 3 {1,1,1} 8 2.497 0.234
Problem 4 {0.5,0.5} 4 2.137 0.244

Based on the results, an apparent enhancement has been added to Newton’s method
by using the hybridized MSSO. The CPU-time needed to satisfy the selected stopping

167
Mathematics 2023, 11, 1473

limit is much better for MSSO than Newton’s method. Even though Newton’s method is
a part of the proposed MSSO, MSSO showed better results because of the mechanism of
SSO in selecting the best initial guess for Newton’s technique as a local search inside the
hybrid algorithm.
It is well known that choosing a starting point that is far from the root of the system
could negatively affect the convergence of Newton’s method. Therefore, since Newton’s
method is included in the MSSO, this could negatively affect MSSO’s convergence as well.
However, based on the mechanism of the MSSO, the algorithm randomly selects 20 agents
that are considered as initial points within a specific interval. In general, optimization
algorithms have more choices to start with compared to Newton’s method. Iterative
methods can benefit from hybridization in selecting initial points because optimization
algorithms can have many initial points. On the other hand, optimization algorithms can
benefit from the fast and accurate convergence of iterative methods.

6. Conclusions
In this work, a hybrid method known as MSSO was introduced for solving systems of
nonlinear equations using Newton’s iterative method as a local search for the Sperm Swarm
Optimization algorithm SSO. The main goal of the MSSO is to solve the problem of New-
ton’s method’s initial guess, the achievement of which results in a better selection of initial
points, enabling it to be applied to a wider variety of real-world applications. Moreover,
Newton’s scheme was used in MSSO as a local search, which improved the accuracy of the
tested solutions. In addition, the MSSO’s convergence speed is substantially improved.
Eight nonlinear systems of varying orders were utilized to illustrate the effectiveness
of the proposed MSSO. The novel MSSO was also compared to six well-known optimization
methods, including the original SSO, BOA, ALO, EO, HHO, and PSO. The Euclidean norm
has been utilized as a fitness function in all benchmarks. According to the results, MSSO
outperforms all other compared algorithms in four metrics: fitness value, solution accuracy,
stability, and speed of convergence. In addition, the consistency of the MSSO is confirmed
by running the methods thirty times. Additionally, the standard deviation showed that
MSSO was the most stable optimization algorithm.
Additionally, we compared the performance of MSSO and Newton’s method on four
problems from the benchmarks. Across all four datasets, the MSSO outperformed Newton’s
method. The MSSO method also overcomes some of Newton’s scheme’s limitations, such
as divergence and selection of initial guesses.
Future work can address some related issues, such as how the suggested method
performs against common optimization benchmarks. Future research will also focus
on solving nonlinear equations arising from real-world applications, such as Burgers’
Equation. In addition, future work needs to address the efficiency of the proposed algorithm
when solving big systems. Finally, the use of a derivative-free iterative method instead
of Newton’s method reduces the computational complexity resulting from the need to
evaluate Newton’s method in each iteration and is an interesting topic that needs to be
focused on in the future.

Author Contributions: Conceptualization, O.S.S. and R.S.; methodology, O.S.S., R.S. and H.S.; vali-
dation, K.A. and I.H.; formal analysis, O.S.S. and R.S.; investigation, O.S.S. and R.S.; resources, O.S.S.,
R.S. and H.S.; data curation, O.S.S. and R.S.; writing—original draft preparation, O.S.S., R.S. and
H.S.; writing—review and editing, K.A. and I.H.; visualization, R.S.; supervision, O.S.S., R.S. and
I.H.; project administration, O.S.S., R.S. and I.H.; funding acquisition, I.H. All authors have read and
agreed to the published version of the manuscript.
Funding: This research and the APC were funded by Universiti Kebangsaan Malaysia, grant number
DIP-2021-018.
Data Availability Statement: The data that support the findings of this study are available on request
from the corresponding author, Sihwail, R.

168
Mathematics 2023, 11, 1473

Conflicts of Interest: The authors declare that there is no conflict of interest regarding the publication
of this paper.

References
1. Traub, J.F. Iterative Methods for the Solution of Equations; Prentice-Hall: Englewood Cliffs, NJ, USA, 1964.
2. Shehadeh, H.A. A Hybrid Sperm Swarm Optimization and Gravitational Search Algorithm (HSSOGSA) for Global Optimization.
Neural Comput. Appl. 2021, 33, 11739–11752. [CrossRef]
3. Gupta, S. Enhanced Harmony Search Algorithm with Non-Linear Control Parameters for Global Optimization and Engineering
Design Problems. Eng. Comput. 2022, 38, 3539–3562. [CrossRef]
4. Zhu, F.; Chen, D.; Zou, F. A Novel Hybrid Dynamic Fireworks Algorithm with Particle Swarm Optimization. Soft Comput. 2021,
25, 2371–2398. [CrossRef]
5. Shehadeh, H.A.; Ahmedy, I.; Idris, M.Y.I. Empirical Study of Sperm Swarm Optimization Algorithm. Adv. Intell. Syst. Comput.
2018, 869, 1082–1104. [CrossRef]
6. Shehadeh, H.A.; Ahmedy, I.; Idris, M.Y.I. Sperm Swarm Optimization Algorithm for Optimizing Wireless Sensor Network
Challenges. In Proceedings of the 6th International Conference on Communications and Broadband Networking, Singapore,
24–26 February 2018; pp. 53–59. [CrossRef]
7. Shehadeh, H.A.; Idris, M.Y.I.; Ahmedy, I.; Ramli, R.; Noor, N.M. The Multi-Objective Optimization Algorithm Based on
Sperm Fertilization Procedure (MOSFP) Method for Solving Wireless Sensor Networks Optimization Problems in Smart Grid
Applications. Energies 2018, 11, 97. [CrossRef]
8. Id, H.A.S.; Yamani, M.; Idris, I.; Ahmedy, I. Multi-Objective Optimization Algorithm Based on Sperm Fertilization Procedure
(MOSFP). Symmetry 2017, 9, 241. [CrossRef]
9. Heidari, A.A.; Mirjalili, S.; Faris, H.; Aljarah, I.; Mafarja, M.; Chen, H. Harris Hawks Optimization: Algorithm and Applications.
Future Gener. Comput. Syst. 2019, 97, 849–872. [CrossRef]
10. Mirjalili, S. The Ant Lion Optimizer. Adv. Eng. Softw. 2015, 83, 80–98. [CrossRef]
11. Arora, S.; Singh, S. Butterfly Optimization Algorithm: A Novel Approach for Global Optimization. Soft Comput. 2018, 23, 715–734.
[CrossRef]
12. Zhang, Y.; Wang, Y.H.; Gong, D.W.; Sun, X.Y. Clustering-Guided Particle Swarm Feature Selection Algorithm for High-
Dimensional Imbalanced Data With Missing Values. IEEE Trans. Evol. Comput. 2022, 26, 616–630. [CrossRef]
13. Faramarzi, A.; Heidarinejad, M.; Stephens, B.; Mirjalili, S. Equilibrium Optimizer: A Novel Optimization Algorithm. Knowl.-Based
Syst. 2020, 191, 105190. [CrossRef]
14. Holland, J.H. Genetic Algorithms. Sci. Am. 1992, 267, 66–72. [CrossRef]
15. Liu, K.; Hu, X.; Yang, Z.; Xie, Y.; Feng, S. Lithium-Ion Battery Charging Management Considering Economic Costs of Electrical
Energy Loss and Battery Degradation. Energy Convers. Manag. 2019, 195, 167–179. [CrossRef]
16. Liu, K.; Zou, C.; Li, K.; Wik, T. Charging Pattern Optimization for Lithium-Ion Batteries with an Electrothermal-Aging Model.
IEEE Trans. Ind. Inform. 2018, 14, 5463–5474. [CrossRef]
17. Zhou, R.H.; Li, Y.G. An Improve Cuckoo Search Algorithm for Solving Nonlinear Equation Group. Appl. Mech. Mater. 2014,
651–653, 2121–2124. [CrossRef]
18. Ariyaratne, M.K.A.; Fernando, T.G.I.; Weerakoon, S. Solving Systems of Nonlinear Equations Using a Modified Firefly Algorithm
(MODFA). Swarm Evol. Comput. 2019, 48, 72–92. [CrossRef]
19. Ren, H.; Wu, L.; Bi, W.; Argyros, I.K. Solving Nonlinear Equations System via an Efficient Genetic Algorithm with Symmetric and
Harmonious Individuals. Appl. Math. Comput. 2013, 219, 10967–10973. [CrossRef]
20. Chang, W. der An Improved Real-Coded Genetic Algorithm for Parameters Estimation of Nonlinear Systems. Mech. Syst. Signal
Process. 2006, 20, 236–246. [CrossRef]
21. Grosan, C.; Abraham, A. A New Approach for Solving Nonlinear Equations Systems. IEEE Trans. Syst. Man Cybern. Part ASyst.
Hum. 2008, 38, 698–714. [CrossRef]
22. Jaberipour, M.; Khorram, E.; Karimi, B. Particle Swarm Algorithm for Solving Systems of Nonlinear Equations. Comput. Math.
Appl. 2011, 62, 566–576. [CrossRef]
23. Mo, Y.; Liu, H.; Wang, Q. Conjugate Direction Particle Swarm Optimization Solving Systems of Nonlinear Equations. Comput.
Math. Appl. 2009, 57, 1877–1882. [CrossRef]
24. Sihwail, R.; Omar, K.; Ariffin, K.A.Z. An Effective Memory Analysis for Malware Detection and Classification. Comput. Mater.
Contin. 2021, 67, 2301–2320. [CrossRef]
25. Sihwail, R.; Omar, K.; Zainol Ariffin, K.; al Afghani, S. Malware Detection Approach Based on Artifacts in Memory Image and
Dynamic Analysis. Appl. Sci. 2019, 9, 3680. [CrossRef]
26. Jadon, S.S.; Tiwari, R.; Sharma, H.; Bansal, J.C. Hybrid Artificial Bee Colony Algorithm with Differential Evolution. Appl. Soft
Comput. J. 2017, 58, 11–24. [CrossRef]
27. Jia, R.; He, D. Hybrid Artificial Bee Colony Algorithm for Solving Nonlinear System of Equations. In Proceedings of
the 2012 8th International Conference on Computational Intelligence and Security, CIS 2012, Guangzhou, China, 17–18
November 2012; pp. 56–60.

169
Mathematics 2023, 11, 1473

28. Aydilek, İ.B. A Hybrid Firefly and Particle Swarm Optimization Algorithm for Computationally Expensive Numerical Problems.
Appl. Soft Comput. J. 2018, 66, 232–249. [CrossRef]
29. Nasr, S.; El-Shorbagy, M.; El-Desoky, I.; Hendawy, Z.; Mousa, A. Hybrid Genetic Algorithm for Constrained Nonlinear Optimiza-
tion Problems. Br. J. Math. Comput. Sci. 2015, 7, 466–480. [CrossRef]
30. Abualigah, L.M.; Khader, A.T.; Hanandeh, E.S.; Gandomi, A.H. A Novel Hybridization Strategy for Krill Herd Algorithm Applied
to Clustering Techniques. Appl. Soft Comput. J. 2017, 60, 423–435. [CrossRef]
31. El-Shorbagy, M.A.; Mousa, A.A.A.; Fathi, W. Hybrid Particle Swarm Algorithm for Multiobjective Optimization: Integrating Particle
Swarm Optimization with Genetic Algorithms for Multiobjective Optimization; Lambert Academic: Saarbrücken, Germany, 2011.
32. Goel, R.; Maini, R. A Hybrid of Ant Colony and Firefly Algorithms (HAFA) for Solving Vehicle Routing Problems. J. Comput. Sci.
2018, 25, 28–37. [CrossRef]
33. Turanoğlu, B.; Akkaya, G. A New Hybrid Heuristic Algorithm Based on Bacterial Foraging Optimization for the Dynamic Facility
Layout Problem. Expert Syst. Appl. 2018, 98, 93–104. [CrossRef]
34. Skoullis, V.I.; Tassopoulos, I.X.; Beligiannis, G.N. Solving the High School Timetabling Problem Using a Hybrid Cat Swarm
Optimization Based Algorithm. Appl. Soft Comput. J. 2017, 52, 277–289. [CrossRef]
35. Chen, X.; Zhou, Y.; Tang, Z.; Luo, Q. A Hybrid Algorithm Combining Glowworm Swarm Optimization and Complete 2-Opt
Algorithm for Spherical Travelling Salesman Problems. Appl. Soft Comput. J. 2017, 58, 104–114. [CrossRef]
36. Marichelvam, M.K.; Tosun, Ö.; Geetha, M. Hybrid Monkey Search Algorithm for Flow Shop Scheduling Problem under Makespan
and Total Flow Time. Appl. Soft Comput. J. 2017, 55, 82–92. [CrossRef]
37. Karr, C.L.; Weck, B.; Freeman, L.M. Solutions to Systems of Nonlinear Equations via a Genetic Algorithm. Eng. Appl. Artif. Intell.
1998, 11, 369–375. [CrossRef]
38. Luo, Y.Z.; Yuan, D.C.; Tang, G.J. Hybrid Genetic Algorithm for Solving Systems of Nonlinear Equations. Chin. J. Comput. Mech.
2005, 22, 109–114.
39. Luo, Y.Z.; Tang, G.J.; Zhou, L.N. Hybrid Approach for Solving Systems of Nonlinear Equations Using Chaos Optimization and
Quasi-Newton Method. Appl. Soft Comput. J. 2008, 8, 1068–1073. [CrossRef]
40. Sihwail, R.; Said Solaiman, O.; Omar, K.; Ariffin, K.A.Z.; Alswaitti, M.; Hashim, I. A Hybrid Approach for Solving Systems of
Nonlinear Equations Using Harris Hawks Optimization and Newton’s Method. IEEE Access 2021, 9, 95791–95807. [CrossRef]
41. Sihwail, R.; Said Solaiman, O.; Zainol Ariffin, K.A. New Robust Hybrid Jarratt-Butterfly Optimization Algorithm for Nonlinear
Models. J. King Saud Univ.—Comput. Inf. Sci. 2022, 34 Pt A, 8207–8220. [CrossRef]
42. Sihwail, R.; Omar, K.; Ariffin, K.A.Z.; Tubishat, M. Improved Harris Hawks Optimization Using Elite Opposition-Based Learning
and Novel Search Mechanism for Feature Selection. IEEE Access 2020, 8, 121127–121145. [CrossRef]
43. Said Solaiman, O.; Hashim, I. Efficacy of Optimal Methods for Nonlinear Equations with Chemical Engineering Applications.
Math. Probl. Eng. 2019, 2019, 1728965. [CrossRef]
44. Said Solaiman, O.; Hashim, I. An Iterative Scheme of Arbitrary Odd Order and Its Basins of Attraction for Nonlinear Systems.
Comput. Mater. Contin. 2020, 66, 1427–1444. [CrossRef]
45. Said Solaiman, O.; Hashim, I. Optimal Eighth-Order Solver for Nonlinear Equations with Applications in Chemical Engineering.
Intell. Autom. Soft Comput. 2021, 27, 379–390. [CrossRef]
46. Said Solaiman, O.; Hashim, I. Two New Efficient Sixth Order Iterative Methods for Solving Nonlinear Equations. J. King Saud
Univ. Sci. 2019, 31, 701–705. [CrossRef]
47. Said Solaiman, O.; Ariffin Abdul Karim, S.; Hashim, I. Dynamical Comparison of Several Third-Order Iterative Methods for
Nonlinear Equations. Comput. Mater. Contin. 2021, 67, 1951–1962. [CrossRef]
48. Adam, S.P.; Alexandropoulos, S.A.N.; Pardalos, P.M.; Vrahatis, M.N. No Free Lunch Theorem: A Review. Springer Optim. Its Appl.
2019, 145, 57–82. [CrossRef]
49. Kennedy, J.; Eberhart, R. Particle Swarm Optimization. In Proceedings of the ICNN’95—International Conference on Neural
Networks, Perth, WA, Australia, 27–1 December 1995; pp. 1942–1948. [CrossRef]
50. El-Shorbagy, M.A.; El-Refaey, A.M. Hybridization of Grasshopper Optimization Algorithm with Genetic Algorithm for Solving
System of Non-Linear Equations. IEEE Access 2020, 8, 220944–220961. [CrossRef]
51. Wang, X.; Li, Y. An Efficient Sixth-Order Newton-Type Method for Solving Nonlinear Systems. Algorithms 2017, 10, 45. [CrossRef]
52. Verschelde, J.; Verlinden, P.; Cools, R. Homotopies Exploiting Newton Polytopes for Solving Sparse Polynomial Systems. SIAM J.
Numer. Anal. 1994, 31, 915–930. [CrossRef]
53. van Hentenryck, P.; Mcallester, D.; Kapur, D. Solving Polynomial Systems Using a Branch and Prune Approach. SIAM J. Numer.
Anal. 1997, 34, 797–827. [CrossRef]
54. Ridha, H.M.; Heidari, A.A.; Wang, M.; Chen, H. Boosted Mutation-Based Harris Hawks Optimizer for Parameters Identification
of Single-Diode Solar Cell Models. Energy Convers. Manag. 2020, 209, 112660. [CrossRef]
55. Elgamal, Z.M.; Yasin, N.M.; Sabri, A.Q.M.; Sihwail, R.; Tubishat, M.; Jarrah, H. Improved Equilibrium Optimization Algorithm
Using Elite Opposition-Based Learning and New Local Search Strategy for Feature Selection in Medical Datasets. Computation
2021, 9, 68. [CrossRef]

170
Mathematics 2023, 11, 1473

56. Shehadeh, H.A.; Mustafa, H.M.J.; Tubishat, M. A Hybrid Genetic Algorithm and Sperm Swarm Optimization (HGASSO) for
Multimodal Functions. Int. J. Appl. Metaheuristic Comput. 2021, 13, 1–33. [CrossRef]
57. Shehadeh, H.A.; Shagari, N.M. Chapter 24: A Hybrid Grey Wolf Optimizer and Sperm Swarm Optimization for Global
Optimization. In Handbook of Intelligent Computing and Optimization for Sustainable Development; John Wiley & Sons, Inc.: Hoboken,
NJ, USA, 2022; pp. 487–507. [CrossRef]

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual
author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to
people or property resulting from any ideas, methods, instructions or products referred to in the content.

171
MDPI
St. Alban-Anlage 66
4052 Basel
Switzerland
www.mdpi.com

Mathematics Editorial Office


E-mail: [email protected]
www.mdpi.com/journal/mathematics

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are
solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s).
MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from
any ideas, methods, instructions or products referred to in the content.
Academic Open
Access Publishing

mdpi.com ISBN 978-3-0365-9215-2

You might also like