Numerical Methods For Solving Nonlinear Equations
Numerical Methods For Solving Nonlinear Equations
Edited by
Maria Isabel Berenguer and Manuel Ruiz Galán
mdpi.com/journal/mathematics
Numerical Methods for Solving
Nonlinear Equations
Numerical Methods for Solving
Nonlinear Equations
Editors
Maria Isabel Berenguer
Manuel Ruiz Galán
Editorial Office
MDPI
St. Alban-Anlage 66
4052 Basel, Switzerland
This is a reprint of articles from the Special Issue published online in the open access journal
Mathematics (ISSN 2227-7390) (available at: https://www.mdpi.com/si/mathematics/Numerical
Methods for Solving Nonlinear Equations).
For citation purposes, cite each article independently as indicated on the article page online and as
indicated below:
Lastname, A.A.; Lastname, B.B. Article Title. Journal Name Year, Volume Number, Page Range.
© 2023 by the authors. Articles in this book are Open Access and distributed under the Creative
Commons Attribution (CC BY) license. The book as a whole is distributed by MDPI under the terms
and conditions of the Creative Commons Attribution-NonCommercial-NoDerivs (CC BY-NC-ND)
license.
Contents
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
Gholamreza Farahmand, Taher Lotfi, Malik Zaka Ullah and Stanford Shateyi
Finding an Efficient Computational Solution for the Bates Partial Integro-Differential Equation
Utilizing the RBF-FD Scheme
Reprinted from: Mathematics 2023, 11, 1123, doi:10.3390/math11051123 . . . . . . . . . . . . . . . 137
Obadah Said Solaiman, Rami Sihwail, Hisham Shehadeh, Ishak Hashim and Kamal Alieyan
Hybrid Newton–Sperm Swarm Optimization Algorithm for Nonlinear Systems
Reprinted from: Mathematics 2023, 11, 1473, doi:10.3390/math11061473 . . . . . . . . . . . . . . . 151
v
About the Editors
Maria Isabel Berenguer
Maria Isabel Berenguer is an Associate Professor in the Department of Applied Mathematics at
the University of Granada, Spain; since 2015, she has also been a member of the university’s Institute
of Mathematics. She is a notable reviewer, and has published a several articles in well-known
journals with high JCR ranking. She has also actively participated in different projects with national
or regional funding. Additionally, she has trained young researchers and tutored international
doctoral students during their term abroad at the University of Granada. Currently, she is a
member of the Editorial Board of MDPI’s journal Mathematics. Her research interests include applied
mathematics, numerical analysis, fixed point theory and inverse problems.
vii
Preface
Many problems that emerge in areas such as medicine, biology, economics, finance, or
engineering can be described in terms of nonlinear equations or systems of such equations, which can
take different forms, from algebraic, differential, integral or integro-differential models to variational
inequalities or equilibrium problems. For this reason, nonlinear problems are one of the most
interesting fields of study in pure and applied mathematics.
However, there is a lack of direct methods that can facilitate the effective resolution of nonlinear
problems, and hence, research interest in their numerical treatment has further consolidated. This
Special Issue have collated manuscripts that address the recent advancements in the aforementioned
area. It contains 10 articles accepted for publication among the 24 submitted.
ix
mathematics
Article
On the Convergence of a New Family of Multi-Point
Ehrlich-Type Iterative Methods for Polynomial Zeros
Petko D. Proinov * and Milena D. Petkova
Faculty of Mathematics and Informatics, University of Plovdiv Paisii Hilendarski, 24 Tzar Asen,
4000 Plovdiv, Bulgaria; [email protected]
* Correspondence: [email protected]
Abstract: In this paper, we construct and study a new family of multi-point Ehrlich-type iterative
methods for approximating all the zeros of a uni-variate polynomial simultaneously. The first member
of this family is the two-point Ehrlich-type iterative method introduced and studied by Trićković
and Petković in 1999. The main purpose of the paper is to provide local and semilocal convergence
analysis of the multi-point Ehrlich-type methods. Our local convergence theorem is obtained by an
approach that was introduced by the authors in 2020. Two numerical examples are presented to show
the applicability of our semilocal convergence theorem.
Keywords: multi-point iterative methods; iteration functions; polynomial zeros; local convergence;
error estimates; semilocal convergence
MSC: 65H04
x ( k +1) = ϕ ( x ( k ) , x ( k −1) , . . . , x ( k − N ) ), k = 0, 1, 2, . . . ,
Academic Editors: Maria Isabel
Berenguer and Manuel Ruiz Galán where N is a fixed natural number, and x (0) , x (−1) , . . . , x (− N ) are N + 1 initial approxima-
tions. In the literature, there are multi-point iterative methods for finding a single zero of
Received: 17 June 2021 a nonlinear equation (see, e.g., [1–7]). This study is devoted to the multi-point iterative
Accepted: 8 July 2021 methods for approximating all the zeros of a polynomial simultaneously (see, e.g., [8–11]).
Published: 12 July 2021
Let us recall the two most popular iterative methods for simultaneous computation
of all the zeros of a polynomial f of degree n ≥ 2. These are Weierstrass’ method [12] and
Publisher’s Note: MDPI stays neutral
Ehrlich’s method [13].
with regard to jurisdictional claims in
Weierstrass’ method is defined by the following iteration:
published maps and institutional affil-
iations.
x ( k +1) = x ( k ) − W f ( x ( k ) ), k = 0, 1, 2, . . . , (1)
x ( k +1) = T ( x ( k ) ), k = 0, 1, 2, . . . , (3)
f ( xi )
Ti ( x ) = xi − (i = 1, . . . , n). (4)
1
f (x i) − f ( xi ) ∑
j =i
xi − x j
Ehrlich’s method has third order convergence. In 1973, this method was rediscovered
by Aberth [14]. In 1970, Börsch-Supan [15] constructed another third-order method for
simultaneous computing all the zeros of a polynomial. However in 1982, Werner [16]
proved that both Ehrlich’s and Börsch-Supan’s methods are identical.
In 1999, Trićković and Petković [9] constructed and studied a two-point version of
Ehrlich’s method. They√ proved that the two-point Ehrlich-type method has the order of
convergence r = 1 + 2.
In the present paper, we introduce an infinite sequence of multi-point Ehrlich-type
iterative methods. We note that the first member of this family of iterative methods is the
two-point Ehrlich-type method constructed in [9]. The main purpose of this paper is to pro-
vide a local and semilocal convergence analysis of the multi-point Ehrlich-type methods.
Our local convergence result (Theorem 2) contains the following information: con-
vergence domain; a priori and a posteriori error estimates; convergence order of every
method of the family. For instance, we prove that for a given natural number N, the order
of convergence of the Nth multi-point Ehrlich-type method is r = r ( N ), where r is the
unique positive solution of the equation
1 + 2( t + . . . + t N ) = t N +1 . (5)
It follows from this√result that the first iterative method ( N = 1) has the order of
convergence r (1) = 1 + 2 which coincides with the above mentioned result of Trićković
and Petković. We √note that each method of the new family has super-quadratic convergence
of order r ∈ [1 + 2, 3). The semilocal convergence result (Theorem 4) states a computer-
verifiable initial condition that guarantees fast convergence of the corresponding method
of the family.
The paper is structured as follows: In Section 2, we introduce the new family of
multi-point iterative methods. Section 3 contains some auxiliary results that underlie the
proofs of the main results. In Section 3, we present a local convergence result (Theorem 2)
for the iterative methods of the new family. This result contains initial conditions as well as
a priori and a posteriori error estimates. In Section 5, we provide a semilocal convergence
result (Theorem 4) with computer verifiable initial conditions. Section 6 provides two
numerical examples to show the applicability of our semilocal convergence theorem and
the convergence behavior of the proposed multi-point iterative methods. The paper ends
with a conclusion section.
2
Mathematics 2021, 9, 1640
In = {1, 2, . . . , n}.
where a0 ∈ K. It is obvious that f possesses a root vector in Kn if and only if it splits over K.
In the following definition, we introduce a real-value function of two vector variables
that plays an essential role in the present study.
f ( xi )
Φi ( x, y) = xi − (i = 1, . . . , n), (6)
1
f (x i) − f ( xi ) ∑ xi − y j
j = i
where DΦ is defined by
1
DΦ = ( x, y) ∈ Kn × Kn : x # y, f ( xi ) − f ( xi ) ∑ = 0 for i ∈ In . (7)
j = i
xi − y j
Theorem 1 (Petković√and Trićkovic [9]). The convergence order of the two-point Ehrlich-type
method (8) is r = 1 + 2 ≈ 2.414.
Φ ( N ) : D N ⊂ Kn × . . . × Kn → Kn
N +1
3
Mathematics 2021, 9, 1640
The sequence ( D N )∞
N = 0 of domains is defined also recursively by setting D0 = K and
n
⎧
⎨
D N = ( x, y, . . . , z) ∈ Kn × . . . × Kn : (y, . . . , z) ∈ D N −1 , x # Φ( N −1) (y, . . . , z)
⎩
N +1
⎫ (10)
1 ⎬
and f ( xi ) − f ( xi ) ∑ = 0 for i ∈ In .
j = i xi − Φ j
( N −1)
(y, . . . , z) ⎭
Note that in the case N = 1, the iterative method (11) coincides with the two-point
Ehrlich-type method (8).
In Section 4, we present a local convergence theorem (Theorem 2) for the methods (11)
with initial conditions that guarantee the convergence to a root vector of f . In the case
N = 1, this result extends Theorem 1 in several directions.
In Section 5, we present a semilocal convergence theorem (Theorem 4) for the fam-
ily (11), which is of practical importance.
3. Preliminaries
In this section, we present two basic properties of the iteration function Φ defined in
Definition 1, which play an important role in obtaining the main result in Section 4.
In what follows, we assume that Kn is endowed with the norm · ∞ defined by
u ∞ = max{|u1 |, . . . , |un |}
u v ⇔ ui ≤ vi for all i ∈ In .
4
Mathematics 2021, 9, 1640
f ( xi ) 1 1 − τi
−∑ = , (14)
f ( xi ) j = i
x i − y j x i − ξi
where τi ∈ K is defined by
yj − ξ j
τi = ( xi − ξ i ) ∑ ( xi − ξ j )( xi − y j )
. (15)
j = i
1 yj − ξ j 1 − τi
xi − ξ i j∑
= − = ,
= i
( x i − ξ j )( x i − y j ) x i − ξi
( n − 1) E ( x ) E ( y )
σ ( x, y) = (16)
(1 − E( x ))(1 − E( x ) − E(y)) − (n − 1) E( x ) E(y)
with domain
D = {( x, y) ∈ K × Kn : (1 − E( x ))(1 − E( x ) − E(y)) > (n − 1) E( x ) E(y) and E( x ) + E(y) < 1},
n
(17)
where E : Kn → R+ is defined by (13).
Lemma 3. Let f ∈ K[z] be a polynomial of degree n ≥ 2 with n simple zeros in K, and let ξ ∈ Kn
be a root vector of f . Suppose x, y ∈ Kn are two vectors such that ( x, y) ∈ D. Then:
(i) ( x, y) ∈ DΦ ;
(ii) Φ( x, y) − ξ σ ( x, y) x − ξ ;
(iii) E(Φ( x, y)) ≤ σ( x, y) E( x ),
where the functions Φ, E and σ are defined by (6), (13) and (16), respectively.
Proof. (i) According to (17), we have E( x ) + E(y) < 1. Then it follows from Lemma 1 that
| xi − y j | ≥ (1 − E( x )) d j (ξ ) > 0 (18)
1
f ( xi ) − f ( xi ) ∑ = 0 (19)
j = i
xi − y j
5
Mathematics 2021, 9, 1640
for i ∈ In . Let i ∈ In be fixed. We shall consider only the non-trivial case f ( xi ) = 0. In this
case, (19) is equivalent to
f ( xi ) 1
− ∑ = 0. (20)
f ( xi ) x − yj
j = i i
On the other hand, it follows from Lemma 2 that (20) is equivalent to τi = 1, where τi is
defined by (15). By Lemma 1 with y = ξ, we obtain
| xi − ξ j | ≥ (1 − E( x )) di (ξ ) > 0 (21)
|y j − ξ j |
|τi | ≤ | xi − ξ i | ∑ | xi − ξ j || xi − y j |
(22)
j = i
1 | xi − ξ i | |y j − ξ j |
≤
(1 − E( x ))(1 − E( x ) − E(y)) di (ξ ) ∑ d j (ξ )
j = i
( n − 1) E ( x ) E ( y )
≤ < 1.
(1 − E( x ))(1 − E( x ) − E(y))
|Φi ( x, y) − ξ i | ≤ σ( x, y) | xi − ξ i | (23)
for all i ∈ In . If xi = ξ i , then (23) holds trivially. Let xi = ξ i . Then, it follows from (21) that
f ( xi ) = 0. It follows from (6), (20) and (14) that
−1
f ( xi ) 1
Φi ( x, y) − ξ i = xi − ξ i − −∑ (24)
f ( xi ) x
j = i i
− yj
xi − ξ i τ
= xi − ξ i − = − i ( x i − ξ i ).
1 − τi 1 − τi
|τi | |τi |
|Φi ( x, y) − ξ i | = |x − ξi | ≤ |x − ξi |
|1 − τi | i 1 − |τi | i
( n − 1) E ( x ) E ( y )
≤ |x − ξi |
(1 − E( x ))(1 − E( x ) − E(y)) − (n − 1) E( x ) E(y) i
= σ( x, y) | xi − ξ i |.
Lemma 4. Let f ∈ K[z] be a polynomial of degree n ≥ 2 with n simple zeros in K, and let ξ ∈ Kn
be a root vector of f . Suppose x, y ∈ Kn are two vectors satisfying
2
max{ E( x ), E(y)} ≤ R = √ , (25)
3+ 8n − 7
6
Mathematics 2021, 9, 1640
E( x ) E(y)
(ii) σ( x, y) ≤ ;
R2
E ( x )2 E ( y )
(iii) E(Φ( x, y)) ≤ .
R2
Hence, it follows from (17) that ( x, y) ∈ D which proves the claim (i). It is easy to show
that R is the unique positive zero of the function φ, defined by
( n − 1) t2
φ(t) = . (27)
(1 − t)(1 − 2t) − (n − 1)t2
( n − 1) E ( x ) E ( y )
σ( x, y) ≤
(1 − R)(1 − 2R) − (n − 1) R2
( n − 1) R2 E( x ) E(y)
=
(1 − R)(1 − 2R) − (n − 1) R2 R2
E( x ) E(y) E( x ) E(y)
= φ( R) = , (28)
R2 R2
which proves (ii). The claim (iii) follows from Lemma 3 (iii) and claim (ii).
D N ={( x, y, . . . , z) : x ∈ Kn , (y, . . . , z) ∈ D N −1 ,
(1 − E( x ))(1 − E( x ) − E(Φ( N −1) (y, . . . , z))) > (n − 1) E( x ) E(Φ( N −1) (y, . . . , z)),
E( x ) + E(Φ( N −1) (y, . . . , z)) < 1},
7
Mathematics 2021, 9, 1640
Proof. Applying Lemma 1 with y = Φ( N −1) (y, . . . , z), we obtain (i). It follows from Defini-
tion 2, Lemma 3 (ii) and Definition 4 that
which proves (ii). From Definition 2, Lemma 3 (iii) and Definition 4, we obtain
Lemma 6. Let f ∈ K[z] be a polynomial of degree n ≥ 2 with n simple zeros in K, and let ξ ∈ Kn
be a root vector of f . Assume N ≥ 1 and x, y, . . . , t, z are N + 1 vectors in Kn such that
2
max{ E( x ), E(y), . . . , E(z)} ≤ R = √ , (30)
3+ 8n − 7
Proof. The proof goes by induction on N. In the case N = 1, Lemma 6 coincides with
Lemma 4. Suppose that for some N ≥ 1 the three claims of the lemma hold for every N + 1
vectors x, y, . . . , t, z ∈ Kn satisfying (30). Let x, y, . . . , t, z ∈ Kn be N + 2 vectors satisfying
( x, y, . . . , t, z) ∈ D N +1 , (31)
E ( x ) E ( y )2 . . . E ( t )2 E ( z )
σN +1 ( x, y, . . . , t, z) ≤ , (32)
R 2( N +1)
E ( x )2 E ( y )2 . . . E ( t )2 E ( z )
E(Φ( N +1) ( x, y, . . . , z) ≤ . (33)
R 2( N +1)
By induction assumption, we obtain (y, . . . , t, z) ∈ D N . By induction assumption (ii) and
(30), we obtain
8
Mathematics 2021, 9, 1640
The inequalities (34) and (35) yield ( x, y, . . . , z) ∈ D N +1 , which proves (31). From Defini-
tion 4, Lemma 4 (ii) and induction assumption (ii), we obtain
which proves (32). Claim (33) follows from Lemma 5 (ii) and claim (32).
Now we are ready to state the first main result in this paper.
max E( x (ν) ) ≤ R.
k− N ≤ ν≤k
which completes the induction. By Lemma 6 (ii), (40) and the definition of r, we obtain the
following estimate
2
σN ( x (k) , x (k−1) , . . . , x (k− N ) ) ≤ E( x (k) ) E( x (k−1) ) · · · E( x (k− N +1) ) E( x (k− N ) )/R2N
k+ N
k + N −1
k +1 2 r k N −1 k + N +1 − r k + N
λ = λr (1+2r+...+2r +r ) = λr
k N
≤ λr λr · · · λr .
9
Mathematics 2021, 9, 1640
x ( k +1) − ξ = Φ ( N ) ( x ( k ) , x ( k −1) , . . . , x ( k − N ) ) − ξ
σN ( x (k) , x (k−1) , . . . , x (k− N ) ) x (k) − ξ
k + N +1 − r k + N
λr x (k) − ξ ,
which proved the a posteriori estimate (37). The a priori estimate (38) can be easily proved
by induction using the estimate (37). Finally, the convergence of the sequence x (k) to a root
vector ξ follows from the estimate (38).
N 1 2 3 4 5 6 7 8 9 10
r( N ) 2.41421 2.83117 2.94771 2.98314 2.99446 2.99816 2.99939 2.99979 2.99993 2.99998
Each iterative method for finding simultaneously all roots of a polynomial f ∈ K[z] of
degree n ≥ 2 is an iterative method in Kn . It searches the roots ξ 1 , . . . , ξ n of the polynomial
f as a vector ξ = (ξ 1 , . . . , ξ n ) ∈ Kn . We have noticed in Section 2 that such a vector ξ is
called a root vector of f . Clearly, a polynomial can have more than one vector of the roots.
On the other hand, we can assume that the vector root is unique up to permutation.
A natural question arises regarding how to measure the distance of an approximation
x ∈ Kn to the zeros of a polynomial. The first step is to identify all vectors whose
components are the same up to permutation. Namely, we define a relation of equivalence
10
Mathematics 2021, 9, 1640
ρ( x, y) = min x − v ∞. (45)
v≡y
Note that ρ is a metric on the set of classes of equivalence. For simplicity, we shall identify
equivalence classes with their representatives.
In what follows, we consider the convergence in Kn with respect to the metric ρ.
Clearly, if a sequence x (k) in Kn is convergent to a vector x ∈ Kn with respect to the norm
· , then it converges to x with respect to the metric ρ. The opposite statement is not true
(see [11]).
Before formulating the main result, we recall a technical lemma.
Lemma 7 ([11]). Let x, ξ, ξ ∈ Kn be such that ξ ≡ ξ. Then there exists a vector x ∈ Kn such that
x ≡ x and
x − ξ x − ξ
=
d(ξ ) . (46)
d(ξ ) ∞
∞
Now we can formulate and prove the second main result of this paper.
where the function E f is defined by (42). Then the polynomial f has only simple zeros and the
multi-point Ehrlich-type iteration (11) is well defined and converges (with respect to the metric ρ)
to a root vector ξ of f with order of convergence r = r ( N ), where r is the unique positive solution
of the Equation (5).
where R is defined in (36). From Theorem 3 and the inequality (48), we conclude that f has
n simple zeros in K and that there exist root vectors ξ (0) , ξ (−1) , . . . ξ (− N ) ∈ Kn such that
x (k) − ξ (k)
max < R. (49)
− N ≤ k ≤ 0 d ( ξ (k )
∞
Let us put ξ (0) = ξ. Since ξ (0) , ξ (−1) , . . . ξ (− N ) are root vectors of f , then ξ (k) ≡ ξ for all
k = 0, −1, . . . , − N. It follows from Lemma 7 that there exist vectors x (0) , x (−1) , . . . , x (− N )
such that x (k) ≡ x (k) and (49) can be represented in the form
x (k) − ξ
max < R. (50)
− N ≤ k ≤ 0 d(ξ )
∞
It follows from Theorem 2 and inequality (50) that the multi-point iterative method (11)
with initial approximations x (0) , x (−1) , . . . , x (− N ) is well defined and converges to ξ. Hence,
the iteration (11) with initial approximations x (0) , x (−1) , . . . , x (− N ) converges with respect
to the metric ρ to the root vector of f .
11
Mathematics 2021, 9, 1640
then f has only simple zeros and the multi-point Ehrlich-type iteration (11) converges to a root
vector ξ of f .
The next result is an immediate consequence of Theorem 5.1 of [19]. It can be used as
a stopping criterion of a large class of iterative methods for approximating all zeros of a
polynomial simultaneously.
where the metric ρ is defined by (45), the function E f is defined by (42), and the function α is
defined by
α(t) = 2/(1 − (n − 2)t + (1 − (n − 2)t)2 − 4t). (54)
6. Numerical Examples
In this section, we present two numerical examples in order to show the applicability
of Theorem 4. Using the convergence criterion (51), we show that at the beginning of the
iterative process it can be proven numerically that the method is convergent under the
given initial approximations.
We apply the first four methods of the family (11) for calculating simultaneously all
the zeros of the selected polynomials. In each example, we calculate the smallest m > 0
that satisfies the convergence criterion (51). In accordance with Theorem 5, we use the
following stop criterion
where μn and ε k are defined by (52) and (53), respectively. To see the convergence behavior
of the methods, we show in the tables ε k+1 in addition to ε k .
In both examples, we take the same polynomials and initial approximations as in [11],
where the initial approximations are chosen quite randomly. This choice gives the oppor-
tunity to compare numerically the convergence behavior of the multi-point Ehrlich-type
methods with those of the multi-point Weierstrass-type methods which are studied in [11].
To present the calculated approximations of high accuracy, we implemented the
corresponding algorithms using the programming package Wolfram Mathematica 10.0
with multiple precision arithmetic.
12
Mathematics 2021, 9, 1640
with zeros −1, 3 and 5i (marked in blue in Figure 1). For N ∈ {1, 2, 3, 4}, the initial approximations
x (0) , x (−1) , . . . , x (− N ) in C3 are given in Table 2, where
The numerical results for Example 1 are presented in Table 3. For instance, for the
multi-point Ehrlich-type method (11) with N = 3, one can see that the convergence condi-
tion (51) is satisfied for m = 6 which guarantees that the considered method is convergent
with order of convergence r = 2.94771. The stopping criterion (55) is satisfied for k = 6
and at the sixth iteration the guaranteed accuracy is 10−16 . At the next seventh iteration,
the zeros of the polynomial f are calculated with accuracy 10−47 .
N m E f ( x(m) ) k E f ( x(k) ) εk ε k +1 r
1 4 0.036247 5 0.000039 9.06336 × 10−14 1.52321 × 10−32 2.41421
2 5 0.001957 5 0.001957 5.97453 × 10−17 5.45631 × 10−48 2.83117
3 6 0.076062 6 0.076062 2.46336 × 10−16 1.05897 × 10−47 2.94771
4 7 0.083021 7 0.083021 6.50717 × 10−17 3.80803 × 10−51 2.98314
with zeros 1, 2, 3, 4, 5, 6, 7 (marked in blue in Figure 2). For given N ∈ {1, 2, 3, 4}, the initial
approximations x (k) ∈ Cn (k = − N, . . . , −1, 0) are chosen with Aberth initial approximations
as follows:
(k) a π 3
xν = − 1 + Rk exp (iθν ), θν = 2ν − , ν = 1, . . . , n, (58)
n n 2
13
Mathematics 2021, 9, 1640
N m E f ( x(m) ) k E f ( x(k) ) εk ε k +1
1 18 0.00526 21 3.48544 × 10−10 4.73454 × 10−16 1.25695 × 10−38
2 6 0.01689 8 7.85062 × 10−6 4.23967 × 10−17 1.06658 × 10−48
3 7 0.01348 8 0.00038 1.12167 × 10−22 6.66169 × 10−65
4 14 0.03215 14 0.03215 6.61642 × 10−24 4.98369 × 10−71
14
Mathematics 2021, 9, 1640
Figure 2. Trajectories of the approximations for Example 2 (N = 3).
7. Conclusions
In this paper, we introduced a new family of multi-points iterative methods for
approximating all the zeros of a polynomial simultaneously. Let us note that the first
member of this family is the two-point Ehrlich-type√ method introduced in 1999 by Trićković
and Petković [9]. Its convergence order is r = 1 + 2 .
We provide a local and semilocal convergence analysis of the new iterative methods.
Our local convergence result (Theorem 2) contains the following information for each
method: convergence order; initial conditions that guarantee the convergence; a priori and
a posteriori error estimates. In√particular, each method of the family has super-quadratic
convergence of order r ∈ [1 + 2, 3). Our semilocal convergence result (Theorem 4) can be
used to numerically prove the convergence of each method for a given polynomial and
initial approximation.
Finally, we would like to note that the local convergence theorem was obtained by a
new approach developed in our previous article [11]. We believe that this approach can be
applied to obtain convergence results for other multi-point iterative methods.
Author Contributions: The authors contributed equally to the writing and approved the final
manuscript of this paper. Both authors have read and agreed to the published version of the manuscript.
Funding: This research was supported by the National Science Fund of the Bulgarian Ministry of
Education and Science under Grant DN 12/12.
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: Not applicable.
Conflicts of Interest: The authors declares no conflict of interest.
15
Mathematics 2021, 9, 1640
References
1. Stewart, G.W. On the convergence of multipoint iterations. Numer. Math. 1994, 68, 143–147. [CrossRef]
2. Abu-Alshaikh, I.; Sahin, A. Two-point iterative methods for solving nonlinear equations. Appl. Math. Comput. 2006, 182, 871–878.
[CrossRef]
3. Ignatova, B.; Kyurkchiev, N.; Iliev, A. Multipoint algorithms arising from optimal in the sense of Kung-Traub iterative procedures
for numerical solution of nonlinear equations. Gen. Math. Notes 2011, 11, 45–79.
4. Argyros, I.K.; González, D. Unified majorizing sequences for Traub-type multipoint iterative procedures. Numer. Algorithms
2013, 64, 549–565. [CrossRef]
5. Tiruneh, A.T.; Ndlela, W.N.; Nkambule, S.J. A two-point Newton method suitable for nonconvergent cases and with super-
quadratic convergence. Adv. Numer. Anal. 2013, 687382. [CrossRef]
6. Petković, M.S.; Neta, B.; Petković, L.D.; Džunić, J. Multipoint methods for solving nonlinear equations: A survey.
Appl. Math. Comput. 2014, 226, 635–660. [CrossRef]
7. Amat, S.; Argyros, I.; Busquier, S.; Hernández-Verón, M.A.; Rubio, M.J. A unified convergence analysis for some two-point type
methods for nonsmooth operators. Mathematics 2019, 7, 701. [CrossRef]
8. Kanno, S.; Kjurkchiev, N.V.; Yamamoto, T. On some methods for the simultaneous determination of polynomial zeros. Jpn. J.
Indust. Appl. Math. 1996, 13, 267–288. [CrossRef]
9. Tričković, S.B.; Petković, M.S. Multipoint methods for the determination of multiple zeros of a polynomial. Novi Sad J. Math.
1999, 29, 221–233.
10. Proinov, P.D.; Petkova, M.D. Convergence of the two-point Weierstrass root-finding method. Jpn. J. Indust. Appl. Math. 2014,
31, 279–292. [CrossRef]
11. Proinov, P.D.; Petkova, M.D. Local and semilocal convergence of a family of multi-point Weierstrass-type root-finding methods.
Mediterr. J. Math. 2020, 17, 107. [CrossRef]
12. Weierstrass, K. Neuer Beweis des Satzes, dass jede ganze rationale Function einer Veränderlichen dargestellt werden kann als ein
Product aus linearen Functionen derselben Veränderlichen. Sitzungsber. Königl. Akad. Wiss. Berlin 1891, II, 1085–1101. [CrossRef]
13. Ehrlich, L. A modified Newton method for polynomials. Commun. ACM 1967, 10, 107–108. [CrossRef]
14. Aberth, O. Iteration methods for finding all zeros of a polynomial simultaneously. Math. Comput. 1973, 27, 339–344. [CrossRef]
15. Börsch-Supan, W. Residuenabschätzung für Polynom-Nullstellen mittels Lagrange-Interpolation. Numer. Math. 1969/70,
14, 287–296. [CrossRef]
16. Werner, W. On the simultaneous determination of polynomial roots. In Iterative Solution of Nonlinear Systems of Equations
(Oberwolfach, 1982); Springer: Berlin, Germany; New York, NY, USA, 1982; Volume 953, pp. 188–202.
17. Proinov, P.D.; Vasileva, M.T. On the convergence of high-order Ehrlich-type iterative methods for approximating all zeros of a
polynomial simultaneously. J. Inequal. Appl. 2015, 2015, 336. [CrossRef]
18. Proinov, P.D. General convergence theorems for iterative processes and applications to the Weierstrass root-finding method.
J. Complex. 2016, 33, 118–144. [CrossRef]
19. Proinov, P.D. Relationships between different types of initial conditions for simultaneous root finding methods. Appl. Math. Lett.
2016, 52, 102–111. [CrossRef]
20. Proinov, P.D.; Ivanov, S.I. Convergence analysis of Sakurai-Torii-Sugiura iterative method for simultaneous approximation of
polynomial zeros. J. Comput. Appl. Math. 2019, 357, 56–70. [CrossRef]
21. Cholakov, S.I.; Vasileva, M.T. A convergence analysis of a fourth-order method for computing all zeros of a polynomial
simultaneously. J. Comput. Appl. Math. 2017, 321, 270–283. [CrossRef]
22. Ivanov, S.I. A unified semilocal convergence analysis of a family of iterative algorithms for computing all zeros of a polynomial
simultaneously. Numer. Algorithms 2017, 75, 1193–1204. [CrossRef]
16
mathematics
Article
Extrapolation Method for Non-Linear Weakly Singular Volterra
Integral Equation with Time Delay †
Li Zhang 1 , Jin Huang 1, *, Hu Li 2 and Yifei Wang 1
Abstract: This paper proposes an extrapolation method to solve a class of non-linear weakly singular
kernel Volterra integral equations with vanishing delay. After the existence and uniqueness of
the solution to the original equation are proved, we combine an improved trapezoidal quadrature
formula with an interpolation technique to obtain an approximate equation, and then we enhance
the error accuracy of the approximate solution using the Richardson extrapolation, on the basis of
the asymptotic error expansion. Simultaneously, a posteriori error estimate for the method is derived.
Some illustrative examples demonstrating the efficiency of the method are given.
Keywords: weakly singular kernel Volterra integral equation; proportional delay; improved trape-
zoidal quadrature formula; Richardson extrapolation; posteriori error estimate
which was discussed and transformed into the second-kind equivalent form
Publisher’s Note: MDPI stays neutral
t
with regard to jurisdictional claims in ∂k(t, s)
k (t, t)y(t) − qk(t, qt)y(qt) + y(s)ds = f (t),
published maps and institutional affil-
qt ∂t
iations.
Another source of motivation comes from the weakly singular delay VIE [7–9]
qt
K (t, s)
y(t) = f (t) + G (s, y(s))ds, t ∈ [0, 1],
0 (qt − s)λ
where λ ∈ (0, 1), K (t, s) is smooth and G (s, y(s)) is a smooth non-linear function. However,
there has not yet been investigated for the case where two integral terms are presented, the
first integral term is the weakly singular Volterra integral and the second integral terms not
only has weak singularity in the left endpoint but also its upper limit is a delay function,
which is challenging to calculate. It is the aim of this paper to fill this gap.
With theoretical and computational advances, some numerical methods for delay
differential equations [10–13], delay integral equations [14], delay integral–differential
equations [15–18], and fractional differential equations with time delay [19–22] have been
investigated widely. Here, we consider the following non-linear weakly singular kernel
VIE with vanishing delay
t θ (t)
y(t) = f (t) + sλ k1 (t, s; y(s))ds + sμ k2 (t, s; y(s))ds, t ∈ I, (1)
0 0
where θ (t) := qt, q ∈ (0, 1), λ, μ ∈ (−1, 0), f (t), k1 (t, s; y(s)), k2 (t, s; y(s)) are r (r ≥ 1,
r ∈ N) times continuously differentiable on I, D × R, Dθ × R, respectively,
D := {(t, s) : 0 ≤ s ≤ t ≤ T } and Dθ := {(t, s) : 0 ≤ s ≤ θ (t) ≤ θ ( T ), t ∈ I }. Ad-
ditionally, k i (t, s; y(s)) (i = 1, 2) satisfy the Lipschitz conditions with respect to y(s) on the
domains, respectively. That is, for fixed s and t, there are two positive constants L j ( j = 1, 2)
which are independent of s and t, such that
Then, Equation (1) possesses a unique solution (see Theorem 1). In this paper, we consider
the case where the solution is smooth.
Some numerical investigations of delay VIE have been conducted, such as discontinu-
ous Galerkin methods [23], collocation methods [24–26], the iterative numerical method [27],
and the least squares approximation method [28]. In [29], an h p version of the pseudo-
spectral method was analyzed, based on the variational form of a non-linear VIE with
vanishing variable delays. The algorithm increased the accuracy by refining the mesh
and/or increasing the degree of the polynomial. Mokhtary et al. [7] used a well-conditioned
Jacobi spectral Galerkin method for a VIE with weakly singular kernels and proportional de-
lay by solving sparse upper triangular non-linear algebraic systems. In [8], the Chebyshev
spectral-collocation method was investigated for the numerical solution of a class of weakly
singular VIEs with proportional delay. An error analysis showed that the approximation
method could obtain spectral accuracy. Zhang et al. [9] used some variable transformations
to change the weakly singular VIE with pantograph delays into new equations defined on
[−1, 1], and then combined it with the Jacobi orthogonal polynomial.
The extrapolation method has been used extensively [30,31]. We apply the extrapola-
tion method for the solution of the non-linear weakly singular kernel VIE with proportional
delay. We prove the existence of the solution to the original equation using an iterative
method, while uniqueness is demonstrated by the Gronwall integral inequality. We obtain
the approximate equation by using the quadrature method based on the improved trape-
zoidal quadrature formula, combining the floor technique and the interpolation technique.
Then, we solve the approximate equation through an iterative method. The existence
of the approximate solution is validated by analyzing the convergence of the iterative
sequence, while uniqueness is shown using a discrete Gronwall inequality. In addition,
we provide an analysis of the convergence of the approximate solution and obtain the
asymptotic expansion of the error. Based on the error asymptotic expansion, the Richardson
extrapolation method is applied to enhance the numerical accuracy of the approximate
solution. Furthermore, we obtain the posterior error estimate of the method. Numerical
18
Mathematics 2021, 9, 1856
experiments effectively support the theoretical analysis, and all the calculations can be
easily implemented.
This paper is organized as follows: In Section 2, the existence and uniqueness of the
solution for (1) are proven. The numerical algorithm is introduced in Section 3. In Section 4,
we prove the existence and uniqueness of the approximate solution. In Section 5, we
provide the convergence analysis of the approximate solution. In Section 6, we obtain
the asymptotic expansion of error, the corresponding extrapolation technique is used for
achieving high precision, and a posterior error estimate is derived. Numerical examples
are described in Section 7. Finally, we outline the conclusions of the paper in Section 8.
Lemma 1 ([32]). Let y(t) and g(t) be non-negative integrable functions, t ∈ [0, T ], A ≥ 0,
satisfying
t
y(t) ≤ A + g(s)y(s)ds,
0
then, for all 0 ≤ t ≤ T,
t
g(s)ds
y(t) ≤ Ae 0 .
Theorem 1. f (t), k1 (t, s; y(s)), k2 (t, s; y(s)) are r (r ≥ 1, r ∈ N) times continuously differ-
entiable on I, D × R, Dθ × R, respectively. Additionally, assume that k i (t, s; y(s)) (i = 1, 2)
satisfies the Lipschitz conditions (2), respectively. Then, Equation (1) has a unique solution.
y0 ( t ) = f ( t ),
t qt
yn (t) = f (t) + sλ k1 (t, s; yn−1 (s))ds + sμ k2 (t, s; yn−1 (s))ds.
0 0
(2L)k−1
|yk (t) − yk−1 (t)| ≤ b t(k−1)(γ+1) . (4)
( k − 1 ) ! ( γ + 1) k −1
Let n = k + 1; then,
19
Mathematics 2021, 9, 1856
t λ qt
y k +1 ( t ) − y k ( t ) ≤ s k1 t, s; yk (s) − k1 t, s; yk−1 (s) ds + sμ k2 t, s; yk (s) − k2 t, s; yk−1 (s) ds
0 0
t t
≤ L1 s λ y k ( s ) − y k −1 ( s ) + L2 sμ yk (s) − yk−1 (s)ds
0 0
t
≤ 2L s yk (s) − yk−1 (s)ds
γ
0
(2L)k
≤b t k ( γ +1) ,
k!(γ + 1)k
that is, the recurrence relation is established when n = k + 1, then the inequality (4) is
also established. Next, we prove that the sequence yn (t) is a Cauchy sequence,
y n ( t ) − y n + m ( t ) ≤ y n +1 ( t ) − y n ( t ) + y n +2 ( t ) − y n +1 ( t ) + · · · + y n + m ( t ) − y n + m −1 ( t )
(2L)n (2L)n+m−1
≤b t n ( γ +1) + · · · + b t(n+m−1)(γ+1)
n!(γ + 1) n ( n + m − 1) ! ( γ + 1) n + m −1
n + m +1
2L i i(γ+1) 1
≤b ∑ (
γ+1
)T
i!
.
i=n
∞
The term ∑ ( γ2L i i (γ+1) 1 is convergent, so the Cauchy sequence { y }
+1 ) T i! n n∈N is con-
i =0
vergent uniformly to y(t). Thus, y(t) is the solution to Equation (1), the existence
is proved.
• Case II. For 1 < s ≤ t ≤ T, the process is similar. Let γ
= max{λ, μ}, when n = 1,
tγ+1
|y2 (t) − y1 (t)| ≤ 2Lb . (5)
+1
γ
(2L)k−1
|yk (t) − yk−1 (t)| ≤ b t(k−1)(γ+1) . (6)
+ 1) k −1
( k − 1) ! ( γ
(2L)k
y k +1 ( t ) − y k ( t ) ≤ b tk(γ+1) ,
k!(γ + 1) k
i.e., the recurrence relation is established when n = k + 1, such that the inequality (6)
is also established. For the sequence yn (t),
n + m +1
yn (t) − yn+m (t) ≤ b ∑ ( 2L )i T i(γ+1) 1 .
i=n
+1
γ i!
∞
Since the term ∑ ( γ2L i i (γ
+1) 1 is convergent, so the Cauchy sequence { y }
+1 ) T i! n n∈N is
i =0
convergent uniformly to y(t). Thus, y(t) is the solution to Equation (1), the existence
is proved.
20
Mathematics 2021, 9, 1856
Now, we prove that the solution to Equation (1) is unique. Let y(t) and v(t) be two dis-
tinct solutions to Equation (1), and denote the difference between them by
w(t) = |y(t) − v(t)|. We obtain
t qt
w(t) = sλ k1 t, s; y(s) − k1 t, s; v(s) ds + sμ k2 t, s; y(s) − k2 t, s; v(s) ds
0 0
t qt
λ
≤ s k1 t, s; y(s) − k1 t, s; v(s) ds + sμ k2 t, s; y(s) − k2 t, s; v(s) ds
0 0
t qt
λ μ
≤ L1 s w(s)ds + L2 s w(s)ds
0 0
t
≤ ( Lsλ + Lsμ )w(s)ds.
0
Let g(s) = Lsλ + Lsμ , then g(s) is a non-negative integrable function, according to
Lemma 1. We obtain w(t) = 0, i.e., y(t) = v(t), the solution to Equation (1) is unique.
β (1 − β )
u(z) = βu( x ) + (1 − β)u(y) − ( x − y)2 u (z) + O(( x − y)3 ). (7)
2
u( x ) = u( βx + (1 − β) x )
= u( βx + (1 − β)y + (1 − β)( x − y))
= u(z + (1 − β)( x − y)) (8)
(1 − β )2
= u(z) + (1 − β)( x − y)u (z) + ( x − y)2 u (z) + O(( x − y)3 ).
2
Similarly, the Taylor expansion of function u(y) at point z is
β2
u(y) = u(z − β( x − y)) = u(z) − β( x − y)u (z) + ( x − y)2 u (z) + O(( x − y)3 ), (9)
2
combining (8) with (9), the proof is completed.
(b− a)
Lemma 3 ([33,34]). Let g(t) ∈ C2r [ a, b] ( r ≥ 1, r ∈ N ), G ( t ) = ( b − t ) λ g ( t ), h = N ,
b
and tk = a + kh for k = 0, · · · , N, as for the integral a G (t)dt. Then, the error of the modified
trapezoidal integration rule
N −1
h
TN ( G ) = G (t0 ) + h ∑ G (tk ) − ζ (−λ) g(b)h1+λ , (10)
2 j =1
21
Mathematics 2021, 9, 1856
r −1
r −1
B2j (2j−1) 2
g ( j ) ( b ) h j + λ +1
EN ( G ) = ∑ (2j)!
G ( a)h2j + ∑ (−1) j ζ (−λ − j)
( j!)
+ O(h2r ), (11)
j =1 j =1
where −1 < λ < 0, ζ is the Riemann–Zeta function and B2j represents the Bernoulli numbers.
= f (ti ) + I1 + I2 + I3 ,
where [qi ] denotes the maximum integer less than qi. According to Lemma 3, we have
t i −1 h
i
I1 = sλ k1 ti , s; y(s) ds ≈ −ζ (−λ)k1 ti , t0 ; y(t0 ) h1+λ + h ∑ tλk k1 ti , tk ; y(tk ) + tiλ k1 ti , ti ; y(ti ) . (13)
0 k =1
2
For I2 and I3 , there are two cases.
• Case I. If [qi ] = 0, then
I2 = 0;
qt qt (14)
i
I3 = sμ k2 ti , s; y(s) ds ≈ −ζ (−μ)(qti )1+μ k2 ti , t0 ; y(t0 ) + i (qti )μ k2 ti , qti ; y(qti ) .
0 2
qti − t[qi] μ
I3 ≈ t[qi] k2 ti , t[qi] ; y(t[qi] ) + (qti )μ k2 ti , qti ; y(qti ) . (15)
2
y(qti ) can be represented by linear interpolation of the adjacent points y(t[qi] ) and y(t[qi]+1 ).
For the node ti = ih, i = 0, 1, · · · , N, since [qi ] ≤ qi ≤ [qi ] + 1, we obtain
t[qi] ≤ qti ≤ t[qi]+1 ; according to Lemma 2, there exists β i ∈ [0, 1] such that
qti = β i t[qi] + (1 − β i )t[qi]+1 . The value of β i = 1 + [qi ] − qi can be calculated easily.
Then, the approximate expression of y(qti ) is
qti − t[qi] μ
I3 ≈ t[qi] k2 ti , t[qi] ; y(t[qi] ) + (qti )μ k2 ti , qti ; β i y(t[qi] ) + (1 − β i )y(t[qi]+1 ) . (17)
2
The approximation equations are as follows
22
Mathematics 2021, 9, 1856
y0 = f ( t0 );
i −1 h
yi ≈ f (ti ) − ζ (−λ)k1 ti , t0 ; y0 h1+λ + h ∑ tλk k1 ti , tk ; yk + tiλ k1 ti , ti ; yi (18)
k =1
2
1+ μ
qti
− ζ (−μ)(qti ) k2 ti , t0 ; y0 + (qti )μ k2 ti , qti ; β i y[qi] + (1 − β i )y[qi]+1 .
2
y0 = f ( t0 );
i −1 h
yi ≈ f (ti ) − ζ (−λ)k1 ti , t0 ; y0 h1+λ + h ∑ tλk k1 ti , tk ; yk + tiλ k1 ti , ti ; yi
k =1
2
(19)
h μ
− ζ (−μ)h1+μ k2 ti , t0 ; y0 + δi + t[qi] k2 ti , t[qi] ; y[qi]
2
qti − t[qi] μ
+ t[qi] k2 ti , t[qi] ; y[qi] + (qti )μ k2 ti , qti ; β i y[qi] + (1 − β i )y[qi]+1 ,
2
where ⎧
⎪
⎨0, [qi ] = 1,
δi ≈ [qi ]−1
μ
⎪
⎩h ∑ t k k 2 ti , t k ; y k , [qi ] ≥ 2.
k =1
y0 = f ( t0 );
i −1 h
yim+1 ≈ f (ti ) − ζ (−λ)k1 ti , t0 ; ỹ0 h1+λ + h ∑ tλk k1 ti , tk ; ỹk + tiλ k1 ti , ti ; yim (20)
k =1
2
1+ μ
qti +1
− ζ (−μ)(qti ) k2 ti , t0 ; ỹ0 + (qti )μ k2 ti , qti ; β i ỹ[qi] + (1 − β i )ym
[qi ]+1
.
2
y0 = f ( t0 );
i −1
h
yim+1 ≈ f (ti ) − ζ (−λ)k1 (ti , t0 ; ỹ0 )h1+λ + h ∑ tλk k1 (ti , tk ; ỹk ) + 2 tiλ k1 (ti , ti ; yim )
k =1
(21)
h μ
− ζ (−μ)h1+μ k2 ti , t0 ; ỹ0 + δi + t[qi] k2 ti , t[qi] ; ỹ[qi]
2
qti − t[qi] μ +1
+ t[qi] k2 (ti , t[qi] ; ỹ[qi] ) + (qti )μ k2 ti , qti ; β i ỹ[qi] + (1 − β i )ym
[qi ]+1
,
2
where
0, [qi ] = 1,
δi ≈ [qi ]−1 μ
h ∑k=1 tk k2 ti , tk ; ỹk , [qi ] ≥ 2.
23
Mathematics 2021, 9, 1856
n −1
wn ≤ h ∑ Bk wk + A, 0 ≤ n ≤ N, (22)
k =1
Theorem 2. Let f (t), k1 (t, s; y(s)), k2 (t, s; y(s)) are four times continuously differentiable on
I, D × R, Dθ × R, respectively. Additionally, y(t) has continuous partial derivatives up to 3 on
I and k i (t, s; y(s)) (i = 1, 2) satisfy Lipschitz conditions (2). Assume that h is sufficiently small,
then the solution to Equation (21) exists and is unique.
Proof. We discuss the existence of the approximate solution under two cases.
• Case I. When [qi ] = 0,
m +1
m h λ
m −1
y − y = t k 1 ( t i , t i ; y m
) − k 1 ( t i , t i ; y )
i i i i i
2
h
≤ L1 tiλ yim − yim−1 .
2
When h is sufficiently small, such that L1 2h tiλ ≤ 12 , then |yim+1 − yim | ≤ 12 |yim − yim−1 |
holds. Therefore, the iterative algorithm is convergent and the limit is the solution to
the approximation equation. The existence of approximation is proved when [qi ] = 0.
Now, we prove the uniqueness of approximation. Suppose yi and xi are both solutions
to Equation (20). Denote the absolute differences as wi = |yi − xi |. We have
w0 =0,
i −1
wi ≤ − ζ (−λ)k1 (ti , t0 ; y0 ) − k1 (ti , t0 ; x0 )h1+λ + h ∑ tλk k1 (ti , tk ; yk ) − k1 (ti , tk ; xk )
k =1
h
+ tiλ k1 (ti , ti ; yi ) − k1 (ti , ti ; xi ) − ζ (−μ)(qt1 )1+μ k2 (ti , t0 ; y0 ) − k2 (ti , t0 ; x0 )
2
qt
+ i (qti )μ k2 (ti , qti ; β i y[qi] + (1 − β i )y[qi]+1 ) − k2 (ti , qti ; β i x[qi] + (1 − β i ) x[qi]+1 )
2
i −1
h qt
≤ L1 h ∑ tλk wk + L1 tiλ wi + L2 i (qti )μ ( β i w[qi] + (1 − β i )w[qi]+1 )
k =1
2 2
i −1
h qti
≤ Lh ∑ tλk wk + L 2 tiλ wi + L 2
(qti )μ (1 − β i )w1 . (23)
k =1
24
Mathematics 2021, 9, 1856
i −1
wi ≤2Lh ∑ tλk wk + Lhtiλ wi + Lqti (qti )μ (1 − βi )w1
k =1
i −1
≤ 2Lhtλk + Lh(qti )μ (1 − β i ) w1 + 2Lh ∑ tλk wk
k =2
i −1
=h ∑ Bk wk ,
k =1
where
2Ltλk + L(qti )μ (1 − β i ), j = 1,
Bk =
2Lhtλk , j = 2, · · · , i − 1.
According to Lemma 4 with A = 0, we have wi = 0, i.e., yi = xi , the solution of
Equation (20) is unique.
• Case II. For [qi ] > 1, we consider the following cases.
(1) The first situation is [qi ] + 1 = i, namely, when i ≤ 1− 1
q , we have
m +1 h
y
i − yim = tiλ k1 (ti , ti ; yim ) − k1 (ti , ti ; yim−1 )
2
qti − t[qi]
−1
+ (qti )μ k2 ti , qti ; β i ỹ[qi] + (1 − β i )ym
[ qi ]+ 1 − k2 ti , qti ; β i ỹ[qi] + (1 − β i )ym
[ qi ]+ 1
2
h qti − t[qi]
≤ L1 tiλ yim − yim−1 + L2 (qti )μ (1 − β i )ym m −1
[qi ]+1 − (1 − β i ) y[qi ]+1
2 2
h
≤ L tiλ + (qti )μ (1 − β i ) yim − yim−1 .
2
Let the step size h be small enough, such that L 2h tiλ + (qti )μ (1 − β i ) ≤ 12 . Then,
we can determine that |yim+1 − yim | ≤ 12 |yim − yim−1 | holds.
(2) The second situation is [qi ] + 1 < i, namely, when i > 1− 1
q , we obtain
m +1
m h λ
m −1
y − y = t k 1 ( t i , t i ; y m
) − k 1 ( t i , t i ; y )
i i i i i
2
h
≤ L1 tiλ yim − yim−1
2
h
≤ L tiλ yim − yim−1 .
2
Let L 2h tiλ ≤ 12 for a sufficiently small h, then |yim+1 − yim | ≤ 12 |yim − yim−1 | holds.
The above two situations show that the iterative algorithm is convergent and that the
limit is the solution to Equation (21).
Next, we prove that the solution to Equation (21) is unique. Suppose yi and xi are both
solutions to Equation (21). Denote the differences as w i = |yi − xi |, i = 1, · · · , N. Then,
we have
25
Mathematics 2021, 9, 1856
0 =0;
w
i −1
i ≤ − ζ (−λ)k1 (ti , t0 ; y0 ) − k1 (ti , t0 ; x0 ) h1+λ + h ∑ tλk k1 (ti , tk ; yk ) − k1 (ti , tk ; xk )
w
k =1
h
+ tiλ k1 (ti , ti ; yi ) − k1 (ti , ti ; xi ) − ζ (−μ)h1+μ k2 (ti , t0 ; y0 ) − k2 (ti , t0 ; x0 )
2
[qi ]−1 h μ
μ
+ h ∑ tk |k2 (ti , tk ; yk ) − k2 (ti , tk ; xk ) + t[qi] |k2 (ti , t[qi] ; y[qi] ) − k2 (ti , t[qi] ; x[qi] )
k =1
2
qti − t[qi] μ
+ (t k2 (ti , t[qi] ; y[qi] ) − k2 (ti , t[qi] ; x[qi] ) (24)
2 [qi]
+ (qti )μ k2 ti , qti ; β i y[qi] + (1 − β i )y[qi]+1 ) − k2 ti , qti ; β i x[qi] + (1 − β i ) x[qi]+1 )
i −1 [qi ]−1
h μ h μ
≤h ∑ tλk L1 wk + 2 tiλ L1 wi + h ∑ k + t[qi] L2 w
t k L2 w
2
[qi]
k =1 k =1
qti − t[qi] μ
+ [qi] + (qti )μ L2 ( β i w
t[qi] L2 w [qi] + (1 − β i )w
[qi]+1 )
2
i −1 [qi ]−1
γ h γ γ h γ
≤ Lh ∑ tk wk + L 2 ti wi + Lh ∑ k + L t[qi] w
tk w
2
[qi]
k =1 k =1
h γ
+ L tγ[qi] w
[qi] + t[qi] β i w
[qi] + (1 − β i )w
[qi]+1 .
2
[qi ]−1 i −1
γ γ γ
i ≤ 4Lh
w ∑ tk wk + (4Lht[qi] + Lhβ i t[qi] )w
[qi] = h ∑ Bk wk ,
k =1 k =1
where γ
4Ltk , j = 1, · · · , [qi ] − 1,
Bk = γ γ
4Lt[qi] + Lβ i t[qi] , j = [qi ].
26
Mathematics 2021, 9, 1856
[qi ]−1 i −1
γ γ γ γ h γ
i ≤ Lh
w ∑ k + Lht[qi] w
tk w [qi] + Lht[qi]+1 w
[qi]+1 + Lh ∑ k + L ti w
tk w
2
i
k =1 k =[qi ]+2
h γ
[qi ]−1
γ h γ γ
+ Lh ∑ tk w
2
[qi] + L t[qi] w
k + L t[qi] w
2
[qi] + t[qi] β i w
[qi] + 1 − β i )w
[qi]+1
k =1
(26)
[qi ]−1 h h γ
γ γ
=2Lh ∑ k
tk w + 2Lht[qi] + L β i tγ[qi] w γ
[qi] + Lht[qi]+1 + L t[qi] (1 − β i ) w[qi]+1
k =1
2 2
i −1
γ h γ
+ Lh ∑ k + L ti w
tk w
2
i .
k =[qi ]+2
γ
Letting h be so small that L 2h ti ≤ 12 , then
[qi ]−1 i −1
γ γ γ γ γ γ
i ≤4Lh
w ∑ k + (4Lht[qi] + Lhβ i t[qi] )w
tk w [qi] + 2Lht[qi]+1 + Lht[qi] (1 − β i ) w[qi]+1 + 2Lh ∑ k
tk w
k =1 k =[qi ]+2
n −1
=h ∑ Bk wk ,
k =1
where ⎧ γ
⎪
⎪ 4Ltk , j = 1, · · · , [qi ] − 1,
⎪
⎪ γ γ
⎨ 4Lt[qi] + Lβ i t[qi] , j = [qi ],
k =
B
⎪ γ γ
⎪
⎪ 2Lt[qi]+1 + Lt[qi] (1 − β i ), j = [qi ] + 1,
⎪
⎩ γ
2Ltk , j = [qi ] + 2, · · · , i − 1.
According to Lemma 4 with A = 0, we have w i = 0, i.e., yi = xi , the solution
of Equation (21) is unique. Combining the above situations, the proof of Theorem 2
is completed.
5. Convergence Analysis
In this section, we will discuss errors caused by the process of obtaining discrete
equations using a quadrature formula and interpolation technique and the errors caused
by solving the discrete equation using iterative algorithms. According to the quadrature
rule, Equation (12) can be expressed as
y ( t0 ) = f ( t0 ),
i −1 h
y(ti ) = f (ti ) − ζ (−λ)k1 ti , t0 ; y(t0 ) h1+λ + h ∑ tλk k1 ti , tk ; y(tk ) + tiλ k1 ti , ti ; y(ti ) + E1,i
k =1
2
27
Mathematics 2021, 9, 1856
[k2 (ti , s; y(s))] s=0
E2,i = [k2 (ti , s; y(s))] s=0 ζ (−μ − 1)h2+γ + ζ (−μ − 2)h3+μ + O(h4+μ )
2!
= T2 (ti )h2+μ + O(h3+μ ),
β(1 − β) 2
E3,i = − h y (qti )(qti )γ k2 ti , qti ; β i y(t[qi] ) + (1 − β i )y(t[qi]+1 )
2
(qti − t[qi] )2 qti ∂2 μ
+ k 2 t i , s; y ( s ) s ds + O(h3 )
12 t[qi] ∂s2
In order to investigate the error between the exact solution and the approximate
solution of Equation (1), we first give the following theorem.
Theorem 3. Under the conditions of Theorem 2, y(ti ) is the exact solution of Equation (1) when
t = ti and yi is the solution of discrete Equation (19) at ti . Assume that h is sufficiently small, then,
the absolute error denote by e1,i = |y(ti ) − yi | has the estimate
28
Mathematics 2021, 9, 1856
i −1
|e1,i | ≤ A + h ∑ Bj |e1,j |, 0 ≤ i ≤ N,
j =1
where
A = 2| T1 (ti )h2+λ + T2 (ti )h2+μ + T3 (ti )h2 + O(h3+γ ) = O(h2+γ ).
By Lemma 4, we have
max |e1,i | ≤ O(h2+γ ).
1≤ i ≤ N
Theorem 4. Under the conditions of Theorem 2, yi is the solution of Equation (19) and yi is the
approximate solution of Equation (1), and yi is defined by (21). The absolute error is denoted by
e2,i = |yi − yi |. Assume that h is sufficiently small, then, there exist two positive constants, C1 and
C2 , which are independent of h = N T
, such that
C1 h, [qi ] + 1 = i,
vi ≤
C2 h, [qi ] + 1 ≤ i.
Proof. Subtracting (21) from (19), we have e2,0 = 0. We consider two cases.
(1) The first case is [qi ] + 1 = i (i.e., when i ≤ 1
1−q ). Then, we have
i −1 [qi ]−1
h μ h μ
e2,i = h ∑ tλk L1 e2,k + 2 tiλ L1 + h ∑ tk L2 e2,k + t[qi] L2 e2,[qi]
2
k =1 k =1
qti − t[qi] μ
+ t[qi] L2 e2,[qi] + (qti )μ L2 ( β i e2,[qi] + (1 − β i ))
2
i −1 [qi ]−1
μ h μ
≤h ∑ tλk Le2,k + h ∑ tk Le2,k + t[qi] Le2,[qi]
2
(29)
k =1 k =1
h μ h h
+ t L + (qti )μ β i e2,[qi] + (qti )μ L(1 − β i ) + tiλ L
2 [qi] 2 2
i −1 1
1
=h ∑ Bk e2,k + (qti )μ L(1 − β i ) + tiλ L h.
k =1
2 2
i −1 [qi ]−1
h μ h μ
e2,i = h ∑ tλk L1 e2,k + 2 tiλ L1 + h ∑ tk L2 e2,k + t[qi] L2 e2,[qi]
2
k =1 k =1
qti − t[qi] μ
+ t[qi] L2 e2,[qi] + (qti )μ L2 ( β i e2,[qi] + (1 − β i )e2,[qi]+1 )
2
i −1 [qi ]−1
h μ h μ
≤h ∑ tλk Le2,k + 2 tiλ L + h ∑ tk Le2,k + t[qi] Le2,[qi]
2
(30)
k =1 k =1
h μ
+ t Le + (qti )μ L( β i e2,[qi] + (1 − β i )e2,[qi]+1 )
2 [qi] 2,[qi]
i −1
1
=h ∑ Bk e2,k + tiλ Lh.
k =1
2
29
Mathematics 2021, 9, 1856
Theorem 5. Under the conditions of Theorem 2, y(ti ) is the exact solution of Equation (1), yi is
the approximate solution of Equation (1) when t = ti , we have
C1 h + O(h2+γ ), [qi ] + 1 = i,
|y(ti ) − yi | ≤ + γ
C2 h + O(h 2 ), [qi ] + 1 ≤ i.
Proof. By Theorems 3 and 4, the absolute error between y(ti ) and yi has the expression
6. Extrapolation Method
In this section, we first describe the asymptotic error expansion and then present an
extrapolation technique for achieving high precision. Finally, a posterior error estimate is
derived.
Theorem 6. Let f (t), k1 (t, s; y(s)), k2 (t, s; y(s)) are four times continuously differentiable on
I, D × R, Dθ × R, respectively. Additionally, y(t) has continuous partial derivatives up to 3 on I
and k i (t, s; y(s)) (i = 1, 2) satisfy Lipschitz conditions (2). There exist functions Ŵi (t)(i = 1, 2, 3)
independent of h, such that we have the following asymptotic expansions:
yi = y(ti ) + Ŵ1 (ti )h2+λ + Ŵ2 (ti ))h2+μ + Ŵ3 (ti )h2 + O(h3+γ ), −1 < λ < 0, −1 < μ ≤ 0. (32)
Proof. Assume that {Ŵk (t), k = 1, 2, 3} satisfy the auxiliary delay equations
t qt
Ŵk (t) = Wk (t) + sλ k1 (t, s; y(s))Ŵk (s)ds + sμ k2 (t, s; y(s))Ŵk (s)ds,
0 0
Let
Ei = ei − W1 (ti )h2+λ + W2 (ti )h2+μ + W3 (ti ) h2 .
Then, we obtain
30
Mathematics 2021, 9, 1856
i −1
h
Ei = − ζ (−λ)h1+λ k1 (ti , t0 ; y(t0 )) E0 + h ∑ tλk k1 (ti , tk ; y(tk ))Ek + 2 tiλ k1 (ti , ti ; y(ti ))Ei
k =1
[qi ]−1
μ h μ
− ζ (−μ)h1+μ k2 (ti , t0 ; y(t0 )) E0 + h ∑ tk k2 (ti , tk ; y(tk )) Ek + t[qi] k2 (ti , t[qi] ; y(t[qi] ) E[qi]
k =1
2
qti − t[qi] μ
+ t[qi] k2 (ti , t[qi] ; y(t[qi] )) E[qi] + (qti )μ k2 ti , qti ; β i y(t[qi] ) E[qi] + (1 − β i )y(t[qi]+1 ) E[qi]+1 .
2
According to Lemma 4, there exists a constant d such that
max | Ei | ≤ dh3+γ .
1≤ i ≤ N
yi = y(ti ) + Ŵ1 (ti )h2+λ + Ŵ2 (ti ))h2+μ + Ŵ3 (ti )h2 + O(h3+γ ).
1, h
22+μ yi 2 − y1,h
y2,h = i
= y(ti ) + Ŵ3 (ti )h2 + O(h3+λ ). (37)
i 22 + μ − 1
A posterior asymptotic error estimate is
h h h h
h 22+λ y 2 − yh yh − y 2 22+λ y 2 − yih yh − y 2
2
yi − y(ti ) = 2+λi i
− y(ti ) + 2i+λ i ≤ 2+λi − y(ti ) + 2i+λ i
2 −1 2 −1 2 −1 2 −1
h
(38)
yh − y 2
i i
= y1,h − y ( t i ) + + O ( h2 )
i 22+λ − 1
h
h
yih −yi2
The error yi2 − y(ti ) is bounded by 22+λ −1
, which is important for constructing
adaptable algorithms.
31
Mathematics 2021, 9, 1856
7. Numerical Experiments
In this section, we illustrate the performance and accuracy of the quadrature method
using the improved trapezoid formula. For ease of notation, we define
E
h
Eh = |y(ti ) − yih |, Ek,i = |y(ti ) − yik,h | (k = 1, 2), Rate = log2 ,
Eh
2
where yih is the approximate solution of Equation (1), yik,h is the approximate solution of
k-th extrapolation, Ek,i is the absolute error between the exact solution and the approximate
solution of k-th extrapolation when t = ti . The procedure was implemented in MATLAB.
with T = 1, λ = − 12 , and q = 0.95. The exact solution is given by y(t) = t and f (t) is determined
by the exact solution.
Applying the algorithm with N = 24 , 25 , 26 , 27 , 28 , the numerical results at t = 0.4 are
presented in Table 1, the CPU time(s) are 0.34, 0.55, 0.98, 1.62, and 3.01 s, respectively. By
comparing Eh and E1,i , we can observe that the accuracy was improved and the extrapolation
algorithm was effective. In the third column, the rate values show that the convergence order was
consistent with the theoretical analysis.
32
Mathematics 2021, 9, 1856
1.1
N=23
1 exact solution
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
8. Conclusions
In this paper, by using the improved trapezoidal quadrature formula and linear inter-
polation, we obtained the approximate equation for non-linear Volterra integral equations
with vanishing delay and weak singular kernels. The approximate solutions were obtained
by an iterative algorithm, which possessed a high accuracy order O(h2+γ ). Additionally,
we analyzed the existence and uniqueness of both the exact and approximate solutions.
The significance of this work was that it demonstrated the efficiency and reliability of the
Richardson extrapolation. The computational findings were compared with the exact solu-
tion: we found that our methods possess high accuracy and low computational complexity,
and the results showed good agreement with the theoretical analysis. For future work, we
can apply this method for solving two-dimensional delay integral equations.
Author Contributions: Conceptualization, J.H. and L.Z.; methodology, J.H. and L.Z.; validation,
J.H. and H.L.; writing—review and editing, L.Z. and Y.W. All authors have read and agreed to the
published version of the manuscript.
33
Mathematics 2021, 9, 1856
Funding: This research was funded by the Program of Chengdu Normal University, grant number
CS18ZDZ02.
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: Not applicable.
Acknowledgments: The authors would like to thank the editor and referees for their careful com-
ments and fruitful suggestions.
Conflicts of Interest: The authors declare no conflict of interest.
References
1. Avaji, M.; Hafshejani, J.; Dehcheshmeh, S.; Ghahfarokhi, D. Solution of delay Volterra integral equations using the Variational
iteration method. J. Appl. Sci. 2012, 12, 196–200. [CrossRef]
2. Williams, L.R.; Leggett, R.W. Nonzero Solutions of Nonlinear Integral Equations Modeling Infectious Disease. Siam J. Math. Anal.
1982, 13, 121. [CrossRef]
3. Volterra, V. On some questions of the inversion of definite integrals. (Sopra alcune questioni di inversione di integrali definiti).
Ann. Mat. Pura Appl. 1897, 25, 139–178. [CrossRef]
4. Yang, K.; Zhang, R. Analysis of continuous collocation solutions for a kind of Volterra functional integral equations with
proportional delay. J. Comput. Appl. Math. 2011, 236, 743–752. [CrossRef]
5. Xie, H.; Zhang, R.; Brunner, H. Collocation methods for general Volterra functional integral equations with vanishing delays.
SIAM J. Sci. Comput. 2011, 33, 3303–3332. [CrossRef]
6. Ming, W.; Huang, C.; Zhao, L. Optimal superconvergence results for Volterra functional integral equations with proportional
vanishing delays. Appl. Math. Comput. 2018, 320, 292–301. [CrossRef]
7. Mokhtary, P.; Moghaddam, B.P.; Lopes, A.M.; Machado, J.A.T. A computational approach for the non-smooth solution of
non-linear weakly singular Volterra integral equation with proportional delay. Numer. Algorithms 2019, 83, 987–1006. [CrossRef]
8. Gu, Z.; Chen, Y. Chebyshev spectral-collocation method for a class of weakly singular Volterra integral equations with proportional
delay. J. Numer. Math. 2014, 22, 311–342. [CrossRef]
9. Zhang, R.; Zhu, B.; Xie, H. Spectral methods for weakly singular Volterra integral equations with pantograph delays. Front.
Math. China 2014, 8, 281–299. [CrossRef]
10. Brunner, H. Collocation Methods for Volterra Integral and Related Functional Differential Equations; Cambridge University Press:
Cambridge, UK, 2004.
11. Zhang, G.; Song, M. Impulsive continuous Runge—Kutta methods for impulsive delay differential equations. Appl. Math.
Comput. 2019, 341, 160–173. [CrossRef]
12. Shu, H.; Xu, W.; Wang, X.; Wu, J. Complex dynamics in a delay differential equation with two delays in tick growth with diapause.
J. Differ. Equ. 2020, 269, 10937–10963. [CrossRef]
13. Fang, J.; Zhan, R. High order explicit exponential Runge—Kutta methods for semilinear delay differential equations. J. Comput.
Appl. Math. 2021, 388, 113279. [CrossRef]
14. Song, H.; Xiao, Y.; Chen, M. Collocation methods for third-kind Volterra integral equations with proportional delays. Appl. Math.
Comput. 2021, 388, 125509.
15. Brunner, H. The numerical solution of weakly singular Volterra functional integro-differential equations with variable delays.
Commun. Pure Appl. Anal. 2006, 5, 261–276. [CrossRef]
16. Huang, C. Stability of linear multistep methods for delay integro-differential equations. Comput. Math. Appl. 2008, 55, 2830–2838.
[CrossRef]
17. Sheng, C.; Wang, Z.; Guo, B. An h p -spectral collocation method for nonlinear Volterra functional integro-differential equations
with delays. Appl. Numer. Math. 2016, 105, 1–24. [CrossRef]
18. Abdi, A.; Berrut, J.P.; Hosseini, S.A. The Linear Barycentric Rational Method for a Class of Delay Volterra Integro-Differential
Equations. J. Sci. Comput. 2018, 75, 1757–1775. [CrossRef]
19. Yaghoobi, S.; Moghaddam, B.P.; Ivaz, K. An efficient cubic spline approximation for variable-order fractional differential
equations with time delay. Nonlinear Dyn. 2017, 87, 815–826. [CrossRef]
20. Rahimkhani, P.; Ordokhani, Y.; Babolian, E. A new operational matrix based on Bernoulli wavelets for solving fractional delay
differential equations. Numer. Algorithms 2017, 74, 223–245. [CrossRef]
21. Rakhshan, S.A.; Effati, S. A generalized Legendre—Gauss collocation method for solving nonlinear fractional differential
equations with time varying delays. Appl. Numer. Math. 2019, 146, 342–360. [CrossRef]
22. Zuniga-Aguilar, C.J.; Gomez-Aguilar, J.F.; Escobar-Jimenez, R.F.; Romero-Ugalde, H.M. A novel method to solve variable-order
fractional delay differential equations based in lagrange interpolations. Chaos Solitons Fractals 2019, 126, 266–282. [CrossRef]
23. Xu, X.; Huang, Q. Superconvergence of discontinuous Galerkin methods for nonlinear delay differential equations with vanishing
delay. J. Comput. Appl. Math. 2019, 348, 314–327. [CrossRef]
34
Mathematics 2021, 9, 1856
24. Bellour, A.; Bousselsal, M. A Taylor collocation method for solving delay integral equations. Numer. Algorithms 2014, 65, 843–857.
[CrossRef]
25. Darania, P.; Sotoudehmaram, F. Numerical analysis of a high order method for nonlinear delay integral equations. J. Comput.
Appl. Math. 2020, 374, 112738. [CrossRef]
26. Khasi, M.; Ghoreishi, F.; Hadizadeh, M. Numerical analysis of a high order method for state-dependent delay integral equations.
Numer. Algorithms 2014, 66, 177–201. [CrossRef]
27. Bica, A.M.; Popescu, C. Numerical solutions of the nonlinear fuzzy Hammerstein-Volterra delay integral equations. Inf. Sci.
2013, 223, 236–255. [CrossRef]
28. Mosleh, M.; Otadi, M. Least squares approximation method for the solution of Hammerstein-Volterra delay integral equations.
Appl. Math. Comput. 2015, 258, 105–110. [CrossRef]
29. Zhang, X. A new strategy for the numerical solution of nonlinear Volterra integral equations with vanishing delays. Appl. Math.
Comput. 2020, 365, 124608.
30. Lima, P.; Diogo, T. Numerical solution of a nonuniquely solvable Volterra integral equation using extrapolation. J. Comput. Appl.
Math. 2002, 140, 537–557. [CrossRef]
31. Sidi, A. Practical Extrapolation Methods; Cambridge University Press: Cambridge, UK, 2003.
32. Tao, L.; Jin, H. High Precision Algorithm for Integral Equations; China Science Press: Beijing, China, 2013.
33. Tao, L.; Yong, H. Extrapolation method for solving weakly singular nonlinear Volterra integral equations of the second kind. J.
Math. Anal. Appl. 2006, 324, 225–237. [CrossRef]
34. Navot, J. A Further Extension of the Euler-Maclaurin Summation Formula. J. Math. Phys. 1962, 41, 155–163. [CrossRef]
35. Tao, L.; Yong, H. A generalization of Gronwall inequality and its application to weakly singular Volterra integral equation of the
second kind. J. Math. Anal. Appl. 2003, 282, 56–62. [CrossRef]
36. Qin, Y. Integral and Discrete Inequalities and Their Applications; Springer International Publishing: Berlin/Heidelberg, Germany,
2016.
35
mathematics
Article
An Iterative Algorithm for Approximating the Fixed Point of a
Contractive Affine Operator
María Isabel Berenguer 1,2, *,† and Manuel Ruiz Galán 1,2,†
Abstract: First of all, in this paper we obtain a perturbed version of the geometric series theorem,
which allows us to present an iterative numerical method to approximate the fixed point of a
contractive affine operator. This result requires some approximations that we obtain using the
projections associated with certain Schauder bases. Next, an algorithm is designed to approximate
the solution of Fredholm’s linear integral equation, and we illustrate the behavior of the method with
some numerical examples.
1. Introduction
The idea of iterative numerical methods is, given a complete metric space X (typically
a Banach space) and a contractive operator T : X −→ X, or at least one which guarantees
Citation: Berenguer, M.I.; Ruiz Galán, the convergence of the Picard iterates, to construct a sequence of approximations of the
M. An Iterative Algorithm for fixed point of that operator x0 = T ( x0 ). The calculation of the Picard iterates is not generally
Approximating the Fixed Point of a easy or even feasible, so several methods which allow us to approximate the elements of
Contractive Affine Operator. the Picard sequence have been proposed. Therefore, a part of the Picard-type iterative
Mathematics 2022, 10, 1012. https:// algorithms are focused on determining, for an element x ∈ X, a value close to T ( x ) and in
doi.org/10.3390/math10071012 this way, successively approximating the iterates. The numerical techniques used are very
Academic Editor: Ioannis K. Argyros
diverse, and the resulting algorithms have numerous applications. Proof of all this are the
recent references [1–16].
Received: 27 February 2022 However, our approach here is completely different: given x, instead of approximating
Accepted: 18 March 2022 successively T ( x ), T 2 ( x ), T 3 ( x ), . . . , which necessarily involves an accumulation of errors,
Published: 22 March 2022 in this paper, we approximate directly T n ( x ) by means of the use of suitable Schauder bases,
Publisher’s Note: MDPI stays neutral transforming it into a simple calculation which, for example, does not involve the resolution
with regard to jurisdictional claims in of systems of algebraic equations or the use of any quadrature formulae because simply
published maps and institutional affil- linear combinations of certain values associated with the operator are calculated. What
iations. is more, motivated by its application for the numerical resolution of the linear Fredholm
integral equation, the operator T is considered to be affine and continuous. This affine
and continuous nature means that, instead of using a fixed-point language, we opted for
resorting to an equivalent version using the geometric series theorem, and more specifically,
Copyright: © 2022 by the authors. our first contribution is to obtain a perturbed version of the same which is susceptible
Licensee MDPI, Basel, Switzerland.
to presenting approximations by means of certain Schauder bases related to the operator.
This article is an open access article
Such an approximation will imply a low computational cost as mentioned above. Thus,
distributed under the terms and
we are going to design an iterative-type algorithm which allows the approximation of the
conditions of the Creative Commons
fixed point of a suitable continuous affine operator.
Attribution (CC BY) license (https://
As we have mentioned, the application that we are presenting consists of a numerical
creativecommons.org/licenses/by/
4.0/).
algorithm to solve the linear Fredholm integral equation, which is chosen for its great versatility.
Theorem 1. Let X be a Banach space, y ∈ X and L ∈ L( X ) with L < 1, and consider the
continuous affine operator A : X −→ X defined by
Ax := y + Lx, ( x ∈ X ).
Proof. Let us first observe that, according to the geometric series theorem, there exists a
unique solution x • ∈ X for the equation Ax = x,
x • = ( I − L)−1 y,
Therefore,
n n n n n n
• j j •
∑ L j y0 − x ≤ ∑ L j y0 − ∑ L y0 + ∑ L y0 − ∑ L y + ∑ L y − x
j j
j =0 j =0 j =0
j =0 j =0
j =0
n n
L n +1
≤ ∑ L j y0 − L j y0 + ∑ L j
y0 − y +
1− L
y
j =0 j =0
n 1 − L n +1 L n +1
j
= ∑ j 0L y − L y 0 +
1− L
y0 − y +
1− L
y ,
j =0
as announced.
38
Mathematics 2022, 10, 1012
j = 0, . . . , n ⇒ L j y0 − L j y0 < ε j , (1)
and that
1 − L n +1 L n +1 ε
y0 − y + y < . (2)
1− L 1− L 2
Then
n
•
∑ L j y0 − x < ε.
j =0
Obviously, (2) is valid as soon as n is large enough and y0 − y is small. For condition (1),
we present some analytical tools in the next section.
In order to derive
n n
∑ L j y0 ≈ ∑ L j y0
j =0 j =0
an approximation as that given in (1) is required. To this end, a possible tool appears
provided by the Schauder bases, since they give an explicit linear approximation of any
element of a Banach space by means of the associated projections, which is compatible with
the continuity and affinity of the operator. What is more, in the case of classic bases, we
easily obtain approximations of (the linear part of) A and its powers.
Thus, before continuing, we revise some of the basic notions of Schauder bases that
we are going to need in the design of our algorithm. A sequence {e j } j∈N in a Banach space
X is a Schauder basis if all the element x ∈ X can be uniquely represented as
∞
x= ∑ αj ej ,
j =1
39
Mathematics 2022, 10, 1012
for a sequence of real {α j } j∈N . If we define for each j ∈ N the linear operator Pj : X −→ X,
known as the j-th projection associated with the basis, as
j
Pj x := ∑ αk ek ,
k =0
for such an x, it is easy to prove, as a consequence of the Baire lemma, that it is a continuous
operator and, in view of the representation of x in terms of the elements of the basis,
lim Pj x − x = 0.
j→∞
where x ∈ X is the unknown function. In view of the previous results, we consider the
continuous and linear operator L : X −→ X defined at each y0 ∈ X as
b
Ly0 := k (·, s)y0 (s)ds.
a
Then, given j ∈ N,
b b
L j y0 = ··· k (·, t1 )k(t1 , t2 ) · · · k (t j−1 , t j )y0 (t j )dt j · · · dt1 .
a a
k (b − a) < 1,
since such a condition is sufficient for the validity of L < 1 and it is very easy to check.
(d)
Furthermore, for each d ∈ N, we fix a Schauder basis {e j } j∈N in C [ a, b]d (if X = C [ a, b]) or
(d)
in L p [ a, b]d (if X = L p [ a, b]) and we denote the projections in this basis as { Pj } j∈N .
With all of this, we are now ready to define the approximate operators L j : for each
x ∈ X and j ∈ N, we take
Now we can apply the corollary 1 since without going any further, each r j is big
enough, L j x − L j x < ε j .
40
Mathematics 2022, 10, 1012
Corollary 2. For any ε > 0 and y0 ∈ X, there are natural numbers n and r0 , . . . , rn in such a way
that if x • is the unique solution to the linear Fredholm integral Equation (3), then
n
•
∑ L j y0 − x < ε,
j =0
Observe that
sol− approx − x • < ε
(d)
and that for an appropriate choice of the bases {e j } j∈N , the calculations are immediate, as
justified below.
Returning to the considered spaces in order to study the linear Fredholm integral
equation, X = C [ a, b] or X = L p [ a, b], we remember how it is possible to tensorially
(d)
construct bases {e j } j∈N in X = C [ a, b]d or X = L p [ a, b]d , respectively, from a basis
(1)
{e j } j∈N in the aforementioned spaces.
Specifically, given d ∈ N, d ≥ 2, we consider in Nd the square ordering introduced
in [17] in a inductive form: for d ≥ 2, (1, 1), (1, 2), (2, 2), (2, 1), (1, 3), (2, 3), (3, 3), (3, 2), . . . ,
and given the ordering o1 , o2 ,. . . of Nd−1 , the order in Nd is (o1 , 1), (o1 , 2), (o2 , 2), (o2 , 1),
(o1 , 3), (o2 , 3), (o3 , 3), . . . . Graphically,
( o1 , 1) / ( o1 , 2) ( o1 , 3) ( o1 , 4)
( o2 , 1) o ( o2 , 2) ( o2 , 3) ( o2 , 4)
( o3 , 1) o ( o3 , 2) o ( o3 , 3) ( o3 , 4)
······ o ( o4 , 3) o ( o4 , 4)
41
Mathematics 2022, 10, 1012
(d)
e j ( t 1 , . . . , t d ) : = e α1 ( t 1 ) · · · e α d ( t d ), ((t1 , . . . , td ) ∈ [ a, b]d ). (5)
(1)
e j (t j ) = 1
and
(1)
1 ≤ k < n ⇒ ek (t j ) = 0.
On the other hand, if A is a non-empty subset of [0, 1] and δA : [0, 1] −→ R is the
function defined in each 0 ≤ t ≤ 1 as
!
1, if t ∈ A
δA (t) :=
0, if t ∈
/A
(1)
e j (·) := ϕ(2k (·) − r ).
In both cases, the tensorial sequences defined as (5) constitute Schauder bases in
their respective spaces, C [ a, b]d and L p [ a, b]d [17,19]. However, what really makes these
bases useful when they are used in our Algorithm 1 is precisely that the calculation of
(d)
the approximate operators L j is very easy, since the basis functions e j are of separate
variables and each factor is immediately integrable. Let us mention that these Schauder
bases allow us to preserve the linearity of the convergence that it is guaranteed by the series
geometric theorem.
5. Numerical Examples
We now show the numerical results obtained in several specific examples. Beforehand,
let us mention that the reordering of a finite number of Schauder basis elements produces
another new Schauder basis, which could be interesting from a computational point of
view. Thus, for each r ∈ N, we reordered the bases of C [ a, b]d and L p [ a, b]d so that the r d
first elements correspond to (α1 , α2 , . . . , αd ) being 1 ≤ αi ≤ r. For these reordered bases,
(d) (d)
we maintain the same previous notation, {e j } j∈Nd for the basis and { Pj } j∈N for the
sequence of projections. Furthermore, given n, r ∈ N, we write
n
x (n,r) := ∑ L j y0 ,
j =0
j + 1 times
where the indices r j involved in the definition of L j are given by τ (r j ) = ( r, . . . , r ).
42
Mathematics 2022, 10, 1012
Table 1. x • − x (n,r) for Example 1 using the usual basis in C [0, 1].
n r=9 r = 17 r = 33
1 0.00435474 0.00216401 0.00162257
2 0.00300495 0.000773636 0.000210816
3 0.00298439 0.000752736 0.00018924
4 0.00298394 0.000752272 0.000188771
Example 2. The following equation is also extracted from the same reference (Example 2, [10]):
1
1
x ( t ) = t2 − t + 1 + ets x (s) ds.
4 0
As in the referenced paper, since the solution of this equation is not known, we consider the operator
F : C [0, 1] → C [0, 1] given by
1
1
F ( x )(t) = x (t) − t2 + t − 1 − ets x (s)ds
4 0
Table 2. F ( x (n,r) ) for Example 2 using the usual basis in C [0, 1].
n r=9 r = 17 r = 33
1 0.122201 0.123111 0.123339
2 0.0399626 0.0413395 0.0416839
3 0.0120328 0.0136133 0.0140084
4 0.00258589 0.00422851 0.0046442
is taken from [20], Example 3. Its solution is x • (t) = t2 − 1. See Table 3 for the error generated by
Algorithm 1.
43
Mathematics 2022, 10, 1012
Table 3. x • − x (n,r) for Example 3 using the usual basis in C [0, 1].
n r=9 r = 17 r = 33
1 0.0627248 0.0593289 0.0584785
2 0.0133747 0.0104771 0.00975651
3 0.00503508 0.00242016 0.00156699
4 0.00362812 0.00109348 0.000463489
Example 4. This is a standard test problem, and it arises in electrostatics (see [21]) where it is
called Love’s equation.
δ 1 x (s)
x (t) = y(t) + ds.
π 0 δ2 + ( t − s )2
1
We consider δ = −1 and y(t) = 1 + (arctg(1 − t) + arctg(t)) as in Example 3.2 of [22]. In this
π
case, the exact solution is x • (t) = 1.
The errors—see Tables 4 nad 5—are similar to those obtained by the Haar wavelet method and
rationalized Haar functions method (see Table 1 in [22]), although their computation requires to
solve some high-order systems of linear equations.
Table 4. x • − x (n,r) for Example 4 using the usual basis in C [0, 1].
n r=9 r = 17 r = 33
1 0.0819571 0.0825347 0.0826789
2 0.0241024 0.0234044 0.023227
3 0.0059927 0.00634808 0.00645712
4 0.00299427 0.00211501 0.0018917
Table 5. x • − x (n,r) for Example 4 using the usual basis in L2 [0, 1].
n r=8 r = 16 r = 32
1 0.0785242 0.0785241 0.0785217
2 0.0235324 0.0235323 0.0235264
3 0.0101296 0.0101295 0.0101140
4 0.0083851 0.0083850 0.0083669
Example 5. Now considering Example 2 of [23] which has solution x • (t) = sin(2πt)
1
x (t) = sin(2πt) + (t2 − t − s2 − s) x (s)ds.
0
We observe that the numerical results obtained with our method (Table 6) significantly improve
those obtained in the reference.
Table 6. x • − x (n,r) for Example 5 using the usual basis in L2 [0, 1].
n r=8 r = 16 r = 32
4 3.17949 × 10−11 4.33093 × 10−11 4.28502 × 10−9
6. Conclusions
In this paper, we present an algorithm for iteratively approximating the fixed point
of a continuous coercive affine operator. Its design is based on a perturbed version of the
classic geometric series theorem, the error control that this provides, and the use of certain
Schauder bases. All of this is illustrated for a wide group of affine problems, the linear
Fredholm integral equations. The low computational cost that our algorithm entails makes
it particularly efficient. All of this is illustrated by several examples. We consider that
44
Mathematics 2022, 10, 1012
future research could be focused on extending the algorithm to solve different types of
integral and even integro-differential equations.
Author Contributions: Conceptualization, M.I.B. and M.R.G.; methodology, M.I.B. and M.R.G.;
software M.I.B. and M.R.G.; validation, M.I.B. and M.R.G.; formal analysis, M.I.B. and M.R.G.;
investigation, M.I.B. and M.R.G.; writing—original draft preparation, M.I.B. and M.R.G.; writing—
review and editing, M.I.B. and M.R.G.; supervision, M.I.B. and M.R.G. All authors have read and
agreed to the published version of the manuscript.
Funding: This research was partially supported by Junta de Andalucía, Project “Convex and
numerical analysis”, reference FQM359, and by the “María de Maeztu” Excellence Unit IMAG,
reference CEX2020-001105-M, funded by MCIN/AEI/10.13039/501100011033/.
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: Not applicable.
Conflicts of Interest: The authors declare no conflict of interest.
References
1. Alguliyev, R.M.; Sukhostat, L.V. Efficient algorithm for big data clustering on single machine. CAAI Trans. Intell. Technol. 2020, 5,
9–14. [CrossRef]
2. Alipour, S.; Mirzaee, F. An iterative algorithm for solving two dimensional nonlinear stochastic integral equations: A combined
successive approximations method with bilinear spline interpolation. Appl. Math. Comput. 2020, 371, 124947. [CrossRef]
3. Asgari, Z.; Toutounian, F.; Babolian, E.; Tohidi, E. LSMR iterative method for solving one- and two-dimensional linear Fredholm
integral equations. Comput. Appl. Math. 2019, 38, 135. [CrossRef]
4. Berenguer, M.I.; Gámez, D. Projected Iterations of Fixed-Point Type to Solve Nonlinear Partial Volterra Integro-Differential
Equations. Bull. Malays. Math. Sci. Soc. 2020, 43, 4431–4442. [CrossRef]
5. Berenguer, M.I.; Garralda-Guillem, A.I.; Ruiz Galán, M. An approximation method for solving systems of Volterra integro-
differential equations. Appl. Numer. Math. 2013, 67, 126–135. [CrossRef]
6. Cominola, A.; Giuliania, M.; Pigab, D.; Castellettia, A.; Rizzoli, A.E. A Hybrid Signature-based Iterative Disaggregation algorithm
for Non-Intrusive Load Monitoring. Appl. Energy 2017, 185, 331–344. [CrossRef]
7. Derviskadic, A.; Romano, P.; Paolone, M. Iterative-Interpolated DFT for Synchrophasor Estimation: A Single Algorithm for P-
and M-Class Compliant PMUs. IEEE Trans. Instrum. Meas. 2018, 67, 547–558. [CrossRef]
8. Ding, F.; Wang, F.; Xua, L.; Wu, M. Decomposition based least squares iterative identification algorithm for multivariate
pseudo-linear ARMA systems using the data filtering. J. Frankl. Inst. 2017, 354, 1321–1339. [CrossRef]
9. Muthuvalu, M.S.; Sulaiman, J. Half-Sweep Arithmetic Mean method with composite trapezoidal scheme for solving linear
Fredholm integral equations. Appl. Math. Comput. 2011, 217, 5442–5448. [CrossRef]
10. Gutiérrez, J.M.; Hernández-Verón, M.A.; Martínez, E. Improved iterative solution of linear fredholm integral equations of second
kind via inverse-free iterative schemes. Mathematics 2020, 8, 1747. [CrossRef]
11. Karimi, S.; Jozi, M. A new iterative method for solving linear Fredholm integral equations using the least squares method. Appl.
Math. Comput. 2015, 250, 744–758. [CrossRef]
12. Qin, X.; Cho, S.Y.; Wang, L. Strong convergence of an iterative algorithm involving nonlinear mappings of nonexpansive and
accretive type. Optimization 2018, 67, 1377–1388. [CrossRef]
13. Rabbani, M.; Das, A.; Hazarika, B.; Arab, R. Existence of solution for two dimensional nonlinear fractional integral equation by
measure of noncompactness and iterative algorithm to solve it. J. Comput. Appl. Math. 2020, 370, 112654. [CrossRef]
14. Ray, S.S.; Sahu, P.K. Numerical Methods for Solving Fredholm Integral Equations of Second Kind. Abstr. Appl. Anal. 2013, 2013,
426916. [CrossRef]
15. Yao, Y.; Postolache, M.; Yao, J.C. An iterative algorithm for solving generalized variational inequalities and fixed points problems.
Mathematics 2019, 7, 61. [CrossRef]
16. Qin, X.; Petruşel, A.; Yao, J.C. CQ iterative algorithms for fixed points of nonexpansive mappings and split feasibility problems in
Hilbert spaces. J. Nonlinear Convex Anal. 2018, 19, 157–165.
17. Gelbaum, B.; de Lamadrid, J.G. Bases of tensor products of Banach spaces. Pac. J. Math. 1961, 11, 1281–1286. [CrossRef]
18. Lindenstrauss, J.; Tzafriri, L. Classical Banach Spaces; Lecture Notes in Mathematics 338; Springer: Berlin/Heidelberg, Germany;
New York, NY, USA, 1973.
19. Pap, E. (Ed.) Handbook of Measure Theory; Elsevier: Amsterdam, The Netherland, 2002; Volume II.
20. Gutiérrez, J.M.; Hernández-Verón, M.A. A Picard-Type Iterative Scheme for Fredholm Integral Equations of the Second Kind.
Mathematics 2021, 9, 83. [CrossRef]
21. Love, E. The electrostatic field of two equal circular conducting disks. Q. J. Mech. Appl. Math. 1949, 2, 420–451. [CrossRef]
45
Mathematics 2022, 10, 1012
22. Amiri, S.; Hajipour, M.; Baleanu, D. On accurate solution of the Fredholm integral equations of the second kind. Appl. Numer.
Math. 2020, 150, 478–490. [CrossRef]
23. Maleknejad, K.; Yousefi, M. Numerical solution of the integral equation of the second kind by using wavelet bases of Hermite
cubic splines. Appl. Math. Comput. 2006, 183, 134–141. [CrossRef]
46
mathematics
Article
Gradient-Based Optimization Algorithm for Solving Sylvester
Matrix Equation
Juan Zhang 1, * and Xiao Luo 2
Abstract: In this paper, we transform the problem of solving the Sylvester matrix equation into an
optimization problem through the Kronecker product primarily. We utilize the adaptive accelerated
proximal gradient and Newton accelerated proximal gradient methods to solve the constrained
non-convex minimization problem. Their convergent properties are analyzed. Finally, we offer
numerical examples to illustrate the effectiveness of the derived algorithms.
Keywords: Sylvester matrix equation; Kronecker product; adaptive accelerated proximal gradient
method; Newton-accelerated proximal gradient method
1. Introduction
Citation: Zhang, J.; Luo, X.; Matrix equations are ubiquitous in signal processing [1], control theory [2], and linear
Gradient-Based Optimization systems [3]. Most time-dependent models accounting for the prediction, simulation, and
Algorithm for Solving Sylvester control of real-world phenomena may be represented as linear or nonlinear dynamical
Matrix Equation. Mathematics 2022, systems. Therefore, the relevance of matrix equations within engineering applications
10, 1040. https://doi.org/10.3390/ largely explains the great effort put forth by the scientific community into their numeri-
math10071040 cal solution. Linear matrix equations have an important role in the stability analysis of
Academic Editors: Maria Isabel linear dynamical systems and the theoretical development of the nonlinear system. The
Berenguer and Manuel Ruiz Galán Sylvester matrix equation was first proposed by Sylvester and produced from the research
of relevant fields in applied mathematical cybernetics. It is a famous matrix equation that
Received: 14 February 2022
occurs in linear and generalized eigenvalue problems for the computation of invariant
Accepted: 22 March 2022
subspaces using Riccati equations [4–6]. The Sylvester matrix equation takes part in linear
Published: 24 March 2022
algebra [7–9], image processing [10], model reduction [11], and numerical methods for
Publisher’s Note: MDPI stays neutral differential equations [12,13].
with regard to jurisdictional claims in We consider the Sylvester matrix equation of the form
published maps and institutional affil-
iations. AX + XB = C, (1)
where A ∈ Rm×m , B ∈ Rn×n , C ∈ Rm×n are given matrices, and X ∈ Rm×n is an unknown
matrix to be solved. We discuss a special form of the Sylvester matrix equation, in which A
Copyright: © 2022 by the authors. and B are symmetric positive definite.
Licensee MDPI, Basel, Switzerland. Recently, there has been a lot of discussion on the solution and numerical calculation
This article is an open access article of the Sylvester matrix equation. The standard methods for solving this equation are the
distributed under the terms and Bartels–Stewart method [14] and the Hessenberg–Schur method [15], which are efficient
conditions of the Creative Commons
for small and dense system matrices. When system matrices are small, the block Krylov
Attribution (CC BY) license (https://
subspace methods [16,17] and global Krylov subspace methods [18] are proposed. These
creativecommons.org/licenses/by/
methods use the global Arnoldi process, block Arnoldi process, or nonsymmetric block
4.0/).
Definition 1. Let Y = (yij ) ∈ Rm×n , Z ∈ R p×q , the Kronecker product of Y and Z be defined by
⎡ ⎤
y11 Z y12 Z ··· y1n Z
⎢ y21 Z y22 Z ··· y2n Z ⎥
⎢ ⎥
Y⊗Z =⎢ . .. .. .. ⎥.
⎣ .. . . . ⎦
ym1 Z ym2 Z · · · ymn Z
48
Mathematics 2022, 10, 1040
vec(YZW ) = (W T ⊗ Y )vec( Z ).
Ā = ( In ⊗ A + B T ⊗ Im ), x̄ = vec( X ), b̄ = vec(C ).
Using the calculation of the matrix differential from [33], we have the following
propositions immediately.
∂tr ( A T X ) ∂tr ( X T A)
Proposition 1. If A = ( aij ) ∈ Rm×n , X = ( xij ) ∈ Rm×n , then ∂X = ∂X = A.
∂tr ( X T AX )
Proposition 2. If A = ( aij ) ∈ Rm×m , X = ( xij ) ∈ Rm×n , then ∂X = AX + A T X.
∂tr ( XX T B)
Proposition 3. If B = (bij ) ∈ Rn×n , X = ( xij ) ∈ Rm×n , then ∂X = XB + XB T .
ϕ( X ) = AX + XB + A T X + XB T − 2C. (4)
2 ϕ ( X ) = A + A T + B + B T . (5)
3. Iterative Methods
In this section, we will introduce the adaptive accelerated proximal gradient (A-APG)
method and the Newton-APG method to solve the Sylvester equation. Moreover, we
compare the A-APG and Newton-APG methods with other existing methods.
min H ( x ) = g( x ) + f ( x ),
x ∈H
49
Mathematics 2022, 10, 1040
where H is the finite-dimensional Hilbert space equipped with the inner product < ·, · >, g
and f are both continuously convex, and f has a Lipschitz constant L. Given initializa-
tions x1 = x0 and t0 = 1, the APG method is
tk = ( 4(t − k − 1)2 + 1)/2,
t k −1 − 1
Yk = Xk + ( Xk − Xk −1 ),
tk
Xk +1 = Proxαg (Yk − α f (Yk )),
where α ∈ (0, L] and the mapping Proxαg (·) : Rn → Rn is defined as
! (
1
Proxαg ( x ) = argmin g(y) + y−x 2
.
y 2α
Since our minimization problem is linear, we choose the explicit scheme. The explicit
scheme is a simple but effective approach for the minimization problem. Given an initial
value Y0 and the step αk , the explicit scheme is
where Yk is the approximation solution. The explicit scheme satisfies the sufficient decrease
property using the gradient descent (GD) method.
Let Xk and Xk−1 be the current and previous states and the extrapolation weight be
wk . Using the explicit method (6), the APG iterative scheme is
wk = k − 2/k + 1,
Yk = (1 + wk ) Xk − wXk−1 , (7)
Yk+1 = Yk − αk ϕ(Yk ).
Together with the standard backtracking, we adopt the step size αk when the following
condition holds:
ϕ(Yk ) − ϕ(Yk+1 ) ≥ η Yk+1 − Yk 2 , (8)
for some η > 0.
Combining (7) and (8), the APG algorithm is summarized in Algorithm 1.
50
Mathematics 2022, 10, 1040
for some γ > 0. If the condition is not met, we restart the APG by setting wk = 0.
The restart APG method (RAPG) is summarized in Algorithm 2.
s k : = Xk − Xk −1 , gk : = ϕ ( Xk ) − ϕ ( Xk −1 ).
tr (skT sk ) tr ( gkT sk )
αk = or . (10)
tr (skT gk ) tr ( gkT gk )
51
Mathematics 2022, 10, 1040
52
Mathematics 2022, 10, 1040
6n3 + O(mn) + O(n2 ), and 4m2 n + 4mn2 + O(mn), respectively. The total computational
complexity is 22m2 n + 32n2 m + 6n3 + O(mn) + O(n2 ) in Algorithm 5 using GD.
4. Convergent Analysis
In this section, we focus on the convergence analysis of A-APG Algorithm 3. The
following proposition is required.
ϕ( X ) − ϕ(Y ) L M X − Y f or X, Y ∈ M.
In recent years, the proximal method based on the Bregman distance has been applied
for solving optimization problems. The proximal operator is
1
Proxαϕ (y) := argmin{ ϕ(y) + X − Xk 2
}.
y 2α
Basically, given the current estimation Xk and step size αk > 0, update Xk+1 via
1
Xk+1 = Prox0α ( Xk − αk ϕ( Xk )) = argmin{ X − ( Xk − α k ϕ ( Xk ) 2
}. (11)
X 2αk
Thus we obtain
1
(X − ( Xk − αk ϕ( Xk ))) = 0,
2αk k+1
which implies that
Xk +1 = Xk − α k ϕ ( Xk ).
This is exactly the explicit scheme in our algorithm.
1
Xk+1 = argmin{ X − ( Xk − α k ϕ ( Xk ) 2 }
X 2α k
1
= argmin{ X − Xk 2 + X − Xk , ϕ( Xk )}
X 2α k
1
= argmin{ X − Xk 2 + X − Xk , ϕ( Xk ) + ϕ( Xk )}.
X 2α k
Then we obtain
1 2
ϕ ( Xk ) X − Xk + Xk+1 − Xk , ϕ( Xk ) + ϕ( Xk )
2αk k+1
2
1 ϕ ( X )
ϕ ( Xk +1 ) + Xk − Xk +1 2 − Xk − Xk +1 2 (12)
2αk 2
1 L
ϕ ( Xk +1 ) + ( − M ) Xk − Xk +1 2 ,
2αk 2
53
Mathematics 2022, 10, 1040
where the second inequality follows from Taylor expansion of ϕ( Xk+1 ). By Equation (12), set
1 1
0 < αk < α := min{ , }, (13)
L M + 2η L M + 2γ
the conditions in linear search Equation (8) and non-restart Equation (9) are both satisfied.
Therefore, the backtracking linear search is well-defined.
where ρ1 = min{η, γ} > 0. Since ϕ is a bounded function, then there exists ϕ∗ such that
ϕ( Xk ) ϕ∗ and ϕ( Xk ) → ϕ∗ as k → +∞. This implies
∞
ρ1 ∑ Xk +1 − Xk 2
ϕ( X0 ) − ϕ∗ < +∞,
k =0
1 2
Yk+1 = argmin{ X − (Yk − αk ϕ(Yk )) },
X 2αk
we have
1
0 = ϕ(Yk ) + (Y − Yk ).
α k k +1
Thus,
1
ϕ(Yk ) = (Y − X k + 1 ) .
αk k
Note that Yk = (1 + wk ) Xk − wk Xk−1 , then
1
ϕ(Yk ) = (1 + w k ) Xk − w k Xk −1 − Xk +1
αk
1
= w ( X − Xk −1 ) + ( Xk − Xk +1 )
αk k k (14)
1
( w Xk − Xk −1 + Xk − Xk +1 )
αmin
= c1 ( Xk +1 − Xk + w Xk − Xk −1 ),
where c1 = α 1 > 0.
min
If k ∈ Ω2 , then
1 2
Xk+1 = argmin{ X − ( Xk − αk ϕ( Xk )) },
X 2αk
54
Mathematics 2022, 10, 1040
Thus
1 1
ϕ ( Xk ) = X − Xk +1 X − Xk +1 = c 1 ( Xk − Xk +1 ), (15)
αk k αmin k
ϕ ( Xk ) c1 ( Xk +1 − Xk + w Xk − Xk −1 ).
2
ϕ ( Xk ) − ϕ ( Xk +1 ) c0 Xk − Xk +1 , k = 1, 2, . . . (16)
Therefore,
lim Xk − Xk+1 = 0.
k →+∞
lim ϕ( Xk j ) = ϕ( X ∗ ), lim ϕ( Xk j ) = ϕ( X ∗ ) = 0,
j→+∞ j→+∞
ψ ( ϕ( x ) − ϕ( x )) ϕ( x ) 1.
Theorem 1. Assume that Propositions 4 and 5 are met. Let { Xk } be the sequence generated by
A-APG Algorithm 3. Then, there exists a point X ∗ ∈ M so that lim Xk = X ∗ and ϕ( X ∗ ) = 0.
k→+∞
Proof. Let ω ( X0 ) be the set of limiting points of the sequence { Xk }. Based on the bound-
edness of { Xk } and the fact that ω ( X0 ) = ∩q∈ N ∪k>q { Xk }, it follows that ω ( X0 ) is a
non-empty and compact set. In addition, by Equation (16), we know that ϕ( X ) is a con-
stant on ω ( X0 ), denoted by ϕ∗ . If there exists some k0 such that ϕ( Xk0 ) = ϕ∗ , then for
∀k > k0 , we have ϕ( Xk ) = ϕ∗ . Next, we assume that ∀k, ϕ( Xk ) > ϕ∗ . Therefore, for
55
Mathematics 2022, 10, 1040
∀, η > 0, ∃l > 0, for ∀k > l we have dist(ω ( X0 ), Xk ) and ϕ∗ < ϕ( Xk ) < ϕ∗ + η i.e.,
for ∀ X ∗ ∈ ω ( X0 ), X ∈ Γη ( X ∗ , ). Applying Proposition 5, for ∀k > l, we have
ψ ( ϕ( Xk ) − ϕ∗ ) ϕ( Xk ) 1.
Then
1
ψ ( ϕ ( Xk ) − ϕ ∗ ) . (17)
c1 ( Xk − Xk −1 + w Xk −1 − Xk −2 )
By the convexity of ψ, it is obvious that
Define
c 0 Xk +1 − Xk 2
k,k+1
c1 ( Xk − Xk −1 + w Xk −1 − Xk −2 )
(19)
Xk +1 − Xk 2
.
c ( Xk − Xk −1 + Xk −1 − Xk −2 )
1
2 Xk +1 − Xk ( Xk − Xk−1 + Xk−1 − Xk−2 ) + 2ck,k+1 .
2
Therefore, for ∀k > l, summing up the above inequality for i = l + 1, . . . , k, we obtain
k
1 k k
2 ∑ Xi + 1 − Xi
2 i=∑
( Xi − Xi−1 + Xi−1 − Xi−2 ) + 2c ∑ i,i+1
i = l +1 l +1 i = l +1
k
1
∑ Xi + 1 − Xi + X l + 1 − X l + X − Xl −1
2 l
i = l +1
+ 2cl +1,k+1 .
k
1
∑ Xi + 1 − Xi X l + 1 − X l + X − Xl −1 + 2cψ( ϕ( Xl ) − ϕ∗ ),
i = l +1
2 l
5. Numerical Results
In this section, we offer two corresponding numerical examples to illustrate the
efficiency of the derived algorithms. All code is written in Python language. Denote
iteration and error by the iteration step and error of the objective function. We take the
matrix order “n” as 128, 1024, 2048, and 4096.
56
Mathematics 2022, 10, 1040
Example 1. Let
⎛ ⎞ ⎛ ⎞
2 −1 1 0.5
⎜ −1 2 −1 ⎟ ⎜0.5 1 0.5 ⎟
⎜ ⎟ ⎜ ⎟
⎜ .. .. .. ⎟ ⎜ .. .. .. ⎟
A1 = ⎜
⎜ . . . ⎟, B1 = ⎜
⎟ ⎜ . . . ⎟
⎟
⎜ .. .. ⎟ ⎜ .. .. ⎟
⎝ . . −1 ⎠ ⎝ . . 0.5 ⎠
−1 2 0.5 1
be tridiagonal matrices in the Sylvester Equation (1). Set the matrix C1 as the identity matrix. The
initial step size is 0.01, which is small enough to iterate. The parameters are η1 = 0.25, ω1 = 0.2
taken from (0,1) randomly. Table 1 and Figure 1 show the numerical results of Algorithms 1–5.
It can be seen that the LGD, A-APG, and Newton-APG Algorithms are more efficient than other
methods. Moreover, the iteration step does not increase when the matrix order increases due to the
same initial value. The A-APG method has higher error accuracy compared with other methods.
The Newton-APG method takes more CPU time and fewer iteration steps than the A-APG method.
The Newton method needs to calculate the inverse of the matrix, while it has quadratic convergence.
From Figure 1, the error curves of the LGD, A-APG, and Newton-APG algorithms are hard to
distinguish. We offer another example below.
57
Mathematics 2022, 10, 1040
Figure 1. The error curves when n = 128, 1024, 2048, 4096 for Example 1.
Remark 1. The difference of the iteration step in Examples 1 and 2 emerges due to the given
different initial values. It can be seen that the LGD, A-APG, and Newton-APG algorithms have
fewer iteration steps. Whether the A-APG method or Newton-APG yields fewer iteration steps
varies from problem to problem. From Examples 1 and 2, we observe that the A-APG method has
higher accuracy, although it takes more time and more iteration steps than the LGD method.
Remark 2. Moreover, we compare the performance of our methods with other methods such as the
conjugate gradient method (CG) in Tables 1 and 2. We take the same initial values and set the error
to 1 × 10−14 . From Tables 1 and 2, it can be seen that the LGD and A-APG methods are more
efficient for solving the Sylvester matrix equation when the order n is small. When n is large, the
LGD and A-APG methods nearly have a convergence rate with the CG method.
Figure 2. The error curves when n = 128, 1024, 2048, 4096 for Example 2.
58
Mathematics 2022, 10, 1040
6. Conclusions
In this paper, we have introduced the A-APG and Newton-APG methods for solving
the Sylvester matrix equation. The key idea is to change the Sylvester matrix equation to
an optimization problem by using the Kronecker product. Moreover, we have analyzed the
computation complexity and proved the convergence of the A-APG method. Convergence
results and preliminary numerical examples have shown that the schemes are promising in
solving the Sylvester matrix equation.
Author Contributions: J.Z. (methodology, review, and editing); X.L. (software, visualization, data
curation). All authors have read and agreed to the published version of the manuscript.
Funding: The work was supported in part by the National Natural Science Foundation of China
(12171412, 11771370), Natural Science Foundation for Distinguished Young Scholars of Hunan
Province (2021JJ10037), Hunan Youth Science and Technology Innovation Talents Project (2021RC3110),
the Key Project of the Education Department of Hunan Province (19A500, 21A0116).
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: Not applicable.
Conflicts of Interest: The authors declare no conflict of interest.
59
Mathematics 2022, 10, 1040
References
1. Dooren, P.M.V. Structured Linear Algebra Problems in Digital Signal Processing; Springer: Berlin/Heidelberg, Germany, 1991.
2. Gajic, Z.; Qureshi, M.T.J. Lyapunov Matrix Equation in System Stability and Control; Courier Corporation: Chicago, IL, USA, 2008.
3. Corless, M.J.; Frazho, A. Linear Systems and Control: An Operator Perspective; CRC Press: Boca Raton, FL, USA, 2003.
4. Stewart, G.W.; Sun, J. Matrix Perturbation Theory; Academic Press: London, UK, 1990.
5. Simoncini, V.; Sadkane, M. Arnoldi-Riccati method for large eigenvalue problems. BIT Numer. Math. 1996, 36, 579–594.
6. Demmel, J.W. Three methods for refining estimates of invariant subspaces. Computing 1987, 38, 43–57.
7. Chen, T.W.; Francis, B.A. Optimal Sampled-Data Control Systems; Springer: London, UK, 1995.
8. Datta, B. Numerical Methods for Linear Control Systems; Elsevier Inc.: Amsterdam, The Netherlands, 2004.
9. Lord, N. Matrix computations. Math. Gaz. 1999, 83, 556–557.
10. Zhao, X.L.; Wang, F.; Huang, T.Z. Deblurring and sparse unmixing for hyperspectral images. IEEE Trans. Geosci. Remote Sens.
2013, 51, 4045–4058.
11. Obinata, G.; Anderson, B.D.O. Model Reduction for Control System Design; Springer Science & Business Media: London, UK, 2001.
12. Bouhamidi, A.; Jbilou, K. A note on the numerical approximate solutions for generalized Sylvester matrix equations with
applications. Appl. Math. Comput. 2008, 206, 687–694.
13. Bai, Z.Z.; Benzi, M.; Chen, F. Modified HSS iteration methods for a class of complex symmetric linear systems. Computing 2010,
87, 93–111.
14. Bartels, R.H.; Stewart, G.W. Solution of the matrix equation AX + XB = C. Commun. ACM 1972, 15, 820–826.
15. Golub, G.H. A Hessenberg-Schur Method for the Problem AX + XB = C; Cornell University: Ithaca, NY, USA, 1978.
16. Robbé, M.; Sadkane, M. A convergence analysis of GMRES and FOM methods for Sylvester equations. Numer. Algorithms 2002,
30, 71–89.
17. Guennouni, A.E.; Jbilou, K.; Riquet, A.J. Block Krylov subspace methods for solving large Sylvester equations. Numer. Algorithms
2002, 29, 75–96.
18. Salkuyeh, D.K.; Toutounian, F. New approaches for solving large Sylvester equations. Appl. Math. Comput. 2005, 173, 9–18.
19. Wachspress, E.L. Iterative solution of the Lyapunov matrix equation. Appl. Math. Lett. 1988, 1, 87–90.
20. Jbilou, K.; Messaoudi, A.; Sadok, H. Global FOM and GMRES algorithms for matrix equations. Appl. Numer. Math. 1999,
31, 49–63.
21. Feng, D.; Chen, T. Gradient Based Iterative Algorithms for Solving a Class of Matrix Equations. IEEE Trans. Autom. Control 2005,
50, 1216–1221.
22. Heyouni, M.; Movahed, F.S.; Tajaddini, A. On global Hessenberg based methods for solving Sylvester matrix equations. Comput.
Math. Appl. 2019, 77, 77–92.
23. Benner, P.; Kürschner, P. Computing real low-rank solutions of Sylvester equations by the factored ADI method. Comput. Math.
Appl. 2014, 67, 1656–1672.
24. Bouhamidi, A.; Hached, M.; Heyouni, M.J.; Bilou, K. A preconditioned block Arnoldi method for large Sylvester matrix equations.
Numer. Linear Algebra Appl. 2013, 20, 208–219.
25. Heyouni, M. Extended Arnoldi methods for large low-rank Sylvester matrix equations. Appl. Numer. Math. 2010, 60, 1171–1182.
26. Agoujil, S.; Bentbib, A.H.; Jbilou, K.; Sadek, E.M. A minimal residual norm method for large-scale Sylvester matrix equations.
Electron. Trans. Numer. Anal. Etna 2014, 43, 45–59.
27. Abdaoui, I.; Elbouyahyaoui, L.; Heyouni, M. An alternative extended block Arnoldi method for solving low-rank Sylvester
equations. Comput. Math. Appl. 2019, 78, 2817–2830.
28. Jbilou, K. Low rank approximate solutions to large Sylvester matrix equations. Appl. Math. Comput. 2005, 177, 365–376.
29. Liang, B.; Lin, Y.Q.; Wei, Y.M. A new projection method for solving large Sylvester equations. Appl. Numer. Math. 2006,
57, 521–532.
30. Jiang, K.; Si, W.; Chen, C.; Bao, C. Efficient numerical methods for computing the stationary states of phase-field crystal models.
SIAM J. Sci. Comput. 2020, 42, B1350–B1377.
31. Beck, A.; Teboulle, M. A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems. SIAM J. Imaging Sci. 2009,
2, 183–202.
32. Bao, C.; Barbastathis, G.; Ji, H.; Shen, Z.; Zhang, Z. Coherence retrieval using trace regularization. SIAM J. Imaging Sci. 2018,
11, 679–706.
33. Magnus, J.R.; Neudecker, H. Matrix Differential Calculus; John Willey & Sons: Hoboken, NJ, USA, 1999.
60
mathematics
Article
Generalized Three-Step Numerical Methods for Solving
Equations in Banach Spaces
Michael I. Argyros 1 , Ioannis K. Argyros 2, *, Samundra Regmi 3, * and Santhosh George 4
Abstract: In this article, we propose a new methodology to construct and study generalized three-step
numerical methods for solving nonlinear equations in Banach spaces. These methods are very general
and include other methods already in the literature as special cases. The convergence analysis of the
specialized methods is been given by assuming the existence of high-order derivatives which are
not shown in these methods. Therefore, these constraints limit the applicability of the methods to
equations involving operators that are sufficiently many times differentiable although the methods
may converge. Moreover, the convergence is shown under a different set of conditions. Motivated by
the optimization considerations and the above concerns, we present a unified convergence analysis
for the generalized numerical methods relying on conditions involving only the operators appearing
in the method. This is the novelty of the article. Special cases and examples are presented to conclude
this article.
Citation: Argyros, M.I.; Argyros, I.K.; Keywords: generalized three-step numerical method; convergence; Banach space
Regmi, S.; George, D. Generalized
Three-Step Numerical Methods for MSC: 49M15; 47H17; 65J15; 65G99; 47H17; 41A25; 49M15
Solving Equations in Banach Spaces.
Mathematics 2022, 10, 2621. https://
doi.org/10.3390/math10152621
1. Introduction
Academic Editors: Maria Isabel
Berenguer and Manuel Ruiz Galán A plethora of applications from diverse disciplines of computational sciences are
converted to nonlinear equations such as
Received: 8 July 2022
Accepted: 26 July 2022 F(x) = 0 (1)
Published: 27 July 2022
using modeling (mathematical) [1–4]. The nonlinear operator F is defined on an open and
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
convex subset Ω of a Banach space X with values in X. The solution of the equation is
published maps and institutional affil-
denoted by x∗ . Numerical methods are mainly used to find x∗ . This is the case since the
iations. analytic form of the solution x∗ is obtained in special cases.
Researchers, as well as practitioners, have proposed numerous numerical methods
under a different set of convergence conditions using high-order derivatives, which are not
present in the methods.
Copyright:
c 2022 by the authors. Let us consider an example.
Licensee MDPI, Basel, Switzerland.
This article is an open access article Example 1. Define the function F on X = [−0.5, 1.5] by
distributed under the terms and
!
conditions of the Creative Commons t3 ln t2 + t5 − t4 , t = 0
F (t) =
Attribution (CC BY) license (https:// 0, t = 0
creativecommons.org/licenses/by/
4.0/).
yn = an = a( xn )
zn = bn = b ( x n , y n ) (2)
x n +1 = c n = c ( x n , y n , z n ),
yn = xn + αn F ( xn )
zn = u n + β n F ( x n ) + γn F ( y n ) (3)
x n +1 = vn + δn F ( xn ) + n F (yn ) + θn F (zn ),
yn = xn − sF ( xn )−1 F ( xn )
zn = xn − O4,n F ( xn ) (5)
x n +1 = zn − O5,n F (zn ),
yn = x n − F ( x n ) −1 F ( x n )
zn = yn − O6,n F ( xn )−1 F (yn ) (6)
x n +1 = zn − O7,n F ( xn )−1 F (zn ),
y n = x n − F ( x n ) −1 F ( x n ). (7)
62
Mathematics 2022, 10, 2621
2
y n = x n − F ( x n ) −1 F ( x n ). (8)
3
Traub-type method (fifth order) [14]:
yn = x n − F ( x n ) −1 F ( x n )
zn = x n − F ( x n ) −1 F ( y n ) (9)
x n +1 = x n − F ( x n ) −1 F ( z n ).
yn = x n − F ( x n ) −1 F ( x n )
xn + yn
x n +1 = xn − 6 F ( xn ) + 4F ( ) F ( y n ) −1 F ( x n ). (11)
2
or
yn = x n − F ( x n ) −1 F ( x n ) (12)
/ 0
3xn + yn xn + yn xn + 3yn −1
x n +1 = xn − 2 2F ( ) − F ( ) + 2F ( ) F ( x n ).
4 2 4
yn = x n − F ( x n ) −1 F ( x n )
/ 0 −1
2xn + yn
x n +1 = xn − 4 3F ( ) + F (yn ) F ( x n ). (13)
3
yn = x n − F ( x n ) −1 F ( x n )
21 2
x n +1 = xn − (3F (yn ) − F ( xn ))−1 + F ( xn )−1 F ( xn ). (14)
3
Corder–Torregrosa method (fifth order) [2]:
2
yn = x n − F ( x n ) −1 F ( x n )
3
1
zn = xn − (3F (yn ) − F ( xn ))−1 (3F (yn ) + F ( xn )) F ( xn )−1 F ( xn ) (15)
2
1 1
x n +1 = zn − ( F (yn ) + F ( xn ))−1 F (zn ).
2 2
or
yn = x n − F ( x n ) −1 F ( x n )
zn = xn − 2( F (yn ) + F ( xn ))−1 F ( xn ) (16)
x n +1 = z n − F ( y n ) −1 F ( z n ).
63
Mathematics 2022, 10, 2621
yn = x n − F ( x n ) −1 F ( x n )
x n +1 = xn − (2F (yn )−1 − F ( xn )−1 ) F ( xn ). (17)
yn = x n − [ w n , x n ; F ] −1 F ( x n )
wn = xn + dF ( xn ), (19)
yn = x n − [ x n , w n ; F ] −1 F ( x n )
x n +1 = yn − ([yn , wn ; F ] + [yn , xn ; F ] − [ xn , wn ; F ])−1 F (yn ). (20)
yn = x n − [ w n , x n ; F ] −1 F ( x n )
zn = M8 ( xn , yn ) (21)
x n +1 = zn − ([zn , xn ; F ] + [zn , yn ; F ] − [yn , xn ; F ])−1 F (zn ),
yn = x n − [ w n , x n ; F ] −1 F ( x n )
zn = yn − (3I − [wn , xn ; F ]( [yn , xn ; F ] + [yn , wn ; F ])) (22)
[ w n , x n ; F ] −1 ) F ( y n )
x n +1 = zn − [zn , yn ; F ]−1 ([wn , xn ; F ]
+[yn , xn ; F ] − [zn , xn ; F ])[wn , xn ; F ]−1 F (zn ).
The local, as well as the semi-local, convergence for methods (4) and (5), were pre-
sented in [17], respectively, using hypotheses relating only to the operators on these meth-
ods. However, the local convergence analysis of method (6) requires the usage of derivatives
or divided differences of higher than two orders, which do not appear in method (6). These
high-order derivatives restrict the applicability of method (6) to equations whose operator
F has high-order derivatives, although method (6) may converge (see Example 1).
Similar restrictions exist for the convergence of the aforementioned methods of order
three or above.
It is also worth noticing that the fifth convergence order method by Sharma [18]
yn = x n − F ( x n ) −1 F ( x n )
zn = yn − 5F ( xn )−1 F (yn ) (23)
1
x n +1 = yn − [9F ( xn )−1 F (yn ) + F ( xn )−1 F (zn )]
5
64
Mathematics 2022, 10, 2621
cannot be handled with the analyses given previously [5–7] for method (4), method (5), or
method (6).
Based on all of the above, clearly, it is important to study the convergence of method (2)
and its specialization method (3) with the approach employed for method (4) or (5). This
way, the resulting unified convergence criteria can apply to their specialized methods listed
or not listed previously. Hence, this is the motivation as well as the novelty of the article.
There are two important types of convergence: the semi-local and the local. The
semi-local uses information involving the initial point to provide criteria, assuring the
convergence of the numerical method, while the local one is based on the information
about the solution to find the radii of the convergence balls.
The local convergence results are vital, although the solution is unknown in general
since the convergence order of the numerical method can be found. This kind of result also
demonstrates the degree of difficulty in selecting starting points. There are cases when the
radius of convergence of the numerical method can be determined without the knowledge
of the solution.
As an example, let X = R. Suppose function F satisfies an autonomous differen-
tial [5,21] equation of the form
H ( F (t)) = F (t),
where H is a continuous function. Notice that H ( F (t∗ )) = F (t∗ ) or F (t∗ ) = H (0). In the
case of F (t) = et − 1, we can choose H (t) = t + 1 (see also the numerical section).
Moreover, the local results can apply to projection numerical methods, such as
Arnoldi’s, the generalized minimum residual numerical method (GMRES), the generalized
conjugate numerical method (GCS) for combined Newton/finite projection numerical
methods, and in relation to the mesh independence principle to develop the cheapest and
most efficient mesh refinement techniques [1,5,11,21].
In this article, we introduce a majorant sequence and use our idea of recurrent functions
to extend the applicability of the numerical method (2). Our analysis includes error bounds
and results on the uniqueness of x∗ based on computable Lipschitz constants not given
before in [5,13,21–24] and in other similar studies using the Taylor series. This idea is very
general. Hence, it applies also to other numerical methods [10,14,22,25].
The convergence analysis of method (2) and method (3) is given in Section 2. Moreover,
the special choices of operators appear in the method in Sections 3 and 4. Concluding
remarks, open problems, and future work complete this article.
ρ = min{ρi } (25)
shall be shown to be a radius of convergence for method (2). Let S1 = [0, ρ). It follows by
the definition of radius ρ that for all t ∈ S1
The notation U ( x, ς) denotes an open ball with center x ∈ X and of radius ς > 0. By
U [ x, ς], we denote the closure of U ( x, ς).
The following conditions are used in the local convergence analysis of the method (2).
Suppose the following:
65
Mathematics 2022, 10, 2621
b( x, y) − x∗ ≤ h2 ( x − x∗ , y − x∗ ) x − x∗
and
c( x, y, z) − x∗ ≤ h3 ( x − x∗ , y − x∗ , z − x∗ ) x − x∗
for all x, y, z ∈ Ω0 = Ω ∩ U ( x∗ , ρ0 ).
(H3) Equations (24) have smallest solutions ρi ∈ S0 − {0};
(H4) U [ x∗ , ρ] ⊂ Ω, where the radius ρ is given by Formula (25).
Next, the main local convergence analysis is presented for method (2).
Theorem 1. Suppose that the conditions (H1)–(H4) hold and x0 ∈ U ( x∗ , r ) − { x∗ }. Then, the
sequence { xn } generated by method (2) is well defined and converges to x∗ . Moreover, the following
estimates hold ∀ n = 0, 1, 2, . . .
y n − x ∗ ≤ h1 ( x n − x ∗ ) x n − x ∗ ≤ x n − x ∗ < ρ (27)
z n − x ∗ ≤ h2 ( x n − x ∗ , y n − x ∗ ) x n − x ∗ ≤ x n − x ∗ (28)
and
x n +1 − x ∗ ≤ h3 ( x n − x ∗ , y n − x ∗ , z n − x ∗ ) x n − x ∗ ≤ x n − x ∗ . (29)
Proof. Let x0 ∈ U ( x∗ , ρ0 ). Then, it follows from the first condition in (H1) the definition of
ρ, (26) (for i = 1) and the first substep of method (2) for n = 0 that
y0 − x∗ ≤ h1 ( x0 − x∗ ) x0 − x∗ ≤ x0 − x∗ < ρ, (30)
z0 − x ∗ ≤ h2 ( x0 − x ∗ , y0 − x ∗ ) x0 − x ∗
≤ h2 ( x0 − x ∗ , y0 − x ∗ )
≤ h2 ( x0 − x ∗ , x0 − x ∗ ) x0 − x ∗ ≤ x0 − x ∗ (31)
and
x1 − x ∗ ≤ h3 ( x0 − x ∗ , y0 − x ∗ , z0 − x ∗ ) x0 − x ∗
≤ h3 ( x0 − x ∗ , x0 − x ∗ , x0 − x ∗ ) x0 − x ∗
≤ x0 − x ∗ ,
showing estimates (28), (29), respectively and the iterates z0 , x1 ∈ U ( x∗ , ρ). By simply
replacing x0 , y0 , z0 , x1 by xk , yk , zk , xk+1 in the preceding calculations, the induction for
estimates (27)–(29) is terminated. Then, from the estimate
xk+1 − x∗ ≤ d xk − x∗ < ρ,
where
d = h3 ( x0 − x∗ , x0 − x∗ , x0 − x∗ ) ∈ [0, 1) (32)
we conclude xk+1 ∈ U [ x∗ , ρ] and limk−→∞ xk = x∗ .
Remark 1. It follows from the proof of Theorem 1 that y, z can be chosen in particular as yn = a( xn )
and zn = b( xn , yn ). Thus, the condition (H2) should hold for all x, a( x ), b( x, y) ∈ Ω0 and not
x, y, z ∈ Ω0 . Clearly, in this case, the resulting functions hi are at least as tight as the functions hi ,
leading to an at least as large radius of convergence ρ̄ as ρ (see the numerical section).
66
Mathematics 2022, 10, 2621
Concerning the semi-local convergence of method (2), let us introduce scalar sequences
{tn }, {sn } and {un } defined for t0 = 0, s0 = η ≥ 0 and the rest of the iterates, depending
on operators a, b, c and F (see how in the next section). These sequences shall be shown to
be majorizing for method (2). However, first, a convergence result for these sequence is
needed.
t n ≤ s n ≤ u n ≤ t n +1 (33)
and
tn ≤ λ (34)
for some λ ≥ 0. Then, the sequence {tn } is convergent to its unique least upper bound t∗ ∈ [0, λ].
Proof. It follows from conditions (33) and (34) that sequence {tn } is nondecreasing and
bounded from above by λ, and as such, it converges to t∗ .
b( xn , yn ) − yn ≤ un − sn
and
c ( x n , y n , z n ) − z n ≤ t n +1 − u n
for all n = 0, 1, 2, . . . and
(H7) U [ x0 , t∗ ] ⊂ Ω.
Then, there exists x∗ ∈ U [ x0 , t∗ ] such that limn−→∞ xn = x∗ .
Proof. It follows by condition (H5) that sequence {tn } is complete as convergent. Thus,
by condition (H6), sequence { xn } is also complete in a Banach space X, and as such, it
converges to some x∗ ∈ U [ x0 , t∗ ] (since U [ x0 , t∗ ] is a closed set).
Remark 2. (i) Additional conditions are needed to show F ( x∗ ) = 0. The same is true for the results
on the uniqueness of the solution.
(ii) The limit point t∗ is not given in the closed form. So, it can be replaced by λ in Theorem 2.
3. Special Cases I
The iterates of method (3) are assumed to exist, and operator F has a divided difference
of order one.
Local Convergence
Three possibilities are presented for the local cases based on different estimates for the
determination of the functions hi . It follows by method (3) that
(P1) yn − x∗ = xn − x∗ + αn F ( xn ) = ( I + αn [ xn , x∗ ; F ])( xn − x∗ ),
zn − x∗ = ( I + γn [yn , x∗ ; F ])(yn − x∗ ) + β n [ xn , x∗ ; F ]( xn − x∗ )
= [( I + γn [yn , x∗ ; F ])( I + αn [ xn , x∗ ; F ]) + β n [ xn , x∗ ; F ]]( xn − x∗ )
67
Mathematics 2022, 10, 2621
and
x n +1 − x ∗ = ( I + θn [zn , x∗ ; F ])(zn − x∗ ) + δn [ xn , x∗ ; F ]( xn − x∗ )
+n [yn , x∗ ; F ](yn − x∗ )
= [( I + θn [zn , x∗ ; F ])( I + γn [yn , x∗ ; F ])( I + β n [ xn , x∗ ; F ])
+δn [ xn , x∗ ; F ] + n [yn , x∗ ; F ]( I + αn [ xn , x∗ ; F ])]( xn − x∗ )
I + α n [ x n , x ∗ ; F ] ≤ h1 ( x n − x ∗ ),
( I + γn [yn , x∗ ; F ])( I + αn [ xn , x∗ ; F ]) + β n [ xn , x∗ ; F ] ≤ h2 ( xn − x∗ , yn − x∗ )
I + α( x )[ x, x∗ ; F ] ≤ h1 ( x − x∗ ) ∀ x ∈ Ω.
h1 ( t ) = sup I + α( x )[ x, x∗ ; F ] .
x ∈Ω, x − x∗ ≤t
yn − x∗ = xn − x∗ − Mn1 F ( xn ) + (αn + Mn ) F ( xn )
= ( I − Mn2 [ xn , x∗ ; F ]) + (αn + Mn2 )[ xn , x∗ ; F ])( xn − x∗ ),
zn − x∗ = (( I − Mn2 [yn , x∗ ; F ]) + (γn + Mn2 )[yn , x∗ ; F ])(yn − x∗ )
and
xn+1 − x∗ = (( I − Mn3 [zn , x∗ ; F ]) + (θn + Mn3 )[zn , x∗ ; F ])(zn − x∗ ).
Thus, the functions hi must satisfy
I + α n ≤ h1 ( x n − x ∗ ),
( I + γn )( I + αn ) ≤ h2 ( xn − x∗ , yn − x∗ )
and
xn+1 − x∗ ≤ ( I + θn )( I + γn )( I + αn ) ≤ h3 ( xn − x∗ , yn − x∗ , zn − x∗ ).
Clearly, the function h1 can be chosen again as in case (P1). The functions h2 and h3
can be defined similarly.
(P3) Assume ∃ function ϕ0 : [0, ∞) −→ R continuous and non-decreasing such that
F ( x∗ )−1 ( F ( x ) − F ( x∗ )) ≤ ϕ0 ( x − x∗ ) ∀ x ∈ Ω.
68
Mathematics 2022, 10, 2621
leading to
1
F ( x ∗ ) −1 F ( x n ) ≤ ϕ0 (θ xn − x∗ )dθ xn − x∗ .
0
Then, by method (3) we obtain, in turn, that
yn − x∗ = [ I + α n F ( x ∗ ) F ( x ∗ ) −1
1
× F ( x∗ + θ ( xn − x∗ ))dθ − F ( x∗ ) + F ( x∗ ) ]( xn − x∗ ),
0
or 1
h1 ( t ) = sup I + α( x ) F ( x∗ + θ ( xn − x∗ ))dθ
x − x∗ ≤t, x ∈Ω 0
or 1
I + α n F ( x ∗ ) (1 + ϕ0 (θ xn − x∗ )dθ ) ≤ h1 ( xn − x∗ )
0
or 1
h1 ( t ) = sup I + α ( x ) F ( x ∗ ) (1 + ϕ0 (θ xn − x∗ )dθ ).
x − x∗ ≤t, x ∈Ω 0
Similarly, for the other two steps, we obtain in the last choice
1
zn − x∗ ≤ I + γn F ( x ∗ ) ( 1 + ϕ0 (θ yn − x∗ )dθ ) yn − x∗
0
1
+ β n F ( x ∗ ) (1 + ϕ0 (θ xn − x∗ )dθ ) xn − x∗
0
and
1
x n +1 − x ∗ ≤ I + θ n F ( x ∗ ) (1 + ϕ0 (θ zn − x∗ )dθ ) zn − x∗
0
1
+ δn F ( x∗ ) (1 + ϕ0 (θ xn − x∗ )dθ ) xn − x∗
0
1
+ n F ( x ∗ ) (1 + ϕ0 (θ yn − x∗ )dθ ) yn − x∗ .
0
or
h2 (s, t) = sup [ I + γ( x ) F ( x∗ )
x − x∗ ≤s, y− x∗ ≤t
1
×(1 + ϕ0 (θt)dθ )t)
0
1
+ β ( x ) F ( x ∗ ) (1 + ϕ0 (θs)dθ )].
0
69
Mathematics 2022, 10, 2621
Finally, concerning the choice of the function h3 , by the third substep of method (3)
1
x n +1 − x ∗ ≤ I + θ n F ( x ∗ ) (1 + ϕ0 (θ zn − x∗ )dθ ) zn − x∗
0
1
+ δn F ( x∗ ) (1 + ϕ0 (θ xn − x∗ )dθ ) xn − x∗
0
1
+ n F ( x ∗ ) (1 + ϕ0 (θ yn − x∗ )dθ ) yn − x∗ ,
0
or
h( x, s, t, u) = sup μ( x, s, t, u),
x − x∗ ≤s, y− x∗ ≤t, z− x∗ ≤u
where
μ( x, s, t, u) = I + θ ( x ) F ( x∗ )
1
×(1 + ϕ0 (θu)dθ )h2 (t, s)
0
1
+ δ ( x ) F ( x ∗ ) (1 + ϕ0 (θs)dθ )
0
1
+ ( x ) F ( x ∗ ) (1 + ϕ0 ((θt)dθ )h1 (s)].
0
The functions h2 and h3 can also be defined with the other two choices as those of
function h1 given previously.
Semi-local Convergence
Concerning this case, we can have instead of the conditions of Theorem 2 (see (H6))
but for method (3)
αn F ( xn ) ≤ sn − tn ,
β n F ( x n ) + γn F ( y n ) ≤ u n − s n
and
δn F ( xn ) + n F (yn ) + θn F (zn ) ≤ tn+1 − un ∀n = 0, 1, 2, . . . .
Notice that under these choices,
yn − xn ≤ sn − tn
zn − yn ≤ un − sn
and
x n +1 − z n ≤ t n +1 − u n .
Then, the conclusions of Theorem 2 hold for method (3). Even more specialized choices
of linear operators appearing on these methods as well as function hi can be found in the
Introduction, the next section, or in [1,2,11,21] and the references therein.
70
Mathematics 2022, 10, 2621
4. Special Cases II
The section contains even more specialized cases of method (2) and method (3). In
particular, we study the local and semi-local convergence first of method (22) and second
of method (20). Notice that to obtain method (22), we set in method (3)
αn = −[ xn , wn ; F ]−1 , un = yn , β n = O, zn = xn+1 ,
γn = ([yn , wn ; F ] + [yn , xn ; F ] − [ xn , wn ; F ])−1 , δn = n = θn = O (36)
and vn = zn .
w0 ( t ) − 1 = 0
h1 ( t ) − 1 = 0
(iii) Equation
w0 ( h 1 ( t ) t ) − 1 = 0
has a smallest solution ρ̄1 ∈ S0 − {0}. Let
71
Mathematics 2022, 10, 2621
(v) Equation
h3 ( t ) − 1 = 0
has a smallest solution ρ3 ∈ S̃1 − {0}, where the function h3 : S̃1 −→ R is defined by
1
1 9(1 + w0 (θh1 (t)t)dθ )h1 (t)
h3 ( t ) = h1 ( t ) + [ 0
5 1 − w0 ( t )
1
(1 + w0 (θh2 (t)t)dθ )h2 (t)].
0
ρ = min{ρ j } j = 1, 2, 3 (37)
is proven to be a radius of convergence for method (2) in Theorem 3. Let S1 = [0, ρ). Then,
it follows by these definitions that ∀ t ∈ S2
0 ≤ w0 ( t ) < 1 (38)
0 ≤ w0 ( h 1 ( t ) t ) < 1 (39)
and
0 ≤ hi (t) < 1. (40)
The conditions required are as follows:
(C1) Equation F ( x ) = 0 has a simple solution x∗ ∈ Ω.
(C2) F ( x∗ )−1 ( F ( x ) − F ( x∗ )) ≤ w0 ( x − x∗ ) ∀ x ∈ Ω.
Set Ω1 = U ( x∗ , ρ0 ) ∩ Ω.
(C3) F ( x∗ )−1 ( F (y) − F ( x )) ≤ w( y − x ) ∀ x, y ∈ Ω1
and
(C4) U [ x0 , ρ] ⊂ Ω.
Next, the main local convergence result follows for method (23).
yn − x∗ ≤ h1 ( xn − x∗ ) xn − x∗ ≤ xn − x∗ < ρ, (41)
z n − x ∗ ≤ h2 ( x n − x ∗ ) x n − x ∗ ≤ x n − x ∗ , (42)
and
x n +1 − x ∗ ≤ h3 ( x n − x ∗ ) x n − x ∗ ≤ x n − x ∗ , (43)
where functions hi are defined previously and the radius ρ is given by Formula (37).
Proof. Let u ∈ U ( x∗ , ρ) − { x∗ }. By using conditions (C1), (C2) and (37), we have that
It follows by (44) and the Banach lemma on invertible operators [11,15] that F (u)−1 ∈
L( X, X ) and
1
F ( u ) −1 F ( x ∗ ) ≤ . (45)
1 − w0 ( x 0 − x ∗ )
If u = x0 , then the iterate y0 is well defined by the first substep of method (23) and we
can write
72
Mathematics 2022, 10, 2621
y0 − x ∗ = x0 − x ∗ − F ( x0 ) −1 F ( x0 )
1
= F ( x0 ) −1 ( F ( x∗ + θ ( x0 − x∗ ))dθ − F ( x0 ))( x0 − x∗ ). (46)
0
In view of (C1)–(C3), (45) (for u = x0 ), (40) (for i = 1) and (46), we obtain in turn that
1
w((1 − θ ) x0 − x∗ )dθ x0 − x∗
y0 − x ∗ ≤ 0
1 − w0 ( x 0 − x ∗ )
≤ h1 ( x0 − x∗ ) x0 − x∗ < x0 − x∗ < ρ. (47)
Thus, the iterate y0 ∈ U ( x∗ , r ) and (41) holds for n = 0. The iterate z0 is well defined
by the second substep of method (23), so we can write
z0 − x ∗ = y0 − x0 − 5F ( x0 )−1 F (y0 )
= y0 − x ∗ − F ( y0 ) −1 F ( y0 )
+ F (y0 )−1 ( F ( x0 ) − F (y0 )) F ( x0 )−1 F (y0 )
−4F ( x0 )−1 F (y0 ). (48)
Notice that linear operator F (y0 )−1 exists by (45) (for u = y0 ). It follows by (37), (40)
(for j = 1), (C3), (45) (for u = x0 , y0 ), in turn that
3 1
w((1 − θ ) y0 − x∗ )dθ
z0 − x ∗ ≤ 0
1 − w0 ( y 0 − x ∗ )
1
w( y0 − x0 )(1 + 0 w0 (θ y0 − x∗ )dθ )
+
(1 − w0 ( x0 − x∗ ))(1 − w0 ( y0 − x∗ ))
1 4
4(1 + 0 w0 (θ y0 − x∗ )dθ
+ y0 − x ∗
1 − w0 ( x 0 − x ∗ )
≤ h2 ( x0 − x ∗ ) x0 − x ∗ ≤ x0 − x ∗ . (49)
Thus, the iterate z0 ∈ U ( x∗ , ρ) and (42) holds for n = 0, where we also used (C1) and
(C2) to obtain the estimate
1
F ( x ∗ ) −1 F ( y0 ) = F ( x ∗ ) −1 [ F ( x∗ + θ (y0 − x∗ ))dθ − F ( x∗ )
0
+ F ( x∗ )](y0 − x∗ )
1
≤ (1 + w0 (θ y0 − x∗ )dθ ) y0 − x∗ .
0
Moreover, the iterate x1 is well defined by the third substep of method (23), so we
can have
1
x1 − x∗ = y0 − x∗ − F ( x0 )−1 (9F (y0 ) + F (z0 )),
5
leading to
1
1 9(1 + w0 (θ y0 − x∗ )dθ ) y0 − x∗
x1 − x ∗ ≤ y0 − x ∗ + 0
5 1 − w0 ( y 0 − x ∗ )
1
+ (1 + w0 ( θ z 0 − x ∗ )dθ ) z0 − x∗
0
≤ h3 ( x0 − x∗ ) x0 − x∗ ≤ x0 − x∗ < ρ. (50)
73
Mathematics 2022, 10, 2621
e−1 2
F (v) = (ev1 − 1, v + v2 , v3 )tr .
2 2
Using this definition, we obtain the derivative as
⎡ ⎤
e v1 0 0
F (v) = ⎣ 0 ( e − 1) v2 + 1 0 ⎦.
0 0 1
Conditions (C1)–(C3) are verified for w0 (t) = (e − 1)t and w(t) = 2(1 + e−1 ) t.
1
Then, the
radii are
74
Mathematics 2022, 10, 2621
Clearly, x∗ = 0 and the conditions (C1)–(C3) hold for w0 (t) = 9t and w(t) = 18t. Then, the
radii are
ρ1 = 0.0556, ρ2 = 0.0089 = ρ and ρ3 = 0.0206.
t0 = 0, s0 = η
1
5 v(θ (sn − tn ))dθ (sn − tn )
un = sn + 0 ,
1 − v0 ( t n )
1
1
t n +1 = un + [(1 + v0 (un + θ (un − sn ))dθ (un − sn ) (54)
1 − v0 ( t n ) 0
1
+3 v(θ (sn − tn ))dθ (sn − tn )]
0
1
1
s n +1 = t n +1 + [ v(θ (tn+1 − tn ))dθ (tn+1 − tn )
1 − v0 ( t n +1 ) 0
1
+(1 + v0 (θtn )dθ (tn+1 − sn )].
0
This sequence is proven to be majorizing for method (23) in Theorem 4. However, first,
we provide a general convergence result for sequence (54).
v0 ( t n ) < 1 (55)
Proof. It follows by (54)–(56) that sequence {tn } is non-decreasing and bounded from
above by τ. Hence, it converges to its unique least upper bound t∗ .
75
Mathematics 2022, 10, 2621
Theorem 4. Suppose that conditions (h1)–(h5) hold. Then, sequence { xn } given by method (23) is
well defined, remains in U [ x0 , t∗ ] and converges to a solution x∗ ∈ U [ x0 , t∗ ] of equation F ( x ) = 0.
Moreover, the following assertions hold:
yn − xn ≤ sn − tn , (57)
zn − yn ≤ un − sn (58)
and
x n +1 − z n ≤ t n +1 − u n . (59)
Proof. Mathematical induction is utilized to show estimates (57)–(59). Using (h1) and
method (23) for n = 0
y0 − x0 = F ( x0 ) −1 F ( x0 ) ≤ η = s0 − t0 ≤ t ∗ .
1
F ( u ) −1 F ( x0 ) ≤ . (60)
1 − v0 ( u − x0
Hence, if we set u = x0 , iterates y0 , z0 and x1 are well defined by method (23) for n = 0.
Suppose iterates xk , yk , zk , xk+1 also exist for all integer values k smaller than n. Then, we
have the estimates
zn − yn = 5 F ( x n ) −1 F ( y n )
1
5 0 v(θ yn − xn )dθ yn − xn
≤
1 − v0 ( x n − x0 )
1
5 0 v(θ sn − tn ))dθ (sn − tn )
≤ = un − sn ,
1 − v0 ( t n )
1
x n +1 − z n = F ( xn )−1 ( F (yn ) − F (zn )) + 3F ( xn )−1 F (yn )
5
1 1 1
≤ [(1 + v0 ( zn − x0 + θ zn − yn )dθ ) yn − xn
1 − v0 ( x n − x0 ) 5 0
1
+3 v(θ yn − xn dθ yn − xn ]
0
≤ t n +1 − u n
and
y n +1 − x n +1 = F ( x n +1 ) −1 F ( x n +1 )
≤ F ( x n +1 ) −1 F ( x0 ) F ( x0 ) −1 F ( x n +1 )
1
1
≤ [ v(θ xn+1 − xn )dθ xn+1 − xn
1 − v0 ( x n +1 − x0 ) 0
1
+(1 + v0 (θ xn − x0 )dθ ) xn+1 − yn ]
0
≤ s n +1 − t n +1 ,
76
Mathematics 2022, 10, 2621
so 1
F ( x0 ) −1 F ( y n ) ≤ v(θ yn − xn )dθ yn − xn
0
and
F ( x n +1 ) = F ( xn+1 ) − F ( xn ) − F ( xn )(yn − xn )
− F ( xn )( xn+1 − xn ) + F ( xn )( xn+1 − xn )
= F ( xn+1 ) − F ( xn ) − F ( xn )( xn+1 − xn ) + F ( xn )( xn+1 − yn ),
so
1
F ( x0 ) −1 F ( x n +1 ) ≤ v(θ xn+1 − xn )dθ xn+1 − xn
0
+(1 + v0 ( xn − x0 )) xn+1 − yn
1
≤ v(θ (tn+1 − tn ))dθ (tn+1 − tn )
0
+(1 + v0 (tn ))(tn+1 − sn ), (61)
z n − x0 ≤ z n − y n + y n − x0
≤ u n − s n + s n − t0 ≤ t ∗
and
x n +1 − x0 ≤ x n +1 − z n + z n − x0
≤ t n +1 − u n + u n − t0 ≤ t ∗ .
Hence, sequence {tn } is majorizing for method (2) and iterates { xn }, {yn }, {zn } belong
in U [ x0 , t∗ ]. The sequence { xn } is complete in Banach space X and as such, it converges to
some x∗ ∈ U [ x0 , t∗ ]. By using the continuity of F and letting n −→ ∞ in (61), we deduce
F ( x∗ ) = 0.
Proposition 2. Suppose:
(i) There exists a solution x∗ ∈ U ( x0 , ρ2 ) of equation F ( x ) = 0 for some ρ2 > 0.
(ii) Condition (h2) holds.
(iii) There exists ρ3 ≥ ρ2 such that
1
v0 ((1 − θ )ρ2 + θρ3 )dθ < 1. (62)
0
Thus, x∗ = y∗ .
77
Mathematics 2022, 10, 2621
The next two examples show how to choose the functions v0 , v, and the parameter η.
Example 5. Consider X = C [0, 1] and Ω = U [0, 1]. Then the problem [5]
Ξ(0) = 0, Ξ(1) = 1,
Ξ = −Ξ − ιΞ2
is also given as integral equation of the form
1
Ξ ( q2 ) = q2 + Θ(q2 , q1 )(Ξ3 (q1 ) + ιΞ2 (q1 ))dq1
0
Consider F : Ω −→ X as
1
[ F ( x )](q2 ) = x (q2 ) − q2 − Θ(q2 , q1 )( x3 (q1 ) + ιx2 (q1 ))dq1 .
0
Choose Ξ0 (q2 ) = q2 and Ω = U (Ξ0 , 0 ). Then, clearly U (Ξ0 , 0 ) ⊂ U (0, 0 + 1), since
Ξ0 = 1. If 2ι < 5. Then, conditions (C1)–(C3) are satisfied for
2ι + 3ρ0 + 6 ι + 6ρ0 + 3
w0 ( t ) = t, w(t) = t.
8 4
Hence, w0 (t) ≤ w(t).
(1 + α) Lt
h1 ( t ) = .
1 − (2 + α ) L0 t
1
ρ=
(1 + α ) L + (2 + α ) L0
1
ρ0 = .
(2 + α)( L0 + L)
78
Mathematics 2022, 10, 2621
The equation
h2 ( t ) − 1 = 0
has a smallest solution ρ ∈ T0 − {0} by the intermediate value theorem, since h2 (0) − 1 = −1
and h2 (t) −→ ∞ as y −→ ρ0− . It shall be shown that R is a radius of convergence for
method (20). It follows by these definitions that ∀t ∈ T0
F ( x∗ )−1 ([v, z; F ] − F ( x∗ )) ≤ L0 ( v − x∗ + z − x∗ )
and
F ( x ) ≤ α x − x∗ .
Set Ω1 = U ( x∗ , ρ) ∩ Ω.
(C3) There exists a positive constant L > 0 such that ∀ x, y, v, z ∈ Ω1
F ( x∗ )−1 ([ x, y; F ] − [v, z; F ]) ≤ L( x − v + y − z )
and
(C4) U [ x0 , ρ] ⊂ Ω.
Next, the local convergence of method (20) is presented using the preceding terminol-
ogy and conditions.
Theorem 5. Under conditions (C1)–(C4), further suppose that x0 ∈ U ( x∗ , ρ). Then, the sequence
{ xn } generated by method (20) is well defined in U ( x∗ , ρ), stays in U ( x∗ , ρ) ∀n = 0, 1, 2, . . . and
is convergent to x∗ so that
y n − x ∗ ≤ h1 ( x n − x ∗ ) x n − x ∗ ≤ x n − x ∗ < Ω (66)
and
x n +1 − x ∗ ≤ h2 ( x n − x ∗ ) x n − x ∗ ≤ x n − x ∗ , (67)
where the functions h1 , h2 and the radius ρ are defined previously.
F ( x∗ )−1 ( A0 − F ( x∗ )) = F ( x∗ )−1 ([ x0 , x0 + F ( x0 ); F ] − F ( x∗ ))
≤ L0 (2 x0 − x ∗ + F ( x0 ) − F ( x ∗ ) )
≤ L0 (2 + α ) x0 − x ∗
< L0 (2 + α)ρ. (68)
It follows by (68) and the Banach lemma on invertible operators [24] that A0−1 ∈
L( X, X ) and
79
Mathematics 2022, 10, 2621
1
A0−1 F ( x∗ ) ≤ . (69)
1 − (2 + α ) L0 x0 − x ∗
Hence, the iterate y0 exists by the first substep of method (20) for n = 0. It follows
from the first substep of method (20), (C2) and (C3), that
y0 − x ∗ ≤ x0 − x∗ − A0−1 F ( x0 )
A0−1 F ( x∗ ) F ( x∗ )−1 ( A0 − ( F ( x0 ) − F ( x∗ )))( x − 0 − x∗ )
≤ A0−1 F ( x∗ ) F ( x∗ )−1 ( A0 − ( F ( x0 ) − F ( x∗ ))) x0 − x ∗
L( x0 − x∗ + F ( x0 ) − F ( x∗ ))
≤ (70)
1 − L0 (2 + α ) x0 − x ∗
≤ h1 ( x0 − x∗ ) x0 − x∗ ≤ x0 − x∗ < ρ.
Thus, the iterate y0 ∈ U ( x∗ , ρ) and (66) holds for n = 0. Similarly, by the second
substep of method (20), we have
1
B0−1 F ( x∗ ) ≤ . (72)
1 − ( L + L0 )(2 + α) x0 − x∗
Thus, the iterate x1 exists by the second sub-step of method (20). Then, as in (70) we
obtain in turn that
x1 − x ∗ ≤ y0 − x∗ − B0−1 F (y0 )
≤ B0−1 F ( x∗ ) F ( x∗ )−1 ( B0 − ( F (y0 ) − F ( x∗ ))) y0 − x ∗
F ( x ∗ )−1 ([y 0 , w0 ; F ] + [ y 0 , x 0 ; F ] − [ x 0 , w0 ; F ] − [ y 0 , x ∗ : F ])
≤
1 − ( L + L0 )(2 + α) x0 − x∗
y0 − x ∗
L(2 + 2α + h2 ( x0 − x∗ )) x0 − x∗
≤ h1 ( x0 − x ∗ )
1 − ( L + L0 )(2 + α) x0 − x∗
x0 − x ∗ (73)
≤ h2 ( x0 − x∗ ) x0 − x∗ ≤ x0 − x∗ < ρ.
Concerning the uniqueness of the solution x∗ (not given in [9]), we provide the result.
80
Mathematics 2022, 10, 2621
Proposition 3. Suppose:
(i) The point x∗ is a simple solution x∗ ∈ U ( x∗ , r ) ⊂ Ω for some r > 0 of equation F ( x ) = 0.
(ii) There exists positive parameter L1 such that ∀y ∈ Ω
F ( x∗ )−1 ([ x∗ , y; F ] − F ( x∗ )) ≤ L1 y − x∗ (75)
Proof. Set P = [ x∗ , y∗ ; F ] for some y∗ ∈ D2 with F (y∗ ) = 0. It follows by (i), (75) and (76)
that
F ( x∗ )−1 ( P − F ( x∗ )) ≤ L1 y∗ − x∗ ) < 1.
Thus, we conclude x∗ = y∗ by the invertability of P and identity P( x∗ − y∗ ) =
F ( x∗ ) − F (y∗ ) = 0.
Remark 4. (i) Notice that not all conditions of Theorem 5 are used in Proposition 3. If they were,
then we can set r1 = ρ.
(ii) By the definition of set Ω1 we have
Ω1 ⊂ Ω. (77)
Therefore, the parameter
L ≤ L2 , (78)
where L2 is the corresponding Lipschitz constant in [1,3,9,19] appearing in the condition ∀x, y, z ∈ Ω
R0 ≤ ρ. (80)
Examples where (77), (78) and (80) are strict can be found in [2,5,11–13,15,21–24].
(1)
f n (t) = Kt2n η + Kt2n−1 η + 2K0 (1 + t + . . . + t2n+1 )η
+K0 (t2n+1 + 2t2n )t2n+1 η + δ − 1,
(2)
f n (t) = Kt2n+1 η + K (t2n+1 + 2t2n )t2n η
+2K0 (1 + t + . . . + t2n+2 )η + δ − 1,
(1)
gn ( t ) = Kt3 + Kt2 − Kt − K + 2K0 (t3 + t4 )
+K0 (t2n+3 + 2tn+2 )t4 η − K0 (t2n+1 + 2t2n )t2 η,
(2)
gn ( t ) = Kt3 + K (t3 + 2t2 )t2n+2 η
+2K0 (t3 + t4 ) − Kt − K (t + 2)t2n η,
(1) (1) (1)
h n +1 ( t ) = gn +1 ( t ) − gn ( t ),
(2) (2) (2)
h n +1 ( t ) = gn +1 ( t ) − gn ( t ),
81
Mathematics 2022, 10, 2621
and polynomials
(1)
g∞ (t) = g1 (t) = Kt3 + Kt2 − Kt − K + 2K0 (t3 + t4 ),
(2)
g∞ (t) = g2 (t) = Kt3 + 2K0 (t3 + t4 ) − Kt = g3 (t)t
and
g(t) = (t − 1)2 (t5 + 4t4 + 6t3 + 6t2 + 5t + 2).
Then, the following auxiliary result connecting these polynomials can be shown.
(1)
hn+1 (t) ≥ 0 ∀ t ∈ [0, ξ 1 ) (85)
and
(2)
hn+1 (t) ≥ 0 ∀ t ∈ [0, ξ 2 ). (86)
Moreover, define functions on the interval T by
(1) (1)
g∞ (t) = lim gn (t) (87)
n−→∞
and
(2) (2)
g∞ (t) = lim gn (t). (88)
n−→∞
Then,
(1)
g∞ (t) = g1 (t) ∀ t ∈ [0, α1 ), (89)
(2)
g∞ (t) = g2 (t) ∀ t ∈ [0, α2 ), (90)
(1) (1)
f n +1 ( t ) ≤ f n (t) + g1 (t)t2n−1 η ∀ t ∈ [0, ξ 1 ), (91)
(2) (2)
f n+1 (t) ≤ f n (t) + g2 (t)t2n η ∀ t ∈ [0, ξ 2 ), (92)
(1) (1)
f n +1 ( ξ 1 ) ≤ f n ( ξ 1 ), (93)
and
(2) (2)
f n +1 ( ξ 2 ) ≤ f n ( ξ 2 ). (94)
Proof. Assertions (81)–(84) hold by the definition of these functions and basic algebra. By
the intermediate value theorem polynomials g1 and g3 have zeros in the interval T − {0},
since g1 (0) = −K, g1 (1) = 4K0 , g2 (0) = −K and g2 (1) = 4K0 . Then, assertions (85) and
(86) follow by the definition of these polynomials and zeros ξ 1 and ξ 2 . Next, assertions (91)
and (94) also follow from (87), (88) and the definition of these polynomials.
82
Mathematics 2022, 10, 2621
K (η + δ)η
t1 = s0 + ,
1 − K0 (2η + δ)
K (tn+1 − tn + sn − tn )(tn+1 − sn )
s n +1 = t n +1 + (95)
1 − K0 (2tn+1 + γn + δ)
K (sn+1 − tn+1 + γn )(sn+1 − tn+1 )
t n +2 = s n +1 + ,
1 − K0 (2sn+1 + δ)
Lemma 4. Suppose
Kη ≤ 1, 0 < ξ 1 , 0 < ξ 2 , a < ξ < 1, (96)
(1)
f 1 (ξ 1 ) ≤0 (97)
and
(1)
f 2 (ξ 2 ) ≤ 0. (98)
η
Then, scalar sequence {tn } is non-decreasing, bounded from above by t∗∗ = 1− ξ ,
and con-
verges to its unique least upper bound t∗ ∈ [0, t∗∗ ]. Moreover, the following error bounds hold
K ( t n +1 − t n + s n − t n )
0< ≤ ξ1, (102)
1 − K0 (2tn+1 + γn + δ)
K ( s n + 1 − t n + 1 + γn )
0< ≤ ξ2, (103)
1 − K0 (2sn+1 + δ)
and
t n ≤ s n ≤ t n +1 . (104)
By the definition of t1 , we obtain
t1 1 − Kη
= > 1,
s0 1 − K0 (2η + δ)
so s 0 < t 1 , and (103) holds for n = 0. Suppose assertions (101)–(103) hold for each
m = 0, 1, 2, 3, . . . , n. By (99) and (100) we have
sm ≤ tm + ξ 2m η ≤ sm−1 + ξ 2m−1 η + ξ 2m η
≤ η + ξη + . . . + ξ 2m η
1 − ξ 2m+1
= η ≤ t∗∗ (105)
1−ξ
and
83
Mathematics 2022, 10, 2621
t m +1 ≤ sm + ξ 2m+1 η ≤ tm + ξ 2m+1 η + ξ 2m η
≤ η + ξη + . . . + ξ 2m+1 η
1 − ξ 2m+2
= η ≤ t∗∗ . (106)
1−ξ
By the induction hypotheses sequences {tm }, {sm } are increasing. Evidently, estimate
(101) holds if
1 − ξ 2m+2
Kξ 2m+1 η + Kξ 2m η + 2K0 ξ η
1−ξ
+K0 ξδ + ξγm K0 − ξ ≤ 0
or
(1)
f m (t) ≤ 0 at t = ξ 1 , (107)
where γm ≤ K (ξ 2m+1 + 2ξ 2m )ξ 2m+1 η 2 .
By (91), (93), and (98) estimate (107) holds.
Similarly, assertion (103) holds if
or
(2)
f m (t) ≤ 0 at t = ξ 2 . (108)
By (92) and (94), assertion (108) holds. Hence, (100) and (103) also hold. Notice that
γn can be written as γn = K ( En + En1 ) En2 , where En = tn+1 − tn > 0, En1 = sn − tn , and
En2 = tn+1 − sn > 0. Hence, we get
Next, a second convergence result for sequence (95) is presented but the sufficient
criteria are weaker but more difficult to verify than those of Lemma 4.
Lemma 5. Suppose
K0 δ < 1, (109)
K0 (2tn+1 + γn + δ) < 1, (110)
and
K0 (2sn+1 + δ) < 1 (111)
1 − K0 δ
hold. Then, sequence {tn } is increasing and bounded from above by t1∗∗ = 2K0 , so it converges to
its unique least upper bound t1∗ ∈ [0, t1∗∗ ].
Proof. It follows from the definition of sequence (95), and conditions (109)–(111).
84
Mathematics 2022, 10, 2621
Suppose
(C1) There exist x0 ∈ Ω, η ≥ 0, δ ∈ [0, 1) such that A0−1 ∈ L( X, X ), A0−1 F ( x0 ) ≤ η, and
F ( x0 ) ≤ δ.
(C2) There exists K0 > 0 such that for all u, v ∈ Ω
A0−1 ([u, v; F ] − A0 ) ≤ K0 ( u − x0 + v − w0 ).
Set Ω0 = U ( x0 , 1− K0 δ
2K0 ) ∩ Ω for K0 δ < 1.
(C3) There exists K > 0 such that for all u, v, ū, v̄ ∈ Ω0
Remark 5. The results in [19] are given in the non-affine form. The benefits of using affine invariant
results over non-affine are well-known [1,5,11,21]. In particular, they assumed A0−1 ≤ β and
(C3)’ [ x, y; F ] − [ x̄, ȳ; F ] ≤ K̄ ( x − x̄ + y − ȳ ) holds for all x, y, x̄ ȳ ∈ Ω. By the definition
of the set Ω0 , we get
Ω0 ⊂ Ω, (112)
so
K0 ≤ βK̄ (113)
and
K ≤ βK̄. (114)
Hence, K can replace βK̄ in the results in [19]. Notice also that using (C3)’ they estimated
1
Bn−+1 1 A0 ≤ (115)
1 − βK̄ (2s̄n+1 + δ)
and
1
A0−1 ( An+1 − A0 ) ≤ , (116)
1 − βK̄ (t̄n+1 − t̄0 ) + γ̄n + δ)
where {t̄n }, {s̄n } are defined for n = 0, 1, 2, . . . by t̄0 = 0, s̄0 = η,
βK̄ (η + δ)η
t̄1 = s̄0 + ,
1 − βK̄ (2s̄0 + δ)
βγ̄
s̄n+1 = t̄n+1 + (117)
1 − βK̄ (2t̄n+1 + γ̄n + δ)
βK̄ (s̄n+1 − t̄n+1 + γ̄n )(s̄n+1 − t̄n+1 )
t̄n+2 = s̄n+1 + ,
1 − βK̄ (2s̄n+1 + δ)
where γ̄n = K̄ (t̄n+1 − t̄n + s̄n − t̄n )(t̄n+1 − s̄n ), δ ≥ γ̄0 . But using the weaker condition (C2) we
obtain respectively,
1
Bn−+1 1 A0 ≤ (118)
1 − K0 (2sn+1 + δ)
and
1
A0−1 ( An+1 − A0 ) ≤ (119)
1 − K0 ( t n + 1 − t 0 + γn + δ )
which are tighter estimates than (115) and (116), respectively. Hence, K0 , K can replace βK̄, β, K̄
and (118), (119) can replace (115), (116), respectively, in the proof of Theorem 3 in [19]. Examples
where (112)–(114) are strict can be found in [1,5,11,21]. Simple induction shows that
85
Mathematics 2022, 10, 2621
These estimates justify the claims made at the introduction of this work along the same lines.
The local results in [19] can also be extended using our technique.
Next, we present the semi-local convergence result for the method (20).
Theorem 6. Suppose that conditions (C) hold. Then, iteration { xn } generated by method (20)
exists in U [ x0 , t∗ ], remains in U [ x0 , t∗ ] and limn−→∞ xn = x∗ ∈ U [ x0 , t∗ ] with F ( x∗ ) = 0, so
that
xn − x∗ ≤ t∗ − tn .
Next, we present the uniqueness of the solution result, where conditions (C) are not
necessarily utilized.
Proof. Let z∗ ∈ Ω1 with F (z∗ ) = 0. Define Q = [ x∗ , z∗ ; F ]. Then, in view of (ii) and (iii),
A0−1 ( Q − A0 ) ≤ K0 ( x∗ − x0 + z∗ − w0 ≤ K0 (r + r ∗ + δ) < 1.
10. Conclusions
The convergence analysis is developed for generalized three-step numerical methods.
The advantages of the new approach include weaker convergence criteria and a uniform
set of conditions utilizing information on these methods in contrast to earlier works on
special cases of these methods, where the existence of high-order derivatives is assumed to
prove convergence. The methodology is very general and does not depend on the methods.
86
Mathematics 2022, 10, 2621
That is why it can be applied to multi-step and other numerical methods that shall be the
topic of future work.
The weak point of this methodology is the observation that the computation of the
majorant functions “h” at this generality is hard in general. Notice that this is not the
case for the special cases of method (2) or method (3) given below them (see, for example,
Examples 4 and 5). As far as we know, there is no other methodology that can be compared
to the one introduced in this article to handle the semi-local or the local convergence of
method (2) or method (3) at this generality.
Author Contributions: Conceptualization, M.I.A., I.K.A., S.R. and S.G.; methodology, M.I.A., I.K.A.,
S.R. and S.G.; software, M.I.A., I.K.A., S.R. and S.G.; validation, M.I.A., I.K.A., S.R. and S.G.; formal
analysis, M.I.A., I.K.A., S.R. and S.G.; investigation, M.I.A., I.K.A., S.R. and S.G.; resources, M.I.A.,
I.K.A., S.R. and S.G.; data curation, M.I.A., I.K.A., S.R. and S.G.; writing—original draft preparation,
M.I.A., I.K.A., S.R. and S.G.; writing—review and editing, M.I.A., I.K.A., S.R. and S.G.; visualization,
M.I.A., I.K.A., S.R. and S.G.; supervision, M.I.A., I.K.A., S.R. and S.G.; project administration, M.I.A.,
I.K.A., S.R. and S.G.; funding acquisition, M.I.A., I.K.A., S.R. and S.G. All authors have read and
agreed to the published version of the manuscript.
Funding: This research received no external funding.
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: Not applicable.
Conflicts of Interest: The authors declare no conflict of interest.
References
1. Appell, J.; DePascale, E.; Lysenko, J.V.; Zabrejko, P.P. New results on Newton-Kantorovich approximations with applications to
nonlinear integral equations. Numer. Funct. Anal. Optim. 1997, 18, 1–17. [CrossRef]
2. Ezquerro, J.A.; Hernandez, M.A. Newton’s Method: An Updated Approach of Kantorovich’s Theory; Birkhäuser: Cham Switzer-
land, 2018.
3. Proinov, P.D. New general convergence theory for iterative processes and its applications to Newton-Kantorovich type theorems.
J. Complex. 2010, 26, 3–42. [CrossRef]
4. Regmi, S.; Argyros, I.K.; George, S.; Argyros, C. Numerical Processes for Approximating Solutions of Nonlinear Equations.
Axioms 2022, 11, 307. [CrossRef]
5. Argyros, I.K. The Theory and Applications of Iteration Methods, 2nd ed.; Engineering Series; CRC Press: Boca Raton, FL, USA; Taylor
and Francis Group: Abingdon, UK, 2022.
6. Zhanlav, K.H.; Otgondorj, K.H.; Sauul, L. A unified approach to the construction of higher-order derivative-free iterative methods
for solving systems of nonlinear equations. Int. J. Comput. Math. 2021.
7. Zhanlav, T.; Chun, C.; Otgondorj, K.H.; Ulziibayar, V. High order iterations for systems of nonlinear equations. Int. J. Comput.
Math. 2020, 97, 1704–1724. [CrossRef]
8. Wang, X. An Ostrowski-type method with memory using a novel self-accelerating parameters. J. Comput. Appl. Math. 2018, 330,
710–720. [CrossRef]
9. Moccari, M.; Lofti, T. On a two-step optimal Steffensen-type method: Relaxed local and semi-local convergence analysis and
dynamical stability. J. Math. Anal. Appl. 2018, 468, 240–269. [CrossRef]
10. Shakhno, S.M.; Gnatyshyn, O.P. On an iterative Method of order 1.839. . . for solving nonlinear least squares problems. Appl. Math.
Comput. 2005, 161, 253–264.
11. Argyros, I.K. Unified Convergence Criteria for Iterative Banach Space Valued Methods with Applications. Mathematics 2021,
9, 1942. [CrossRef]
12. Potra, F.-A.; Pták, V. Nondiscrete Induction and Iterative Processes; Pitman Publishing: Boston, MA, USA, 1984.
13. Cordero, A.; Torregrosa, J.R. Variants of Newton’s method using fifth-order quadrature formulas. Appl. Math. Comput. 2007, 190,
686–698. [CrossRef]
14. Traub, J.F. Iterative Methods for the Solution of Equations; Prentice Hall: Hoboken, NJ, USA, 1964.
15. Kantorovich, L.V.; Akilov, G.P. Functional Analysis; Pergamon Press: Oxford, UK, 1982.
16. Xiao, X.; Yin, H. Achieving higher order of convergence for solving systems of nonlinear equations. Appl. Math. Comput. 2017,
311, 251–261. [CrossRef]
17. Sharma, J.R.; Arora, H. Efficient derivative-free numerical methods for solving systems of nonlinear equations. Comput. Appl.
Math. 2016, 35, 269–284. [CrossRef]
87
Mathematics 2022, 10, 2621
18. Sharma, J.R.; Guha, R.K. Simple yet efficient Newton-like method for systems of nonlinear equations. Calcolo 2016, 53, 451–473.
[CrossRef]
19. Noor, M.A.; Waseem, M. Some iterative methods for solving a system of nonlinear equations. Comput. Math. Appl. 2009, 57,
101–106. [CrossRef]
20. Wang, X.; Zhang, T. A family of Steffensen type methods with seventh-order convergence. Numer. Algor. 2013, 62, 429–444.
[CrossRef]
21. Argyros, I.K.; Magréñan, A.A. A Contemporary Study of Iterative Methods; Elsevier: Amsterdam, The Netherlands; Academic Press:
New York, NY, USA, 2018.
22. Grau-Sanchez, M.; Grau, A.; Noguera, M. Ostrowski type methods for solving system of nonlinear equations. Appl. Math. Comput.
2011, 218, 2377–2385. [CrossRef]
23. Homeier, H.H.H. A modified Newton method with cubic convergence: The multivariate case. J. Comput. Appl. Math. 2004, 169,
161–169. [CrossRef]
24. Kou, J.; Wang, X.; Li, Y. Some eight order root finding three-step methods. Commun. Nonlinear Sci. Numer. Simul. 2010, 15, 536–544.
[CrossRef]
25. Verma, R. New Trends in Fractional Programming; Nova Science Publisher: New York, NY, USA, 2019.
88
mathematics
Article
A Methodology for Obtaining the Different Convergence
Orders of Numerical Method under Weaker Conditions
Ioannis K. Argyros 1 , Samundra Regmi 2 , Stepan Shakhno 3, * and Halyna Yarmola 4
Abstract: A process for solving an algebraic equation was presented by Newton in 1669 and later
by Raphson in 1690. This technique is called Newton’s method or Newton–Raphson method and is
even today a popular technique for solving nonlinear equations in abstract spaces. The objective of
this article is to update developments in the convergence of this method. In particular, it is shown
that the Kantorovich theory for solving nonlinear equations using Newton’s method can be replaced
by a finer one with no additional and even weaker conditions. Moreover, the convergence order two
is proven under these conditions. Furthermore, the new ratio of convergence is at least as small. The
same methodology can be used to extend the applicability of other numerical methods. Numerical
experiments complement this study.
L ( x0 )−1 L( x0 ) ≤ s,
(K2) ∃ parameter M1 > 0 : Lipschitz condition
1
s≤
2M1
and
(K4) B[ x0 , ρ] ⊂ D, where parameter ρ > 0 is given later. √
1 − 2M1 s1−
Denote B[ x0 , r ] := { x ∈ D : x − x0 ≤ r } for r > 0. Set ρ = r1 = .
M1
There are many variants of Kantorovich’s convergence result for NM. One of these
results follows [4,7,20].
x n +1 − x n ≤ u n +1 − u n .
1 1
Moreover, the convergence is linear if s = and quadratic if s < . Further-
2M1 2M1
solution is unique B[ x0 , r1 ] in the first case and in B( x0 , r2 ) in the second case where
more, the √
1 + 1 − 2M1 s
r2 = and scalar sequence {un } is given as
M1
M1 (un − un−1 )2
u0 = 0, u1 = s, un+1 = un + .
2(1 − M1 un )
c( x ) = x3 − a
1
for D = B( x0 , 1 − a) and parameter a ∈ (0, ). Select initial point x0 = 1. Conditions (K) give
2
1−a
s= and M1 = 2(2 − a). It follows that estimate
3
1−a 1
>
3 4(2 − a )
1
holds ∀ a ∈ (0, ). That is condition (K3) is not satisfied. Therefore convergence is not assured by
2
this theorem. However, NM may converge. Hence, clearly, there is a need to improve the results
based on the conditions K.
By looking at the crucial sufficient condition (K3) for the convergence, (K4) and the
majorizing sequence given by Kantorovich in the preceding Theorem 1 one sees that if
the Lipschitz constants M1 is replaced by a smaller one, say L > 0, than the convergence
domain will be extended, the error distances xn+1 − xn , xn − x ∗ will be tighter and the
location of the solution more accurate. This replacement will also lead to fewer Newton
iterates to reach a certain predecided accuracy (see the numerical Section). That is why with
the new methodology, a new domain is obtained inside D that also contains the Newton
90
Mathematics 2022, 10, 2931
M0 ≤ M1 , (3)
M ≤ M1 (4)
since
D0 ⊂ D. (5)
Notice also since parameters M0 and M are specializations of parameter M1 , M1 = M1 ( D ),
M0 = M0 ( D ), but M = M( D0 ). Therefore, no additional work is required to find M0 and
M0
M (see also [22,23]). Moreover the ratio can be very small (arbitrarily). Indeed,
M
91
Mathematics 2022, 10, 2931
q4
−→ 0.
q2
Preceding items indicate the times (at most) one is improving the other. These are the
extensions given in this aforementioned references. However, it turns out that parameter L
can replace M1 in these papers (see Section 3). Denote by Ñ, q̃ the corresponding items. It
follows
q̃1 M q̃ q̃
= −→ 0, 2 −→ 0, 3 −→ 0
q1 M1 q2 q3
M0 M
for −→ 0 and −→ 0. Hence, the new results also extend the ones in the afore-
M1 M1
mentioned references. Other extensions involve tighter majorizing sequences for NM (see
Section 2) and improved uniqueness report for solution x ∗ (Section 3). The applications
appear in Section 4 followed by conclusions in Section 5.
2. Majorizations
Let K0 , M0 , K, M be given positive parameters and s be a positive variable. The real
K ( t1 − t0 )2
sequence {tn } defined for t0 = 0, t1 = s, t2 = t1 + and ∀n = 0, 1, 2, . . . by
2 ( 1 − K0 t 1 )
M ( t n +1 − t n )2
t n +2 = t n +1 + (6)
2(1 − M0 tn+1 )
plays an important role in the study of NM, we adopted the notation tn (s) = tn
∀n = 1, 2, . . . . That is why some convergence results for it are listed in what follows
next in this study.
1
K0 t1 < 1 and tn+1 < (7)
M0
hold ∀ n = 1, 2, . . . . Then, the following assertions hold
1
t n < t n +1 < (8)
M0
1
and ∃ t∗ ∈ [s, ] such that lim tn = t∗ .
M0 n→∞
Proof. The definition of sequence {tn } and the condition (7) implies (8). Moreover, increas-
1
ing sequence {tn } has as an upper bound. Hence, it is convergent to its (unique) least
M0
upper bound t∗ .
Next, stronger convergence criteria are presented. However, these criteria are easier to
verify than conditions of Lemma 1. Define parameter δ by
2M
δ= 5 . (9)
M+ M2 + 8M0 M
This parameter plays a role in the following results.
Case: K0 = M0 and K = M.
Part (i) of the next auxiliary result relates to the Lemma in [19].
1
s≤ (10)
2M2
92
Mathematics 2022, 10, 2931
holds, where
1 5
M2 = ( M + 4M0 + M2 + 8M0 M). (11)
4
Then, the following assertions hold
(i) Estimates
t n +1 − t n ≤ δ ( t n − t n −1 ) (12)
1 − δ n +1 s
tn < s< (13)
1−δ 1−δ
hold. Moreover, conclusions of Lemma 1 are true for sequence {tn }. The sequence, {tn } converges
s
linearly to t∗ ∈ (0, ]. Furthermore, if for some μ > 0
1−δ
μ
s< . (14)
(1 + μ) M2
Then, the following assertions hold
(ii)
M
t n +1 − t n ≤ (1 + μ)(tn − tn−1 )2 (15)
2
and
1 n
t n +1 − t n ≤ (αs)2 , (16)
α
M
where α = (1 + μ) and the conclusions of Lemma 1 for sequence {tn } are true. The sequence,
2
{tn } converges quadratically to t∗ .
M M
0< ≤ (1 + μ ) (17)
2(1 − M0 tn+1 ) 2
μ
is true. This estimate is true for n = 1, since it is equivalent to M0 s ≤ . But this is
1+μ
μM0 μ
true by M0 ≤ 2M2 , condition (11) and inequality ≤ . Then, in view of
(1 + μ)2M2 1+μ
estimate (13), estimate (17) certainly holds provided that
(1 + μ) M0 (1 + δ + . . . + δn+1 )s − μ ≤ 0. (18)
This estimate motivates the introduction of recurrent polynomials pn which are defined
by
pn (t) = (1 + μ) M0 (1 + t + . . . + tn+1 )s − μ, (19)
∀t ∈ [0, 1). In view of polynomial pn assertion (18) holds if
pn (t) ≤ 0 at t = δ. (20)
93
Mathematics 2022, 10, 2931
(1 + μ) M0 s
p∞ (t) = − μ. (23)
1−t
Hence, assertion (20) holds if
p∞ (t) ≤ 0 at t = δ, (24)
or equivalently
5
μ M2 + 8M0 M − M
M0 s ≤ 5 ,
1 + μ M2 + 8M0 M + M
which can be rewritten as condition (14). Therefore, the induction for assertion (17) is
completed. That is assertion (15) holds by the definition of sequence {tn } and estimate (15).
It follows that
so
n −1
≤ α1+2+2 +...+2
2 n
t n +1 − t n s2
1 n
= (αs)2 .
α
μ 2
Notice also that Mμ < 4M2 , then < , so αs < μ.
(1 + μ) M1 M (1 + μ )
Remark 1. (1) The technique of recurrent polynomials in part (i) is used: to produce convergence
condition (11) and a closed form upper bound on sequence {tn } (see estimate (13)) other
1
than and t∗ (which is not given in closed form). This way we also established the linear
M0
convergence of sequence {tn }. By considering condition (14) but being able to use estimate
(13) we establish the quadratic convergence of sequence {tn } in part (ii) of Lemma 2.
(2) If μ = 1, then (14) is the strict version of condition (10).
(3) Sequence {tn } is tighter than the Kantorovich sequence {un } since M0 ≤ M1 and M ≤ M1 .
2M s
Concerning the ration of convergence αs this is also smaller than r = √ 1
(1 − 1 − 2M1 s)2
given in the Kantorovich Theorem [19]. Indeed, by these definitions αs < r provided that
4M
μ ∈ (0, μ1 ), where μ1 = √ 1 − 1. Notice that
M (1 + 1 − 2M1 s)2
5 4M1
(1 + 1 − 2M1 s)2 < (1 + 1)2 = 4 ≤ ,
M
so μ1 > 0.
Part (i) of the next auxiliary result relates to a Lemma in [19]. The case M0 = M has been
studied in the introduction. So, in the next Lemma we assume M0 = M in part (ii).
94
Mathematics 2022, 10, 2931
1 − δ n +1 t − t1
t n +2 ≤ s + (t2 − t1 ) < t∗∗ = s + 2 s, ∀n = 1, 2, . . . . (27)
1−δ 1−δ
Moreover, conclusions of Lemma 1 are true for sequence {tn }. The sequence {tn } converges
linearly to t∗ ∈ (0, t∗∗ ]. Define parameters h0 by
√
2( M0 M + 8M02 + M0 M ) h0
h0 = √ , M̄3 = ,
M( M M + 8M + M M + 4M )
2 2
0 0 0 0
μ
γ = 1 + μ, β = , d = 2(1 − δ )
1+μ
and
M0
μ= .
2M3 − M0
(ii) Suppose
M0
M0 < M ≤ (28)
θ
and (25) hold, where θ ≈ 0.6478 is the smallest solution of scalar equation 2z4 + z − 1 = 0.
Then, the conclusions of Lemma 2 also hold for sequence {tn }. The sequence converges
quadratically to t∗ ∈ (0, t∗∗ ].
(iii) Suppose
1 1
M≥ M0 and s < (29)
θ 2 M̄3
hold. Then, the conclusions of Lemma 2 are true for sequence {tn }. The sequence {tn } con-
verges quadratically to t∗ ∈ (0, t∗∗ ].
(iv) M0 > M and (25) hold. Then, M̄3 ≤ M3 and the conclusions of Lemma 2 are true for
sequence {tn }. The sequence {tn } converges quadratically to t∗ ∈ (0, t∗∗ ].
M Mγ
≤ .
2(1 − M0 tn+1 ) 2
It suffices
1 − δn
γM0 s + ( t2 − t1 ) + 1 − γ ≤ 0
1−δ
95
Mathematics 2022, 10, 2931
or
pn (t) ≤ 0 at t = δ, (30)
where
It follows that
γM0 (t2 − t1 )
p∞ (t) = + γM0 s + 1 − γ.
1−t
So, (30) holds provided that
p∞ (t) ≤ 0 at t = δ. (31)
x2
+x ≤ β
2(1 − x )(1 − δ)
or
( d − 1) x 2 + (1 + β ) x − β ≤ 0
or
5
1+β− (1 − β)2 + 4βd
x≤
2(1 − d )
or
5
1+β− (1 − β)2 + 4βd
s≤ . (32)
2(1 − d )
1
Claim. The right hand side of assertion (31) equals . Indeed, this is true if
M2
2M0 (1 − d)
1 + β − (1 − β)2 + 4βd =
M2
or
2M0 (1 − d)
1+β− = (1 − β)2 + 4βd
2M3
or by squaring both sides
or
M0 (1 − d) M (1 − d ) M0
β 1− −d = 0 1−
2M3 2M3 2M3
or
96
Mathematics 2022, 10, 2931
M0 M0 M
β 1− (1 − d ) = 1 − (1 − d ) 0
2M3 2M3 2M3
or
M0
β=
2M3
or
μ M0
=
1+μ 2M3
or
M0
μ= ,
2M3 − M0
which is true. Notice also that
5
1
2M3 − M0 =
(4M0 + M0 M + M0 M + 8M02 )
4
1 5
= ( M0 M + M0 M + 8M02 ) > 0
4
√
M0 M + M0 M + 8M02 − 4M0 √
and 2M3 − 2M0 > 0, since 2M3 − M0 = , M0 < M0 M
4
and 3M0 < M0 M + 8M02 (by condition (25)). Thus, μ ∈ (0, 1). It remains to show
M
α= (1 + μ ) s < 1
2
or by the choice of μ and M2
M2 M0
1+ s<1
2 2M3 − M0
or
1
s< . (33)
2 M̄3
Claim. M̄3 ≤ M3 . By the definition of parameters M2 and M̄3 it must be shown that
√ √
M ( M0 M + M0 M + 8M02 + 4M0 M0 M + M0 M + 8M02 + 4M0
√ ≤
2( M0 M + M0 M + 8M02 ) 4
M0
or if for y =
M
√
2− y≤ y + 8y2 . (34)
√ √
By (28) 2 − y > 0, so estimate (34) holds if 2y2 + y − 1 ≥ 0 or
√
2z4 + z − 1 ≥ 0 for z = y.
However, the last inequality holds by (28). The claimed is justified. So, estimate (33)
holds by (25) and this claim.
(iii) It follows from the proof in part (ii). However, this time M2 ≤ M̄2 follows from (29).
Notice also that according to part (ii) condition (25) implies (29). Moreover, according
to part (iii) condition (29) implies (25).
97
Mathematics 2022, 10, 2931
(iv) As in case (ii) estimate (34) must be satisfied. If M0 ≥ 4M, then the estimate (34)
√
holds, since 2 − y ≤ 0. If M < M0 < 4M then again M0 > θ M, so estimate (34) or
equivalently 2z + z − 1 > 0 holds.
2
Case. Parameters K0 and K are not equal to M0 . Comments similar to Remark 1 can
follow for Lemma 3.
It is convenient to define parameter δ0 by
K ( t2 − t1 )
δ0 =
2 ( 1 − K0 t 2 )
and the quadratic polynomial ϕ by
1 1
= 5 .
2h2 M0 + M2 + M0 (K − 2K0 )
Define parameter M4 by
! (
1 1 1
= min , . (35)
M4 h1 h2
Part (i) of the next auxiliary result relates to Lemma 2.1 in [22].
Lemma 4. Suppose
1
s≤ (36)
2M4
holds, where parameter M4 is given by Formula (35). Then, the following assertions hold
(i) Estimates
Ks2
tn+2 − tn+1 ≤ δ0 δn−1 ,
2 ( 1 − K0 s )
and
1 − δn δ
tn+2 ≤ s + 1 + δ0 (t2 − t1 ) ≤ t̄ = s + 1 + 0 ( t2 − t1 ).
1−δ 1−δ
Moreover, conclusions of Lemma 2 are true for sequence {tn }. The sequence {tn } converges
linearly to t∗ ∈ (0, t̄].
98
Mathematics 2022, 10, 2931
(ii) Suppose
δ0 (t2 − t1 )
M0 +s ≤ β, (37)
1−δ
2
s< (38)
(1 + μ ) M
and (36) hold for some μ > 0. Then, the conclusions of Lemma 3 are true for sequence {tn }.
The sequence {tn } converges quadratically to t∗ ∈ (0, t̄].
M M
≤ γ
2(1 − M0 tn+1 ) 2
holds provided that
pn (t) ≤ 0 at t = δ. (39)
Define function p∞ : [0, 1) −→ R by
γM0 δ0 (t2 − t1 )
p∞ (t) = + γM0 s − γ.
1−t
Hence, estimate (39) holds provided that
p∞ (t) ≤ 0 at t = δ.
However, this assertion holds, since μ ∈ (0, 1). Moreover, the definition of α and
condition (38) of the Lemma 4 imply
M
αs = (1 + μ ).
2
Hence, the sequence {tn } converges quadratically to t∗ .
Remark 2. Conditions (36)–(38) can be condensed and a specific choice for μ can be given as
1 1
follows: Define function f : 0, −→ R by
K0
δ0 (t)(t2 (t) − t1 (t))
f (t) = 1 − M0 +t .
1−δ
It follows by this definition
1 −
f (0) = 1 > 0, f (t) −→ −∞ as t −→ .
K0
99
Mathematics 2022, 10, 2931
1
Denote by μ2 the smallest solution of equation f (t) = 0 in 0, . Then, by choosing
K0
μ = μ2 conditions (37) holds as equality. Then, if follows that if we solve the first condition in (37)
for “s", then conditions (36)–(38) can be condensed as
! (
1 2
s ≤ s1 min , . (40)
M4 (2 + μ2 ) M
2
If s1 = , then condition (40) should hold as a strict inequality to show quadratic
(2 + μ2 ) M
convergence.
3. Semi-Local Convergence
Sequence {tn } given by (6) was shown to be majorizing for { xn } and tighter than
{un } under conditions of Lemmas in [19,22,23], respectively. These Lemmas correspond
to part (i) of Lemma 1, Lemma 3 and Lemma 4, respectively. However, by asking the
initial approximation s to be bounded above by a slightly larger bound the quadratic order
of convergence is recovered. Hence, the preceding Lemmas can replace the order ones,
respectively in the semi-local proofs for NM in these references. The parameter K0 and K
are connected to x0 and L as follows
(K7) ∃ parameter K0 > 0 such that for x1 = x0 − L ( x0 )−1 L( x0 )
L ( x0 )−1 (L ( x1 ) − L ( x0 )) ≤ K0 x1 − x0 ,
(K8) ∃ parameter K such that ∀ξ ∈ [0, 1], ∀ x, y ∈ D0 ,
1
L
( x ) −1
(L
( x + ξ ( y − x )) − L
( x )) dξ ≤ K y−x .
0 0 2
Note that K0 ≤ M0 and K ≤ M. The convergence criteria in Lemmas 1, 3 and 4 do not
necessarily imply each other in each case. That is why we do not only rely on Lemma 4 to
show the semi-local convergence of NM. Consider the following three sets of conditions:
(A1): (K1), (K4), (K5), (K6) and conditions of Lemma 1 hold for ρ = t∗ , or
(A2): (K1), (K4) (K5), (K6), conditions of Lemma 2 hold with ρ = t∗ , or
(A3): (K1), (K4) (K5), (K6), conditions of Lemma 3 hold with ρ = t∗ , or
(A4): (K1), (K4) (K5), (K6), conditions of Lemma 4 hold with ρ = t∗ .
The upper bounds of the limit point given in the Lemmas and in closed form can
replace ρ in condition (K4). The proof are omitted in the presentation of the semi-local
convergence of NM since the proof is given in the aforementioned references [19,20,22,23]
with the exception of quadratic convergence given in part (ii) of the presented Lemmas.
x n +1 − x n ≤ t n +1 − t n
and
x ∗ − xn ≤ t∗ − tn .
The convergence ball is given next. Notice, however that we do not use all conditions Ai.
M0
(ρ0 + ρ1 ) < 1. (41)
2
100
Mathematics 2022, 10, 2931
J ( x ∗ − x∗ ) = L( x ∗ ) − L( x∗ ) = 0.
If conditions of Theorem 2 hold, set ρ0 = ρ.
4. Numerical Experiments
Two experiments are presented in this Section.
1−a
Example 3. Recall Example 1 (with L( x ) = c( x )). Then, the parameters are s = ,
1 23
a+5 1
K0 = , M0 = 3 − a, M1 = 2(2 − a). It also follows D0 = B(1, 1 − a) ∩ B 1, =
3 M0
1 1 2 1
B 1, , so K = M = 2 1 + . Denote by Ti , i = 1, 2, 3, 4 the set of values a for
M0 3−a
which conditions (K3), ( N2) − N4) are satisfied. Then, by solving these inequalities for a :
T1 = ∅, T2 = [0.4648, 0.5), T3 = [0.4503, 0.5), and T4 = [0.4272, 0.5), respectively.
1
The domain can be further extended. Choose a = 0.4, then, = 0.3846. The following
M0
Table 1 shows, that the conditions of Lemma 1, since K0 t < 1 and M0 tn+1 < 1 ∀ n = 1, 2, . . ..
n 1 2 3 4 5 6 7 8
tn 0.2000 0.2865 0.3272 0.3425 0.3455 0.3456 0.3456 0.3456
101
Mathematics 2022, 10, 2931
Notice that M0 < M1 and M < M1 . The Kantorovich convergence condition (K3) is not
fulfilled, since 2M1 s ≈ 1.0968 > 1. Hence, convergence of converge NM is not assured by the
Kantorovich criterion. However, the new conditions (N2)–(N4) are fulfilled, since q2 s ≈ 0.9749 < 1,
q3 s ≈ 0.9320 < 1, q4 s ≈ 0.8723 < 1.
The following Table 2 shows, that the conditions of Lemma 1 are fulfilled, since K0 t < 1 and
M0 tn+1 < 1 ∀n = 1, 2, . . ..
n 1 2 3 4 5 6
tn 0.1569 0.2154 0.2266 0.2271 0.2271 0.2271
Example 5. Let U = V = C [0, 1] be the domain of continuous real functions defined on the
interval [0, 1]. Set D = B[ x0 , 3], and define operator L on D as
1
L(v)(v1 ) = v(v1 ) − y(v1 ) − N (v1 , t)v3 (t)dt, v ∈ C [0, 1], v1 ∈ [0, 1], (43)
0
z ∈ C [0, 1], v1 ∈ [0, 1]. Pick x0 (v1 ) = y(v1 ) = 1. The norm-max is used. It then follows from
(43)–(45) that L ( x0 )−1 ∈ L( B2 , B1 ),
L( x ) = e x + 2x − 1.
e x0 + 2x − 1
The equation L( x ) = 0 has the solution x ∗ = 0. The parameters are s =
0
,
e x0 + 2
M0 = M1 = e, K0 = K = M = e 0 e and
x + 1
1 1 12 1 1 12
D0 = (−1, 1) ∩ x0 − , x0 + = x0 − , x0 + .
e e e e
Let us choose x0 = 0.15. Then, s ≈ 0.1461. Conditions (K3) and (N2) are fulfilled. The
majorizing sequences {tn } (6) and {un } from Theorem 1 are:
102
Mathematics 2022, 10, 2931
n | x n +1 − x n | | tn +1 − tn | | u n +1 − u n |
0 1.4607 × 10−1 1.4607 × 10−1 1.4607 × 10−1
1 3.9321 × 10−3 2.3721 × 10−2 4.8092 × 10−2
2 2.5837 × 10−6 8.7693 × 10−4 6.6568 × 10−3
3 1.1126 × 10−12 1.2039 × 10−6 1.3262 × 10−4
4 0 2.2688× 10−12 5.2681× 10−8
Let us choose x0 = 0.2. Then, s ≈ 0.1929. In this case condition (K3) is not held, but (N2)
holds. The majorizing sequence {tn } (6) is:
n | x n +1 − x n | | tn +1 − tn |
0 1.929 × 10−1 1.929 × 10−1
1 7.0934 × 10−3 4.9769 × 10−2
2 8.4258 × 10−6 6.4204 × 10−3
3 1.1832 × 10−11 1.1263 × 10−4
4 0 3.4690 × 10−8
5. Conclusions
We developed a comparison between results on the semi-local convergence of NM.
There exists an extensive literature on the convergence analysis of NM. Most convergence
results are based on recurrent relations, where the Lipschitz conditions are given in affine
or non-affine invariant forms.The new methodology uses recurrent functions. The idea
is to construct a domain included in the one used before which also contains the Newton
iterates. That is important, since the new results do not require additional conditions. This
way the new sufficient convergence conditions are weaker in the Lipschitz case, since they
rely on smaller constants. Other benefits include tighter error bounds and more precise
uniqueness of the solution results. The new constants are special cases of earlier ones. The
methodology is very general making it suitable to extend the usage of other numerical
methods under Hölder or more generalized majorant conditions. This will be the topic of
our future work.
Author Contributions: Conceptualization I.K.A.; Methodology I.K.A.; Investigation S.R., I.K.A., S.S.
and H.Y. All authors have read and agreed to the published version of the manuscript.
Funding: This research received no external funding.
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: Not applicable.
Conflicts of Interest: The authors declare no conflict of interest.
References
1. Appell, J.; DePascale, E.; Lysenko, J.V.; Zabrejko, P.P. New results on Newton-Kantorovich approximations with applications to
nonlinear integral equations. Numer. Funct. Anal. Optim. 1997, 18, 1–17. [CrossRef]
2. Traub, J.F. Iterative Methods for the Solution of Equations; Prentice Hall: Hoboken, NJ, USA, 1964.
3. Ezquerro, J.A.; Hernández-Verón, M.A. Newton’s Method: An Updated Approach of Kantorovich’s Theory. Frontiers in Mathematics;
Birkhäuser/Springer: Cham, Switzerland, 2017.
4. Kantorovich, L.V.; Akilov, G.P. Functional Analysis; Pergamon Press: Oxford, UK, 1982.
103
Mathematics 2022, 10, 2931
5. Potra, F.A.; Pták, V. Nondiscrete induction and iterative processes. In Research Notes in Mathematics; Pitman (Advanced Publishing
Program): Boston, MA, USA, 1984; Volume 103.
6. Verma, R. New Trends in Fractional Programming; Nova Science Publisher: New York, NY, USA, 2019.
7. Yamamoto, T. Historical developments in convergence analysis for Newton’s and Newton-like methods. J. Comput. Appl. Math.
2000, 124, 1–23. [CrossRef]
8. Zhanlav, T.; Chun, C.; Otgondorj, K.H.; Ulziibayar, V. High order iterations for systems of nonlinear equations. Int. J. Comput.
Math. 2020, 97, 1704–1724. [CrossRef]
9. Sharma, J.R.; Guha, R.K. Simple yet efficient Newton-like method for systems of nonlinear equations. Calcolo 2016, 53, 451–473.
[CrossRef]
10. Grau-Sanchez, M.; Grau, A.; Noguera, M. Ostrowski type methods for solving system of nonlinear equations. Appl. Math. Comput.
2011, 218, 2377–2385. [CrossRef]
11. Homeier, H.H.H. A modified Newton method with cubic convergence: The multivariate case. J. Comput. Appl. Math. 2004, 169,
161–169. [CrossRef]
12. Kou, J.; Wang, X.; Li, Y. Some eight order root finding three-step methods. Commun. Nonlinear Sci. Numer. Simul. 2010, 15, 536–544.
[CrossRef]
13. Nashed, M.Z.; Chen, X. Convergence of Newton-like methods for singular operator equations using outer inverses. Numer. Math.
1993, 66, 235–257. [CrossRef]
14. Wang, X. An Ostrowski-type method with memory using a novel self-accelerating parameters. J. Comput. Appl. Math. 2018, 330,
710–720. [CrossRef]
15. Moccari, M.; Lofti, T. On a two-step optimal Steffensen-type method: Relaxed local and semi-local convergence analysis and
dynamical stability. J. Math. Anal. Appl. 2018, 468, 240–269. [CrossRef]
16. Sharma, J.R.; Arora, H. Efficient derivative-free numerical methods for solving systems of nonlinear equations. Comput. Appl.
Math. 2016, 35, 269–284. [CrossRef]
17. Noor, M.A.; Waseem, M. Some iterative methods for solving a system of nonlinear equations. Comput. Math. Appl. 2009, 57,
101–106. [CrossRef]
18. Shakhno, S.M. On a two-step iterative process under generalized Lipschitz conditions for first-order divided differences. J. Math.
Sci. 2010, 168, 576–584. [CrossRef]
19. Argyros, I.K. On the Newton-Kantorovich hypothesis for solving equations. J. Comput. Math. 2004, 169, 315–332. [CrossRef]
20. Argyros, I.K. Unified Convergence Criteria for Iterative Banach Space Valued Methods with Applications. Mathematics 2021, 9,
1942. [CrossRef]
21. Proinov, P.D. New general convergence theory for iterative processes and its applications to Newton-Kantorovich type theorems.
J. Complex. 2010, 26, 3–42. [CrossRef]
22. Argyros, I.K.; Hilout, S. On an improved convergence analysis of Newton’s scheme. Appl. Math. Comput. 2013, 225, 372–386.
23. Argyros, I.K.; Hilout, S. Weaker conditions for the convergence of Newton’s scheme. J. Complex. 2012, 28, 364–387. [CrossRef]
104
mathematics
Article
Constructing a Class of Frozen Jacobian Multi-Step Iterative
Solvers for Systems of Nonlinear Equations
R. H. Al-Obaidi and M. T. Darvishi ∗
Abstract: In this paper, in order to solve systems of nonlinear equations, a new class of frozen Jacobian
multi-step iterative methods is presented. Our proposed algorithms are characterized by a highly
convergent order and an excellent efficiency index. The theoretical analysis is presented in detail.
Finally, numerical experiments are presented for showing the performance of the proposed methods,
when compared with known algorithms taken from the literature.
Keywords: iterative method; frozen Jacobian multi-step iterative method; system of nonlinear
equations; high-order convergence
MSC: 65Hxx
1. Introduction
Approximating a locally unique solution α of the nonlinear system
F (x) = 0 (1)
Citation: Al-Obaidi, R.H.; Darvishi,
has many applications in engineering and mathematics [1–4]. In (1), we have n equations
M.T. Constructing a Class of Frozen
with n variables. In fact, F is a vector-valued function with n variables. Several problems
Jacobian Multi-Step Iterative Solvers
arising from the different areas in natural and applied sciences take the form of systems of
for Systems of Nonlinear Equations.
Mathematics 2022, 10, 2952. https://
nonlinear Equation (1) that need to be solved, where F (x) = ( f 1 (x), f 2 (x), · · · , f n (x)) such
doi.org/10.3390/math10162952
that for all k = 1, 2, · · · , n, f k is a scalar nonlinear function. Additionally, there are many
real life problems for which, in the process of finding their solutions, one needs to solve
Academic Editors: Maria Isabel a system of nonlinear equations, see for example [5–9]. It is known that finding an exact
Berenguer and Manuel Ruiz Galán
solution αt = (α1 , α2 , · · · , αn ) of the nonlinear system (1) is not an easy task, especially
Received: 1 July 2022 when the equation contains terms consisting of logarithms, trigonometric and exponential
Accepted: 8 August 2022 functions, or a combination of transcendental terms. Hence, in general, one cannot find
Published: 16 August 2022 the solution of Equation (1) analytically, therefore, we have to use iterative methods. Any
iterative method starts from one approximation and constructs a sequence such that it
Publisher’s Note: MDPI stays neutral
converges to the solution of the Equation (1) (for more details, see [10]).
with regard to jurisdictional claims in
The most commonly used iterative method to solve (1) is the classical Newton method,
published maps and institutional affil-
given by
iations.
x ( k +1) = x ( k ) − J F ( x ( k ) ) −1 F ( x ( k ) ),
where JF (x) (or F (x)) is the Jacobian matrix of function F, and x(k) is the k-th approxima-
tion of the root of (1) with the initial guess x(0) . It is well known
√ that Newton’s method is
Copyright: © 2022 by the authors.
a quadratic convergence method with the efficiency index 2 [11]. The third and higher-
Licensee MDPI, Basel, Switzerland.
This article is an open access article
order methods such as the Halley and Chebyshev methods [12] have little practical value
distributed under the terms and
because of the evaluation of the second Frechèt-derivative. However, third and higher-
conditions of the Creative Commons order multi-step methods can be good substitutes because they require the evaluation of
Attribution (CC BY) license (https:// the function and its first derivative at different points.
creativecommons.org/licenses/by/ In the recent decades, many authors tried to design iterative procedures with better effi-
4.0/). ciency and higher order of convergence than the Newton scheme, see, for example, ref. [13–24]
and references therein. However, the accuracy of solutions is highly dependent on the
efficiency of the utilized algorithm. Furthermore, at each step of any iterative method,
we must find the exact solution of an obtained linear system which is expensive in actual
applications, especially when the system size n is very large. However, the proposed
higher-order iterative methods are futile unless they have high-order convergence. There-
fore, the important aim in developing any new algorithm is to achieve high convergence
order with requiring as small as possible the evaluations of functions, derivatives and
matrix inversions. Thus, here, we focus on the technique of the frozen Jacobian multi-step
iterative algorithms. It is shown that this idea is computationally attractive and economical
for constructing iterative solvers because the inversion of the Jacobian matrix (regarding
LU-decomposition) is performed once. Many researchers have reduced the computational
cost of these algorithms by frozen Jacobian multi-step iterative techniques [25–28].
In this work, we construct a new class of frozen Jacobian multi-step iterative methods
for solving the nonlinear systems of equations. This is a high-order convergent algorithm
with an excellent efficiency index. The theoretical analysis is presented completely. Further,
by solving some nonlinear systems, the ability of the methods is compared with some
known algorithms.
The rest of this paper is organized as follows. In the following section, we present
our new methods with obtaining of their order of convergence. Additionally, their com-
putational efficiency are discussed in general. Some numerical examples are considered
in Sections 3 and 4 to show the asymptotic behavior of these methods. Finally, a brief
concluding remark is presented in Section 5.
In (2), for an m-step method (m > 1), one needs m function evaluations and only
one Jacobian evaluation. Further, the number of LU decompositions is one. The order
of convergence for such FJA method is m + 1. In the right-hand side column of (2), the
algorithm is briefy described.
In the following subsections, by choosing two different values for m, a third- and a
fourth-order frozen Jacobian multi-step iterative algorithm are presented.
y ( k ) = x ( k ) − J F ( x ( k ) ) −1 F ( x ( k ) ),
(3)
x(k+1) = x(k) − JF (x(k) )−1 ( F (y(k) ) + F (x(k) )),
we denote this by M3 .
106
Mathematics 2022, 10, 2952
Definition 1 ([29]). Let F be an operator which maps a Banach space X into a Banach space Y. If
there exists a bounded linear operator T from X into Y such that
F (x + y) − F (x) − T (y)
lim = 0,
y →0 y
1 1 1
F (x(n) ) = F (α) + F (α) En + F (α) En2 + F (α) En3 + F (α) En4 + . . .
2! 3! 4!
as α is the root of F so F (α) = 0. As a matter of fact, one may yield the following equations
of F (x(n) ) and F (x(n) ) in a neighborhood of α by using Taylor’s series expansions [32],
/ 0
F (x(n) ) = F (α) En + C2 En2 + C3 En3 + C4 En4 + C5 En5 + O|| En6 || , (4)
/ 0
F (x(n) ) = F (α) I + 2C2 En + 3C3 En2 + 4C4 En3 + 5C5 En4 + 6C6 En5 + O|| En6 || , (5)
F (x(n) )−1 F (x(n) ) = En − C2 En2 + (2C22 − 2C3 ) En3 + (−4C23 + 7C2 C3 − 3C4 ) En4
+(−32C25 + 8C24 − 20C22 C3 + 10C2 C4 + 6C32 − 4C5 ) En5 + O|| En6 ||.
107
Mathematics 2022, 10, 2952
Thus,
Finally, since
we have
Clearly, the error Equation (8) shows that the order of convergence of the frozen
Jacobian multi-step iterative method (3) is three. This completes the proof.
where p is the rate of convergence and c stands for the total computational cost per iteration
in terms of the number of functional evaluations, such that c = (rn + mn2 ) where r refers
to the number of function evaluations needed per iteration and m is the number of Jacobian
matrix evaluations needed per iteration.
It is well known that the computation of LU factorization by any of the existing
methods in the literature normally needs 2n3 /3 flops in floating point operations, while
the floating point operations to solve two triangular systems needs 2n2 flops.
The second criterion is the flops-like efficiency index (FLEI) which was defined by
Montazeri et al. [34] as
1
FLEI = p c
where p is the order of convergence of the method, c denotes the total computational
cost per loop in terms of the number of functional evaluations, as well as the cost of LU
factorization for solving two triangular systems (based on the flops).
As the first comparison, we compare M3 with the third-order method given by
Darvishi [35], which is denoted as M3,1
y ( k ) = x ( k ) − J F ( x ( k ) ) −1 F ( x ( k ) ),
x(k+1) = x(k) − 2( JF (x(k) ) + JF (y(k) ))−1 F (x(k) ).
108
Mathematics 2022, 10, 2952
The second iterative method shown by M3,2 is the following third-order method
introduced by Hernández [36]
y ( k ) = x ( k ) − J F ( x ( k ) ) −1 F ( x ( k ) ),
x(k+1) = x(k) + 12 JF (x(k) )−1 ( JF (y(k) ) − 3JF (x(k) )) × JF (x(k) )−1 F (x(k) ).
Finally, the following third-order iterative method, M3,4 , ref. [38] is considered
The computational efficiency of our third-order method revealed that our method, M3 ,
is the best one in respect with methods M3,1 , M3,2 , M3,3 and M3,4 , as presented in Table 1,
and Figures 1 and 2.
Figure 1. The classical efficiency index for methods M3 , M3,1 , M3,2 , M3,3 and M3,4 .
109
Mathematics 2022, 10, 2952
Figure 2. The flops-like efficiency index for methods M3 , M3,1 , M3,2 , M3,3 and M3,4 .
y ( k ) = x ( k ) − J F ( x ( k ) ) −1 F ( x ( k ) ),
z(k) = x(k) − JF (x(k) )−1 ( F (y(k) ) + F (x(k) )), (9)
x(k+1) = y(k) − JF (x(k) )−1 ( F (z(k) ) + F (y(k) )).
In the following subsections, the order of convergence and efficiency indices are
obtained for the method described in (9).
Hence,
/
( F (z(k) ) + F (y(k) )) = F (α) C2 En2 + 2C3 En3 + (−4C23 + 3C4 ) En4
0 (10)
+(32C25 + 18C24 − 20C22 C3 + 3C5 ) En5 + O|| En6 || .
110
Mathematics 2022, 10, 2952
Since we have x(k+1) = y(k) − JF (x(k) ))−1 ( F (z(k) ) + F (y(k) )) from (6) and (11), the
following result is obtained
x(k+1) = α + (4C23 ) En4 + (32C25 − 26C24 + 20C22 C3 + C5 ) En5 + O|| En6 ||. (12)
This completes the proof, since error Equation (12) shows that the order of convergence
of the frozen Jacobian multi-step iterative method (9) is four.
the fourth-order iterative method M4,2 given by Darvishi and Barati [40],
y ( k ) = x ( k ) − J F ( x ( k ) ) − 1
F ( x ( k ) ),
z ( k ) = x ( k ) − J F ( x ( k ) ) −1 F ( y ( k ) ) + F ( x ( k ) ) ,
/ 0 −1
(x(k) + z(k) )
x(k+1) = x(k) − 16 JF (x(k) ) + 23 JF ( 2 ) + 1
6 J F ( z (k) ) F ( x ( k ) ),
The computational efficiency of our fourth-order method showed that our method M4
is better than methods M4,1 , M4,2 , M4,3 and M4,4 as the comparison results are presented in
Table 2, and Figures 3 and 4. As we can see from Table 2, the indices of our method M4 are
better than similar ones in methods M4,1 , M4,2 , M4,3 and M4,4 . Furthermore, Figures 3 and 4
show the superiority of our method in respect with the another schemes.
111
Mathematics 2022, 10, 2952
Figure 3. The classical efficiency index for methods M4 , M4,1 , M4,2 , M4,3 and M4,4 .
Figure 4. The Flops-like efficiency index for methods M4 , M4,1 , M4,2 , M4,3 and M4,4 .
3. Numerical Results
In order to check the validity and efficiency of our proposed frozen Jacobian multi-
step iterative methods, three test problems are considered to illustrate convergence and
computation behaviors such as efficiency index and some another indices of the frozen
Jacobian multi-step iterative methods. Numerical computations have been performed using
variable precision arithmetic that uses floating point representation of 100 decimal digits of
mantissa in MATLAB. The computer specifications are: Intel(R) Core(TM) i7-1065G7 CPU
1.30 GHz with 16.00 GB of RAM on Windows 10 pro.
The exact zero of F (x) = ( f 1 (x), f 2 (x), . . . , f n (x))t = 0 is (0, 0, . . . , 0)t . To solve (13),
we set the initial guess as (0.78, 0.78, . . . , 0.78)t . The stopping criterion is selected as
|| f (x(k) )|| ≤ 10−3 .
112
Mathematics 2022, 10, 2952
Experiment 2. The next test problem is the following system of nonlinear equations [44],
The exact root of F (x) = 0 is (1, 1, . . . , 1)t . To solve (14), the initial guess is taken as
(2, 2, . . . , 2)t . The stopping criterion is selected as || f (x(k) )|| ≤ 10−8 .
Experiment 3. The last test problem is the following nonlinear system [9],
f i (x) = xi 2 xi+1 − 1, i = 1, 2, . . . , n − 1,
(15)
f n (x) = xn 2 x1 − 1,
with the exact solution (1, 1, . . . , 1)t . To solve (15), the initial guess and the stopping criterion
are respectively considered as (3, 3, . . . , 3)t and || f (x(k) )|| ≤ 10−8 .
Table 3 shows the comparison results between our third-order frozen Jacobian two-
step iterative method M3 and some third-order frozen Jacobian iterative methods, namely,
M3,1 , M3,2 , M3,3 and M3,4 . For all test problems, two different values for n are considered,
namely, n = 50, 100. As this table shows, in all cases, our method works better than the
others. Similarly, in Table 4, CPU time and number of iterations are presented for our
fourth-order method, namely, M4 and methods M4,1 , M4,2 , M4,3 and M4,4 . Similar to M3 ,
the CPU time for M4 is less than the CPU time for the other methods. These tables show
superiority of our methods in respect with the other ones. In Tables 3 and 4, it shows the
number of iterations.
113
Mathematics 2022, 10, 2952
4. Another Comparison
In the previous parts, we presented some comparison results between our methods
M3 and M4 with some another frozen Jacobian multi-step iterative methods from third- and
fourth-order methods. In this section, we compare our presented methods with three other
methods which are fourth- and fifth-order ones. As Tables 5 and 6 and Figures 5 and 6
show, our methods are also better than these methods.
J F ( x ( k ) ) θ1 = F ( x ( k ) ),
y ( k ) = x ( k ) − θ1 ,
J F ( x ( k ) ) θ2 = F ( y ( k ) ),
JF ( x ( k ) ) θ3 = JF ( y ( k ) ) θ2 ,
x(k+1) = y(k) − 2θ2 + θ3 .
y ( k ) = x ( k ) − J F ( x ( k ) ) −1 F ( x ( k ) ),
z ( k ) = y ( k ) − J F ( x ( k ) ) −1 F ( y ( k ) ),
x ( k +1) = z ( k ) − J F ( x ( k ) ) −1 F ( z ( k ) ).
JF ( x ( k ) ) θ1 = F ( x ( k ) ),
y ( k ) = x ( k ) − θ1 ,
J F ( x ( k ) ) θ2 = F ( y ( k ) ),
z(k) = y(k) − 3θ2 ,
JF ( x ( k ) ) θ3 = JF ( z ( k ) ) θ2 ,
JF ( x ( k ) ) θ4 = JF ( z ( k ) ) θ3 ,
x(k+1) = y(k) − 74 θ2 + 12 θ3 + 14 θ4 .
114
Mathematics 2022, 10, 2952
Methods M3 M4 MA MB MC
No. of steps 2 3 2 3 3
Order of convergence 3 4 4 4 5
Functional evaluations 2n + n2 3n + n2 2n + 2n2 3n + n2 2n + 2n2
31/(2n+n ) 41/(3n+n ) 41/(2n+2n ) 41/(3n+n ) 51/(2n+2n )
2 2 2 2 2
The classical efficiency index (IE)
No. of LU decompositions 1 1 1 1 1
Cost of LU decompositions 2n3 2n3 2n3 2n3 2n3
3 3 3 3 3
2n3 2n3 5n3 2n3 5n3
3 + 4n 3 + 6n 3 + 4n 3 + 6n 3 + 4n
Cost of linear systems (based on flops) 2 2 2 2 2
3 3 3 3 3
Flops-like efficiency index (FLEI) 1/( 3 +5n2 +2n)
2n 1/( 3 +7n2 +3n)
2n 1/( 3 +6n2 +2n)
5n 1/( 3 +7n2 +3n)
2n 1/( 3 +6n2 +2n)
5n
3 4 4 4 5
115
Mathematics 2022, 10, 2952
5. Conclusions
In this article, two new frozen Jacobian two- and three-step iterative methods to solve
systems of nonlinear equations are presented. For the first method, we proved that the order
of convergence is three, while for the second one, a fourth-order convergence is proved.
By solving three different examples, one may see our methods work as well. Further, the
CPU time of our methods is less than some selected frozen Jacobian multi-step iterative
methods in the literature. Moreover, other indices of our methods such as number of steps,
functional evaluations, the classical efficiency index, and so on, are better than these indices
for other methods. This class of the frozen Jacobian multi-step iterative methods can be a
pattern for new research on the frozen Jacobian iterative algorithms.
Author Contributions: Investigation, R.H.A.-O. and M.T.D.; Project administration, M.T.D.; Re-
sources, R.H.A.-O.; Supervision, M.T.D.; Writing—original draft, M.T.D. All authors have read and
agreed to the published version of the manuscript.
Funding: This research received no external funding.
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: Not applicable.
Acknowledgments: The authors would like to thank the editor of the journal and three anonymous
reviewers for their generous time in providing detailed comments and suggestions that helped us to
improve the paper.
Conflicts of Interest: The authors declare no conflict of interest.
References
1. Fay, T.H.; Graham, S.D. Coupled spring equations. Int. J. Math. Educ. Sci. Technol. 2003, 34, 65–79. [CrossRef]
2. Petzold, L. Automatic selection of methods for solving stiff and non stiff systems of ordinary differential equations. SIAM J. Sci.
Stat. Comput. 1983, 4, 136–148. [CrossRef]
3. Ehle, B.L. High order A-stable methods for the numerical solution of systems of D.E.’s. BIT Numer. Math. 1968, 8, 276–278.
[CrossRef]
4. Wambecq, A. Rational Runge-Kutta methods for solving systems of ordinary differential equations. Computing 1978, 20, 333–342.
[CrossRef]
5. Liang, H.; Liu, M.; Song, M. Extinction and permanence of the numerical solution of a two-prey one-predator system with
impulsive effect. Int. J. Comput. Math. 2011, 88, 1305–1325. [CrossRef]
6. Harko, T.; Lobo, F.S.N.; Mak, M.K. Exact analytical solutions of the Susceptible-Infected-Recovered (SIR) epidemic model and of
the SIR model with equal death and birth rates. Appl. Math. Comput. 2014, 236, 184–194. [CrossRef]
7. Zhao, J.; Wang, L.; Han, Z. Stability analysis of two new SIRs models with two viruses. Int. J. Comput. Math. 2018, 95, 2026–2035.
[CrossRef]
8. Kröger, M.; Schlickeiser, R. Analytical solution of the SIR-model for the temporal evolution of epidemics, Part A: Time-independent
reproduction factor. J. Phys. A Math. Theor. 2020, 53, 505601. [CrossRef]
9. Ullah, M.Z.; Behl, R.; Argyros, I.K. Some high-order iterative methods for nonlinear models originating from real life problems.
Mathematics 2020, 8, 1249. [CrossRef]
10. Argyros, I.K. Concerning the “terra incognita” between convergence regions of two Newton methods. Nonlinear Anal. 2005, 62,
179–194. [CrossRef]
11. Drexler, M. Newton Method as a Global Solver for Non-Linear Problems. Ph.D Thesis, University of Oxford, Oxford, UK, 1997.
12. Guti’errez, J.M.; Hern’andez, M.A. A family of Chebyshev-Halley type methods in Banach spaces. Bull. Aust. Math. Soc. 1997, 55,
113–130. [CrossRef]
13. Cordero, A.; Jordán, C.; Sanabria, E.; Torregrosa, J.R. A new class of iterative processes for solving nonlinear systems by using one
divided differences operator. Mathematics 2019, 7, 776. [CrossRef]
14. Stefanov, S.M. Numerical solution of systems of non linear equations defined by convex functions. J. Interdiscip. Math. 2022, 25,
951–962. [CrossRef]
15. Lee, M.Y.; Kim, Y.I.K. Development of a family of Jarratt-like sixth-order iterative methods for solving nonlinear systems with
their basins of attraction. Algorithms 2020, 13, 303. [CrossRef]
16. Cordero, A.; Jordán, C.; Sanabria-Codesal, E.; Torregrosa, J.R. Design, convergence and stability of a fourth-order class of iterative
methods for solving nonlinear vectorial problems. Fractal Fract. 2021, 5, 125. [CrossRef]
116
Mathematics 2022, 10, 2952
17. Amiri, A.; Cordero, A.; Darvishi, M.T.; Torregrosa, J.R. A fast algorithm to solve systems of nonlinear equations. J. Comput. Appl.
Math. 2019, 354, 242–258. [CrossRef]
18. Argyros, I.K.; Sharma, D.; Argyros, C.I.; Parhi, S.K.; Sunanda, S.K. A family of fifth and sixth convergence order methods for
nonlinear models. Symmetry 2021, 13, 715. [CrossRef]
19. Singh, A. An efficient fifth-order Steffensen-type method for solving systems of nonlinear equations. Int. J. Comput. Sci. Math.
2018, 9, 501–514. [CrossRef]
20. Ullah, M.Z.; Serra-Capizzano, S.; Ahmad, F. An efficient multi-step iterative method for computing the numerical solution of
systems of nonlinear equations associated with ODEs. Appl. Math. Comput. 2015, 250, 249–259. [CrossRef]
21. Pacurar, M. Approximating common fixed points of Pres̆ic-Kannan type operators by a multi-step iterative method. An. St. Univ.
Ovidius Constanta 2009, 17, 153–168.
22. Rafiq, A.; Rafiullah, M. Some multi-step iterative methods for solving nonlinear equations. Comput. Math. Appl. 2009, 58,
1589–1597. [CrossRef]
23. Aremu, K.O.; Izuchukwu, C.; Ogwo, G.N.; Mewomo, O.T. Multi-step iterative algorithm for minimization and fixed point
problems in p-uniformly convex metric spaces. J. Ind. Manag. Optim. 2021, 17, 2161. [CrossRef]
24. Soleymani, F.; Lotfi, T.; Bakhtiari, P. A multi-step class of iterative methods for nonlinear systems. Optim. Lett. 2014, 8, 1001–1015.
[CrossRef]
25. Qasim, U.; Ali, Z.; Ahmad, F.; Serra-Capizzano, S.; Ullah, M.Z.; Asma, M. Constructing frozen Jacobian iterative methods for
solving systems of nonlinear equations, associated with ODEs and PDEs using the homotopy method. Algorithms 2016, 9, 18.
[CrossRef]
26. Amat, S.; Busquier, S.; Grau, À.; Grau-Sánchez, M. Maximum efficiency for a family of Newton-like methods with frozen
derivatives and some applications. Appl. Math. Comput. 2013, 219, 7954–7963. [CrossRef]
27. Kouser, S.; Rehman, S.U.; Ahmad, F.; Serra-Capizzano, S.; Ullah, M.Z.; Alshomrani, A.S.; Aljahdali, H.M.; Ahmad, S.; Ahmad, S.
Generalized Newton multi-step iterative methods GMNp,m for solving systems of nonlinear equations. Int. J. Comput. Math.
2018, 95, 881–897. [CrossRef]
28. Ahmad, F.; Tohidi, E.; Ullah, M.Z.; Carrasco, J.A. Higher order multi-step Jarratt-like method for solving systems of nonlinear
equations: Application to PDEs and ODEs. Comput. Math. Appl. 2015, 70, 624–636. [CrossRef]
29. Kaplan, W.; Kaplan, W.A. Ordinary Differential Equations; Addison-Wesley Publishing Company: Boston, MA, USA, 1958.
30. Emmanuel, E.C. On the Frechet derivatives with applications to the inverse function theorem of ordinary differential equations.
Aian J. Math. Sci. 2020, 4, 1–10.
31. Ortega, J.M.; Rheinboldt, W.C.; Werner, C. Iterative Solution of Nonlinear Equations in Several Variables; Society for Industrial and
Applied Mathematics: Philadelphia, PA, USA, 2000.
32. Behl, R.; Bhalla, S.; Magre nán, A.A.; Kumar, S. An efficient high order iterative scheme for large nonlinear systems with dynamics.
J. Comput. Appl. Math. 2022, 404, 113249. [CrossRef]
33. Ostrowski, A.M. Solutions of the Equations and Systems of Equations; Prentice-Hall: England Cliffs, NJ, USA; New York, NY,
USA, 1964.
34. Montazeri, H.; Soleymani, F.; Shateyi, S.; Motsa, S.S. On a new method for computing the numerical solution of systems of
nonlinear equations. J. Appl. Math. 2012, 2012, 751975. [CrossRef]
35. Darvishi, M.T. A two-step high order Newton-like method for solving systems of nonlinear equations. Int. J. Pure Appl. Math.
2009, 57, 543–555.
36. Hernández, M.A. Second-derivative-free variant of the Chebyshev method for nonlinear equations. J. Optim. Theory Appl. 2000, 3,
501–515. [CrossRef]
37. Babajee, D.K.R.; Dauhoo, M.Z.; Darvishi, M.T.; Karami, A.; Barati, A. Analysis of two Chebyshev-like third order methods free
from second derivatives for solving systems of nonlinear equations. J. Comput. Appl. Math. 2010, 233, 2002–2012. [CrossRef]
38. Noor, M.S.; Waseem, M. Some iterative methods for solving a system of nonlinear equations. Comput. Math. Appl. 2009, 57,
101–106. [CrossRef]
39. Sharma, J.R.; Guha, R.K.; Sharma, R. An efficient fourth order weighted-Newton method for systems of nonlinear equations.
Numer. Algorithms 2013, 62, 307–323. [CrossRef]
40. Darvishi, M.T.; Barati, A. A fourth-order method from quadrature formulae to solve systems of nonlinear equations. Appl. Math.
Comput. 2007, 188, 257–261. [CrossRef]
41. Soleymani, F.,Regarding the accuracy of optimal eighth-order methods. Math. Comput. Modell. 2011, 53, 1351–1357. [CrossRef]
42. Cordero, A.; Hueso, J.L.; Martínez, E.; Torregrosa, J.R. A modified Newton-Jarratt’s composition. Numer. Algorithms 2010, 55,
87–99. [CrossRef]
43. Shin, B.C.; Darvishi, M.T.; Kim, C.H.,A comparison of the Newton–Krylov method with high order Newton-like methods to solve
nonlinear systems. Appl. Math. Comput. 2010, 217, 3190–3198. [CrossRef]
44. Waziri, M.Y.; Aisha, H.; Gambo, A.I. On performance analysis of diagonal variants of Newton’s method for large-scale systems of
nonlinear equations. Int. J. Comput. Appl. 2011, 975, 8887.
117
mathematics
Article
Approximation of the Fixed Point of the Product of Two
Operators in Banach Algebras with Applications to Some
Functional Equations
Khaled Ben Amara 1 , Maria Isabel Berenguer 2,3, * and Aref Jeribi 1
1 Department of Mathematics, Faculty of Sciences of Sfax, University of Sfax, Sfax 3000, Tunisia
2 Department of Applied Mathematics, E.T.S. de Ingeniería de Edificación, University of Granada,
18071 Granada, Spain
3 Institute of Mathematics (IMAG), University of Granada, 18071 Granada, Spain
* Correspondence: [email protected]
Abstract: Making use of the Boyd-Wong fixed point theorem, we establish a new existence and
uniqueness result and an approximation process of the fixed point for the product of two nonlinear
operators in Banach algebras. This provides an adequate tool for deriving the existence and
uniqueness of solutions of two interesting type of nonlinear functional equations in Banach algebras,
as well as for developing an approximation method of their solutions. In addition, to illustrate the
applicability of our results we give some numerical examples.
Keywords: Banach algebras; fixed point theory; functional equations; Schauder bases
x = A ( x ) · B ( x ), (3)
where A and B are nonlinear operators map a nonempty closed convex subset Ω ⊂ E
into E.
A hybrid fixed point result to (3) was proved by Dhage in [12] and since then, several
extensions and generalizations of this result have been achieved. See [13–15] and the
references therein. These results can be used to achieves the existence of solutions. Although
the explicit calculation of the fixed point is difficult in most cases, the previous cited results
are regarded as one of the most powerful tools to give an approximation of the fixed
point by a computational method and to develop numerical methods that allow us to
approximate the solution of these equations.
In Banach spaces, several works deals with developing numerical techniques in
order to approximate the solutions of integral and integro–differential equations, by using
different methods such as the Chebyshev polynomial [16], the secant-like methods [17],
using Schauder’s basis [18,19], the parameterization method [20], the wavelet methods [21],
a collocation method in combination with operational matrices of Berstein polynomials [22],
the contraction principle and a suitable quadrature formula [23], the variational iteration
method [24], etc.
Since the Banach algebras represents a practical framework for several equations such
as (1) and (2), and in general (3), the purposes of this paper are twofold. Firstly, to present,
under suitable conditions, a method to approximate the fixed point of a hybrid equation of
type (3), by means of the product and composition of operators defined in a Banach algebra.
Secondly, to set forth and apply the proposed method to obtain an approximation of the
solutions of (1) and (2).
The structure of this work is as follows: in Section 2 we present some definitions and
auxiliary results; in Section 3 we derive an approximation method for the fixed point of the
hybrid Equation (3); in Sections 4 and 5, we apply our results to prove the existence and the
uniqueness of solution of (1) and (2), we give an approximation method for these solutions
and moreover, we establish some numerical examples to illustrate the applicability of our
results. Finally, some conclusions are quoted in Section 6.
2. Analytical Tools
In this section, we provide some concepts and results that we will need in the following
sections. The first analytical tool to be used comes from the theory of the fixed point. Let X
be a Banach space with norm · and the zero element θ. We denote by B( x, r ) the closed
ball centered at x with radius r. We write Br to denote B(θ, r ). For any bounded subset Ω of
X, the symbol Ω denotes the norm of a set Ω, i.e., Ω = sup{ x , x ∈ Ω}.
Let us introduce the concept of D -Lipschitzian mappings which will be used in
the sequel.
Ax − Ay ≤ φ( x − y ) ∀ x, y ∈ X
120
Mathematics 2022, 10, 4179
where ϕ : [0, ∞) → [0, ∞) is a continuous function such that ϕ(r ) < r if r > 0. Then A has a
unique fixed point x̃ ∈ X and for any x0 ∈ X, the sequence { An ( x0 )}n∈N converges to x̃.
On the other hand, Schauder bases will constitute the second essential tool. We recall
that a Schauder basis in a Banach space E is a sequence {en }n∈N ⊂ E such that for every
x ∈ E, there is a unique sequence { an }n∈N ⊂ R such that
x= ∑ an en .
n ≥1
This notion produces the concept of the sequence of projections Pn : E → E, defined by the
formula
n
Pn ∑ a k ek = ∑ ak ek ,
k ≥1 k =1
Moreover, in view of the Baire category Theorem [27], that for all n ≥ 1, en∗ and Pn are
continuous. This yields, in particular, that
lim Pn ( x ) − x = 0.
n→∞
121
Mathematics 2022, 10, 4179
A( x ) B ( x ) − B ( y ) + B ( y ) A ( x ) − A ( y ) ≤ A ( X ) ψ ( x − y ) + B ( X ) ϕ ( x − y ).
This implies that A · B defines a nonlinear contraction with D -function
Applying the cited Boyd-Wong’s fixed point Theorem, we obtain the desired result.
Boyd-Wong’s fixed point Theorem expresses the fixed point of A · B as the limit of the
sequence {( A · B)n ( x0 )}n∈N with x0 ∈ X. If it is possible explicitly compute ( A · B)n ( x0 ),
then for each n, the expression ( A · B)n ( x0 ) would be an approximation of the fixed point.
But in the practice, this explicit calculation use to be not possible. For that, our aim is to
propose another approximation of the fixed point which simple to calculate. We will need
the following lemma.
m −1
∑ φm− p ( A · B) ◦ Tp−1 ◦ . . . ◦ T1 ( x0 ) − Tp ◦ . . . ◦ T1 ( x0 ) +
p =1
ε
( A · B) ◦ Tm−1 ◦ . . . ◦ T1 ( x0 ) − Tm ◦ . . . ◦ T1 ( x0 ) ≤ . (5)
2
Then,
x̃ − Tm ◦ . . . ◦ T1 ( x0 ) ≤ ε.
( A · B)m ( x0 ) − Tm ◦ . . . ◦ T1 ( x0 ) ≤
( A · B)m−1 ◦ ( A · B)( x0 ) − ( A · B)m−1 ◦ T1 ( x0 )
+ ( A · B)m−2 ◦ ( A · B) ◦ T1 ( x0 ) − ( A · B)m−2 ◦ T2 ◦ T1 ( x0 ) + · · · +
+ ( A · B) ◦ ( A · B) ◦ Tm−2 ◦ . . . ◦ T1 ( x0 ) − ( A · B) ◦ Tm−1 ◦ . . . ◦ T1 ( x0 )
+ ( A · B) ◦ Tm−1 ◦ . . . ◦ T1 ( x0 ) − Tm ◦ . . . ◦ T1 ( x0 ) .
( A · B)m ( x0 ) − Tm ◦ . . . ◦ T1 ( x0 ) ≤
m −1
∑ φm− p ( A · B) ◦ Tp−1 ◦ . . . ◦ T1 ( x0 ) − Tp ◦ . . . ◦ T1 ( x0 )
p =1
+ ( A · B) ◦ Tm−1 ◦ . . . ◦ T1 ( x0 ) − Tm ◦ . . . ◦ T1 ( x0 ) .
122
Mathematics 2022, 10, 4179
x̃ − Tm ◦ . . . ◦ T1 ( x0 ) ≤
m −1
∑ φm− p ( A · B) ◦ Tp−1 ◦ . . . ◦ T1 ( x0 ) − Tp ◦ . . . ◦ T1 ( x0 )
p =1
Taking into account the above lemma, observe that, under the previous hypotheses,
x ∗ = Tm ◦ . . . ◦ T1 ( x0 ) ≈ x̃
( A · B)m ( x0 ) ≈ Tm ◦ . . . ◦ T1 ( x0 ).
Schauder bases are the tool we will use next to build such operators. Concretely, for the
case of problems (1) and (2), which can be written as a fixed point problem x = A( x ) · B( x ),
where B is given by an integral operator, we will choice to approximate only the power
terms of the operator B which is difficult to compute in general, unlike operator A which is
easy to calculate and does not need to approximate their power terms. For this reason, we
specifically propose the following scheme, in which we will construct S1 , S2 ,· · · , Sm :
x0
↓
( A · B)( x0 ) ≈ T1 ( x0 ) = A( x0 ) · S1 ( x0 )
↓ ↓
( A · B )2 ( x0 ) ≈ T2 ◦ T1 x0 = ( A · S2 ) ◦ T1 ( x0 )
.. .. ..
. . .
.. .. ..
. . .
↓ ↓
( A · B ) m ( x0 ) ≈ Tm ◦ . . . ◦ T1 ( x0 ) = ( A · Sm ) ◦ Tm−1 ◦ . . . ◦ T1 ( x0 ) ≈ x̃
Remark 1. The above scheme is constructed as follows. In the first term, we approximate B( x0 )
by S1 ( x0 ), then we obtain T1 ( x0 ) := A( x0 ) · S1 ( x0 ) as an approximation of the first term of the
Picard iterate, A( x0 ) · B( x0 ). In the second term of our scheme, we approximate the second term
of the Picard iterate, ( A · B)2 ( x0 ) = A(( A · B)( x0 )) · B(( A · B)( x0 )). So we obtain the second
term of our scheme by combining the first term T1 ( x0 ), with an approximation of the operator
B, which denoted by S2 , and consequently we obtain a second term of our scheme T2 ◦ T1 ( x0 ) =
( A · S2 )( T1 ( x0 )) which approximate ( A · B)2 ( x0 ).
123
Mathematics 2022, 10, 4179
and
(iii) There is a constant δ > 0 such that supx∈R,| x|≤r | f (0, x )|−1 ≤ δ.
Throughout this section, Ω will denote the closed ball Br of C ( J ), where r is defined in
the above assumption (ii ). Observe that Ω is a non-empty, closed, convex and bounded
subset of C ( J ).
M A MB ≤ r and
M A δLμ t + M A δ2 |α(0)| Mμ + MB α ∞ ϕ(t) + M A γ(·) L1 ψ ( t ) < t, ∀t > 0,
Proof. Notice that the problem of the existence of a solution to (1) can be formulated in the
following fixed point problem x = A( x ) · B( x ), where A, B are given for x ∈ C ( J ) by
Let x ∈ Ω and t, t ∈ J. Since f is D -lipschitzian with respect to the second variable and is
continuous with respect to the first variable, then by using the inequality
| f (t, x (t)) − f (t , x (t ))| ≤ | f (t, x (t)) − f (t , x (t))| + | f (t , x (t)) − f (t , x (t ))|,
This proves the claim. Our strategy is to apply Theorem 2 to show the existence and the
uniqueness of a fixed point for the product A · B in Ω which in turn is a continuous solution
for problem (1).
For this purpose, we will claim, first, that A and B are D -lipschitzian mappings on Ω.
The claim regarding A is clear in view of assumption (ii ), that is A is D -lipschitzian with
D -function Φ such that
Φ(t) = α ∞ ϕ(t), t ∈ J.
124
Mathematics 2022, 10, 4179
We corroborate now the claim for B. Let x, y ∈ Ω, and let t ∈ J. By using our assumptions,
we obtain
|( B( x ))(t) − ( B(y))(t)| =
t
1 1
μ ( x ) − μ ( y ) + g ( s, x ( s )) − g ( s, y ( s )) ds ≤
f (0, x (0)) f (0, y(0)) 0
Lμ |α(0)|
x−y + Lμ r + |μ(0)| ϕ( x − y )+
| f (0, x (0))| | f (0, x (0)) f (0, y(0))|
t
|γ(s)|ψ(| x (s) − y(s)|)ds ≤
0
δLμ x − y + δ |α(0)| Lμ r + |μ(0)| ϕ( x − y ) + γ(·)
2
L1 ψ ( x − y ).
Taking the supremum over t, we obtain that B is D -lipschitzian with D -function Ψ such
that
Ψ(t) = δLμ t + δ2 |α(0)| Lμ r + |μ(0)| ϕ(t) + γ(·) L1 ψ(t), t ∈ J.
On the other hand, bearing in mind assumption (i ), by using the above discussion we can
see that A(Ω) and B(Ω) are bounded with bounds M A and MB respectively. Taking into
account the estimate M A MB ≤ r, we obtain that A · B maps Ω into Ω.
Since
t
1
|( B( x ))(t)| ≤ μ( x ) + | g(s, x (s))|ds
f (0, x (0)) 0
t t
≤ δ(|μ( x ) − μ(0)| + |μ(0)|) + | g(s, x (s)) − g(s, 0)|ds + | g(s, 0)|ds
0 0
t t
≤ δ( Lμ x + |μ(0)|) + |γ(s)|ψ(| x (s)|)ds + | g(s, 0)|ds,
0 0
B( x ) ≤ δ( Lμ x + |μ(0)|) + ρ γ ∞ ψ (r ) + ρ g(·, 0) ∞ = MB .
Now, applying Theorem 2, we infer that (1) has one and only one solution x̃ in Ω, and
for each x0 ∈ Ω we have
125
Mathematics 2022, 10, 4179
In what follows we will assume that the hypotheses of the Theorem 3 are satisfied.
S p : C( J ) −→ C ( J )
x −→ S p ( x )
defined as t
1
S p ( x )(t) = μ( x ) + Pn p (U0 ( x ))(s)ds,
f (0, x (0)) 0
Remark 2. For p ≥ 1 and any n p ∈ N that we use for defining Tp , the operator Tp maps Ω into
Ω, since just keep in mind that for x ∈ Ω, we have
t
1
Tp ( x )(t) = A( x )(t) μ( x ) + Pn p (U0 ( x ))(s)ds ≤
f (0, x (0)) 0
t
| f (t, x (t))| δ|μ( x )| + Pn p (U0 ( x ))(s)ds ,
0
and proceeding as in the above subsection and using the fact that Pn p is a bounded linear operator
on C ( J ), we get 1 2
Tp ( x )(t) ≤ M A δ|μ( x )| + ρ
Pn p (U0 ( x )) ≤
3 4
M A δ( Lμ r + |μ(0)|) + ρ sup | g(s, x (s))| ≤ M A MB < r.
s∈ J
Theorem 4. Let x̃ be the unique solution to the nonlinear problem (1). Let x0 ∈ Ω and ε > 0, then
there exist m ∈ N and ni ∈ N to construct Ti for i = 1, . . . , m, in such a way that
x̃ − Tm ◦ . . . ◦ T1 ( x0 ) ≤ ε.
( A · B)m ( x0 ) − x̃ ≤ ε/2.
126
Mathematics 2022, 10, 4179
and A p : C ( J ) → C ( J ) by
A p ( x )(s) := f s, Tp ◦ . . . ◦ T1 ( x )(s) , s ∈ J, x ∈ C ( J ).
m −1
∑ Θm− p ( A · B) ◦ Tp−1 ◦ . . . ◦ T1 ( x0 ) − Tp ◦ . . . ◦ T1 ( x0 ) +
p =1
( A · B) ◦ Tm−1 ◦ . . . ◦ T1 ( x0 ) − Tm ◦ . . . ◦ T1 ( x0 ) ≤ ε/2.
Since Θ is a nondecreasing continuous mapping, and taking into account the convergence
of the projection operators associated to the Schauder basis, for all 1 ≤ p ≤ m we obtain
Θm− p ρM A Pn p (U p−1 ( x0 )) − U p−1 ( x0 ) ≤ ε/2m,
m −1
∑ Θm− p ( A · B) ◦ Tp−1 ◦ . . . ◦ T1 ( x0 ) − Tp ◦ . . . ◦ T1 ( x0 ) +
p =1
( A · B) ◦ Tm−1 ◦ . . . ◦ T1 ( x0 ) − Tm ◦ . . . ◦ T1 ( x0 ) ≤
m −1
∑ Θm− p ρM A Pn p (U p−1 ( x0 )) − U p−1 ( x0 ) + ρM A Pnm (Um−1 ( x0 )) − Um−1 ( x0 ) ≤ ε/2.
p =1
127
Mathematics 2022, 10, 4179
numbers (see Table 1 in [18] and [28,29] for details). In following examples, we will denote
x ∗ = Tm ◦ . . . ◦ T1 ( x0 ) with m = 4 and n1 = · · · = nm = l with l = 9 or l = 33.
Example 1. Consider the nonlinear differential equation with a nonlocal initial condition
⎧
⎪
⎪ d x (t)
⎪
⎪ = ae− x(t) , t ∈ J,
⎪
⎨ dt f (t, x (t))
(12)
⎪
⎪
⎪
⎪ 3
⎪
⎩ x (0) = b sup | x (t)| + ,
t∈ J 4
b
where 0 < a < 1/ log(2) and f (t, x ) = .
1 + ae−b t
Let us define the mappings g : J × R → R and μ : C ( J ) → R by
g(t, x ) = ae− x , t ∈ J, x ∈ R
and
μ(u) = b sup |u(t)| + 3/4 , u ∈ C ( J ).
t∈ J
Let R be small enough such that a(log(2) + R) < 1. Let x, y ∈ [− R, R], by an elementary
calculus we can show that the functions f and g satisfy the condition (ii), with α(t) = ϕ(t) = 0,
γ(t) = ae R (1 − e−t ), and ψ(t) = t.
On the other hand, we have that μ is Lipschizian with a Lipschiz constant Lμ = b, and
1
sup [ f (0, x )]−1 ≤ δ = .
x,| x |≤ R b
128
Mathematics 2022, 10, 4179
Example 2. Consider the nonlinear differential equation with a nonlocal initial condition
⎧
⎪ d x (t)
⎪
⎪ = a( x (t))2 , t ∈ J,
⎨ dt f (t, x (t))
(13)
⎪
⎪
⎪
⎩ x (0) = 1/(4b) sup | x (t)|2 ,
t∈ J
b ( t + 1)
where a, b are positive constants such that ab2 < 3 and f (t, x ) = .
ab2
1+ 3 ( x /b
3 3 − 1)
Let us define the mappings g : J × R → R and μ : C ( J ) → R by
a ( t + 1) R2
we can show that f and g satisfy the condition (ii) with α(t) = 2 , γ(t) = 2aR,
1 − 3b
a
( R3 + b3 )
and ϕ(t) = ψ(t) = t.
On the other hand, we have that
R
|μ(u) − μ(v)| ≤ u−v .
2b
Now, applying Theorem 3, in order to obtain that (13), with a is small enough, has a unique solution
in BR with R = 1/2. We can check that the solution is x̃ (t) = b(t + 1). Table 2 shows the numerical
results of the proposed method for a = 0.05, b = 1/4 and x0 (t) = 12 t.
129
Mathematics 2022, 10, 4179
and
Throughout this section, Ω will denote the closed ball Br of C ( J ), where r is defined in
the above assumption (ii ).
First, we will establish the following result which shows the existence and uniqueness
of a solution.
where
Proof. By using similar arguments to those in the above section, we can show that A
and B define D -lipschitzian mappings from Ω into C ( J ), with D -functions α ∞ ϕ and
ρ γ ∞ ψ, respectively. Also it is easy to see that A(Ω) and B(Ω) are bounded with bounds,
respectively, M A and MB . Taking into account our assumptions, we deduce that A · B maps
Ω into Ω.
Notice that A · B defines a nonlinear contraction with D -function
Now, an application of Theorem 2 yields that (2) has one and only one solution x̃ in Ω, and
for each x0 ∈ Ω we have
130
Mathematics 2022, 10, 4179
proceeding essentially as in the above section and using the fact that Pn p is a bounded linear operator
on C ( J × J ), we get
Tp ( x )(t) ≤ M A |q(t)| + ρ
Pn p (U0 ( x )) ≤
MA q ∞ + ρ sup | K ( t, s, x ( τ ( s )))| ≤ M A MB .
t,s∈ J
Accordingly, under the hypotheses of the Theorem 5, the mapping Tp maps Ω into Ω. In particular,
for m ≥ 1, the operator Tm ◦ . . . ◦ T1 maps Ω into Ω.
Analogously as we did in the previous section, the following result allow us to justify
it is possible to choose n1 , n2 , . . . in order that T1 , T2 , . . . can be used to approximate the
unique solution to Equation (2).
Theorem 6. Let x̃ be the unique solution to the nonlinear Equation (2). Let x0 ∈ Ω and ε > 0,
then there exists m ∈ N and ni ∈ N to construct Ti for i = 1, . . . , m, such that
x̃ − Tm ◦ . . . ◦ T1 ( x0 ) ≤ ε.
( A · B)m ( x0 ) − x̃ ≤ ε/2.
and A p : C ( J ) → C ( J ) by
A p ( x )(s) := f s, Tp ◦ . . . ◦ T1 ( x )(s) , s ∈ J, x ∈ C ( J ).
131
Mathematics 2022, 10, 4179
Proceeding essentially, as in the Theorem 4, and taking into account (15) together with
Remark 3 the desired thesis can be proved.
where γ(t, s) = 2R, and ψ(t) = t. An application of Theorem 5, yields that (17) has a unique
solution in BR , with R = 3. In fact the solution is x̃ (t) = b(t + 1).
Using the proposed method with a = 0.1, b = 0.1 and x0 (t) = t2 , we obtain Table 3.
Similarly to that above, (18) can be written as a fixed point problem with the same notations in (14).
Let R > 0 and let x, y ∈ [− R, R]. By an elementary calculus we can show that the functions f and
g satisfy the condition (ii), with α(t) = ae R , γ(t) = (1 + tan2 (1 − c) R), and ϕ(t) = (1 − e−t )
and ψ(t) = tan(1 − c)t.
Apply Theorem 5, (18), with a small enough and c = 1 − a, has a unique solution in
BR with R = 3, in fact the solution is x̃ (t) = t. We obtain the results given in Table 4 for
a = 0.01, b = 1, R = 3, and x0 (t) = sin(t).
132
Mathematics 2022, 10, 4179
x +1
(19)
K (t, s, x ) = e−(t+1)u du,
0
q(t) = ( b + t )2 .
Let 0 < R < 1 and let x, y ∈ [− R, R]. By an elementary calculus, we can show that f and g
2t
satisfy the condition (ii ), with α(t) = ϕ(t) = 0, ψ(t) = 0 e−s ds, and γ(t, s) = t+1 1 e(t+1)( R−1) .
Taking a = 0.1, b = 1, and applying Theorem 5, the problem has a unique solution in
BR = { x ∈ C ([0, 1]); x ≤ R}, in fact the solution is x̃ (t) = at. We obtain the results given in
Table 5.
6. Conclusions
In this paper we have presented a numerical method, based on the use of Schauder’s
bases, to solve hybrid nonlinear equations in Banach algebras. To do this, we have used
133
Mathematics 2022, 10, 4179
Boyd-Wong’s theorem to establish the existence and uniqueness of a fixed point for the
product of two nonlinear operators in Banach algebra (Theorem 2). The method is applied
to a wide class of nonlinear hybrid equations such as the ones we have illustrated by means
of several numerical examples.
The possibility of applying this process or a similar idea to other types of hybrid
equations or systems of such equations is open and we hope to discuss this in the near future.
Author Contributions: Conceptualization, K.B.A. and M.I.B.; methodology, K.B.A., M.I.B. and A.J.;
software, K.B.A. and M.I.B.; validation, K.B.A. and M.I.B.; formal analysis, K.B.A., M.I.B. and A.J.;
investigation, K.B.A. and M.I.B.; writing—original draft preparation, K.B.A. and M.I.B.; writing—review
and editing, K.B.A. and M.I.B.; supervision, K.B.A., M.I.B. and A.J. All authors have read and agreed to
the published version of the manuscript.
Funding: The research of Aref Jeribi and Khaled Ben Amara has been partially supported by the
University of Sfax (Tunisia). The research of María Isabel Berenguer has been partially supported by
Junta de Andalucía (Spain), Project Convex and numerical analysis, reference FQM359, and by the María
de Maeztu Excellence Unit IMAG, reference CEX2020-001105-M, funded by MCIN/AEI/10.13039/
501100011033/.
Data Availability Statement: Not applicable.
Acknowledgments: This work was partially carried out during the first author’s visit to the
Department of Applied Mathematics, University of Granada. The authors wish to thank the
anonymous referees for their useful comments. They also acknowledge the financial support of the
University of Sfax (Tunisia), the Consejería de Conocimiento, Investigación y Universidad, Junta de
Andalucía (Spain) and the María de Maeztu Excellence Unit IMAG (Spain).
Conflicts of Interest: The authors declare no conflict of interest.
References
1. Deimling, K. Nonlinear Functional Analysis; Springer: Berlin/Heidelberg, Germany, 1985; ISBN 3-540-13928-1.
2. Djebali, S.; Sahnoun, Z. Nonlinear alternatives of Schauder and Krasnosel’skii types with applications to Hammerstein integral
equations in L1 -spaces. J. Differ. Equ. 2010, 249, 2061–2075. [CrossRef]
3. Byszewski, L. Theorems about the existence and uniqueness of solutions of a semilinear evolution nonlocal Cauchy problem. J.
Math. Anal. Appl. 1991, 162, 494–505. [CrossRef]
4. Byszewski, L. Existence and Uniqueness of Mild and Classical Solutions of Semilinear Functional-Differential Evolution Nonlocal Cauchy
Problem; Selected Problems of Mathematics; Cracow University of Technology: Krakow, Poland, 1995
5. Deng, K. Exponential decay of solutions of semilinear parabolic equations with nonlocal initial conditions. J. Math. Anal. Appl.
1993, 179, 630–637. [CrossRef]
6. Dhage, B.C. Multi-valued mappings and fixed points I. Nonlinear Funct. Anal. Appl. 2005, 10, 359–378. [CrossRef]
7. O’Regan, D. New fixed point results for 1-set contractive set-valued maps. Comput. Math. Appl. 1998, 35, 27–34. [CrossRef]
8. Ben Amar, A.; Chouayekh, S.; Jeribi, A. Fixed point theory in a new class of Banach algebras and application. Afr. Mat. 2013, 24,
705–724. [CrossRef]
9. Dhage, B.C. On some nonlinear alternatives of Leray-Schauder type and functional integral equations. Arch. Math. (Brno) 2006,
42, 11–23.
10. Jeribi, A.; Kaddachi, N.; Krichen, B. Existence results for a system of nonlinear integral equations in Banach algebras under weak
topology. Fixed Point Theory 2017, 18, 247–267. [CrossRef]
11. Jeribi, A.; Krichen, B. Nonlinear functional analysis in Banach spaces and Banach algebras: Fixed point theory under weak
topology for nonlinear operators and block operator matrices with applications. In Monographs and Research Notes in Mathematics;
CRC Press/Taylor and Francis: Boca Raton, FL, USA, 2015.
12. Dhage, B.C. On some variants of Schauder’s fixed point principle and applications to nonlinear integral equations. J. Math. Phy.
Sci. 1988, 25, 603–611.
13. Cichon, M.; Metwali, M.M.A. On a fixed point theorem for the product of operators. J. Fixed Point Theory Appl. 2016, 18, 753–770.
[CrossRef]
14. Dhage, B.C. On a fixed point theorem in Banach algebras with aplications. Appl. Math. Lett. 2005, 18, 273–280. [CrossRef]
15. Dhage, B.C. A hybrid fixed point theorem in Banach algebras with applications. Commun. Appl. Nonlinear Anal. 2006, 13, 71–84.
16. Akyüz-Daşcioǧlu, A.; Sezer, M. Chebyshev polynomial solutions of systems of higher-order linear Fredholm-Volterra integro-
differential equations. J. Frankl. Inst. 2005, 342, 688–701. [CrossRef]
17. Argyros, I.K.; Ezquerro, J.A.; Hernández, M.A.; Hilout, S.; Romero, N.; Velasco, A.I. Expanding the applicability of secant like
methods for solving nonlinear equations. Carphatian J. Math. 2015, 31, 11–30. [CrossRef]
134
Mathematics 2022, 10, 4179
18. Berenguer, M.I.; Gámez, D.; López Linares, A.J. Fixed point techniques and Schauder bases to approximate the solution of the
first order nonlinear mixed Fredholm-Volterra integro-differential equation. J. Comput. Appl. Math. 2013, 252, 52–61. [CrossRef]
19. Berenguer, M.I.; Gámez, D. A computational method for solving a class of two dimensional Volterra integral equations. J. Comput.
Appl. Math. 2017, 318, 403–410. [CrossRef]
20. Dzhumabaev, D.S. On one approach to solve the linear boundary value problems for Fredholm integro differential equations.
J. Comput. Appl. Math. 2016, 294, 342–357. [CrossRef]
21. Heydari, M.H.; Hooshmandasl, M.R.; Mohammadi, F.; Cattani, C. Wavelets method for solving systems of nonlinear singular
fractional Volterra integro-differential equations. Commun. Nonlinear Sci. 2014, 19, 37–48. [CrossRef]
22. Maleknejad, K.; Basirat, B.; Hashemizadeh, E. A Berstein operational matrix approach for solving a system of high order linear
Volterra-Fredholm integro-differential equations. Math. Comput. Model. 2012, 55, 1363–1372. [CrossRef]
23. Micula, S. On some iterative numerical methods for a Volterra functional integral equation of the second kind. J. Fixed Point
Theory Appl. 2017, 19, 1815–1824. [CrossRef]
24. Saberi-Nadjafi, J.; Tamamgar, M. The variational iteration method: A highly promising method for solving the system of
integro-differential equations. Comput. Math. Appl. 2008, 56, 346–351. [CrossRef]
25. Boyd, D.W.; Wong, J.S.W. On nonlinear contractions. Proc. Am. Math. Soc. 1969, 20, 458–464. [CrossRef]
26. Pata, V. A fixed point theorem in metric spaces. J. Fixed Point Theory Appl. 2011, 10, 299–305. [CrossRef]
27. Brezis, H. Functional Analysis, Sobolev Spaces and Partial Differential Equations; Universitext; Springer: New York, NY, USA, 2011;
ISBN 978-0-387-70913-0.
28. Gelbaum, B.R.; Gil de Lamadrid, J. Bases of tensor products of Banach spaces. Pac. J. Math. 1961, 11, 1281–1286. [CrossRef]
29. Semadeni, Z. Product Schauder bases and approximation with nodes in spaces of continuous functions. Bull. Acad. Polon. Sci.
1963, 11, 387–391.
135
mathematics
Article
Finding an Efficient Computational Solution for the Bates
Partial Integro-Differential Equation Utilizing the
RBF-FD Scheme
Gholamreza Farahmand 1 , Taher Lotfi 1, *, Malik Zaka Ullah 2 and Stanford Shateyi 3, *
Abstract: This paper proposes a computational solver via the localized radial basis function finite
difference (RBF-FD) scheme and the use of graded meshes for solving the time-dependent Bates
partial integro-differential equation (PIDE) arising in computational finance. In order to avoid facing
a large system of discretization systems, we employ graded meshes along both of the spatial variables,
which results in constructing a set of ordinary differential equations (ODEs) of lower sizes. Moreover,
an explicit time integrator is used because it can bypass the need to solve the large discretized linear
systems in each time level. The stability of the numerical method is discussed in detail based on the
eigenvalues of the system matrix. Finally, numerical tests revealed the accuracy and reliability of the
presented solver.
∂u( x, y, τ ) 1 ∂2 u( x, y, τ ) 1 2 ∂2 u( x, y, τ )
= yx2 + σ y
∂τ 2 ∂x2 2 ∂y2
∂2 u( x, y, τ )
+ ρσyx
∂x∂y (2)
∂u( x, y, τ ) ∂u( x, y, τ )
+ (−λξ − q + r ) x + κ (θ − y)
∂x ∂y
∞
− (λ + r )u( x, y, τ ) + λ u( x, y, τ )b()d = Au( x, y, τ ),
0
Au( x, y, τ ) = A D u( x, y, τ ) + λA I u( x, y, τ ), (3)
in which A D and A I stand for the differential and integral1 portions of2 the PIDE prob-
(ln()−γ)2
lem. The probability density function is b() = √ 1 exp − 2σ̂2 , where it reads
∞ 2π σ̂
u( x, y, 0) = (0, x − K )+ , (4)
wherein K is the strike price. The payoff for a put option could be written similarly. The
point is that the initial condition is written only on x and does not rely on the second
independent variable of the PIDE, i.e., y.
The side conditions for x and y could be given as follows [12]:
u( x, y, τ ) 0, x → 0, (5)
u( x, y, τ ) xmax exp (−qτ ) − K exp (−rτ ), x → xmax , (6)
∂u( x, y, τ )
0, y → +ymax . (7)
∂y
Note that for the case when y = 0, the PIDE (2) is degenerate and no boundaries should
be incorporated while xmax and ymax are large constants. Similarly for the put option, the
boundary conditions are described by the following:
138
Mathematics 2023, 11, 1123
The Bates PIDE (2) is given on ( x, y, τ ) ∈ [0, +∞) × [0, +∞) ×(0, T ]. To solve our high-
dimensional linear PDE, we must truncate the unbounded domain while quite delicately
ignoring the error caused by imposing the boundary conditions. This can be pursued
as follows:
Ω = [0, xmax ] × [0, ymax ], (11)
wherein xmax , ymax are fixed values. The values for xmax , and ymax should be considered
large enough to be able to neglect the effect of imposing artificial boundary conditions
or imposing the boundary conditions for truncated domains. Some choices are Ω =
[0, 4K ] × [0, 1] or Ω = [0, 3K ] × [0, 1].
Assume that { xi }im=1 is a mesh of nodes for x. The hyperbolic stretching of nodes [13]
can be expressed as follows (1 ≤ i ≤ m):
xi = c sinh( β i ) + K, (12)
wherein c > 0 stands for a fixed value that controls the density around x = K and m 3.
In implementations, one can employ c as in [14], i.e., c = K/5. This puts a focus around
the strike price, in which the initial condition of the PIDE has nonsmoothness. Moreover,
{ β i }im=0 stands for the uniform points given by the following:
K
β i = (i − 1)Δβ + sinh−1 − , 1 ≤ i ≤ m, (13)
c
wherein Δβ = (m − 1)−1 sinh−1 S−c K − sinh−1 −cK .
Also, if {y j }nj=1 is a partition for y, then this stretching strategy can be expressed by
the following:
y j = sinh(ς j )ν, 1 ≤ j ≤ n, (14)
wherein ν > 0 is a fixed value that controls the density around y = 0 and n 3. Basically,
we use ν = K/500 [14]. Additionally, the ς j are equidistant nodes provided by ς j =
−1 K
(Δς)( j − 1), and for any 1 ≤ j ≤ n we have Δς = n− 1
1 sinh ν .
Numerical solution methods generally utilize the discretization means to realize the
approximate calculation. When the computational domain/interval is partitioned more
finely, the calculated result is closer to the theoretical solution. Indeed, the time required for
the calculation increases. For high-dimensional PIDE problems with kink behavior at the
initial conditions, sometimes special solvers such as high-order sparse numerical methods
are necessary, see, e.g., [15]. Noting finite difference (FD) methods are discussed in [16,17].
In this paper, the main aim is to propose a novel computational method for resolving
(2) via the radial basis function generated finite difference (RBF-FD) methodology [18].
This is mostly because (2) is a (1+2)D problem with variable coefficients, in which there
is one cross derivatives. Hence, the computational solvers should be constructed for this
aim with much attention. In fact, the motivation of this work lies in the fact that literature
lacks the application of efficient RBF-FD methodology which result in fast and sparse
procedures for solving the Bates PIDE model. Hence, such an application and investigation
on the theoretical stability issues will help price option under stochastic volatility in equity
markets.
The RBF-FD formulations in this paper, see, e.g., [19], are written so they can be
applied to graded meshes in which there is a clear concentration on the hot zone. The
procedure taken here is to employ tensor grids and then time discretize the semi-discretized
constructed problem. We note that the present work is related to the pioneering works
in [20–22]. Meanwhile, these works motivate us to propose a new variant of the RBF-FD
scheme for the Bates PIDE problem that competes with these efficient works.
In this paper, after reviewing the well-resulted maps for generating graded meshes
along spatial variables with a clear focus around the hot area, the rest of this article is
unfolded as follows. The RBF-FD formulas associated with the GMQ RBF are given in
139
Mathematics 2023, 11, 1123
where the underlined x shows a vector quantity in the dimension d and 1 ≤ k ≤ m for some
set of test functions Λ( x ). It is noted that the extension of RBF-FD methodology for solving
computational finance models was revived by the works of Soleymani and co-authors, see
for instance [23,24].
Now, we consider the famous generalized multiquadric RBF (GMQ RBF) as follows
([25] Chapter 4):
ψ
L[Λ(y j )] ∑ α i Λ ( y i ), j = 1, 2, . . . , ψ. (17)
i =1
This gives us ψ unknowns for ψ equations while the solutions will be αi . For computing
the 1st derivative, three graded nodes are considered (ψ = 3) as comes next: {yi − h, yi , yi +
wh}, w > 0, h > 0, and find (17) as follows:
g ( y i ) α i −1 g ( y i −1 ) + α i g ( y i ) + α i +1 g ( y i +1 ). (18)
Noting that we assume that the function g is smooth sufficiently. In estimating the 1st
derivative of a function, the analytical weighting coefficients associated to this RBF can be
given as follows [22]:
ω p2 (9 − 6l ) − h2 (l − 1)(4(l − 5)ω − 10l + 29)
α i −1 = , (19)
3p2 h(2l − 3)(ω + 1)
2
(ω − 1) p (6l − 9) + 4h2 (l − 5)(l − 1)ω
αi = , (20)
3p2 h(2l − 3)ω
p2 (6l − 9) − h2 (l − 1)ω (2l (5ω − 2) − 29ω + 20)
α i +1 = . (21)
3p2 h(2l − 3)ω (ω + 1)
140
Mathematics 2023, 11, 1123
i +1
g (yi ) ∑ Θ j g ( y j ), (22)
j = i −1
Also noting that the given RBF-FD formulations are valid for the interior nodes and
at boundary points, similar formulations must be constructed. We give the derivation for
the independent variable y and it would be similar for the other cases. The formulations
(19)–(21) and (23)–(24) are useful for the rows two to the row before the last one, while for
the 1st and the last rows of the derivative matrices (30) and (31), the weighting coefficients
could not be valid on boundaries and sided estimations should be incorporated. Hence, by
the work [21] on the stencil {y1 , y2 , y3 }, we have:
g ( y1 ) = g [ y2 , y1 ] − g [ y3 , y2 ] + g [ y3 , y1 ] + O ( y2 − y1 )2 , (26)
and
g ( y m ) = − g [ y m −1 , y m −2 ] + g [ y m −2 , y m ] + g [ y m , y m −1 ] + O ( y m −1 − y m )2 , (27)
wherein g[l, p] = ( g(l ) − g( p))/(l − p). In a similar manner, for the four nodes {{y1 , g(y1 )},
{y2 , g(y2 )}, {y3 , g(y3 )}, {y4 , g(y4 )}}, we can obtain
where δyl,q = yl − yq , h is the maximum space width for the considered stencil nodes.
Similarly, we have:
141
Mathematics 2023, 11, 1123
1st and 2nd derivatives of the function in approximating the PDE problem (2) via semi-
discretization are considered on non-uniform stencils given in Section 2 as comes next:
⎧
⎪
⎪ αi,j using (19) i − j = 1,
⎨
αi,j using (20) i − j = 0,
Mx = (30)
⎪ αi,j using (21)
⎪
⎩
j − i = 1,
0 otherwise,
and ⎧
⎪
⎪ Θi,j using (23) i − j = 1,
⎨
Θi,j using (24) i − j = 0,
Mxx = (31)
⎪ Θi,j using (25)
⎪ j − i = 1,
⎩
0 otherwise.
Consider the N × N unit matrix I = Ix ⊗ Iy , while N = m × n, Ix and Iy are unit
matrices of appropriate sizes. The MOL can be resulted in the following coefficient matrix
for the 1 + 2 dimensional PIDE:
1 1
B = Y X 2 ( Mxx ⊗ In ) + σ2 Y ( Im ⊗ Myy ) + ρσY X Mx,y
2 2 (32)
+ (−λξ − q + r )X ( Mx ⊗ In ) + κ (θ IN − Y )( Im ⊗ My ) − (−λ + r ) IN ,
where ⊗ stands for the Kronecker product. The square matrices Mx , My , Mz , Mxx , and
Myy , are constructed by the associated weights similarly. Additionally, the sparse diagonal
matrices Y and X are written as:
Y = Ix ⊗ diag(y1 , y2 , · · · , yn ), (33)
X = diag( x1 , x2 , · · · , xm ) ⊗ Iy . (34)
Here the weights corresponding the cross derivative term in the structure of the PIDE (2)
can be obtained by employing the Kronecker product as follows:
Mx,y = Mx ⊗ My . (35)
Now it is possible to find the following system of ODEs for pricing (2):
u (τ ) = Bu(τ ). (36)
Now, note that we can use the work of [22,29] to discretize the integral part as follows. By a
linear interpolation for u( x, y, τ ) among the adaptive numerical grid nodes, the nonlocal
integral given in (2) can be solved using
∞
A I (u) = u( x, y, τ )b()d. (37)
0
Employing z = x, one can transform (37) into the integral below:
∞ z 1
A I (u) = u(z, y, τ )b dz. (38)
0 x x
m −1
Ai ( u ) ∑ Qi,l , (39)
l =1
142
Mathematics 2023, 11, 1123
⎛ ⎛ ⎛ ⎞
− ln xxli + γ + σ̂2
1 ⎝ σ̂2 ⎝ ⎝ ⎠
Qi,l = exp γ + −erf √
2Δxl 2 2σ̂
⎛ ⎞⎞ ⎛ ⎛ ⎞ (41)
x
− ln lx+i 1 + γ + σ̂2 γ − ln xxl
+erf⎝ √ ⎠⎠ xi (u( xl , y, τ ) − u( xl +1 , y, τ )) + ⎝erf⎝ √ ⎠
i
2σ̂ 2σ̂
⎛ ⎞⎞ ⎞⎞
x l +1
γ − ln xi
−erf⎝ √ ⎠⎠( xl +1 u( xl , y, τ ) − xl u( xl +1 , y, τ ))⎠⎠,
2σ̂
Theorem 1. Let us assume that (43) satisfies the Lipschitz condition, then we have a conditional
time-stable iteration process using (44) for solving (43).
143
Mathematics 2023, 11, 1123
Proof. Considering the time-stepping method (44) on the set of ODEs (43) gives:
(ζ B̄)0 (ζ B̄)1 (ζ B̄)2
u ι +1 = + + uι . (46)
1 1 2
have modulus less than or equal to one. Viewing (46) as an iterative map, it would be clear
(ζ B̄ )2
that the eigenvalues of this matrix are 1 + ζ B̄i + 2i , where B̄i are the eigenvalues of
matrix B̄. Thus, for i = 1, 2, . . . , m, the A-stability is simplified to
2
1 + ζ B̄i + (ζ B̄i ) ≤ 1. (48)
2
Therefore, our proposed method is time-stable if the time step size ζ reads as (48). The
stability function in (48) shows a conditional stable behavior for (44). Using (48) along with
ζ > 0 we have the following:
2
0<ζ≤ , (49)
Re(λmax ( B̄))
where Re(·) is the real part and λmax (·) is the largest eigenvalue (in the absolute value
sense). Note that we also obtain
while 1/2
1/2
Re( B̄i )(ζRe( B̄i ) + 2) Re( B̄i )(ζRe( B̄i ) + 2)
ξi = 2 − − . (51)
ζ3 ζ
These inequalities on the real and the imaginary parts of the eigenvalues will determine
the conditional time stability bounds of the proposed solver when pricing (2). This ends
the proof.
To discuss about the advantages of the proposed approach, we briefly express that our
solver has now been expressed all in matrix notations as in (43) which is a system of linear
ODEs. When it couples by the ODE solver (44) with the stability condition (50), it solves (2)
and the stability relied only on the largest eigenvalue of the system matrix.
5. Numerical Aspects
The goal here is to resolve (2) for at-the-money options, i.e., the value of u at x0 = K
and also y0 = 0.04 and K = 100$. The comparing methods are given below:
• The 2nd-order FD scheme with equidistant stencils for space and the explicit 1st order
Euler’s scheme denoted by FDM,
• The method of scalable algebraic multigrid discussed in [34] and shown by AFFT.
• The scheme recently proposed by Soleymani et al. in [21] based on efficient non-
uniform procedure denoted by SM.
• The presented solver in Sections 2–4 shown via RBF-FD-PM in this section.
Noting that all the programs have been written carefully under similar conditions in
Mathematica 13 [35,36]. Here, the whole CPU time (for constructing the meshes, the
derivative matrices, the set of ODEs and employing the time-stepping method) is in second.
144
Mathematics 2023, 11, 1123
We use more number of nodes along x than y, since its computational domain is larger. The
criterion given below is used for computing the errors
ε i,j,ι = uapprox ( xi , y j , τι ) − uref ( x, y, τ ), (52)
where uapprox and uref are the approximate and exact solutions. uref is selected from the
already well-known literature [14,34].
It is remarked that one efficient way to compute the shape parameter is to calculate it
adaptively via the number of discretization points, the numerical domain as well as the
structure of the PIDE problem. Hence, here we use (1 ≤ i ≤ m − 1):
p = 4 max{Δxi }, (53)
where Δxi are the increments along the variable mesh. We can write and use (53) similarly
for the other variable. Throughout the tables of this paper, a E-b stands for the scientific
notation a × 10−b .
Example 1 ([14]). Let us investigate the computational results for the call option of (2) using the
following settings:
The reference price, which is obtained by the FFT approach [14], is 8.894869 at the
point ( x0 , y0 ) = (100, 0.04). The numerical truncated domain is Ω = [0, 3K ] × [0, 1] and
ψ = 1.5. Economically speaking, the values for the variance (for domain truncating) that
are larger than one are not significant. The results in this case are provided in Table 1, which
shows the superiority of the proposed solver RBF-FD-PM.
Example 2 ([34]). Let us investigate the computational results of a European put option for (2)
using the following settings:
145
Mathematics 2023, 11, 1123
Table 1. Cont.
The reference prices for specific locations of the domain are 11.302917 at (90, 0.04, 0.5),
6.589881 at (100, 0.04, 0.5) and 4.191455 at (110, 0.04, 0.5) using [34]. The convergence results
are provided in Tables 2 and 3 and confirm the superiority of the proposed solver with
ψ = 2 in this paper.
The FDM solver is back-of-the-envelope accounting because it is clear that the uniform
grids for the PIDE problem are not a fast calculation to obtain highly accurate prices. To
check the stability and positivity of the numerical solution for RBF-FD-PM, the numerical
solution for Example 2 is plotted in Figure 1, which shows the stable behavior of RBF-FD-
PM using m = 16, n = 8 and k = 1001.
Figure 1. Numerical solution of Example 2 using the RBF-FD-PM solver when τ = 0 on the left and
τ = 0.5 on the right. Green points show the location of the graded discretization points on the red
curve, which is the numerical solution.
The reason for providing Figure 1 is twofold. We must first reveal that the numerical
solution obtained by RBF-FD-PM using some m and n is stable and does not have oscil-
lations. This is important since the PIDE model has a mixed derivative term, which can
lead to oscillations in the numerical solution as long as a careless numerical method is
employed. Second, we must reveal how the graded meshes (the green points in Figure 1)
located on the numerical solution are obtained by employing an automatic interpolation
on the obtained solutions.
An inquiry might arise by analyzing the results in Tables 1 and 2. tt is not easy to
find out the advantages of the proposed approach since the numerical values are given for
different values of the parameters m, n, N and k + 1. In fact, larger time step sizes (lower
k) are taken for SM and RBF-FD-PM since their ODE solver, i.e., (44), has a larger stability
146
Mathematics 2023, 11, 1123
region, and the overall solvers must be compared by fixing an accuracy for the errors and
then checking the computational times.
To also show how the instability may ruin the numerical pricing using the stability
bound (49), we provide the numerical results of solving (2) by the RBF-FD-PM using m = 16
and n = 8, but with k = 25 uniform discretization nodes along time in Figure 2. This shows
that all the involved solvers have some limitations, but the proposed solver sounds more
efficient than others.
However, due to nonsmoothness at the strike price in the initial condition, it might be
useful to employ a time-stepping solver that works on graded meshes over time with more
focus at the beginning of the starting time, i.e., zero (the solution near the initial time point
has a weak singularity). One such method is the Rannacher time-marching method [37].
Although such an application will help our solver a lot, we will try to focus on this in
forthcoming related works.
147
Mathematics 2023, 11, 1123
6. Concluding Remarks
PIDEs arise in the mathematical modeling of many processes in different fields of
engineering and finance. This paper has presented an approximate solution of the linear
Bates PIDE with clear application in financial option pricing using a local integral term. The
solution method was considered on graded meshes at which there is a clear concentration of
the discretization nodes on the financially important are of the problem. Then, an RBF-FD
solver using semi-discretization via sparse arrays have been constructed for solving the
Bates PIDE. The numerical results were furnished and supported the theoretical discussions.
These results have been provided in Tables 1 and 2 which implicitly state that the proposed
approach can compete the most efficient solver (SM) for the same purpose. Additionally, the
prospects for future research can be focused on how to obtain RBF-FD weights on stencils
having five/six adjacent nodes on graded meshes or employing the Rannacher time-
marching method in order to obtain higher accuracies for solving the PIDE problem (2).
Author Contributions: Conceptualization, T.L. and G.F.; formal analysis, G.F.; T.L. and S.S.; funding
acquisition, M.Z.U. and T.L.; investigation, T.L., M.Z.U. and S.S.; methodology, G.F. and S.S.; supervi-
sion, T.L.; validation, M.Z.U. and S.S.; writing—original draft, G.F., M.Z.U. and S.S.; writing—review
and editing, T.L., M.Z.U. and S.S. All authors have read and agreed to the published version of the
manuscript.
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: For the data-availability statement, we state that data sharing is not
applicable to this article as no new data were created in this study.
Acknowledgments: The third author states that: The Deanship of Scientific Research (DSR) at
King Abdulaziz University (KAU), Jeddah, Saudi Arabia, has funded this project, under grant no.
(KEP-MSc: 65-130-1443). The authors are very much thankful to three anonymous referees for their
suggestions, which helped to improve this paper.
Conflicts of Interest: The authors declare that they have no known competing financial interests or
personal relationships that could have appeared to influence the work reported in this paper.
References
1. Bates, D. Jumps and stochastic volatility: The exchange rate processes implicit in Deutsche mark options. Rev. Fin. Stud. 1996, 9,
69–107. [CrossRef]
2. Itkin, A. Pricing Derivatives Under Lévy Models: Modern Finite-Difference and Pseudo–Differential Operators Approach, Birkhäuser Basel;
Springer: Berlin/Heidelberg, Germany, 2017.
3. Kim, K.-H.; Sin, M.-G. Efficient hedging in general Black-Scholes model. Finan. Math. Appl. 2014, 3, 1–9.
4. Ghanadian, A.; Lotfi, T. Approximate solution of nonlinear Black-Scholes equation via a fully discretized fourth-order method.
Aims Math. 2020, 5, 879–893. [CrossRef]
5. Heston, S. A closed-form solution for options with stochastic volatility with applications to bond and currency options. Rev.
Finan. Stud. 1993, 6, 327–343. [CrossRef]
6. Chang, Y.; Wang, Y.; Zhang, S. Option pricing under double Heston jump-diffusion model with approximative fractional stochastic
volatility. Mathematics 2021, 9, 126. [CrossRef]
7. Gómez-Valle, L.; Martínez-Rodríguez, J. Including jumps in the stochastic valuation of freight derivatives. Mathematics 2021, 9,
154. [CrossRef]
8. Liu, J.; Yan, J. Convergence rate of the high-order finite difference method for option pricing in a Markov regime-switching
jump-diffusion model. Fractal Fract. 2022, 6, 409. [CrossRef]
148
Mathematics 2023, 11, 1123
9. Chen, K.-S.; Huang, Y.-C. Detecting jump risk and jump-diffusion model for Bitcoin options pricing and hedging. Mathematics
2021, 9, 2567. [CrossRef]
10. Hellmuth, K.; Klingenberg, C. Computing Black Scholes with uncertain volatility-a machine learning approach. Mathematics 2022,
10, 489. [CrossRef]
11. Ballestra, L.; Sgarra, C. The evaluation of American options in a stochastic volatility model with jumps: An efficient finite element
approach. Comput. Math. Appl. 2010, 60, 1571–1590. [CrossRef]
12. Ballestra, L.; Cecere, L. A fast numerical method to price American options under the Bates model. Comput. Math. Appl. 2016, 72,
1305–1319. [CrossRef]
13. Kluge, T. Pricing Derivatives in Stochastic Volatility Models Using the Finite Difference Method. Ph.D. Thesis, TU Chemnitz,
Chemnitz, Germany, 2002.
14. in ’t Hout, K.J.; Foulon, S. ADI finite difference schemes for option pricing in the Heston model with correlation. Int. J. Numer.
Anal. Model. 2010, 7, 303–320.
15. Balajewicz, M.; Toivanen, J. Reduced order models for pricing European and American optionsunder stochastic volatility and
jump-diffusion models. J. Comput. Sci. 2017, 20, 198–204. [CrossRef]
16. Duffy, D. Finite Difference Methods in Financial Engineering: A Partial Differential Equation Approach; Wiley: England, UK, 2006.
17. Düring, B.; Fournié, M. High-order compact finite difference scheme for option pricing in stochastic volatility models. J. Comput.
Appl. Math. 2012, 236, 4462–4473. [CrossRef]
18. Soleymani, F.; Ullah, M. A multiquadric RBF-FD scheme for simulating the financial HHW equation utilizing exponential
integrator. Calcolo 2018, 55, 51. [CrossRef]
19. Milovanović, S.; von Sydow, L. A high order method for pricing of financial derivatives using radial basis function generated
finite differences. Math. Comput. Simul. 2020, 174, 205–217. [CrossRef]
20. Milovanović, S.; von Sydow, L. Radial basis function generated finite differences for option pricing problems. Comput. Math. Appl.
2018, 75, 1462–1481. [CrossRef]
21. Soleymani, F.; Barfeie, M. Pricing options under stochastic volatility jump model: Astable adaptive scheme. Appl. Numer. Math.
2019, 145, 69–89. [CrossRef]
22. Soleymani, F.; Zhu, S. RBF-FD solution for a financial partial-integro differential equation utilizing the generalized multiquadric
function. Comput. Math. Appl. 2021, 82, 161–178. [CrossRef]
23. Soleymani, F.; Itkin, A. Pricing foreign exchange options under stochastic volatility and interest rates using an RBF-FD method. J.
Comput. Sci. 2019, 37, 101028. [CrossRef]
24. Itkin, A.; Soleymani, F. Four-factor model of quanto CDS with jumps-at-default and stochastic recovery. J. Comput. Sci. 2021, 54,
101434. [CrossRef]
25. Fasshauer, G. Meshfree Approximation Methods with MATLAB; World Scientific Publishing Co.: Singapore, 2007.
26. Bayona, V.; Moscoso, M.; Carretero, M.; Kindelan, M. RBF-FD formulas and convergence properties. J. Comput. Phys. 2010, 229,
8281–8295. [CrossRef]
27. Meyer, G. The Time-Discrete Method of Lines for Options and Bonds, A PDE Approach; World Scientific Publishing: Hackensack, NJ,
USA, 2015.
28. Knapp, R. A method of lines framework in Mathematica. J. Numer. Anal. Indust. Appl. Math. 2008, 3, 43–59.
29. Salmi, S.; Toivanen, J. An iterative method for pricing American options under jump-diffusion models. Appl. Numer. Math. 2011,
61, 821–831. [CrossRef]
30. Michel, V.; Thomann, D. TVD-MOOD schemes based on implicit-explicit time integration. Appl. Math. Comput. 2022, 433, 127397.
31. Shymanskyi, V.; Protsyk, Y. Simulation of the heat conduction process in the Claydite-Block construction with taking into account
the fractal structure of the material. In Proceedings of the 2018 IEEE 13th International Scientific and Technical Conference on
Computer Sciences and Information Technologies (CSIT), Lviv, Ukraine, 11–14 September 2018.
32. Sayfidinov, O.; Bognár, G.; Kovács, E. Solution of the 1D KPZ equation by explicit methods. Symmetry 2022, 14, 699. [CrossRef]
33. Butcher, J. Numerical Methods for Ordinary Differential Equations, 2nd ed.; Wiley: England, UK, 2008.
34. Salmi, S.; Toivanen, J.; von Sydow, L. An IMEX-scheme for pricing options under stochastic volatility models with jumps. SIAM J.
Sci. Comput. 2014, 36, 817–834. [CrossRef]
35. Mangano, S. Mathematica Cookbook; O’Reilly Media: Newton, MA, USA, 2010.
36. Wellin, P.; Gaylord, R.; Kamin, S. An Introduction to Programming with Mathematica; Cambridge University Press: Cambridge,
UK, 2005.
37. Giles, M.; Carter, R. Convergence analysis of Crank-Nicolson and Rannacher time-marching. J. Comput. Financ. 2006, 9, 89–112.
[CrossRef]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual
author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to
people or property resulting from any ideas, methods, instructions or products referred to in the content.
149
mathematics
Article
Hybrid Newton–Sperm Swarm Optimization Algorithm for
Nonlinear Systems
Obadah Said Solaiman 1 , Rami Sihwail 2, *, Hisham Shehadeh 3 , Ishak Hashim 1,4 and Kamal Alieyan 2
1 Department of Mathematical Sciences, Faculty of Science & Technology, Universiti Kebangsaan Malaysia,
Bangi 43600, Selangor, Malaysia; [email protected] (O.S.S.); [email protected] (I.H.)
2 Department of Cyber Security, Faculty of Computer Science & Informatics, Amman Arab University,
Amman 11953, Jordan; [email protected]
3 Department of Computer Information System, Faculty of Computer Science & Informatics, Amman Arab
University, Amman 11953, Jordan; [email protected]
4 Nonlinear Dynamics Research Center (NDRC), Ajman University, Ajman P.O. Box 346, United Arab Emirates
* Correspondence: [email protected]; Tel.: +962-7-8814-2270
Abstract: Several problems have been solved by nonlinear equation systems (NESs), including
real-life issues in chemistry and neurophysiology. However, the accuracy of solutions is highly
dependent on the efficiency of the algorithm used. In this paper, a Modified Sperm Swarm Optimiza-
tion Algorithm called MSSO is introduced to solve NESs. MSSO combines Newton’s second-order
iterative method with the Sperm Swarm Optimization Algorithm (SSO). Through this combination,
MSSO’s search mechanism is improved, its convergence rate is accelerated, local optima are avoided,
and more accurate solutions are provided. The method overcomes several drawbacks of Newton’s
method, such as the initial points’ selection, falling into the trap of local optima, and divergence.
In this study, MSSO was evaluated using eight NES benchmarks that are commonly used in the
literature, three of which are from real-life applications. Furthermore, MSSO was compared with
several well-known optimization algorithms, including the original SSO, Harris Hawk Optimization
(HHO), Butterfly Optimization Algorithm (BOA), Ant Lion Optimizer (ALO), Particle Swarm Opti-
mization (PSO), and Equilibrium Optimization (EO). According to the results, MSSO outperformed
Citation: Said Solaiman, O.; Sihwail, the compared algorithms across all selected benchmark systems in four aspects: stability, fitness
R.; Shehadeh, H.; Hashim, I.; Alieyan, values, best solutions, and convergence speed.
K. Hybrid Newton–Sperm Swarm
Optimization Algorithm for
Keywords: nonlinear systems; Newton’s method; iterative methods; sperm swarm optimization
Nonlinear Systems. Mathematics 2023,
algorithm; optimization algorithm
11, 1473. https://doi.org/10.3390/
math11061473
MSC: 65D99; 65H10; 65K10
Academic Editors: Maria
Isabel Berenguer and Manuel Ruiz
Galán
1. Introduction
Received: 6 February 2023
Revised: 11 March 2023 Many issues in the natural and applied sciences are represented by systems of non-
Accepted: 14 March 2023 linear equations F ( X ) = 0 that require solving, where F ( X ) = ( f 1 , f 2 , . . . , f n ) such that f i
Published: 17 March 2023 is nonlinear for all i = 1, 2, . . . , n. It is well known that determining the precise solution
α = (α1 , α2 , . . . , αn )t to the nonlinear system F ( X ) = 0 is a difficult undertaking, especially
when the equation comprises terms made up of logarithmic, exponential, trigonometric,
or a mix of any transcendental terms. Thus, finding approximate solutions to this type of
Copyright: © 2023 by the authors.
problem has emerged as a need. The iterative methods, including Newton’s method, are
Licensee MDPI, Basel, Switzerland.
some of the most famous methods for finding approximate solutions to nonlinear equation
This article is an open access article
systems (NESs) [1]. Alternatively, optimization algorithms have been applied in attempts
distributed under the terms and
to extract the root solution of nonlinear systems.
conditions of the Creative Commons
In the last ten years, various optimization algorithms have been developed. Those
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
methods can be divided into four primary categories: human-based methods, swarm-based
4.0/).
152
Mathematics 2023, 11, 1473
than attempting to examine NESs. In a relatively recent study, Sihwail et al. [40] developed
a hybrid algorithm known as NHHO to solve arbitrary NESs of equations that combine
Harris Hawks’ optimization method and Newton’s method. Very recently, Sihwail et al. [41]
proposed a new algorithm for solving NESs of equations in which Jarratt’s iterative ap-
proach and the Butterfly optimization algorithm were combined to create the new scheme
known as JBOA.
A hybrid algorithm can leverage the benefits of one method while overcoming the
drawbacks of the other. However, most hybrid methods face problems with premature
convergence due to the technique used in the original algorithms [42]. As a result, choosing
a dependable combination of algorithms to produce an efficient hybrid algorithm is a
crucial step.
One of the more recent swarm-based methods is Sperm Swarm Optimization (SSO),
which is based on the mobility of flocks of sperm to fertilize an ovum. There are various
benefits of SSO, which can be listed as follows [2,5,6]:
• The capability of exploitation of SSO is very robust.
• Several kinds of research have validated its simplicity, efficiency, and ability to con-
verge to the optimal solution.
• Its theory can be applied to a wide range of problems in the areas of engineering
and science.
• Its mathematical formulation is easy to implement, understand, and utilize.
However, most NESs simulate different data science and engineering problems that
have more than one solution. Hence, it is difficult to give accurate solutions to these prob-
lems. Like other optimization algorithms, SSO may fall into a local minimum (solution)
instead of the optimal solution. As a result, we developed a hybrid approach that incor-
porates Newton’s iterative scheme with the SSO algorithm to mitigate the drawback. It is
worth mentioning that Newton’s method is the first known iterative scheme for solving
nonlinear equations using the successive approximation technique. According to Newton’s
method, the correct digits nearly double each time a step is performed, referred to as the
second order of convergence.
Newton’s method is highly dependent on choosing the correct initial point. To achieve
good convergence toward the root, the starting point, like other iterative approaches, must
be close enough to the root. The scheme may converge slowly or diverge if the initial point
is incorrect. Consequently, Newton’s method can only perform limited local searches in
some cases.
For the reasons outlined above, a hybrid SSO algorithm (MSSO) has been proposed to
solve NESs, where Newton’s method is applied to improve the search technique and SSO
is used to enhance the selection of initial solutions and make global search more efficient.
It is not the concern of this study to demonstrate that hybridizing the SSO and New-
ton’s methods performs better than other optimization algorithms such as PSO or genetic
algorithms. However, this work aims to highlight the benefits of hybridizing an opti-
mization algorithm with an iterative method. This is to enhance the iterative method’s
accuracy in solving nonlinear systems and reduce its complexity. Further, it is also able to
overcome several drawbacks of Newton’s method, such as initial point selection, trapping
in local optima, and divergence problems. Moreover, hybridization in MSSO is beneficial
in finding better roots for the selected NSEs. Optimization algorithms alone are unlikely to
provide precise solutions compared to iterative methods such as Newton’s method and
Jarratt’s method.
The proposed modification improves the initial solution distribution in the search
space domain. Moreover, compared to the random distribution used by the original
technique, Newton’s approach improves the computational accuracy of SSO and accelerates
its convergence rate. Hence, this research paper aims to improve the accuracy of NES
solutions. The following are the main contributions of this paper:
153
Mathematics 2023, 11, 1473
2. Background
2.1. Standard Sperm Swarm Optimization (SSO) Algorithm
SSO is a newly created swarm-based technique proposed by Shehadeh et al. [2,5,6]
that draws inspiration from the actions of a group of sperm as they fertilize an ovum.
In the process of fertilization, a single sperm navigates a path against overwhelming
odds to merge with an egg (ova). In general, there are 130 million sperm involved in the
insemination process. Eventually, one of these sperm will fertilize the ovum. Based on
Shehadeh et al. [6], the procedure of fertilization can be summarized as follows:
A male’s reproductive system releases the sperm into the cervix, where the fertilization
process starts. Each sperm is given a random location inside the cervix to begin the
fertilization process as part of this task. Further, every sperm has two velocities on the
Cartesian plane. The initial velocity value of sperm denotes this velocity. The procedure of
fertilization is demonstrated in Figure 1.
From this point, every sperm in the swarm is ready to swim until it reaches the outer
surface of the ovum. Scientists found that the sperm float on the surface as a flock or swarm,
moving from the zone of low temperature to the area of high temperature. Moreover,
they observed that the ovum triggers a chemical to pull the swarm; this is known as a
chemotactic process. According to researchers, these cells also beat at the same frequency
as the tail movements through the grouping. The ovum and its location in the fallopian
154
Mathematics 2023, 11, 1473
tubes are illustrated in Figure 1. Based on Shehadeh et al. [6], this velocity is denoted by
the personal best velocity of the sperm.
Usually, in a typical scenario, one sperm can fertilize an ovum. Based on that,
Shehadeh et al. [2,5–8] calls this sperm the winner. The winner and the flock of sperm
are illustrated in Figure 2.
The best answer is found and obtained using this strategy, which makes use of a
group of sperm (potential solutions) floating over the whole search area. Concurrently, the
possible solutions will consider the most suitable sperm in their path, who will be the victor
(the sperm that is closest to the egg). Alternatively, the flock will consider on the winner’s
position and the position of its prior best solution. Thus, every sperm enhances its initial
zone across the optimum area by taking into consideration its current velocity, current
location, and the location of both the global’s best solution (the winner) and the sperm’s
best solution. Mathematically speaking, in SSO, the flock updated their sites according to
the following formula:
x i +1 ( t ) = x i ( t ) + v i ( t ) (1)
where
• vi is the velocity of potential solution i at iteration t;
• xi is the current position of possible solution i at iteration t;
Three velocities can be used to calculate the sperm’s velocity: the initial velocity of a
potential solution, the personal best solution, and the global best solution.
First is the initial velocity of sperm, which takes a random value based on the velocity
dumping parameter and the pH value of the initial location. The model can be calculated
by applying the following formula:
Second is a personal best location for the potential solution, adjusted in memory based
on the prior location until it is closest to the optimal value. However, this velocity can be
changed based on the pH and temperature values. The following formula may be used to
calculate this model:
Current _Best _Solution = Log10 ( pH_Rand2 ) · Log10 ( Temp_Rand1 ) · xsbesti []− xi [] (3)
Third, the global best solution is simulated by the winner, which is denoted by the
closest sperm to the ovum. The mathematical model of the winning velocity of the potential
solution Vi(t) can be represented in Equation (4). The flock of sperm and the value of the
winner are depicted in Figure 2.
155
Mathematics 2023, 11, 1473
Global_Best_Solution(the_winner ) = Log10 ( pH_Rand3 ) · Log10 ( Temp_Rand2 ) · xsgbesti []− xi [] (4)
Based on the theory of SSO, both pH and temperature affect the velocity rule. The pH
changes depending on the woman’s attitude, whether depressed or happy, and on the food
consumed. The value of the pH parameter falls in a range between seven and fourteen.
Alternatively, the temperature ranges from 35.1 to 38.5 ◦ C according to blood pressure
circulation in the reproductive system [7].
Further, SSO is a swarm-based method that simulates the metaphor of natural fer-
tilization. SSO, however, has a few disadvantages in terms of efficiency. Applied to a
broad search domain, SSO is prone to getting trapped in local optima [2], which is one
of its main drawbacks. Therefore, improvements are needed to enhance the method’s
exploration process.
156
Mathematics 2023, 11, 1473
(NFL)” theorem [48]. By using Newton’s method, the proposed MSSO outperforms the
original SSO in terms of solving nonlinear equation systems. In MSSO, Newton’s methods
are used as a local search to enhance the search process, as shown in Figure 3.
When Newton’s method is applied to the sperm position, at each iteration, the fitness
value of the potential solution is compared to the fitness of the location calculated by
Newton’s scheme. The newly computed location by Newton’s method is shown in Figure 3
as ( Xn+1 ).
In each iteration, MSSO employs both the SSO algorithm and Newton’s method. The
SSO first determines the most optimal sperm location among the twenty initial locations as
an optimal candidate location. The optimal candidate location is then fed into Newton’s
method. In other words, the output from SSO is considered a potential solution or a tem-
porary solution. The obtained solution is then treated as an input for Newton’s method.
Newton’s method as an iterative method calculates the next candidate solution based on
Equation (6). Newton’s method’s ability to find a better candidate is very high since it is a
second-order convergence method. However, in order to avoid a local optimal solution,
the candidate solution obtained from Newton’s method (Xn+1 ) is compared to the solution
calculated by SSO (Xsperm ). Thus, the location with the lowest fitness value determines
the potential solution to the problem. The next iteration is then performed based on the
current most promising solution. Algorithm 1 shows the pseudocode for the suggested
157
Mathematics 2023, 11, 1473
MSSO algorithm.
The initialization, exploitation, and exploration phases of the SSO method are shown
in the algorithm. The alterations specified in the red box are implemented at the end of
each iteration. We compare Newton’s location with the sperm’s optimal location based on
their fitness values and select the one that has the best fitness value.
Computational Complexity
The complexity of the new MSSO’s can be obtained by adding up the SSO’s complexity
and Newton’s method’s complexity. At first glance, Newton’s technique is overly com-
plicated compared to optimization methods. At each iteration, one has to solve a N × N
system of linear models, which is time-consuming because every Jacobian calculation
requires n2 scalar function evaluations. As a result, combining Newton’s approach with
any optimization process is likely to make it more complicated.
On the other hand, combining SSO with Newton’s technique did not significantly
increase processing time. However, the MSSO can overcome Newton’s method limitations,
including selecting the starting points and divergence difficulties. As a result, the MSSO is
superior at solving nonlinear equation systems.
The MSSO’s time complexity is influenced by the initial phase, the process of updat-
ing the position of the sperm, and the use of Newton’s scheme. The complexity of the
initialization process is O(S), where S is the total number of sperm. The updating process,
which includes determining the optimal solution and updating sperm positions, has a
complexity equal to O(I × S) + O(I × S × M), where I and M represent the maximum
number of iterations and the complexity of the tested benchmark equation respectively.
Furthermore, Newton’s scheme complexity is calculated as O(I × T), where T is the compu-
tation time. Consequently, the proposed MSSO has an overall computational complexity of
O(S × (I + IM + 1) + IT).
Every improvement certainly has a cost. The principal objective of the proposed
hybrid algorithm is to enhance the fitness value and the convergence speed of the existing
algorithms. However, as a result of adding one algorithm to another, the complexity and
the time cost of the hybrid algorithm are increased compared to the original algorithm.
Eventually, a tradeoff between the merits and disadvantages should be considered while
using any algorithm.
158
Mathematics 2023, 11, 1473
4. Numerical Tests
Eight nonlinear systems of several orders were selected as indicators to clarify the
efficiency and capability of the new hybrid MSSO scheme. Comparisons between MSSO
and the other six well-known optimization algorithms have been performed. Those op-
timization algorithms are the original SSO [2], HHO [9], PSO [49], ALO [10], BOA [11],
and EO [13]. For consistency, all selected systems used in the comparisons are arbitrary
problems that are common in the literature, for instance, [19,21,40,44,50–53].
The comparison between the optimization algorithms is based on the fitness value of
each algorithm in each benchmark. A solution with less fitness value is more accurate than
a solution with a higher fitness value. Hence, the most effective optimization algorithm is
the one that solves with the least fitness value. The fitness function used in the comparison
is the Euclidean norm, also called the square norm or norm-2. Using the Euclidean norm,
we can determine the distance from the origin, which is expressed as follows:
Fitness = F ( x ) 2 = f 12 + f 22 + . . . + f n2 , (7)
Similar settings have been used in all benchmarks to guarantee a fair comparison of
all selected algorithms. The parameter values of all optimization algorithms have been
fine-tuned to improve the performance of the algorithms. The best solution was chosen
by every optimization method 30 times. Search agents (population size) have been set to
20 and the maximum iteration to 50. Furthermore, the best solution with the least fitness
value is chosen if there is more than one solution for a particular benchmark. In the end,
for lack of space, answers are shortened to 11 decimal places.
Calculations were conducted using MATLAB software version R2020a with the default
variable precision of 16 digits. This was on an Intel Core i5 processor running at 2.2 GHz
and 8 GB of RAM under the Microsoft Windows 8 operating system.
Problem 1: Let us consider the first problem to be the following nonlinear system of
two equations: !
x1 + 1 − ex2 = 0,
F1 ( X ) =
x1 + cos( x2 ) − 2 = 0,
For this system, the precise solution is given by α = {1.3401918575555883401 . . . ,
0.8502329164169513268 . . .}t . After running the algorithms 30 times, MSSO significantly
surpassed all other optimization algorithms in the comparison. Table 1 shows that the
proposed hybrid MSSO algorithm has attained the best solution with the least fitness value
equaling zero. This means that the solution obtained by MSSO is an exact solution for the
given system.
MSSO HHO [9] PSO [49] ALO [10] BOA [11] EO [13] SSO [2]
x1 1.34019185756 1.34020535556 1.34019185727 1.34019196042 1.34359319240 1.34019194567 1.34502836805
x2 0.85023291642 0.85023195766 0.85023291632 0.85023300025 0.85138606082 0.85023289034 0.85355356706
Fitness 0 2.1212 × 10−5 2.2401 × 10−10 1.0147 × 10−7 2.6296 × 10−3 1.8396 × 10−7 3.7618 × 10−3
Problem 2: The second benchmark is the system of two nonlinear equations given by:
!
2 − e x1 + tan−1 x2 = 0,
F2 ( X ) =
tan−1 x12 + x22 − 5 = 0,
Here, the exact zero for the system in this problem is given by α = ( 1.1290650391602 . . . ,
1.9300808629035 . . .)t . As shown in Table 2, it is evident that MSSO achieved the exact
solution of this system with a fitness value of zero. It also outperformed all other algorithms
with a substantial difference, especially in comparison with SSO, BOA, and HHO.
159
Mathematics 2023, 11, 1473
MSSO HHO [9] PSO [49] ALO [10] BOA [11] EO [13] SSO [2]
x1 1.12906503916 1.12903302177 1.12906503916 1.12906515112 1.12588512395 1.12906504185 1.14402014766
x2 1.93008086290 1.93011297982 1.93008086290 1.93008085965 1.93375637741 1.93008086329 1.92067058635
Fitness 0 0.000117763 8.01 × 10−15 4.22 × 10−7 0.012716315 1.1201 × 10−8 0.048651092
MSSO HHO [9] PSO [49] ALO [10] BOA [11] EO [13] SSO [2]
x1 0.90956949452 0.90449212115 0.89176809239 0.85453639710 0.83212389642 0.90775456824 1.03817572093
x2 0.66122683227 0.66642798414 0.67275154835 0.69673611158 0.69808559231 0.66254037960 0.56914672488
x3 1.57583414391 1.57229467736 1.56169705842 1.53258611089 1.52262989677 1.57448413869 1.69602879530
Fitness 0 0.005442108 0.004315295 0.013715754 0.036158224 0.000699083 0.061770954
The precise solution of the nonlinear system in this problem is equal to α = (0.351733711249 . . . ,
0.351733711249 . . . , 0.351733711249 . . .)t . The best solution achieved by the compared
schemes for the given system is illustrated in Table 4. The proposed MSSO found a precise
answer, with zero as a fitness value. ALO recorded the second-best solution with a fitness
value of 2.27 × 10−6 , while the rest of the compared algorithms were far from the exact
answer. Again, the proposed MSSO has proved it has an efficient local search mechanism.
Hence, it can achieve more accurate solutions for nonlinear systems.
MSSO HHO [9] PSO [49] ALO [10] BOA [11] EO [13] SSO [2]
x1 0.35173371125 0.36165321762 0.35083292352 0.35172698088 0.38459199838 0.35086562122 0.37260511330
x2 0.35173371125 0.35137717774 0.35226253114 0.35173655019 0.33171697596 0.35200965295 0.34576550099
x3 0.35173371125 0.34410796587 0.35213140099 0.35173726704 0.34030291514 0.35226146573 0.33588500543
Fitness 0 0.005300022 0.000379475 2.2674 × 10−6 0.016625262 0.000294859 0.010254721
Problem 5: The next benchmark is the following system of two nonlinear equations:
!
x1 + e x2 − cos( x2 ) = 0,
F5 ( X ) =
3x1 − sin( x1 ) − x2 = 0,
This nonlinear system has the trivial solution α = (0, 0)t . Table 5 illustrates the comparison
between the different optimization algorithms for the given system. Compared with the
other algorithms, the original SSO and HHO achieved excellent results, with fitness values
160
Mathematics 2023, 11, 1473
of 5.36 × 10−15 and 6.92 × 10−14 , respectively. However, MSSO outperformed both of them
and delivered the exact solution for the given system.
MSSO HHO [9] PSO [49] ALO [10] BOA [11] EO [13] SSO [2]
x 3.6298689 × 10−22 1.0162783 × 10−14 −2.0631743 × 10−8 −2.0631743 × 10−8 0.00019546601 −1.0265357 × 10−14 −5.8109345 × 10−16
y 7.2597377 × 10−22 -4.1451213 × 10−14 2.4507340 × 10−7 2.4507340 × 10−7 1.1132830 × 10−5 1.0797593 × 10−13 −3.9989603 × 10−15
Fitness 0 6.92 × 10−14 3.64 × 10−7 3.64 × 10−7 4.32 × 10−4 1.61 × 10−13 5.36 × 10−15
Problem 6: The sixth system considered for the comparison is an interval arithmetic
benchmark [53] given by the following system of ten equations:
⎧
⎪
⎪ x1 − 0.25428722 − 0.18324757x4 x3 x9 = 0,
⎪
⎪
⎪
⎪ x2 − 0.37842197 − 0.16275449x1 x10 x6 = 0,
⎪
⎪
⎪
⎪ x3 − 0.27162577 − 0.16955071x1 x2 x10 = 0,
⎪
⎪
⎪
⎪ x4 − 0.19807914 − 0.15585316x7 x1 x6 = 0,
⎪
⎨
x5 − 0.44166728 − 0.19950920x7 x6 x3 = 0,
F6 ( X ) =
⎪ x6 − 0.14654113 − 0.18922793x8 x5 x10 = 0,
⎪
⎪
⎪
⎪
⎪ x7 − 0.42937161 − 0.21180486x2 x5 x8 = 0,
⎪
⎪
⎪
⎪ x8 − 0.07056438 − 0.17081208x1 x7 x6 = 0,
⎪
⎪
⎪
⎪ x9 − 0.34504906 − 0.19612740x10 x6 x8 = 0,
⎪
⎩
x10 − 0.42651102 − 0.21466544x4 x8 x1 = 0,
−10 ≤ x1 , x2 , . . . , x10 ≤ 10.
In this benchmark, MSSO has proven its efficiency. Table 6 clearly shows the significant
differences between MSSO and the other compared algorithms. MSSO achieved the best
solution with a fitness value of 5.21 × 10−17 , while all different algorithms achieved
solutions far from the exact answer. When we compare the fitness values of the hybrid
MSSO and the original SSO, we can see how substantial modifications were made to the
local search mechanism of the original SSO to produce the hybrid MSSO.
MSSO HHO [9] PSO [49] ALO [10] BOA [11] EO [13] SSO [2]
x1 0.25783339370 0.34365751785 0.25784839865 0.26464526597 0.33136834430 0.25516109743 0.20435054402
x2 0.38109715460 0.33753782972 0.38110810543 0.40023813660 0.38789340931 0.37760106529 0.28412716608
x3 0.27874501735 0.29465973836 0.27883198050 0.30288150337 0.21629745964 0.27543881117 0
x4 0.20066896423 0.25159175619 0.20067772983 0.19561671789 0.11897384735 0.20247039332 4.6624555 × 10−14
x5 0.44525142484 0.29083336278 0.44529373708 0.42832138835 0.43899648474 0.44562023380 0.21484320995
x6 0.14918391997 0.17861978035 0.14916957364 0.13017287705 0.11989963467 0.14456849647 0.04811561607
x7 0.43200969898 0.45287147997 0.43201094116 0.42448051059 0.41892967958 0.43104930617 0.46906778944
x8 0.07340277778 0.12886919949 0.07336337021 0.08657096366 0.00941718057 0.07245346262 0.04141333025
x9 0.34596682688 0.41390929124 0.34597891260 0.35142553752 0.31940825594 0.34552658400 0.44010425014
x10 0.42732627599 0.31843020513 0.42732508540 0.40501764912 0.31956474381 0.42687560151 0.45420039449
Fitness 5.21 × 10−17 0.238337 0.000107027 0.049509462 0.182742367 0.007434684 0.448061654
161
Mathematics 2023, 11, 1473
In Table 7, the comparison for this system shows that MSSO has the least fitness value of
7.09 × 10−21, while PSO and EO have fitness values of 2.85 × 10−9 and 3.45 × 10−8, respectively.
MSSO HHO [9] PSO [49] ALO [10] BOA [11] EO [13] SSO [2]
x1 1.8492683 × 10−7 4.8416050 × 10−6 1.9790922 × 10−5 1.0594162 × 10−5 1 × 10−22 1.0078652 × 10−5 1 × 10−22
x2 1.5794030 × 10−7 1 × 10−22 1.3635593 × 10−15 3.3602174 × 10−7 1 × 10−22 3.5661155 × 10−8 1 × 10−22
x3 1.3864372 × 10−5 1.6731599 × 10−5 3.1002047 × 10−5 3.0337169 × 10−5 4.1974292 × 10−6 3.0993197 × 10−5 3.8503515 × 10−6
x4 7.1476236 × 10−11 9.8490309 × 10−6 5.7239289 × 10−10 1.4332843 × 10−8 6.9533835 × 10−6 9.9663562 × 10−6 8.9634004 × 10−7
x5 6.6527288 × 10−21 1 × 10−22 1.3480554 × 10−18 1 × 10−22 1 × 10−22 1.0400080 × 10−22 1 × 10−22
x6 2.4773409 × 10−6 1 × 10−22 4.8969622 × 10−6 2.6156272 × 10−7 2.6521256 × 10−6 1.0305788 × 10−9 2.7036389 × 10−7
x7 4.9999643 × 10−6 1 × 10−22 4.9991846 × 10−6 5.0388691 × 10−6 1 × 10−22 9.0975075 × 10−10 4.1982561 × 10−6
x8 1.7135628 × 10−5 1.1426668 × 10−5 1.1003359 × 10−10 7.7891263 × 10−7 2.1564327 × 10−5 2.1008819 × 10−22 2.2239423 × 10−5
x9 4.7151213 × 10−7 5.6143966 × 10−6 2.0556945 × 10−7 7.7518075 × 10−6 3.3294486 × 10−6 9.9131258 × 10−6 6.2906500 × 10−6
x10 2.2079329 × 10−6 2.4214874 × 10−6 2.7225791 × 10−15 6.7696638 × 10−7 1 × 10−22 1.9269643 × 10−8 1 × 10−22
Fitness 7.09 × 10−21 3.23 × 10−6 2.85 × 10−9 1.73 × 10−7 6.22 × 10−6 3.45 × 10−8 6.91 × 10−6
There is more than one exact solution to this system. Table 8 shows that the proposed
MSSO algorithm achieved the most accurate solution with a fitness value of 1.18 × 10−24 ,
and the PSO algorithm achieved second place with a fitness value of 5.26 × 10−7 . In
contrast, the rest of the algorithms recorded answers that differ significantly from the exact
solution. Further, NESs in problems 6–8 prove the flexibility of the proposed hybrid MSSO
as it remains efficient even in a wide interval [−10, 10].
MSSO HHO [9] PSO [49] ALO [10] BOA [11] EO [13] SSO [2]
x1 0.68279148724 0.52702319411 0.76960300904 0.28887548289 0.95829879077 0.26693676403 1.00511003439
x2 0.50647432076 0.29250343550 0.66834059064 0.24588295652 0.10377244360 0.73023242916 −0.14156714998
x3 −0.7306132937 0.84391409892 0.63852284443 −0.95725516399 0.20563151204 −0.96364982722 0.12921880541
x4 −0.8622550449 0.96128971140 −0.74385526431 0.96902915055 −0.98879741269 0.68370357562 0.99423873612
x5 3.8805276 × 10−19 −0.01763142313 −5.5341563 × 10−7 0.00262835607 −0.02586929684 −0.00260602535 0.01451788346
x6 −3.013005 × 10−19 −0.00227648751 −2.3175063 × 10−7 0.00282255517 0.01218071672 −0.00190637065 −0.00244565414
Fitness 1.18 × 10−24 0.020764231 5.26 × 10−7 0.001489 0.048684 0.002456 0.032031
162
Mathematics 2023, 11, 1473
The comparison results in all benchmarks confirm the hypothesis that we have men-
tioned in the first section; that is, that the hybridization of two algorithms inherits the
efficient merits of both algorithms (SSO and Newton’s methods). This can be seen by
looking at the comparison results between the MSSO and the original SSO, where the
MSSO has outperformed the original SSO in all selected benchmarks. The reason for this
remarkable performance is the use of Newton’s method as a local search, which strengthens
the hybrid’s capability to avoid the local optimum in Problems 1–5 (where MSSO has
obtained the exact solution), and significantly improves the obtained fitness values in
Problems 6-8. The comparisons indicate that the proposed hybrid algorithm MSSO has
avoided being trapped in the local optima in all problems, compared with the majority of
the other algorithms.
Table 9. The comparison results of the average 30-run solution for all problems.
MSSO HHO [9] PSO [49] ALO [10] BOA [11] EO [13] SSO [2]
Problem 1 2.2709 × 10−16 0.0022869 2.2927 × 10−6 6.2912 × 10−7 0.039573 9.3817 × 10−5 0.050354
Problem 2 4.7855 × 10−16 0.13913 0.037009 0.11703 0.12076 0.060332 0.27432
Problem 3 1.1842 × 10−16 0.038848 2.1189 × 10−12 7.5657 × 10−6 0.20288 3.3604 × 10−5 0.14209
Problem 4 1.1102 × 10−17 0.052119 0.0055764 0.042144 0.063878 0.015678 0.12467
Problem 5 0 2.706 × 10−9 0.011783 0.058917 0.0035033 0.011783 2.1635E-09
Problem 6 5.2147 × 10−17 0.37777 0.00092349 0.16493 0.36299 0.037007 0.56687
Problem 7 4.7872 × 10−9 4.4292 × 10−5 2.5874 × 10−6 3.4687 × 10−6 2.0904 × 10−5 2.3701 × 10−6 1.1393 × 10−5
Problem 8 0.010581 0.18797 0.01278 2.9582 0.13696 0.014989 0.11305
Mean (F-test) 1 5.375 2.625 4.375 5.5 3.375 5.625
Rank 1 5 2 4 6 3 7
According to Table 9, MSSO has surpassed all other compared algorithms. The average
fitness values of MSSO and the original SSO show a significant difference in all benchmarks.
Consequently, this improvement confirms the flexibility of the hybrid MSSO in seeking the
best solution without being entrapped by local optima. Furthermore, as shown in Table 9,
MSSO outperforms all of the other compared algorithms, particularly for problems 2, 4, 6,
and 8.
Additionally, the algorithm is considered consistent and stable if it maintains consis-
tency over 30 runs. The average of the solutions must, therefore, be the same as or very
close to the best solution in order to achieve consistency. It has been demonstrated in
this study that MSSO consistency has been maintained for all selected problems. More-
over, the average standard deviation achieved by each algorithm is shown in Table 10,
in which smaller values of standard deviation indicate more stability. The hybrid MSSO
demonstrated stable results in most of the selected problems.
MSSO HHO [9] PSO [49] ALO [10] BOA [11] EO [13] SSO [2]
Problem 1 7.7097 × 10−16 0.0035697 4.4735 × 10−6 8.3914 × 10−7 0.046935 0.00012725 0.030655
Problem 2 1.1592 × 10−16 0.062883 0.021589 0.069769 0.058383 0.058383 0.085942
Problem 3 3.4857 × 10−16 0.039902 3.889 × 10−12 8.6567 × 10−6 0.18297 5.8122 × 10−5 0.069194
Problem 4 6.0809 × 10−17 0.031155 0.0043532 0.039841 0.022589 0.045526 0.082401
Problem 5 0 9.7731 × 10−9 0.064539 0.13399 0.0026465 0.064539 4.3991 × 10−9
Problem 6 0 0.076223 0.00068543 0.061743 0.064055 0.038589 0.07177
Problem 7 6.8883 × 10−9 2.0068 × 10−5 4.9339 × 10−6 2.582 × 10−6 1.0476 × 10−5 2.8284 × 10−6 2.3389 × 10−6
Problem 8 0.057522 0.11493 0.0348 12.8391 0.051742 0.016544 0.050028
Mean (F-test) 1.5 5.25 2.875 4.625 4.875 3.75 4.875
Rank 1 7 2 4 5.5 3 5.5
163
Mathematics 2023, 11, 1473
Furthermore, the significance of MSSO improvements was examined using the statis-
tical t-test in Table 11. Improvements were considered significant if the p-value was less
than 0.05; otherwise, they were not. Results show that all algorithms have p-values lower
than 0.05 in all tested problems, except for HHO, which has a single value above 0.05 in
Problem 5. It is evident from this that MSSO has a higher level of reliability than competing
algorithms. Further, MSSO’s solutions are significantly more accurate than those of other
algorithms since the majority of its p-values are close to 0. The results demonstrate that the
MSSO is a robust search method capable of finding precise solutions. Moreover, it is able to
avoid local optimal traps and immature convergence.
HHO [9] PSO [49] ALO [10] SSO [2] EO [13] BOA [11]
Problem 1 7.425 × 10−5 0.0032362 7.305 × 10−5 6.322 × 10−11 0.00053712 0.032655
Problem 2 1.396 × 10−8 0.0078581 0.00018975 7.512 × 10−17 0.0053228 0.002542
Problem 3 2.1404 × 10−8 0.00038876 1.7706 × 10−6 2.171 × 10−13 5.5596 × 10−5 0.000194
Problem 4 6.4406 × 10−10 5.0472 × 10−5 1.2494 × 10−7 6.948 × 10−11 0.013013 6.918 × 10−11
Problem 5 0.32558 9.7628 × 10−8 0.0012288 0.32563 0.001503 4.3991 × 10−6
Problem 6 1.8665 × 10−26 3.7433 × 10−5 1.1361 × 10−16 2.345 × 10−26 2.9061 × 10−10 2.855 × 10−22
Problem 7 3.7355 × 10−12 0.0044676 3.9777 × 10−9 7.247 × 10−21 0.00029067 2.3389 × 10−6
Problem 8 2.3479 × 10−12 0.049502 0.00014768 1.574 × 10−9 2.3043 × 10−6 0.000028
Moreover, one of the criteria that is considered when comparing algorithms is their
speed of convergence. Figure 4 indicates that MSSO enhanced the convergence speed of the
original SSO in all problems. It also shows that MSSO achieves the best solution with much
fewer iterations than the other algorithms. Consequently, the superiority of the proposed
MSSO is confirmed.
It is known that any optimization method has some constraints that slow down the
algorithm from finding the optimum solution or, in some cases, prevent it from finding the
solution. HHO, for instance, probably attains local optima instead of the optimal answer.
SSO quickly falls into a local minimum of systems of nonlinear equations, which consists
of a set of models [2]. PSO has some drawbacks, such as a lack of population variety and
the inability to balance local optima and global optima [54]. The EO method, on the other
hand, does not function well for large-scale situations [55].
The novel hybrid algorithm MSSO’s convergence speed is attributed to combining
Newton’s iterative method as a local search and the SSO algorithm. On the one hand, MSSO
benefits from the originality of Newton’s method, which was developed to find solutions to
nonlinear equation systems. On the other hand, SSO ensures appropriate initial solutions
for Newton’s method by employing search agents. Furthermore, Newton’s method features
a second-order of convergence, which implies that the scheme converges to approximately
two significant digits in each iteration [1]. Thus, the hybridization between Newton’s
method and the SSO algorithm inherits the merits from both sides to produce an efficient
algorithm that can overcome the main disadvantages [56,57].
It is worth noting that the default precision value of the variable in MATLAB was used
for all calculations in this study, which is 16 digits of precision. This precision is timesaving
compared with more significant digits. However, in some situations, this may impact the
outcome. In MATLAB, the function ”vpa” may be used to enhance variable precision.
Thus, increasing the number of digits can improve the accuracy of the findings, but this is
a time-consuming operation. More details and examples of this case can be seen in [40].
In this research, the use of ”vpa” has increased the accuracy of the results in Problem 5,
Problem 7, and Problem 8.
164
Mathematics 2023, 11, 1473
Figure 4. The convergence speed for the eight problems based on an average of 30 runs.
165
Mathematics 2023, 11, 1473
both strategies were examined for problems 1–4. Tables 12–15 compare the fitness values
achieved by MSSO and Newton’s method using three randomly chosen starting points. We
examined both strategies for comparison purposes at iteration 5, iteration 7, and iteration 10.
In addition, variables of 1200-digit precision in all selected problems were used to clarify
the solutions’ accuracy. If, as noted earlier, the number of digits is increased, the findings
may also improve.
166
Mathematics 2023, 11, 1473
with one initial point, MSSO starts with 20 search agents. The superiority of the MSSO is
demonstrated by the accuracy of its solutions. In addition, the time required to reach the
convergence criteria is less in MSSO. Having 20 random initial solutions clearly requires
more time for Newton’s method. Therefore, this is another reason why hybridizing both
SSO and Newton’s method is better than depending on one of them.
Moreover, the speed of convergence towards the best solution is astonishing. MSSO
can choose the best initial point in a few iterations and move quickly toward the global
optima. Figure 5 shows the convergence speed of problems 1–4 for the first five iterations
on an average of 30 runs.
Figure 5. The convergence speed of problems 1–4 for five iterations based on an average of 30 runs.
Table 16. Comparing Newton’s method and MSSO in terms of average time (in seconds).
Newton MSSO
Problem
Initial Guess Iteration ε ≤ 1 × 10 −15 ε ≤ 1 × 10 −15
Problem 1 {0,0} 5 1.389 0.283
Problem 2 {1.5,1.5} 5 1.871 0.179
Problem 3 {1,1,1} 8 2.497 0.234
Problem 4 {0.5,0.5} 4 2.137 0.244
Based on the results, an apparent enhancement has been added to Newton’s method
by using the hybridized MSSO. The CPU-time needed to satisfy the selected stopping
167
Mathematics 2023, 11, 1473
limit is much better for MSSO than Newton’s method. Even though Newton’s method is
a part of the proposed MSSO, MSSO showed better results because of the mechanism of
SSO in selecting the best initial guess for Newton’s technique as a local search inside the
hybrid algorithm.
It is well known that choosing a starting point that is far from the root of the system
could negatively affect the convergence of Newton’s method. Therefore, since Newton’s
method is included in the MSSO, this could negatively affect MSSO’s convergence as well.
However, based on the mechanism of the MSSO, the algorithm randomly selects 20 agents
that are considered as initial points within a specific interval. In general, optimization
algorithms have more choices to start with compared to Newton’s method. Iterative
methods can benefit from hybridization in selecting initial points because optimization
algorithms can have many initial points. On the other hand, optimization algorithms can
benefit from the fast and accurate convergence of iterative methods.
6. Conclusions
In this work, a hybrid method known as MSSO was introduced for solving systems of
nonlinear equations using Newton’s iterative method as a local search for the Sperm Swarm
Optimization algorithm SSO. The main goal of the MSSO is to solve the problem of New-
ton’s method’s initial guess, the achievement of which results in a better selection of initial
points, enabling it to be applied to a wider variety of real-world applications. Moreover,
Newton’s scheme was used in MSSO as a local search, which improved the accuracy of the
tested solutions. In addition, the MSSO’s convergence speed is substantially improved.
Eight nonlinear systems of varying orders were utilized to illustrate the effectiveness
of the proposed MSSO. The novel MSSO was also compared to six well-known optimization
methods, including the original SSO, BOA, ALO, EO, HHO, and PSO. The Euclidean norm
has been utilized as a fitness function in all benchmarks. According to the results, MSSO
outperforms all other compared algorithms in four metrics: fitness value, solution accuracy,
stability, and speed of convergence. In addition, the consistency of the MSSO is confirmed
by running the methods thirty times. Additionally, the standard deviation showed that
MSSO was the most stable optimization algorithm.
Additionally, we compared the performance of MSSO and Newton’s method on four
problems from the benchmarks. Across all four datasets, the MSSO outperformed Newton’s
method. The MSSO method also overcomes some of Newton’s scheme’s limitations, such
as divergence and selection of initial guesses.
Future work can address some related issues, such as how the suggested method
performs against common optimization benchmarks. Future research will also focus
on solving nonlinear equations arising from real-world applications, such as Burgers’
Equation. In addition, future work needs to address the efficiency of the proposed algorithm
when solving big systems. Finally, the use of a derivative-free iterative method instead
of Newton’s method reduces the computational complexity resulting from the need to
evaluate Newton’s method in each iteration and is an interesting topic that needs to be
focused on in the future.
Author Contributions: Conceptualization, O.S.S. and R.S.; methodology, O.S.S., R.S. and H.S.; vali-
dation, K.A. and I.H.; formal analysis, O.S.S. and R.S.; investigation, O.S.S. and R.S.; resources, O.S.S.,
R.S. and H.S.; data curation, O.S.S. and R.S.; writing—original draft preparation, O.S.S., R.S. and
H.S.; writing—review and editing, K.A. and I.H.; visualization, R.S.; supervision, O.S.S., R.S. and
I.H.; project administration, O.S.S., R.S. and I.H.; funding acquisition, I.H. All authors have read and
agreed to the published version of the manuscript.
Funding: This research and the APC were funded by Universiti Kebangsaan Malaysia, grant number
DIP-2021-018.
Data Availability Statement: The data that support the findings of this study are available on request
from the corresponding author, Sihwail, R.
168
Mathematics 2023, 11, 1473
Conflicts of Interest: The authors declare that there is no conflict of interest regarding the publication
of this paper.
References
1. Traub, J.F. Iterative Methods for the Solution of Equations; Prentice-Hall: Englewood Cliffs, NJ, USA, 1964.
2. Shehadeh, H.A. A Hybrid Sperm Swarm Optimization and Gravitational Search Algorithm (HSSOGSA) for Global Optimization.
Neural Comput. Appl. 2021, 33, 11739–11752. [CrossRef]
3. Gupta, S. Enhanced Harmony Search Algorithm with Non-Linear Control Parameters for Global Optimization and Engineering
Design Problems. Eng. Comput. 2022, 38, 3539–3562. [CrossRef]
4. Zhu, F.; Chen, D.; Zou, F. A Novel Hybrid Dynamic Fireworks Algorithm with Particle Swarm Optimization. Soft Comput. 2021,
25, 2371–2398. [CrossRef]
5. Shehadeh, H.A.; Ahmedy, I.; Idris, M.Y.I. Empirical Study of Sperm Swarm Optimization Algorithm. Adv. Intell. Syst. Comput.
2018, 869, 1082–1104. [CrossRef]
6. Shehadeh, H.A.; Ahmedy, I.; Idris, M.Y.I. Sperm Swarm Optimization Algorithm for Optimizing Wireless Sensor Network
Challenges. In Proceedings of the 6th International Conference on Communications and Broadband Networking, Singapore,
24–26 February 2018; pp. 53–59. [CrossRef]
7. Shehadeh, H.A.; Idris, M.Y.I.; Ahmedy, I.; Ramli, R.; Noor, N.M. The Multi-Objective Optimization Algorithm Based on
Sperm Fertilization Procedure (MOSFP) Method for Solving Wireless Sensor Networks Optimization Problems in Smart Grid
Applications. Energies 2018, 11, 97. [CrossRef]
8. Id, H.A.S.; Yamani, M.; Idris, I.; Ahmedy, I. Multi-Objective Optimization Algorithm Based on Sperm Fertilization Procedure
(MOSFP). Symmetry 2017, 9, 241. [CrossRef]
9. Heidari, A.A.; Mirjalili, S.; Faris, H.; Aljarah, I.; Mafarja, M.; Chen, H. Harris Hawks Optimization: Algorithm and Applications.
Future Gener. Comput. Syst. 2019, 97, 849–872. [CrossRef]
10. Mirjalili, S. The Ant Lion Optimizer. Adv. Eng. Softw. 2015, 83, 80–98. [CrossRef]
11. Arora, S.; Singh, S. Butterfly Optimization Algorithm: A Novel Approach for Global Optimization. Soft Comput. 2018, 23, 715–734.
[CrossRef]
12. Zhang, Y.; Wang, Y.H.; Gong, D.W.; Sun, X.Y. Clustering-Guided Particle Swarm Feature Selection Algorithm for High-
Dimensional Imbalanced Data With Missing Values. IEEE Trans. Evol. Comput. 2022, 26, 616–630. [CrossRef]
13. Faramarzi, A.; Heidarinejad, M.; Stephens, B.; Mirjalili, S. Equilibrium Optimizer: A Novel Optimization Algorithm. Knowl.-Based
Syst. 2020, 191, 105190. [CrossRef]
14. Holland, J.H. Genetic Algorithms. Sci. Am. 1992, 267, 66–72. [CrossRef]
15. Liu, K.; Hu, X.; Yang, Z.; Xie, Y.; Feng, S. Lithium-Ion Battery Charging Management Considering Economic Costs of Electrical
Energy Loss and Battery Degradation. Energy Convers. Manag. 2019, 195, 167–179. [CrossRef]
16. Liu, K.; Zou, C.; Li, K.; Wik, T. Charging Pattern Optimization for Lithium-Ion Batteries with an Electrothermal-Aging Model.
IEEE Trans. Ind. Inform. 2018, 14, 5463–5474. [CrossRef]
17. Zhou, R.H.; Li, Y.G. An Improve Cuckoo Search Algorithm for Solving Nonlinear Equation Group. Appl. Mech. Mater. 2014,
651–653, 2121–2124. [CrossRef]
18. Ariyaratne, M.K.A.; Fernando, T.G.I.; Weerakoon, S. Solving Systems of Nonlinear Equations Using a Modified Firefly Algorithm
(MODFA). Swarm Evol. Comput. 2019, 48, 72–92. [CrossRef]
19. Ren, H.; Wu, L.; Bi, W.; Argyros, I.K. Solving Nonlinear Equations System via an Efficient Genetic Algorithm with Symmetric and
Harmonious Individuals. Appl. Math. Comput. 2013, 219, 10967–10973. [CrossRef]
20. Chang, W. der An Improved Real-Coded Genetic Algorithm for Parameters Estimation of Nonlinear Systems. Mech. Syst. Signal
Process. 2006, 20, 236–246. [CrossRef]
21. Grosan, C.; Abraham, A. A New Approach for Solving Nonlinear Equations Systems. IEEE Trans. Syst. Man Cybern. Part ASyst.
Hum. 2008, 38, 698–714. [CrossRef]
22. Jaberipour, M.; Khorram, E.; Karimi, B. Particle Swarm Algorithm for Solving Systems of Nonlinear Equations. Comput. Math.
Appl. 2011, 62, 566–576. [CrossRef]
23. Mo, Y.; Liu, H.; Wang, Q. Conjugate Direction Particle Swarm Optimization Solving Systems of Nonlinear Equations. Comput.
Math. Appl. 2009, 57, 1877–1882. [CrossRef]
24. Sihwail, R.; Omar, K.; Ariffin, K.A.Z. An Effective Memory Analysis for Malware Detection and Classification. Comput. Mater.
Contin. 2021, 67, 2301–2320. [CrossRef]
25. Sihwail, R.; Omar, K.; Zainol Ariffin, K.; al Afghani, S. Malware Detection Approach Based on Artifacts in Memory Image and
Dynamic Analysis. Appl. Sci. 2019, 9, 3680. [CrossRef]
26. Jadon, S.S.; Tiwari, R.; Sharma, H.; Bansal, J.C. Hybrid Artificial Bee Colony Algorithm with Differential Evolution. Appl. Soft
Comput. J. 2017, 58, 11–24. [CrossRef]
27. Jia, R.; He, D. Hybrid Artificial Bee Colony Algorithm for Solving Nonlinear System of Equations. In Proceedings of
the 2012 8th International Conference on Computational Intelligence and Security, CIS 2012, Guangzhou, China, 17–18
November 2012; pp. 56–60.
169
Mathematics 2023, 11, 1473
28. Aydilek, İ.B. A Hybrid Firefly and Particle Swarm Optimization Algorithm for Computationally Expensive Numerical Problems.
Appl. Soft Comput. J. 2018, 66, 232–249. [CrossRef]
29. Nasr, S.; El-Shorbagy, M.; El-Desoky, I.; Hendawy, Z.; Mousa, A. Hybrid Genetic Algorithm for Constrained Nonlinear Optimiza-
tion Problems. Br. J. Math. Comput. Sci. 2015, 7, 466–480. [CrossRef]
30. Abualigah, L.M.; Khader, A.T.; Hanandeh, E.S.; Gandomi, A.H. A Novel Hybridization Strategy for Krill Herd Algorithm Applied
to Clustering Techniques. Appl. Soft Comput. J. 2017, 60, 423–435. [CrossRef]
31. El-Shorbagy, M.A.; Mousa, A.A.A.; Fathi, W. Hybrid Particle Swarm Algorithm for Multiobjective Optimization: Integrating Particle
Swarm Optimization with Genetic Algorithms for Multiobjective Optimization; Lambert Academic: Saarbrücken, Germany, 2011.
32. Goel, R.; Maini, R. A Hybrid of Ant Colony and Firefly Algorithms (HAFA) for Solving Vehicle Routing Problems. J. Comput. Sci.
2018, 25, 28–37. [CrossRef]
33. Turanoğlu, B.; Akkaya, G. A New Hybrid Heuristic Algorithm Based on Bacterial Foraging Optimization for the Dynamic Facility
Layout Problem. Expert Syst. Appl. 2018, 98, 93–104. [CrossRef]
34. Skoullis, V.I.; Tassopoulos, I.X.; Beligiannis, G.N. Solving the High School Timetabling Problem Using a Hybrid Cat Swarm
Optimization Based Algorithm. Appl. Soft Comput. J. 2017, 52, 277–289. [CrossRef]
35. Chen, X.; Zhou, Y.; Tang, Z.; Luo, Q. A Hybrid Algorithm Combining Glowworm Swarm Optimization and Complete 2-Opt
Algorithm for Spherical Travelling Salesman Problems. Appl. Soft Comput. J. 2017, 58, 104–114. [CrossRef]
36. Marichelvam, M.K.; Tosun, Ö.; Geetha, M. Hybrid Monkey Search Algorithm for Flow Shop Scheduling Problem under Makespan
and Total Flow Time. Appl. Soft Comput. J. 2017, 55, 82–92. [CrossRef]
37. Karr, C.L.; Weck, B.; Freeman, L.M. Solutions to Systems of Nonlinear Equations via a Genetic Algorithm. Eng. Appl. Artif. Intell.
1998, 11, 369–375. [CrossRef]
38. Luo, Y.Z.; Yuan, D.C.; Tang, G.J. Hybrid Genetic Algorithm for Solving Systems of Nonlinear Equations. Chin. J. Comput. Mech.
2005, 22, 109–114.
39. Luo, Y.Z.; Tang, G.J.; Zhou, L.N. Hybrid Approach for Solving Systems of Nonlinear Equations Using Chaos Optimization and
Quasi-Newton Method. Appl. Soft Comput. J. 2008, 8, 1068–1073. [CrossRef]
40. Sihwail, R.; Said Solaiman, O.; Omar, K.; Ariffin, K.A.Z.; Alswaitti, M.; Hashim, I. A Hybrid Approach for Solving Systems of
Nonlinear Equations Using Harris Hawks Optimization and Newton’s Method. IEEE Access 2021, 9, 95791–95807. [CrossRef]
41. Sihwail, R.; Said Solaiman, O.; Zainol Ariffin, K.A. New Robust Hybrid Jarratt-Butterfly Optimization Algorithm for Nonlinear
Models. J. King Saud Univ.—Comput. Inf. Sci. 2022, 34 Pt A, 8207–8220. [CrossRef]
42. Sihwail, R.; Omar, K.; Ariffin, K.A.Z.; Tubishat, M. Improved Harris Hawks Optimization Using Elite Opposition-Based Learning
and Novel Search Mechanism for Feature Selection. IEEE Access 2020, 8, 121127–121145. [CrossRef]
43. Said Solaiman, O.; Hashim, I. Efficacy of Optimal Methods for Nonlinear Equations with Chemical Engineering Applications.
Math. Probl. Eng. 2019, 2019, 1728965. [CrossRef]
44. Said Solaiman, O.; Hashim, I. An Iterative Scheme of Arbitrary Odd Order and Its Basins of Attraction for Nonlinear Systems.
Comput. Mater. Contin. 2020, 66, 1427–1444. [CrossRef]
45. Said Solaiman, O.; Hashim, I. Optimal Eighth-Order Solver for Nonlinear Equations with Applications in Chemical Engineering.
Intell. Autom. Soft Comput. 2021, 27, 379–390. [CrossRef]
46. Said Solaiman, O.; Hashim, I. Two New Efficient Sixth Order Iterative Methods for Solving Nonlinear Equations. J. King Saud
Univ. Sci. 2019, 31, 701–705. [CrossRef]
47. Said Solaiman, O.; Ariffin Abdul Karim, S.; Hashim, I. Dynamical Comparison of Several Third-Order Iterative Methods for
Nonlinear Equations. Comput. Mater. Contin. 2021, 67, 1951–1962. [CrossRef]
48. Adam, S.P.; Alexandropoulos, S.A.N.; Pardalos, P.M.; Vrahatis, M.N. No Free Lunch Theorem: A Review. Springer Optim. Its Appl.
2019, 145, 57–82. [CrossRef]
49. Kennedy, J.; Eberhart, R. Particle Swarm Optimization. In Proceedings of the ICNN’95—International Conference on Neural
Networks, Perth, WA, Australia, 27–1 December 1995; pp. 1942–1948. [CrossRef]
50. El-Shorbagy, M.A.; El-Refaey, A.M. Hybridization of Grasshopper Optimization Algorithm with Genetic Algorithm for Solving
System of Non-Linear Equations. IEEE Access 2020, 8, 220944–220961. [CrossRef]
51. Wang, X.; Li, Y. An Efficient Sixth-Order Newton-Type Method for Solving Nonlinear Systems. Algorithms 2017, 10, 45. [CrossRef]
52. Verschelde, J.; Verlinden, P.; Cools, R. Homotopies Exploiting Newton Polytopes for Solving Sparse Polynomial Systems. SIAM J.
Numer. Anal. 1994, 31, 915–930. [CrossRef]
53. van Hentenryck, P.; Mcallester, D.; Kapur, D. Solving Polynomial Systems Using a Branch and Prune Approach. SIAM J. Numer.
Anal. 1997, 34, 797–827. [CrossRef]
54. Ridha, H.M.; Heidari, A.A.; Wang, M.; Chen, H. Boosted Mutation-Based Harris Hawks Optimizer for Parameters Identification
of Single-Diode Solar Cell Models. Energy Convers. Manag. 2020, 209, 112660. [CrossRef]
55. Elgamal, Z.M.; Yasin, N.M.; Sabri, A.Q.M.; Sihwail, R.; Tubishat, M.; Jarrah, H. Improved Equilibrium Optimization Algorithm
Using Elite Opposition-Based Learning and New Local Search Strategy for Feature Selection in Medical Datasets. Computation
2021, 9, 68. [CrossRef]
170
Mathematics 2023, 11, 1473
56. Shehadeh, H.A.; Mustafa, H.M.J.; Tubishat, M. A Hybrid Genetic Algorithm and Sperm Swarm Optimization (HGASSO) for
Multimodal Functions. Int. J. Appl. Metaheuristic Comput. 2021, 13, 1–33. [CrossRef]
57. Shehadeh, H.A.; Shagari, N.M. Chapter 24: A Hybrid Grey Wolf Optimizer and Sperm Swarm Optimization for Global
Optimization. In Handbook of Intelligent Computing and Optimization for Sustainable Development; John Wiley & Sons, Inc.: Hoboken,
NJ, USA, 2022; pp. 487–507. [CrossRef]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual
author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to
people or property resulting from any ideas, methods, instructions or products referred to in the content.
171
MDPI
St. Alban-Anlage 66
4052 Basel
Switzerland
www.mdpi.com
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are
solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s).
MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from
any ideas, methods, instructions or products referred to in the content.
Academic Open
Access Publishing