System Identification Using Multilayer Differential Neural Networks: A New Result

Volume 2012, Article ID 529176,20pages doi:10.1155/2012/529176

Research Article

System Identification Using Multilayer Differential Neural Networks: A New Result

J. Humberto P ´erez-Cruz,

A. Y. Alanis,

Jos ´e de Jes ´us Rubio,

and Jaime Pacheco

1Centro Universitario de Ciencias Exactas e Ingenier´ıas, Universidad de Guadalajara, Boulevard Marcelino Garc´ıa Barrag´an No. 1421, 44430 Guadalajara, JAL, Mexico

2Secci´on de Estudios de Posgrado e Investigaci´on, ESIME-UA, IPN, Avenida de las Granjas No. 682, 02250 Santa Catarina, NL, Mexico

Correspondence should be addressed to J. Humberto P´erez-Cruz,phhhantom2001@yahoo.com.mx Received 22 November 2011; Revised 30 January 2012; Accepted 2 February 2012

Academic Editor: Hector Pomares

Copyrightq2012 J. Humberto P´erez-Cruz et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

In previous works, a learning law with a dead zone function was developed for multilayer differential neural networks. This scheme requires strictly a priori knowledge of an upper bound for the unmodeled dynamics. In this paper, the learning law is modified in such a way that this condition is relaxed. By this modification, the tuning process is simpler and the dead-zone function is not required anymore. On the basis of this modification and by using a Lyapunov-like analysis, a stronger result is here demonstrated: the exponential convergence of the identification error to a bounded zone. Besides, a value for upper bound of such zone is provided. The workability of this approach is tested by a simulation example.

1. Introduction

During the last four decades system identification has emerged as a powerful and effective alternative to the first principles modeling1–4. By using the first approach, a satisfactory mathematical model of a system can be obtained directly from an input and output experimental data set 5. Ideally no a priori knowledge of the system is necessary since this is considered as a black box. Thus, the time employed to develop such model is reduced significantly with respect to a first principles approach. For the linear case, system identification is a problem well understood and enjoys well-established solutions 6.

However, the nonlinear case is much more challenging. Although some proposals have been presented 7, the class of considered nonlinear systems can result very limited. Due to their capability of handling a more general class of systems and due to advantages such as the fact of not requiring linear in parameters and persistence of excitation assumptions8,

artificial neural networksANNshave been extensively used in identification of nonlinear systems 9–12. Their success is based on their capability of providing arbitrarily good approximations to any smooth function13–15as well as their massive parallelism and very fast adaptability16,17.

An artificial neural network can be simply considered as a nonlinear generic mathematical formula whose parameters are adjusted in order to represent the behavior of a static or dynamic system18. These parameters are called weights. Generally speaking, ANN can be classified as feedforwardstaticones, based on the back propagation technique 19 or as recurrent dynamic ones 17. In the first network type, system dynamics is approximated by a static mapping. These networks have two major disadvantages: a slow learning rate and a high sensitivity to training data. The second approachrecurrent ANN incorporates feedback into its structure. Due to this feature, recurrent neural networks can overcome many problems associated with static ANN, such as global extrema search, and consequently have better approximation properties. Depending on their structure, recurrent neural networks can be classified as discrete-time ones or differential ones.

The first deep insight about the identification of dynamic systems based on neural networks was provided by Narendra and Parthasarathy 20. However, none-stability analyses of their neuroidentifier were presented. Hunt et al. 21 called attention to determine the convergence, stability and robustness of the algorithms based on neural networks for identification and control. This issue was addressed by Polycarpou and Ioannou 16, Rovithakis and Christodoulou17, Kosmatopoulos et al. 22, and Yu and Poznyak 23. Given different structures of continuous-time neural networks, the stability of their algorithms could be proven by using Lyapunov-like analysis. All aforementioned works considered only the case of single-layer networks. However, as it is known, this kind of networks does not necessarily satisfy the property of universal function approximation24.

And although the activation functions of single-layer neural networks are selected as a basis set in such a way that this property can be guaranteed, the approximation error can never be made smaller than a lower bound24. This drawback can be overcome by using multilayer neural networks. Due to this better capability of function approximation, the case multilayer was considered in25for feedforward networks and for continuous time recurrent neural networks for first time in26and subsequently in27. By using Lyapunov-like analysis and a dead-zone function, boundedness for the identification error could be guaranteed in26.

The following upper bound for the “average” identification error was reported,

lim sup

T→ ∞

1 T

_T

0

1− f0 Υ

λ_min

P^−1/2Q₀P^−1/2P^1/2Δt

Δ^T_tQ0Δtdt≤f0 Υ, 1.1

whereΔtis the identification error,Q₀is a positive definite matrix,f₀is a upper bound for the modeling error,Υis an upper bound for a deterministic disturbance, and·is a dead-zone function defined as

z

⎧⎨

⎩

z z≥0,

0 z <0. 1.2

Although, in28, open-loop analysis based on the passivity method for a multilayer neural network was carried out and certain simplifications were accomplished, the main result about

the aforementioned identification error could not be modified. In 29, the application of the multilayer scheme for control was explored. Since previous works 26–29 are based on this “average” identification error, one could wonder about the real utility of this result.

Certainly, boundedness for this kind of error does not guarantee thatΔtbelongs toL2orL_∞. Besides, none value for upper bound of identification error norm is provided. Likewise, none information about the speed of the convergence process is presented. Another disadvantage of this approach is that the upper bound for the modeling errorf0 must be strictly known a priori in order to implement the learning laws for the weight matrices. In order to avoid these drawbacks, in this paper, we propose to modify the learning laws employed in26in such a way that their implementation does not require anymore the knowledge of an upper bound for the modeling error. Besides, on the basis of these new learning laws, a stronger result is here guaranteed: the exponential convergence of the identification error norm to a bounded zone. The workability of the scheme developed in this paper is tested by simulation.

2. Multilayer Neural Identifier

Consider that the nonlinear system to be identified can be represented by

˙

xtfxt, ut, t ξt, 2.1 wherex_t ∈ ⁿ is the measurable state vector fort ∈ : {t:t≥ 0}, ut ∈ ^qis the control input,f : ⁿ× ^q × → ⁿ is an unknown nonlinear vector function which represents the nominal dynamics of the system, and ξt ∈ ⁿ represents a deterministic disturbance.

fxt, u_t, trepresents a very ample class of systems including affine and nonaffine-in-control nonlinear systems. However, when the control input appears in a nonlinear fashion in the system state equation2.1, throughout this paper, such nonlinearity with respect to the input is assumed known and represented byγ·:^q → ^s.

Consider the following parallel structure of multilayer neural network d

dtxtAxtW1,tσV1,txt W2,tφV2,txtγut, 2.2 where xt ∈ ⁿ is the state of the neural network,ut ∈ ^q is the control input, A ∈ ^n×n is a Hurwitz matrix which can be specified by the designer, the matricesW_1,t ∈ ^n×mand W_2,t ∈ ^n×rare the weights of output layers, the matricesV_1,t∈ ^m×nandV_2,t ∈ ^r×nare the weights of hidden layers,σ·is the activation vector-function with sigmoidal components, that is,σ·: σ1·, . . . , σm·^T,

σjv: aσj

1exp

−_m

i1c_σj,iv_i −dσj, forj1, . . . , m, 2.3 whereaσj,cσj,i, anddσj are positive constants which can be specified by the designer,φ·: ^r → ^r×sis also a sigmoidal function, that is,

φijz: a_φij 1exp

−_r

l1c_φij,lz_l −dφij fori1, . . . , r, j1, . . . , s, 2.4

whereaφij,cφij,l, anddφij are positive constants which can be specified by the designer,γ·: ^q → ^srepresents the nonlinearity with respect to the input—if it exists—which is assumed a priori known for the system 2.1. It is important to mention that m and r, that is, the number of neurons forσ·and the number of rows forφ·, respectively, can be selected by the designer.

The problem of identifying system 2.1 based on the multilayer differential neural network2.2consists of, given the measurable state xt and the inputut, adjusting on line the weightsW1,t, W2,t, V1,t, andV2,tby proper learning laws such that the identification error Δt:x_t−x_tcan be reduced.

Hereafter, it is considered that the following assumptions are valid;

A1System2.1satisfies theuniform ontLipschitz condition, that is,

fx, u, t−fz, v, t≤L₁x−zL₂u−v; x, z∈ ⁿ;u, v∈ ^q; 0≤L₁, L₂ <∞. 2.5

A2The differences of functionsσ·andφ·fulfil the generalized Lipschitz conditions

σ^T_tΛ1σt≤Δ^T_tΛσΔt, γ^Tutφ^T_tΛ2φtγut≤Δ^T_tΛφΔtγut², 2.6 where

σ_t:σ

V₁⁰x_t

−σ V₁⁰x_t

, φ_t:φ V₂⁰x_t

−φ V₂⁰x_t

, 2.7

Λ1 ∈ ^m×m,Λ2 ∈ ^r×r,Λσ ∈ ^n×n,Λφ ∈ ^n×nare known positive definite matrices, V₁⁰ ∈ ^m×n and V₂⁰ ∈ ^r×n are constant matrices which can be selected by the designer.

Asσ·andφ·fulfil the Lipschitz conditions and from Lemma A.1 proven in26 the following is true:

σ_t:σV1,txt−σ V10xt

DσV1,txtνσ,

σ_tγut:

φV2,tx_t−φ V₂⁰x_t

γut

2.8

^s

i1

φiV2,txt−φi

V₂⁰xt

γiut

s i1

DiφV2,txtviφ

γiut, 2.9

where

D_σ ∂σY

∂Y

YV₁⁰xt

∈ ^m×m, D_iφ ∂φ_iZ

∂Z

ZV₂⁰xt

∈ ^r×r, 2.10

ν_σ ∈ ^mandν_iφ ∈ ⁿare unknown vectors but bounded byνσ²_Λ1 ≤ l₁V_1,tx_t²_Λ1, νiφ²_Λ

2 ≤ l₂V_2,tx_t²_Λ2, respectively;V_1,t : V_1,t−V₁⁰,V_2,t : V_2,t −V₂⁰,l₁ andl₂ are

positive constants which can be defined asl1 : 4L²_g,1,l2 : 4L²_g,2, whereLg,1 and L_g,2are global Lipschitz constants forσ·andφ_i·, respectively.

A3The nonlinear functionγ·is such thatγut² ≤ uwhereuis a known positive constant.

A4Unmodeled dynamicsftis bounded by ft²

Λ3

≤f0f1xt²_Λ3, 2.11

wheref₀andf₁are known positive constants andΛ3 ∈ ^n×n is a known positive definite matrix and f_t can be defined as f_t : fxt, u_t, t−Ax_t− W₁⁰σV₁⁰x_t − W₂⁰φV₂⁰xtγut; W₁⁰ ∈ ^n×m andW₂⁰ ∈ ^n×r are constant matrices which can be selected by the designer.

A5The deterministic disturbance is bounded, that is,ξt²_Λ4 ≤Υ,Λ4is a known positive definite matrix.

A6The following matrix Riccati equation has a unique, positive definite solutionP:

A^TPP AP RPQ0, 2.12

where R2W₁⁰Λ⁻¹₁

W₁⁰_T

2W₂⁰Λ⁻¹₂ W₂⁰_T

Λ⁻¹₃ Λ⁻¹₄ , Q ΛσuΛφQ₀, 2.13

Q₀is a positive definite matrix which can be selected by the designer.

Remark 2.1. Based on30,31, it can be established that the matrix Riccati equation2.12has a unique positive definite solutionPif the following conditions are satisfied;

aThe pairA, R^1/2is controllable, and the pairQ^1/2, Ais observable.

bThe following matrix inequality is fulfilled:

1 4

A^TR⁻¹−R⁻¹A R

A^TR⁻¹−R⁻¹A_T

≤A^TR⁻¹A−Q. 2.14

Both conditions can relatively easily be fulfilled ifAis selected as a stable diagonal matrix.

A7It exists a bounded controlut, such that the closed-loop system is quadratic stable, that is, it exists a Lyapunov functionV⁰>0 and a positive constantλsuch that

∂V⁰

∂x fxt, u_t, t≤ −λxt². 2.15 Additionally, the inequalityλ≥f₁Λ3must be satisfied.

Now, consider the learning law:

W˙_1,t−s_tK₁PΔtσ^TV1,tx_t s_tK₁PΔtx_t^TV_1,t^TD^T_σ, W˙2,t−stK2PΔtγ^Tutφ^TV2,txt stK2PΔtx^T_tV_2,t^T

s i1

γiutD^T_iφ ,

V˙_1,t−s_tK₃D_σ^TW_1,t^TPΔtx_t^T−s_tl1

2K₃Λ1V_1,tx_tx^T_t, V˙2,t−stK4

s i1

γiutD_iφ^T

W_2,t^TPΔtx^T_t −stsl2u

2 K4Λ2V2,txtx^T_t,

2.16

wheresis the number of columns corresponding toφ·, K1∈ ^n×n, K2∈ ^n×n, K3∈ ^m×m, andK₄∈ ^r×r are positive definite matrices which are selected by the designer.s_tis a dead- zone function which is defined as

st:

1− μ

P^1/2Δt

, z

⎧⎨

⎩

z z≥0, 0 z <0,

μ f0 Υ λ_min

P^−1/2Q₀P^−1/2.

2.17

Based on this learning law, the following result was demonstrated in26.

Theorem 2.2. If the assumptions (A1)–(A7) are satisfied and the weight matricesW1,t,W2,tV1,t, and V_2,tof the neural network2.2are adjusted by the learning law2.16, then

athe identification error and the weights are bounded:

Δt, W_1,t, W_2,t, V_1,t, V_2,t∈L_∞, 2.18

bthe identification errorΔtsatisfies the following tracking performance:

lim sup

T→ ∞

1 T

_T

0

1− f0 Υ

λmin

P^−1/2Q0P^−1/2P^1/2Δt

Δ^T_tQ₀Δtdt≤f₀ Υ. 2.19

In order to prove this result, the following nonnegative function was utilized:

V_t:V0P^1/2Δt−μ₂

tr

W_1,t^TK⁻¹₁ W_1,t tr

W_2,t^TK₂⁻¹W_2,t tr

V_1,t^TK₃⁻¹V_1,t tr

V_2,t^TK₄⁻¹V2,t

,

2.20

whereW_1,t:W_1,t−W₁⁰; W_2,t:W_2,t−W₂⁰.

3. Exponential Convergence of the Identification Process

Consider that the assumptionsA1–A3andA5-A6are still valid but the assumption A4is slightly modified as follows.

B4In a compact setΩ∈ ⁿ, unmodeled dynamicsftis bounded byft²_Λ3 ≤f0where f0is a constant not necessarily a priori known.

Remark 3.1. B4 is a common assumption in the neural network literature 17, 22. As mentioned inSection 2,ftis given byft:fxt, ut, t−Axt−W₁⁰σV₁⁰xt−W₂⁰φV₂⁰xtγut. Note thatW₁⁰σV₁⁰xtandW₂⁰φV₂⁰xtγutare bounded functions becauseσ·andφ·are sigmoidal functions. Asx_tbelongs toΩ, clearlyx_tis also bounded. Therefore, assumption B4 implies implicitly thatfxt, ut, tis a bounded function in a compact setΩ∈ ⁿ.

Although certainly assumption B4is more restrictive than assumption A4, from now on, assumptionA7is not needed anymore.

In this paper, the following modification to the learning law2.16is proposed:

W˙1,t−2k1PΔtσ^TV1,txt 2k1PΔtx^T_tV_1,t^TD^T_σ−α 2W1,t, W˙_2,t−2k₂PΔtγ^Tutφ^TV2,tx_t 2k₂PΔtx_t^TV_2,t^T

s i1

γ_iutD^T_iφ

−α 2W_2,t, V˙1,t−2k3D^T_σW_1,t^TPΔtx^T_t −k3l1Λ1V1,txtx_t^T− α

2V1,t, V˙2,t−2k4

s i1

γiutD^T_iφ

W_2,t^TPΔtx^T_t −k4sl2uΛ2V2,txtx^T_t −α 2V2,t,

3.1

where k1, k2, k3, and k4 are positive constants which are selected by the designer; P is the solution of the Riccati equation given by 2.12;α : λminP^−1/2Q0P^−1/2;s is the number of columns corresponding toφ·. By using the constants k₁, k₂, k₃, and k₄ in 3.1 instead of the matricesK₁, K₂, K₃, andK₄ in2.16, the tuning process of the neural network2.2 is simplified. Besides, none dead-zone function is now required. Based on the learning law 3.1, the following result is here established.

Theorem 3.2. If the assumptions (A1)–(A3), (B4), (A5)-(A6) are satisfied and the weight matrices W1,t,W2,t,V1,t, andV2,tof the neural network2.2are adjusted by the learning law3.1, then

athe identification error and the weights are bounded:

Δt, W1,t, W2,t, V1,t, V2,t∈L_∞, 3.2

bthe norm of identification error converges exponentially to a region bounded given by

tlim→ ∞xt−xt ≤

f0 Υ

αλ_minP. 3.3

Proof ofTheorem 3.2. Before beginning analysis, the dynamics of the identification error Δt

must be determined. The first derivative ofΔtis dΔt

dt d

dtxt−xt. 3.4

Note that an alternative representation for2.1could be calculated as follows:

˙

xtAxtW₁⁰σ V₁⁰xt

W₂⁰φ V₂⁰xt

γut ftξt. 3.5

Substituting2.2and3.5into3.4yields

Δ˙tAx_tW_1,tσV1,tx_t W_2,tφV2,tx_tγut−Ax_t−W₁⁰σ V₁⁰x_t

−W₂⁰φ V₂⁰x_t

γut−f_t−ξ_t, AΔtW1,tσV1,txt−W₁⁰σ

V₁⁰xt

W2,tφV2,txtγut

−W₂⁰φ V₂⁰xt

γut−ft−ξt.

3.6

Subtracting and adding the terms W₁⁰σV1,tx_t, W₁⁰σV₁⁰x_t, W₂⁰φV2,tx_tγut, and W₂⁰φV₂⁰x_tγutand considering thatW_1,t :W_1,t−W₁⁰,W_2,t :W_2,t−W₂⁰,σ_t : σV10x_t− σV10x_t, φ_t : φV20x_t − φV20x_t, σ_t : σV1,tx_t − σV10x_t, φ_tγut : φV2,tx_t − φV20xtγut,3.6can be expressed as

Δ˙tAΔtW1,tσV1,txt−W₁⁰σV1,txt W₁⁰σV1,txt−W₁⁰σ V₁⁰xt

W₁⁰σ V₁⁰xt

−W₁⁰σ V₁⁰xt

W2,tφV2,txtγut−W₂⁰φV2,txtγut W₂⁰φV2,txtγut

−W₂⁰φ V₂⁰xt

γut W₂⁰φ V₂⁰xt

γut−W₂⁰φ V₂⁰xt

γut−ft−ξt

AΔtW_1,tσV1,tx_t W₁⁰σ_tW₁⁰σ_tW_2,tφV2,tx_tγut W₂⁰φ_tγut W₂⁰φ_tγut−f_t−ξ_t,

Δ˙tAΔtW_1,tσV1,tx_t W_2,tφV2,tx_tγut W₁⁰σ_tW₂⁰φ_tγut W₁⁰σ_t W₂⁰φ_tγut−f_t−ξ_t.

3.7

In order to begin analysis, the following nonnegative function is selected:

V_t Δ^T_tPΔt 1 2k1

tr

W_1,t^TW_1,t 1

2k2

tr

W_2,t^TW_2,t

1 2k₃tr

V_1,t^TV1,t

1 2k₄ tr

V_2,t^TV2,t

,

3.8

where P is a positive solution for the Riccati matrix equation given by 2.12. The first derivative ofV_tis

V˙t d dt

Δ^T_tPΔt

d dt

1 2k₁tr

W_1,t^TW1,t

d dt

1 2k₂tr

W_2,t^TW2,t

d dt

1 2k3tr

V_1,t^TV1,t

d dt

1 2k4tr

V_2,t^TV2,t

.

3.9

Each term of3.9will be calculated separately. Ford/dtΔ^T_tPΔt, d

dt

Δ^T_tPΔt

2Δ^T_tPΔ˙t. 3.10

substituting3.7into3.10yields d

dt

Δ^T_tPΔt

2Δ^T_tP AΔt2Δ^T_tPW_1,tσV1,tx_t 2Δ^T_tPW_2,tφV2,tx_tγut

2Δ^T_tP W₁⁰σ_t2Δ^T_tP W₂⁰φ_tγut 2Δ^T_tP W₁⁰σ_t2Δ^T_tP W₂⁰φ_tγut

−2Δ^T_tPf_t−2Δ^T_tP ξ_t.

3.11

The terms 2Δ^T_tP W₁⁰σ_t, 2Δ^T_tP W₂⁰φ_tγut, −2Δ^T_tPf_t, and −2Δ^T_tP ξ_t in 3.11 can be bounded using the following matrix inequality proven in26:

X^TYY^TX≤X^TΓ⁻¹XY^TΓY, 3.12 which is valid for anyX, Y ∈ ^n×kand for any positive definite matrix 0 < Γ Γ^T ∈ ^n×n. Thus, for 2Δ^T_tP W₁⁰σ_tand considering assumptionA2,

2Δ^T_tP W₁⁰σ_t Δ^T_tP W₁⁰σ_tσ_t^T W₁⁰T

PΔt

≤Δ^T_tP W₁⁰Λ⁻¹₁ W₁⁰_T

PΔtσ^T_tΛ1σt

≤Δ^T_tP W₁⁰Λ⁻¹₁ W₁⁰_T

PΔt Δ^T_tΛσΔt.

3.13

For 2Δ^T_tP W₂⁰φtγut,and considering assumptionsA2andA3 2Δ^T_tP W₂⁰φtγut Δ^T_tP W₂⁰φtγut γ^Tutφ^T_t

W₂⁰T

PΔt

≤Δ^T_tP W₂⁰Λ⁻¹₂ W₂⁰_T

PΔtγ^Tutφ^T_tΛ2φ_tγut

≤Δ^T_tP W₂⁰Λ⁻¹₂ W₂⁰T

PΔtuΔ^T_tΛφΔt.

3.14

By using3.12and given assumptionsB4andA5,−2Δ^T_tPftand−2Δ^T_tP ξtcan be bounded, respectively, by

−2Δ^T_tPf_t−Δ^T_tPf_t−f_t^TPΔt≤Δ^T_tPΛ⁻¹₃ PΔtf_t^TΛ3f_t

≤Δ^T_tPΛ⁻¹₃ PΔtf₀,

−2Δ^T_tP ξt−Δ^T_tP ξt−ξ^T_tPΔt≤Δ^T_tPΛ⁻¹₄ PΔtξ^T_tΛ4ξt

≤Δ^T_tPΛ⁻¹₄ PΔt Υ.

3.15

Considering2.8, 2Δ^T_tP W₁⁰σtcan be developed as

2Δ^T_tP W₁⁰σ_t2Δ^T_tP W₁⁰D_σV_1,tx_t2Δ^T_tP W₁⁰ν_σ. 3.16

By simultaneously adding and subtracting the term 2Δ^T_tP W_1,tD_σV_1,tx_t into the right-hand side of3.16,

2Δ^T_tP W₁⁰σ_t2Δ^T_tP W_1,tD_σV_1,tx_t−2Δ^T_tPW_1,tD_σV_1,tx_t2Δ^T_tP W₁⁰ν_σ. 3.17

By using3.12and considering assumptionA2, the termΔ^T_tP W₁⁰νσcan be bounded as

2Δ^T_tP W₁⁰νσ Δ^T_tP W₁⁰νσν^T_σ W₁⁰_T

PΔt≤Δ^T_tP W₁⁰Λ⁻¹₁ W₁⁰_T

PΔtν_σ^TΛ1νσ

≤Δ^T_tP W₁⁰Λ⁻¹₁ W₁⁰_T

PΔtl₁V_1,tx_t²

Λ1

.

3.18

And consequently, 2Δ^T_tP W₁⁰σtis bounded by

2Δ^T_tP W₁⁰σ_t≤2Δ^T_tP W_1,tD_σV_1,tx_t−2Δ^T_tPW_1,tD_σV_1,tx_tΔ^T_tP W₁⁰Λ⁻¹₁ W₁⁰_T

PΔtl1V_1,tx_t²

Λ1

. 3.19

For 2Δ^T_tP W₂⁰φ_tγutand considering2.9,

2Δ^T_tP W₂⁰φ_tγut 2Δ^T_tP W₂⁰ s

i1

DiφV2,txtνiφ

γiut

2Δ^T_tP W₂⁰ s

i1

DiφV2,txtγiut 2Δ^T_tP W₂⁰ s

i1

νiφγiut.

3.20

Adding and subtracting the term 2Δ^T_tP W2,t_s

i1DiφV2,txtγiut into the right-hand side of 3.20,

2Δ^T_tP W₂⁰φ_tγut 2Δ^T_tP W_2,t s

i1

D_iφV_2,tx_tγ_iut−2Δ^T_tPW_2,t s

i1

D_iφV_2,tx_tγ_iut

2Δ^T_tP W₂⁰ s

i1

νiφγiut.

3.21

By using3.12, 2Δ^T_tP W₂⁰_s

i1ν_iφγ_iutcan be bounded by

2Δ^T_tP W₂⁰ s i1

ν_iφγ_iut Δ^T_tP W₂⁰ s

i1

ν_iφγ_iut

_s

i1

ν_iφγ_iut _T

W₂⁰_T PΔt

≤Δ^T_tP W₂⁰Λ⁻¹₂ W₂⁰_T

PΔt _s

i1

νiφγiut _T

Λ2

s i1

νiφγiut,

3.22

but considering that

_s

i1

νiφγiut _T

Λ2

s i1

νiφγiut≤s s

i1

γ_i²utν_iφ^TΛ2νiφ 3.23

and from assumptionsA2andA3, the following can be concluded:

2Δ^T_tP W₂⁰ s

i1

ν_iφγ_iut≤Δ^T_tP W₂⁰Λ⁻¹₂ W₂⁰_T

PΔtsl₂uV_2,tx_t²

Λ2

. 3.24

Thus, 2Δ^T_tP W₂⁰φ_tγutis bounded by

2Δ^T_tP W₂⁰φ_tγut≤2Δ^T_tP W_2,t s

i1

D_iφV_2,tx_tγ_iut−2Δ^T_tPW_2,t s

i1

D_iφV_2,tx_tγ_iut

Δ^T_tP W₂⁰Λ⁻¹₂ W₂⁰_T

PΔtsl₂uV_2,tx_t²

Λ2

.

3.25