Research ArticleAn Efficient Hybrid Conjugate Gradient Method withthe Strong Wolfe-Powell Line Search

Ahmad Alhawarat1 Mustafa Mamat2 Mohd Rivaie3 and Zabidin Salleh1

1School of Informatics and Applied Mathematics Universiti Malaysia Terengganu 21030 Kuala Terengganu Terengganu Malaysia2Faculty of Informatics and Computing Universiti Sultan Zainal Abidin 21300 Kuala Terengganu Terengganu Malaysia3Department of Computer Science and Mathematics Universiti Teknologi MARA (UITM) Terengganu Campus Kuala Terengganu21080 Kuala Terengganu Terengganu Malaysia

Correspondence should be addressed to Ahmad Alhawarat abadee2010yahoocom

Received 11 March 2015 Revised 6 July 2015 Accepted 7 July 2015

Academic Editor Haipeng Peng

Copyright copy 2015 Ahmad Alhawarat et al This is an open access article distributed under the Creative Commons AttributionLicense which permits unrestricted use distribution and reproduction in any medium provided the original work is properlycited

Conjugate gradient (CG) method is an interesting tool to solve optimization problems in many fields such as design economicsphysics and engineering In this paper we depict a new hybrid of CG method which relates to the famous Polak-Ribiere-Polyak(PRP) formula It reveals a solution for the PRP case which is not globally convergent with the strong Wolfe-Powell (SWP) linesearch The new formula possesses the sufficient descent condition and the global convergent properties In addition we furtherexplained about the cases where PRP method failed with SWP line search Furthermore we provide numerical computations forthe new hybrid CG method which is almost better than other related PRP formulas in both the number of iterations and the CPUtime under some standard test functions

1 Introduction

The nonlinear conjugate gradient (CG) method is a usefultool to find the minimum value for unconstrained optimiza-tion problems Consider the following form

min 119891 (119909) | 119909 isin 119877119899 (1)

where 119891 119877119899 rarr 119877 is continuously differentiable functionand its gradient is denoted by 119892(119909) = nabla119891(119909) The CGmethodis to find a sequence of points 119909

119896 119896 ge 1 starting from

initial point 1199091 isin 119877119899 which is given as the following iterative

formula

119909119896+1 = 119909119896 +120572119896119889119896 (2)

where 119909119896is the current iteration point and 120572

119896gt 0 is the step

length obtained by some line search The search direction 119889119896

is defined by

119889119896=

minus119892119896 119896 = 1

minus119892119896+ 120573119896119889119896minus1 119896 gt 1

(3)

where 119892119896= 119892(119909

119896) and 120573

119896is known as CG method formula

or coefficientTo find the step length (120572

119896) we can use the exact line

search which is given by

119891 (119909119896+120572119896119889119896) = min119891 (119909

119896+120572119889119896) 120572 ge 0 (4)

In fact (4) is not an effective line search since it needsheavy computations in function and gradient evaluationsTherefore we prefer to use inexpensive line searchThe strongWolfe-Powell (SWP) line search [1 2] which is given asfollows

119891 (119909119896+120572119896119889119896) le 119891 (119909

119896) + 120575120572

119896119892119879119896119889119896 (5)

100381610038161003816100381610038161003816119892 (119909119896+120572119896119889119896)119879

119889119896

100381610038161003816100381610038161003816le minus 120590

10038161003816100381610038161003816119892119879

119896119889119896

10038161003816100381610038161003816 (6)

where 0 lt 120575 lt 120590 lt 1 is to find an approximation of 120572119896where

the descent property (see (14))must be satisfied and no longersearching in the direction when 119909

119896is far from the solution

Thus by using SWP line search we inherit the advantagesof exact line search with inexpensive and low computational

Hindawi Publishing CorporationMathematical Problems in EngineeringVolume 2015 Article ID 103517 7 pageshttpdxdoiorg1011552015103517

2 Mathematical Problems in Engineering

cost However different choices of 120572119896and 119889

119896imply different

CG methods In fact the SWP line search is a strong versionof weak Wolfe-Powell (WWP) line search where the latter isgiven by (5) and

119892 (119909119896+120572119896119889119896)119879

119889119896ge 120590119892119879119896119889119896 (7)

The CG method has been developed recently based on itssimplicity numerical efficiency and low memory require-ments Thus it is used widely in engineering medical scienceand other fields As an application in engineering we canuse CG method to solve some real life problem similar tothat mentioned in [3] The CG method is limited for thefunctions where their gradient is availableThus the heuristicalgorithm [4] can be used as an alternative method to findthe solution for general functions A heuristic algorithm isto find an approximation solution for the objective functionswith accepted time In addition the heuristic algorithms canbe applied without using computers We refer the reader tosee some applications for this algorithm in [5ndash7]

The most popular formulas for 120573119896are Hestenes-Stiefel

(HS) [8] Fletcher-Reeves (FR) [9] Polak-Ribiere-Polyak(PRP) [10] andWei et al (WYL) [11] respectively as follows

120573HS119896=119892119879119896(119910119896minus1)

119889119879119896minus1119910119896minus1

(8)

120573FR119896=100381710038171003817100381711989211989610038171003817100381710038172

1003817100381710038171003817119892119896minus110038171003817100381710038172 (9)

120573PRP119896=119892119879119896119910119896minus1

1003817100381710038171003817119892119896minus110038171003817100381710038172 (10)

120573WYL119896

=119892119879119896(119892119896minus (1003817100381710038171003817119892119896

1003817100381710038171003817 1003817100381710038171003817119892119896minus1

1003817100381710038171003817) 119892119896minus1)1003817100381710038171003817119892119896minus1

10038171003817100381710038172 (11)

where 119910119896minus1 = 119892119896 minus 119892119896minus1

Hestenes-Stiefel [8] proposed the first formula for solvingthe quadratic functions in 1952 Fletcher and Reeves [9]presented the first formula (9) for nonlinear functions in1964 The convergence properties of FR method with exactline search were obtained by Zoutendijk [12] Al-Baali [13]proved that FR method is globally convergent with the SWPline search when 120590 lt 12 Later on Guanghui et al [14]extended the result to120590 le 12The global convergence of PRPmethod (10) with the exact line search was proved by Elijahand Ribiere in [10] Powell [15] gave out a counterexampleshowing that there exists nonconvex function where PRPmethod does not converge globally even when the exactline search is used Powell suggested using nonnegative PRPmethod to reveal this problem Gilbert and Nocedal [16]proved that nonnegative PRP (PRP+) method that is 120573

119896=

max120573PRP119896 0 is globally convergent under complicated line

searches However there is no guarantee that PRP+ is conver-gent with SWP line search for general nonlinear functions

Touati-Ahmed and Storey [17] suggest the following hybridmethod

120573TS119896=

120573PRP119896 if 0 le 120573PRP

119896le 120573FR119896

120573FR119896 otherwise

(12)

In 2006 Wei et al [11] presented a new positive CG method(11) which is quite similar to original PRP method whichhas been studied in both exact and inexact line search Manymodifications have appeared such as the following [18ndash20]respectively

120573VHS119896

=100381710038171003817100381711989211989610038171003817100381710038172minus (1003817100381710038171003817119892119896

1003817100381710038171003817 1003817100381710038171003817119892119896minus1

1003817100381710038171003817) 119892119896119892119896minus1119889119879119896minus1119910119896minus1

120573NPRP119896

=100381710038171003817100381711989211989610038171003817100381710038172minus (1003817100381710038171003817119892119896

1003817100381710038171003817 1003817100381710038171003817119892119896minus1

1003817100381710038171003817)1003816100381610038161003816119892119896119892119896minus1

10038161003816100381610038161003817100381710038171003817119892119896minus1

10038171003817100381710038172

120573DPRP119896

=100381710038171003817100381711989211989610038171003817100381710038172minus (1003817100381710038171003817119892119896

1003817100381710038171003817 1003817100381710038171003817119892119896minus1

1003817100381710038171003817)1003816100381610038161003816119892119896119892119896minus1

1003816100381610038161003816119898 1003816100381610038161003816119892119879

119896119889119896minus11003816100381610038161003816 +1003817100381710038171003817119892119896minus1

10038171003817100381710038172

where 119898 ge 1

(13)

Recently many CG formulas were constructed in order to getthe efficiency and robustness For more about the latest CGmethods we refer the reader to see [21 22]

One of the important rules in CG methods is the descentcondition that is if one can prove

119892119879119896119889119896lt 0 (14)

then we have a guarantee for119891(119909119896+1) lt 119891(119909119896) If we extended

(14) to the following form

119892119879119896119889119896le minus 119888 1003817100381710038171003817119892119896

10038171003817100381710038172 119896 ge 0 119888 gt 0 (15)

then (15) is called the sufficient descent conditionThis paper is organized as follows in Section 2 we will

present the current problem for PRP and nonnegative PRPmethod with SWP line search Later on we will suggest thenew hybrid CG formula and its simplifications Furthermorewe will establish the global convergence properties withthe SWP line search in Section 3 Numerical results withconclusion will be presented in Sections 4 and 5 respectively

2 Motivation and the Hybrid Formula

The PRP formula is one of the best CG methods in thiscentury However as we mentioned before this methodfails to solve some standard test problems for nonconvexfunctions even the exact line search is used Thus the maincontribution of this paper is to extend using PRP formula inseveral cases with SWP line search under mild condition andrestart the CG algorithm by using NPRP CG formula whenPRP failed to satisfy that condition

The following discussion illustrates the cases in whichPRP method fails and succeeds with SWP line search to

Mathematical Problems in Engineering 3

obtain the convergence propertiesThe PRPmethod could besimplified as follows

120573PRP119896=119892119879119896(119892119896minus 119892119896minus1)

1003817100381710038171003817119892119896minus110038171003817100381710038172 =

100381710038171003817100381711989211989610038171003817100381710038172minus 119892119879119896119892119896minus1

1003817100381710038171003817119892119896minus110038171003817100381710038172 (16)

Therefore we have the following cases

Case A If 119892119879119896119892119896minus1 gt 0 then we have the following two

possibilities

Case A1 If 1198921198962 gt 119892119879

119896119892119896minus1 then we have 0 le 120573PRP

119896le 120573FR119896 In

this case PRPmethod is efficient and has global convergenceproperties

Case A2 If 1198921198962 lt 119892119879

119896119892119896minus1 then we have 120573PRP

119896lt 0 In this

case based on [16] we fail to obtain the global convergenceproperties for nonconvex functions

Case B If 119892119879119896119892119896minus1 lt 0 then

120573PRP119896=119892119879119896(119892119896minus 119892119896minus1)

1003817100381710038171003817119892119896minus110038171003817100381710038172 =

100381710038171003817100381711989211989610038171003817100381710038172+10038161003816100381610038161003816119892119879

119896119892119896minus110038161003816100381610038161003816

1003817100381710038171003817119892119896minus110038171003817100381710038172

= 120573FR119896+

10038161003816100381610038161003816119892119879

119896119892119896minus110038161003816100381610038161003816

1003817100381710038171003817119892119896minus110038171003817100381710038172

(17)

In Case B there is no guarantee that this method will satisfythe sufficient descent condition

For the next discussion we will discuss the nonnegativePRP method which is given as follows

120573PRP+119896

= max 120573PRP119896 0 ie

120573PRP+119896

= max Case A1 Case A2 Case B 0 (18)

Therefore we have a problem inCase B To solve this problemGilbert and Nocedal [16] used another line search to satisfythe convergence properties In addition if 120573PRP+

119896= 0 the

CG method returns to the steepest descent method whichis sometimes a weak tool to find the optimum point forfunctions Furthermore we can notice that

120573TS119896=

Case A1

120573FR119896 otherwise

(19)

So from PRP method we can use only Case A1To improve the above ideas we suggest the following

hybrid method

120573PRPlowastlowast

119896=

120573PRP119896=119892119879119896(119892119896minus 119892119896minus1)

1003817100381710038171003817119892119896minus110038171003817100381710038172 if 1003817100381710038171003817119892119896

10038171003817100381710038172gt10038161003816100381610038161003816119892119879

119896119892119896minus110038161003816100381610038161003816

120573NPRP119896

=100381710038171003817100381711989211989610038171003817100381710038172minus (1003817100381710038171003817119892119896

1003817100381710038171003817 1003817100381710038171003817119892119896minus1

1003817100381710038171003817)1003816100381610038161003816119892119896119892119896minus1

10038161003816100381610038161003817100381710038171003817119892119896minus1

10038171003817100381710038172 otherwise

(20)

If 120573PRPlowastlowast

119896= (119892

1198962 minus 119892119879

119896119892119896minus1)119892119896minus1

2 then under thecondition 119892

1198962 gt |119892119879

119896119892119896minus1| we obtain

0 lt 120573PRPlowastlowast

119896=100381710038171003817100381711989211989610038171003817100381710038172minus 119892119879119896119892119896minus1

1003817100381710038171003817119892119896minus110038171003817100381710038172 le

100381710038171003817100381711989211989610038171003817100381710038172+10038161003816100381610038161003816119892119879

119896119892119896minus110038161003816100381610038161003816

1003817100381710038171003817119892119896minus110038171003817100381710038172

lt2 1003817100381710038171003817119892119896

10038171003817100381710038172

1003817100381710038171003817119892119896minus110038171003817100381710038172

(21)

And if

120573PRPlowastlowast

119896= 120573NPRP119896

=

100381710038171003817100381711989211989610038171003817100381710038172minus (1003817100381710038171003817119892119896

1003817100381710038171003817 1003817100381710038171003817119892119896minus1

1003817100381710038171003817)10038161003816100381610038161003816119892119879

119896119892119896minus110038161003816100381610038161003816

1003817100381710038171003817119892119896minus110038171003817100381710038172 (22)

we obtain

0 le 120573PRPlowastlowast

119896lt100381710038171003817100381711989211989610038171003817100381710038172

1003817100381710038171003817119892119896minus110038171003817100381710038172 = 120573

FR119896 (23)

One of the advantages for 120573PRPlowastlowast

119896is that we can use Case A1

and Case B under the condition 1198921198962 gt |119892119879

119896119892119896minus1|

Choosing 120573NPRP119896

as a restart CG formula in (20) if thecondition 119892

1198962 gt |119892119879

119896119892119896minus1| is not satisfied that means

100381710038171003817100381711989211989610038171003817100381710038172le10038161003816100381610038161003816119892119879

119896119892119896minus110038161003816100381610038161003816 (24)

Thus NPRP method is a suitable nonnegative value to useThe following algorithm is an algorithm of CG method

with the new coefficient 120573PRPlowastlowast

119896

Algorithm 1 Consider the followingStep 1 Initialization given 1199091 set 119896 = 1Step 2 Compute 120573

119896based on (20)

Step 3 Compute 119889119896based on (3)

Step 4 Compute 120572119896based on (5) and (6)

Step 5 Update new point based on (2)Step 6 Convergent test and stopping criteria if 119892

119896 le 120576 then

stop otherwise go to Step 2 with 119896 = 119896 + 1

4 Mathematical Problems in Engineering

3 The Global Convergence Properties for120573PRP

lowastlowast

119896with SWP Line Search

The following standard assumptions are necessary for thiswork

Assumption 1 The level set Χ = 119909 isin 119877119899 119891(119909) le 119891(1199091)with 1199091 to the starting point of the iterative method (2) isbounded

Assumption 2 In some open convex neighborhood119873 ofΧ119891is continuous and differentiable and its gradient is Lipschitzcontinuous that is for any 119909 119910 isin 119873 there exists a constant119871 gt 0 such that 119892(119909) minus 119892(119910) le 119871119909 minus 119910

The following lemma is one of the most important lem-maswhich is used to prove the global convergence properties

Lemma 2 (see [12]) Suppose Assumptions 1 and 2 are trueConsider any form of (2) and (3) with 120572

119896computed by WWP

line search direction 119889119896 is descent for all 119896 ge 1 theninfin

sum119896=1

cos2120579119896

100381710038171003817100381711989211989610038171003817100381710038172lt infin (25)

where

cos 120579119896= minus

(119892119879119896119889119896)

1003817100381710038171003817119889119896100381710038171003817100381710038171003817100381710038171198921198961003817100381710038171003817 (26)

Equation (25) is known as Zoutendijk condition

The following discussion will discuss the global conver-gence properties for 120573PRP

lowastlowast

119896with SWP line search

Case 1 (120573PRPlowastlowast

119896= 120573NPRP119896

) Since 0 le 120573NPRP119896

le 120573FR119896

the proof ofthe global convergence properties is similar to 120573FR

119896 We refer

the reader to see [13 14]

Case 2 (120573PRPlowastlowast

119896= 120573PRP119896

where 1198921198962 gt |119892119879

119896119892119896minus1|) In this case

we have

0 le 120573PRPlowastlowast

119896=100381710038171003817100381711989211989610038171003817100381710038172minus 119892119879119896119892119896minus1

1003817100381710038171003817119892119896minus110038171003817100381710038172 le

100381710038171003817100381711989211989610038171003817100381710038172+10038161003816100381610038161003816119892119879

119896119892119896minus110038161003816100381610038161003816

1003817100381710038171003817119892119896minus110038171003817100381710038172

le2 1003817100381710038171003817119892119896

10038171003817100381710038172

1003817100381710038171003817119892119896minus110038171003817100381710038172

(27)

The following theoremdemonstrates120573PRPlowastlowast

119896in Case 2 satisfies

the sufficient descent condition with SWP line search

Theorem 3 Consider the sequences 119892119896and 119889

119896are constructed

by Algorithm 1 and 120572119896is computed by (5) and (6) if 120590 isin

(0 14] Then (15) holds

Proof From (3) for 119896 = 1 it is obvious Suppose it is trueuntil 119896 minus 1 that is 119892119879

119896minus1119889119896minus1 = minus119892119896minus12 Multiply (3) by 119892

119896

we obtain

119892119879119896119889119896= 119892119879119896(minus119892119879119896+120573119896119889119896minus1) = minus

100381710038171003817100381711989211989610038171003817100381710038172+120573119896119892119879119896119889119896minus1 (28)

Dividing both sides by 119892119896minus1

2 and by using Case 2 and (6)then we obtain

minus 1+ 2120590119892119879119896minus1119889119896minus11003817100381710038171003817119892119896minus1

10038171003817100381710038172 le

119892119879119896119889119896

100381710038171003817100381711989211989610038171003817100381710038172 le minus 1minus 2120590

119892119879119896minus1119889119896minus11003817100381710038171003817119892119896minus1

10038171003817100381710038172 (29)

From (29) we obtain

minus119896minus1sum119895=0(2120590)119895 le

119892119879119896119889119896

100381710038171003817100381711989211989610038171003817100381710038172 le minus 2+

119896minus1sum119895=0(2120590)119895 (30)

Since119896minus1sum119895=0(2120590)119895 le 1 minus (2120590)119896

1 minus 2120590 (31)

we have

minus1 minus (2120590)119896

1 minus 2120590le119892119879119896119889119896

100381710038171003817100381711989211989610038171003817100381710038172 le minus 2+

1 minus (2120590)119896

1 minus 2120590 (32)

If 120590 le 14 we get

1 minus (2120590)119896

1 minus 2120590lt 2 (33)

Let

119888 = 2minus 1 minus (120590)119896

1 minus 120590 (34)

then

119888 minus 2 le119892119879119896119889119896

100381710038171003817100381711989211989610038171003817100381710038172 le minus 119888 (35)

Thus

119892119879119896119889119896le minus 119888 1003817100381710038171003817119892119896

10038171003817100381710038172 (36)

where 119888 isin (0 1) The proof is complete

Gilbert andNocedal [16] presented an important theorem(Theorem 4) to find the global convergence properties fornonnegative PRPmethod if the descent condition is satisfiedFurthermore [16] presented nice property called Propertylowast asfollows

119875119903119900119901119890119903119905119910lowast Consider a CG method of forms (2) and (3) andsuppose 0 lt 120574 le 119892

119896 le 120574 we say that the CG method

possesses Propertylowast if there exists constant 119887 gt 1 and 120582 gt 0such that for all 119896 ge 1 we get |120573

119896| le 119887 and if 119909

119896minus119909119896minus1 le 120582

we obtain

10038161003816100381610038161205731198961003816100381610038161003816 le

12119887 (37)

Theorem 4 Consider any CG method of forms (2) and (3)achieves the following properties

(I) 120573119896ge 0

(II) The sufficient descent condition (15) holds

Mathematical Problems in Engineering 5

(III) The Zoutendijk condition (25) is satisfied by the linesearch

(IV) 119875119903119900119901119890119903119905119910lowast holds

(V) Assumptions 1 and 2 hold

Then the iterates are globally convergent

The next lemma shows that if the gradients are boundedaway from zero and Propertylowast holds then a certain fractionof steps cannot be too small The proof is given in [16]However we state it for readability

Lemma 5 Consider a CG algorithm as defined in (2) and (3)with the parameter 120573PRP

lowastlowast

119896 If Assumptions 1 and 2 are satisfied

then 119875119903119900119901119890119903119905119910lowast holds

Proof Let 119887 = 212057421205742 ge 1 and 120582 le 12057422119871120574119887

100381610038161003816100381610038161003816120573PRP

lowastlowast

119896

100381610038161003816100381610038161003816le

1003816100381610038161003816100381610038161003816100381610038161003816

100381710038171003817100381711989211989610038171003817100381710038172minus 119892119879119896119892119896minus1

1003817100381710038171003817119892119896minus110038171003817100381710038172

1003816100381610038161003816100381610038161003816100381610038161003816le

100381710038171003817100381711989211989610038171003817100381710038172+10038161003816100381610038161003816119892119879

119896119892119896minus110038161003816100381610038161003816

1003817100381710038171003817119892119896minus110038171003817100381710038172

le10038171003817100381710038171198921198961003817100381710038171003817 (10038171003817100381710038171198921198961003817100381710038171003817 +1003817100381710038171003817119892119896minus1

1003817100381710038171003817)1003817100381710038171003817119892119896minus1

10038171003817100381710038172 le

21205742

1205742= 119887 gt 1

(38)

Using 119904119896 le 120582 we have

100381610038161003816100381610038161003816120573PRP

lowastlowast

119896

100381610038161003816100381610038161003816le

1003816100381610038161003816100381610038161003816100381610038161003816

119892119879119896(119892119896minus 119892119896minus1)

1003817100381710038171003817119892119896minus110038171003817100381710038172

1003816100381610038161003816100381610038161003816100381610038161003816le100381710038171003817100381711989211989610038171003817100381710038171003817100381710038171003817119892119896 minus 119892119896minus1

10038171003817100381710038171003817100381710038171003817119892119896minus1

10038171003817100381710038172 (39)

By Assumption 2

100381610038161003816100381610038161003816120573PRP

lowastlowast

119896

100381610038161003816100381610038161003816le119871120582120574

1205742le

12119887 (40)

The proof is complete

Lemma 6 The CG formula presented in Case 2 has thefollowing properties

(1) 120573PRPlowastlowast

119896gt 0 since the condition 119892

1198962 gt |119892119879

119896119892119896minus1| forces

the CG formula in (20) to be nonnegative

(2) 120573PRPlowastlowast

119896satisfies 119875119903119900119901119890119903119905119910lowast based on Lemma 5

(3) 120573PRPlowastlowast

119896satisfies the sufficient descent condition based on

Theorem 3 and 119888 isin (0 1)

By using Theorems 3 and 4 and Lemma 6 we havethe following convergence result The proof is similar toTheorem 43 which is presented in [16]

Theorem 7 Suppose that Assumption 1 holds Consider theCG method of forms (2) and (3) and 120573

119896as in Case 2

where 120572119896is computed by (5) and (6) with 120590 le 14 then

lim119896rarrinfin

inf 119892119896 = 0

t

e0

e1

e2

e3

Ps(t)

00

02

04

06

08

10

FR

FR

NPRPWYLVHS

VHSNPRPWYL

PRPlowastlowast

PRPlowastlowast

PRP+

PRP+

Figure 1 Performance profile based on the number of iterations

4 Numerical Results and Discussions

To evaluate the efficiency of the new method we selectedsome of the test functions in Table 1 from CUTEr [24]Neculai [23] and Adorio and Diliman [25] We performed acomparison with other CG methods including VHS NPRPPRP+ FR and PRPlowastlowast formulas The tolerance 120576 is selectedto 10minus6 for all algorithms to investigate the rapidity of theiterationmethods towards the optimal solutionThe gradientvalue is used as the stopping criteria Here the stoppingcriteria considered 119892

119896 le 10minus6 We considered the method

failed if the number of iterations exceeds 1000 timesIn Table 1 we selected different initial points for every

function Thus this demonstrated that this method can beused in several real life functions from other fields such asengineering and medical science as we mentioned before Inaddition different dimensions from 500 until 10000 are usedWe also choose from different group of functions We usedMatlab 79 subroutine program with CPU processor Intel (R)Core (TM) i3 CPU and 2GB DDR2 RAM under SWP linesearch to find the optimumpointThe efficiency comparisonsresults are shown in Figures 1 and 2 respectively using aperformance profile introduced by Dolan and More [26]

This performance measure was introduced to compare aset of solvers 119878 on a set of problems 119875 Assuming 119899

119904solvers

and 119899119901problems in 119878 and 119875 respectively the measure 119905

119901119904

is defined as the computation time (eg the number ofiterations or the CPU time) required to solve problem 119901 bysolver 119904

To create a baseline for comparison the performance ofsolver 119904 on problem119901 is scaled by the best performance of anysolver in 119878 on the problem using the ratio

119903119901119904=

119905119901119904

min 119905119901119904 119904 isin 119878

(41)

6 Mathematical Problems in Engineering

Table 1 A list of problem functions used with the SWP condition with 120575 = 001 and 120590 = 01

Number Function Dimensions Initial points

1 Extended White amp Holstfunction [23] 500 1000 5000 10000 (minus12 1 minus12 1 minus12 1) (5 5 5) (10 10 10) (15 15 15)

2 Extended Rosenbrockfunction [24] 500 1000 5000 10000 (minus12 1 minus12 1 minus12 1) (5 5 5) (10 10 10) (15 15 15)

3 Six-hump function [25] 2 (1 1) (5 5) (10 10) (15 15)

4 Extended Beale function[23] 500 1000 5000 10000 (minus1 minus1 minus1) (05 05 05) (10 10 10) (1 1 1)

5 Three-hump function [25] 2 (1 1) (5 5) (10 10) (15 15)

6 Extended Himmelblaufunction [23] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)

7 Diagonal 2 function [23] 500 1000 5000 10000 (02 02 02) (025 025 025) (05 05 05) (1 1 1)8 NONSCOMP function [24] 500 1000 5000 10000 (1 1 1) (minus1 minus1 minus1) (minus2 minus2 minus2) (minus5 minus5 minus5)

9 Extended DENSCHNBfunction [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)

10 Booth function [25] 2 (1 1) (5 5) (10 10) (15 15)

11 Extended quadratic penaltyfunction QP2 [24] 100 200 300 400 (2 2 2) (5 5 5) (10 10 10) (15 15 15)

12 DIXMAANA function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)13 DIXMAANB function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)14 DIXMAANC function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)15 DIXMAAND function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)16 DIXMAANE function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)17 DIXMAANF function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)18 DIXMAANG function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)19 NONDIA function [24] 500 1000 5000 10000 (minus2 minus2 minus2) (minus1 minus1 minus1) (0 0 0) (1 1 1)

20 Extended Tridiagonal 1function [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)

21 DQDRTIC function [24] 500 1000 5000 10000 (minus1 minus1 minus1) (1 1 1) (2 2 2) (3 3 3)22 Diagonal 4 function [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)23 DQDRTIC function [24] 10 20 30 40 (1 1 1) (minus1 minus1 minus1) (1 1 1) (2 2 2)

24 Extended Cliff function[24] 100 200 300 400 (1 1 1) (5 5 5) (10 10 10) (15 15 15)

Suppose that a parameter 119903119872ge 119903119901119904

for all 119901 119904 is chosen and119903119901119904= 119903119872

if and only if solver 119904 does not solve problem 119901Because we would like to obtain an overall assessment of theperformance of a solver we defined the measure

120588119904(119905) =

1119899119901

size 119901 isin119875 119903119901119904le 119905 (42)

Thus 120588119904(119905) is the probability for solver 119904 isin 119878 that the

performance ratio 119903119901119904

is within a factor 119905 isin 119877 of the bestpossible ratio If we define the function 119901

119904as the cumulative

distribution function for the performance ratio then theperformance measure 119901

119904 119877 rarr [0 1] for a solver is

nondecreasing and piecewise continuous from the right Thevalue of 119901

119904(1) is the probability that the solver has the best

performance of all of the solvers In general a solverwith highvalues of 119901(119905) which would appear in the upper right cornerof the figure is preferable

Based on the left side of Figures 1 and 2 the PRPlowastlowastformula is above the other curves Therefore it is the most

efficient method among related PRP methods in terms ofefficiency and robustness In Figure 2 we see that the curveof PRPlowastlowast is still the best but the efficiency is not goodas the number of iterations since we use the complicatedhybrid algorithm leads to high CPU time Thus using highprocessors computers to find the solution will be moreefficient since the number of iterations decreased rapidlyunder PRPlowastlowast method

5 Conclusion

In this paper we proposed hybrid conjugate gradient methodby using nonnegative PRP and NPRP formulas with the SWPline search which extended the cases of using PRP methodunder mild condition The global convergence property isestablished and it is very simple Our numerical results hadshown that the hybrid method is the best when compared toother related PRP CG methods

Mathematical Problems in Engineering 7

t

e0

e1

e2

e3

Ps(t)

00

02

04

06

08

10

FR

FR

NPRP

VHSNPRPWYL

WYLVHS

PRPlowastlowast

PRPlowastlowast

PRP+

PRP+

Figure 2 Performance profile based on the CPU time

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgments

The authors are grateful to the editor and the anony-mous reviewers for their comments and suggestions whichimproved this paper substantially They would also like tothank The Ministry of Education Malaysia (MOE) for fund-ing this research under The Fundamental Research GrantScheme (Grant no 59256)

References

[1] P Wolfe ldquoConvergence conditions for ascent methodsrdquo SIAMReview vol 11 no 2 pp 226ndash235 1969

[2] PWolfe ldquoConvergence conditions for ascent methods II somecorrectionsrdquo SIAM Review vol 13 no 2 pp 185ndash188 1971

[3] J CaiQ Li L LiH Peng andYYang ldquoA fuzzy adaptive chaoticant swarm optimization for economic dispatchrdquo InternationalJournal of Electrical Power and Energy Systems vol 34 no 1 pp154ndash160 2012

[4] KNatalliaAn Introduction toHeuristic Slgorithms Departmentof Informatics and Telecommunications 2005

[5] L LiH Peng J Kurths Y Yang andH J Schellnhuber ldquoChaos-order transition in foraging behavior of antsrdquo Proceedings of theNational Academy of Sciences of the United States of Americavol 111 no 23 pp 8392ndash8397 2014

[6] M Wan L Li J Xiao C Wang and Y Yang ldquoData clusteringusing bacterial foraging optimizationrdquo Journal of IntelligentInformation Systems vol 38 no 2 pp 321ndash341 2012

[7] M Wan C Wang L Li and Y Yang ldquoChaotic ant swarmapproach for data clusteringrdquo Applied Soft Computing Journalvol 12 no 8 pp 2387ndash2393 2012

[8] M R Hestenes and E Stiefel ldquoMethods of conjugate gradientsfor solving linear systemsrdquo Journal of Research of the NationalBureau of Standards vol 49 no 6 pp 409ndash436 1952

[9] R Fletcher and C M Reeves ldquoFunction minimization byconjugate gradientsrdquo The Computer Journal vol 7 no 2 pp149ndash154 1964

[10] P Elijah and G Ribiere ldquoNote sur la convergence de methodesde directions conjugueesrdquo Revue francaise drsquoinformatique et derecherche operationnelle vol 3 no 1 pp 35ndash43 1969

[11] Z Wei S Yao and L Liu ldquoThe convergence properties of somenew conjugate gradient methodsrdquo Applied Mathematics andComputation vol 183 no 2 pp 1341ndash1350 2006

[12] G Zoutendijk ldquoNonlinear programming computationalmeth-odsrdquo Integer and Nonlinear Programming vol 143 no 1 pp 37ndash86 1970

[13] M Al-Baali ldquoDescent property and global convergence ofthe FletchermdashReeves method with inexact line searchrdquo IMAJournal of Numerical Analysis vol 5 no 1 pp 121ndash124 1985

[14] L Guanghui H Jiye and Y Hongxia ldquoGlobal convergence ofthe fletcher-reeves algorithm with inexact linesearchrdquo AppliedMathematics-A Journal of Chinese Universities vol 10 no 1 pp75ndash82 1995

[15] M J D Powell ldquoNonconvex minimization calculations and theconjugate gradient methodrdquo in Numerical Analysis vol 1066of Lecture Notes in Mathematics pp 122ndash141 Springer BerlinGermany 1984

[16] J C Gilbert and J Nocedal ldquoGlobal convergence properties ofconjugate gradient methods for optimizationrdquo SIAM Journal onOptimization vol 2 no 1 pp 21ndash42 1992

[17] D Touati-Ahmed and C Storey ldquoEfficient hybrid conjugategradient techniquesrdquo Journal of OptimizationTheory and Appli-cations vol 64 no 2 pp 379ndash397 1990

[18] Y Shengwei ZWei andH Huang ldquoA note aboutWYLrsquos conju-gate gradientmethod and its applicationsrdquoAppliedMathematicsand Computation vol 191 no 2 pp 381ndash388 2007

[19] L Zhang ldquoAn improved Wei-Yao-Liu nonlinear conjugategradient method for optimization computationrdquoApplied Math-ematics and Computation vol 215 no 6 pp 2269ndash2274 2009

[20] Z Dai and F Wen ldquoAnother improvedWeindashYaondashLiu nonlinearconjugate gradient method with sufficient descent propertyrdquoAppliedMathematics andComputation vol 218 no 14 pp 7421ndash7430 2012

[21] M Rivaie M Mamat L W June and I Mohd ldquoA new classof nonlinear conjugate gradient coefficients with global conver-gence propertiesrdquo Applied Mathematics and Computation vol218 no 22 pp 11323ndash11332 2012

[22] A AlhawaratMMamatM Rivaie and IMohd ldquoA newmodi-fication of nonlinear conjugate gradient coefficients with globalconvergence propertiesrdquo International Journal of MathematicalComputational Statistical Natural and Physical Engineeringvol 8 no 1 pp 54ndash60 2014

[23] A Neculai ldquoAn unconstrained optimization test functionscollectionrdquo Advanced Modeling and Optimization vol 10 no 1pp 147ndash161 2008

[24] I Bongartz A R Conn N Gould P L Toint and I Bon-gartz Constrained and Unconstrained Testing EnvironmentDepartement de Mathematique 1993

[25] E P Adorio and U Diliman ldquoMvf-multivariate test functionslibrary in c for unconstrained global optimizationrdquo 2005

[26] E D Dolan and J J More ldquoBenchmarking optimization soft-ware with performance profilesrdquo Mathematical Programmingvol 91 no 2 pp 201ndash213 2002

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

2 Mathematical Problems in Engineering

cost However different choices of 120572119896and 119889

119896imply different

CG methods In fact the SWP line search is a strong versionof weak Wolfe-Powell (WWP) line search where the latter isgiven by (5) and

119892 (119909119896+120572119896119889119896)119879

119889119896ge 120590119892119879119896119889119896 (7)

The CG method has been developed recently based on itssimplicity numerical efficiency and low memory require-ments Thus it is used widely in engineering medical scienceand other fields As an application in engineering we canuse CG method to solve some real life problem similar tothat mentioned in [3] The CG method is limited for thefunctions where their gradient is availableThus the heuristicalgorithm [4] can be used as an alternative method to findthe solution for general functions A heuristic algorithm isto find an approximation solution for the objective functionswith accepted time In addition the heuristic algorithms canbe applied without using computers We refer the reader tosee some applications for this algorithm in [5ndash7]

The most popular formulas for 120573119896are Hestenes-Stiefel

(HS) [8] Fletcher-Reeves (FR) [9] Polak-Ribiere-Polyak(PRP) [10] andWei et al (WYL) [11] respectively as follows

120573HS119896=119892119879119896(119910119896minus1)

119889119879119896minus1119910119896minus1

(8)

120573FR119896=100381710038171003817100381711989211989610038171003817100381710038172

1003817100381710038171003817119892119896minus110038171003817100381710038172 (9)

120573PRP119896=119892119879119896119910119896minus1

1003817100381710038171003817119892119896minus110038171003817100381710038172 (10)

120573WYL119896

=119892119879119896(119892119896minus (1003817100381710038171003817119892119896

1003817100381710038171003817 1003817100381710038171003817119892119896minus1

1003817100381710038171003817) 119892119896minus1)1003817100381710038171003817119892119896minus1

10038171003817100381710038172 (11)

where 119910119896minus1 = 119892119896 minus 119892119896minus1

Hestenes-Stiefel [8] proposed the first formula for solvingthe quadratic functions in 1952 Fletcher and Reeves [9]presented the first formula (9) for nonlinear functions in1964 The convergence properties of FR method with exactline search were obtained by Zoutendijk [12] Al-Baali [13]proved that FR method is globally convergent with the SWPline search when 120590 lt 12 Later on Guanghui et al [14]extended the result to120590 le 12The global convergence of PRPmethod (10) with the exact line search was proved by Elijahand Ribiere in [10] Powell [15] gave out a counterexampleshowing that there exists nonconvex function where PRPmethod does not converge globally even when the exactline search is used Powell suggested using nonnegative PRPmethod to reveal this problem Gilbert and Nocedal [16]proved that nonnegative PRP (PRP+) method that is 120573

119896=

max120573PRP119896 0 is globally convergent under complicated line

searches However there is no guarantee that PRP+ is conver-gent with SWP line search for general nonlinear functions

Touati-Ahmed and Storey [17] suggest the following hybridmethod

120573TS119896=

120573PRP119896 if 0 le 120573PRP

119896le 120573FR119896

120573FR119896 otherwise

(12)

In 2006 Wei et al [11] presented a new positive CG method(11) which is quite similar to original PRP method whichhas been studied in both exact and inexact line search Manymodifications have appeared such as the following [18ndash20]respectively

120573VHS119896

=100381710038171003817100381711989211989610038171003817100381710038172minus (1003817100381710038171003817119892119896

1003817100381710038171003817 1003817100381710038171003817119892119896minus1

1003817100381710038171003817) 119892119896119892119896minus1119889119879119896minus1119910119896minus1

120573NPRP119896

=100381710038171003817100381711989211989610038171003817100381710038172minus (1003817100381710038171003817119892119896

1003817100381710038171003817 1003817100381710038171003817119892119896minus1

1003817100381710038171003817)1003816100381610038161003816119892119896119892119896minus1

10038161003816100381610038161003817100381710038171003817119892119896minus1

10038171003817100381710038172

120573DPRP119896

=100381710038171003817100381711989211989610038171003817100381710038172minus (1003817100381710038171003817119892119896

1003817100381710038171003817 1003817100381710038171003817119892119896minus1

1003817100381710038171003817)1003816100381610038161003816119892119896119892119896minus1

1003816100381610038161003816119898 1003816100381610038161003816119892119879

119896119889119896minus11003816100381610038161003816 +1003817100381710038171003817119892119896minus1

10038171003817100381710038172

where 119898 ge 1

(13)

Recently many CG formulas were constructed in order to getthe efficiency and robustness For more about the latest CGmethods we refer the reader to see [21 22]

One of the important rules in CG methods is the descentcondition that is if one can prove

119892119879119896119889119896lt 0 (14)

then we have a guarantee for119891(119909119896+1) lt 119891(119909119896) If we extended

(14) to the following form

119892119879119896119889119896le minus 119888 1003817100381710038171003817119892119896

10038171003817100381710038172 119896 ge 0 119888 gt 0 (15)

then (15) is called the sufficient descent conditionThis paper is organized as follows in Section 2 we will

present the current problem for PRP and nonnegative PRPmethod with SWP line search Later on we will suggest thenew hybrid CG formula and its simplifications Furthermorewe will establish the global convergence properties withthe SWP line search in Section 3 Numerical results withconclusion will be presented in Sections 4 and 5 respectively

2 Motivation and the Hybrid Formula

The PRP formula is one of the best CG methods in thiscentury However as we mentioned before this methodfails to solve some standard test problems for nonconvexfunctions even the exact line search is used Thus the maincontribution of this paper is to extend using PRP formula inseveral cases with SWP line search under mild condition andrestart the CG algorithm by using NPRP CG formula whenPRP failed to satisfy that condition

The following discussion illustrates the cases in whichPRP method fails and succeeds with SWP line search to

Mathematical Problems in Engineering 3

obtain the convergence propertiesThe PRPmethod could besimplified as follows

120573PRP119896=119892119879119896(119892119896minus 119892119896minus1)

1003817100381710038171003817119892119896minus110038171003817100381710038172 =

100381710038171003817100381711989211989610038171003817100381710038172minus 119892119879119896119892119896minus1

1003817100381710038171003817119892119896minus110038171003817100381710038172 (16)

Therefore we have the following cases

Case A If 119892119879119896119892119896minus1 gt 0 then we have the following two

possibilities

Case A1 If 1198921198962 gt 119892119879

119896119892119896minus1 then we have 0 le 120573PRP

119896le 120573FR119896 In

this case PRPmethod is efficient and has global convergenceproperties

Case A2 If 1198921198962 lt 119892119879

119896119892119896minus1 then we have 120573PRP

119896lt 0 In this

case based on [16] we fail to obtain the global convergenceproperties for nonconvex functions

Case B If 119892119879119896119892119896minus1 lt 0 then

120573PRP119896=119892119879119896(119892119896minus 119892119896minus1)

1003817100381710038171003817119892119896minus110038171003817100381710038172 =

100381710038171003817100381711989211989610038171003817100381710038172+10038161003816100381610038161003816119892119879

119896119892119896minus110038161003816100381610038161003816

1003817100381710038171003817119892119896minus110038171003817100381710038172

= 120573FR119896+

10038161003816100381610038161003816119892119879

119896119892119896minus110038161003816100381610038161003816

1003817100381710038171003817119892119896minus110038171003817100381710038172

(17)

In Case B there is no guarantee that this method will satisfythe sufficient descent condition

For the next discussion we will discuss the nonnegativePRP method which is given as follows

120573PRP+119896

= max 120573PRP119896 0 ie

120573PRP+119896

= max Case A1 Case A2 Case B 0 (18)

Therefore we have a problem inCase B To solve this problemGilbert and Nocedal [16] used another line search to satisfythe convergence properties In addition if 120573PRP+

119896= 0 the

CG method returns to the steepest descent method whichis sometimes a weak tool to find the optimum point forfunctions Furthermore we can notice that

120573TS119896=

Case A1

120573FR119896 otherwise

(19)

So from PRP method we can use only Case A1To improve the above ideas we suggest the following

hybrid method

120573PRPlowastlowast

119896=

120573PRP119896=119892119879119896(119892119896minus 119892119896minus1)

1003817100381710038171003817119892119896minus110038171003817100381710038172 if 1003817100381710038171003817119892119896

10038171003817100381710038172gt10038161003816100381610038161003816119892119879

119896119892119896minus110038161003816100381610038161003816

120573NPRP119896

=100381710038171003817100381711989211989610038171003817100381710038172minus (1003817100381710038171003817119892119896

1003817100381710038171003817 1003817100381710038171003817119892119896minus1

1003817100381710038171003817)1003816100381610038161003816119892119896119892119896minus1

10038161003816100381610038161003817100381710038171003817119892119896minus1

10038171003817100381710038172 otherwise

(20)

If 120573PRPlowastlowast

119896= (119892

1198962 minus 119892119879

119896119892119896minus1)119892119896minus1

2 then under thecondition 119892

1198962 gt |119892119879

119896119892119896minus1| we obtain

0 lt 120573PRPlowastlowast

119896=100381710038171003817100381711989211989610038171003817100381710038172minus 119892119879119896119892119896minus1

1003817100381710038171003817119892119896minus110038171003817100381710038172 le

100381710038171003817100381711989211989610038171003817100381710038172+10038161003816100381610038161003816119892119879

119896119892119896minus110038161003816100381610038161003816

1003817100381710038171003817119892119896minus110038171003817100381710038172

lt2 1003817100381710038171003817119892119896

10038171003817100381710038172

1003817100381710038171003817119892119896minus110038171003817100381710038172

(21)

And if

120573PRPlowastlowast

119896= 120573NPRP119896

=

100381710038171003817100381711989211989610038171003817100381710038172minus (1003817100381710038171003817119892119896

1003817100381710038171003817 1003817100381710038171003817119892119896minus1

1003817100381710038171003817)10038161003816100381610038161003816119892119879

119896119892119896minus110038161003816100381610038161003816

1003817100381710038171003817119892119896minus110038171003817100381710038172 (22)

we obtain

0 le 120573PRPlowastlowast

119896lt100381710038171003817100381711989211989610038171003817100381710038172

1003817100381710038171003817119892119896minus110038171003817100381710038172 = 120573

FR119896 (23)

One of the advantages for 120573PRPlowastlowast

119896is that we can use Case A1

and Case B under the condition 1198921198962 gt |119892119879

119896119892119896minus1|

Choosing 120573NPRP119896

as a restart CG formula in (20) if thecondition 119892

1198962 gt |119892119879

119896119892119896minus1| is not satisfied that means

100381710038171003817100381711989211989610038171003817100381710038172le10038161003816100381610038161003816119892119879

119896119892119896minus110038161003816100381610038161003816 (24)

Thus NPRP method is a suitable nonnegative value to useThe following algorithm is an algorithm of CG method

with the new coefficient 120573PRPlowastlowast

119896

Algorithm 1 Consider the followingStep 1 Initialization given 1199091 set 119896 = 1Step 2 Compute 120573

119896based on (20)

Step 3 Compute 119889119896based on (3)

Step 4 Compute 120572119896based on (5) and (6)

Step 5 Update new point based on (2)Step 6 Convergent test and stopping criteria if 119892

119896 le 120576 then

stop otherwise go to Step 2 with 119896 = 119896 + 1

4 Mathematical Problems in Engineering

3 The Global Convergence Properties for120573PRP

lowastlowast

119896with SWP Line Search

The following standard assumptions are necessary for thiswork

Assumption 1 The level set Χ = 119909 isin 119877119899 119891(119909) le 119891(1199091)with 1199091 to the starting point of the iterative method (2) isbounded

Assumption 2 In some open convex neighborhood119873 ofΧ119891is continuous and differentiable and its gradient is Lipschitzcontinuous that is for any 119909 119910 isin 119873 there exists a constant119871 gt 0 such that 119892(119909) minus 119892(119910) le 119871119909 minus 119910

The following lemma is one of the most important lem-maswhich is used to prove the global convergence properties

Lemma 2 (see [12]) Suppose Assumptions 1 and 2 are trueConsider any form of (2) and (3) with 120572

119896computed by WWP

line search direction 119889119896 is descent for all 119896 ge 1 theninfin

sum119896=1

cos2120579119896

100381710038171003817100381711989211989610038171003817100381710038172lt infin (25)

where

cos 120579119896= minus

(119892119879119896119889119896)

1003817100381710038171003817119889119896100381710038171003817100381710038171003817100381710038171198921198961003817100381710038171003817 (26)

Equation (25) is known as Zoutendijk condition

The following discussion will discuss the global conver-gence properties for 120573PRP

lowastlowast

119896with SWP line search

Case 1 (120573PRPlowastlowast

119896= 120573NPRP119896

) Since 0 le 120573NPRP119896

le 120573FR119896

the proof ofthe global convergence properties is similar to 120573FR

119896 We refer

the reader to see [13 14]

Case 2 (120573PRPlowastlowast

119896= 120573PRP119896

where 1198921198962 gt |119892119879

119896119892119896minus1|) In this case

we have

0 le 120573PRPlowastlowast

119896=100381710038171003817100381711989211989610038171003817100381710038172minus 119892119879119896119892119896minus1

1003817100381710038171003817119892119896minus110038171003817100381710038172 le

100381710038171003817100381711989211989610038171003817100381710038172+10038161003816100381610038161003816119892119879

119896119892119896minus110038161003816100381610038161003816

1003817100381710038171003817119892119896minus110038171003817100381710038172

le2 1003817100381710038171003817119892119896

10038171003817100381710038172

1003817100381710038171003817119892119896minus110038171003817100381710038172

(27)

The following theoremdemonstrates120573PRPlowastlowast

119896in Case 2 satisfies

the sufficient descent condition with SWP line search

Theorem 3 Consider the sequences 119892119896and 119889

119896are constructed

by Algorithm 1 and 120572119896is computed by (5) and (6) if 120590 isin

(0 14] Then (15) holds

Proof From (3) for 119896 = 1 it is obvious Suppose it is trueuntil 119896 minus 1 that is 119892119879

119896minus1119889119896minus1 = minus119892119896minus12 Multiply (3) by 119892

119896

we obtain

119892119879119896119889119896= 119892119879119896(minus119892119879119896+120573119896119889119896minus1) = minus

100381710038171003817100381711989211989610038171003817100381710038172+120573119896119892119879119896119889119896minus1 (28)

Dividing both sides by 119892119896minus1

2 and by using Case 2 and (6)then we obtain

minus 1+ 2120590119892119879119896minus1119889119896minus11003817100381710038171003817119892119896minus1

10038171003817100381710038172 le

119892119879119896119889119896

100381710038171003817100381711989211989610038171003817100381710038172 le minus 1minus 2120590

119892119879119896minus1119889119896minus11003817100381710038171003817119892119896minus1

10038171003817100381710038172 (29)

From (29) we obtain

minus119896minus1sum119895=0(2120590)119895 le

119892119879119896119889119896

100381710038171003817100381711989211989610038171003817100381710038172 le minus 2+

119896minus1sum119895=0(2120590)119895 (30)

Since119896minus1sum119895=0(2120590)119895 le 1 minus (2120590)119896

1 minus 2120590 (31)

we have

minus1 minus (2120590)119896

1 minus 2120590le119892119879119896119889119896

100381710038171003817100381711989211989610038171003817100381710038172 le minus 2+

1 minus (2120590)119896

1 minus 2120590 (32)

If 120590 le 14 we get

1 minus (2120590)119896

1 minus 2120590lt 2 (33)

Let

119888 = 2minus 1 minus (120590)119896

1 minus 120590 (34)

then

119888 minus 2 le119892119879119896119889119896

100381710038171003817100381711989211989610038171003817100381710038172 le minus 119888 (35)

Thus

119892119879119896119889119896le minus 119888 1003817100381710038171003817119892119896

10038171003817100381710038172 (36)

where 119888 isin (0 1) The proof is complete

Gilbert andNocedal [16] presented an important theorem(Theorem 4) to find the global convergence properties fornonnegative PRPmethod if the descent condition is satisfiedFurthermore [16] presented nice property called Propertylowast asfollows

119875119903119900119901119890119903119905119910lowast Consider a CG method of forms (2) and (3) andsuppose 0 lt 120574 le 119892

119896 le 120574 we say that the CG method

possesses Propertylowast if there exists constant 119887 gt 1 and 120582 gt 0such that for all 119896 ge 1 we get |120573

119896| le 119887 and if 119909

119896minus119909119896minus1 le 120582

we obtain

10038161003816100381610038161205731198961003816100381610038161003816 le

12119887 (37)

Theorem 4 Consider any CG method of forms (2) and (3)achieves the following properties

(I) 120573119896ge 0

(II) The sufficient descent condition (15) holds

Mathematical Problems in Engineering 5

(III) The Zoutendijk condition (25) is satisfied by the linesearch

(IV) 119875119903119900119901119890119903119905119910lowast holds

(V) Assumptions 1 and 2 hold

Then the iterates are globally convergent

The next lemma shows that if the gradients are boundedaway from zero and Propertylowast holds then a certain fractionof steps cannot be too small The proof is given in [16]However we state it for readability

Lemma 5 Consider a CG algorithm as defined in (2) and (3)with the parameter 120573PRP

lowastlowast

119896 If Assumptions 1 and 2 are satisfied

then 119875119903119900119901119890119903119905119910lowast holds

Proof Let 119887 = 212057421205742 ge 1 and 120582 le 12057422119871120574119887

100381610038161003816100381610038161003816120573PRP

lowastlowast

119896

100381610038161003816100381610038161003816le

1003816100381610038161003816100381610038161003816100381610038161003816

100381710038171003817100381711989211989610038171003817100381710038172minus 119892119879119896119892119896minus1

1003817100381710038171003817119892119896minus110038171003817100381710038172

1003816100381610038161003816100381610038161003816100381610038161003816le

100381710038171003817100381711989211989610038171003817100381710038172+10038161003816100381610038161003816119892119879

119896119892119896minus110038161003816100381610038161003816

1003817100381710038171003817119892119896minus110038171003817100381710038172

le10038171003817100381710038171198921198961003817100381710038171003817 (10038171003817100381710038171198921198961003817100381710038171003817 +1003817100381710038171003817119892119896minus1

1003817100381710038171003817)1003817100381710038171003817119892119896minus1

10038171003817100381710038172 le

21205742

1205742= 119887 gt 1

(38)

Using 119904119896 le 120582 we have

100381610038161003816100381610038161003816120573PRP

lowastlowast

119896

100381610038161003816100381610038161003816le

1003816100381610038161003816100381610038161003816100381610038161003816

119892119879119896(119892119896minus 119892119896minus1)

1003817100381710038171003817119892119896minus110038171003817100381710038172

1003816100381610038161003816100381610038161003816100381610038161003816le100381710038171003817100381711989211989610038171003817100381710038171003817100381710038171003817119892119896 minus 119892119896minus1

10038171003817100381710038171003817100381710038171003817119892119896minus1

10038171003817100381710038172 (39)

By Assumption 2

100381610038161003816100381610038161003816120573PRP

lowastlowast

119896

100381610038161003816100381610038161003816le119871120582120574

1205742le

12119887 (40)

The proof is complete

Lemma 6 The CG formula presented in Case 2 has thefollowing properties

(1) 120573PRPlowastlowast

119896gt 0 since the condition 119892

1198962 gt |119892119879

119896119892119896minus1| forces

the CG formula in (20) to be nonnegative

(2) 120573PRPlowastlowast

119896satisfies 119875119903119900119901119890119903119905119910lowast based on Lemma 5

(3) 120573PRPlowastlowast

119896satisfies the sufficient descent condition based on

Theorem 3 and 119888 isin (0 1)

By using Theorems 3 and 4 and Lemma 6 we havethe following convergence result The proof is similar toTheorem 43 which is presented in [16]

Theorem 7 Suppose that Assumption 1 holds Consider theCG method of forms (2) and (3) and 120573

119896as in Case 2

where 120572119896is computed by (5) and (6) with 120590 le 14 then

lim119896rarrinfin

inf 119892119896 = 0

t

e0

e1

e2

e3

Ps(t)

00

02

04

06

08

10

FR

FR

NPRPWYLVHS

VHSNPRPWYL

PRPlowastlowast

PRPlowastlowast

PRP+

PRP+

Figure 1 Performance profile based on the number of iterations

4 Numerical Results and Discussions

To evaluate the efficiency of the new method we selectedsome of the test functions in Table 1 from CUTEr [24]Neculai [23] and Adorio and Diliman [25] We performed acomparison with other CG methods including VHS NPRPPRP+ FR and PRPlowastlowast formulas The tolerance 120576 is selectedto 10minus6 for all algorithms to investigate the rapidity of theiterationmethods towards the optimal solutionThe gradientvalue is used as the stopping criteria Here the stoppingcriteria considered 119892

119896 le 10minus6 We considered the method

failed if the number of iterations exceeds 1000 timesIn Table 1 we selected different initial points for every

function Thus this demonstrated that this method can beused in several real life functions from other fields such asengineering and medical science as we mentioned before Inaddition different dimensions from 500 until 10000 are usedWe also choose from different group of functions We usedMatlab 79 subroutine program with CPU processor Intel (R)Core (TM) i3 CPU and 2GB DDR2 RAM under SWP linesearch to find the optimumpointThe efficiency comparisonsresults are shown in Figures 1 and 2 respectively using aperformance profile introduced by Dolan and More [26]

This performance measure was introduced to compare aset of solvers 119878 on a set of problems 119875 Assuming 119899

119904solvers

and 119899119901problems in 119878 and 119875 respectively the measure 119905

119901119904

is defined as the computation time (eg the number ofiterations or the CPU time) required to solve problem 119901 bysolver 119904

To create a baseline for comparison the performance ofsolver 119904 on problem119901 is scaled by the best performance of anysolver in 119878 on the problem using the ratio

119903119901119904=

119905119901119904

min 119905119901119904 119904 isin 119878

(41)

6 Mathematical Problems in Engineering

Table 1 A list of problem functions used with the SWP condition with 120575 = 001 and 120590 = 01

Number Function Dimensions Initial points

1 Extended White amp Holstfunction [23] 500 1000 5000 10000 (minus12 1 minus12 1 minus12 1) (5 5 5) (10 10 10) (15 15 15)

2 Extended Rosenbrockfunction [24] 500 1000 5000 10000 (minus12 1 minus12 1 minus12 1) (5 5 5) (10 10 10) (15 15 15)

3 Six-hump function [25] 2 (1 1) (5 5) (10 10) (15 15)

4 Extended Beale function[23] 500 1000 5000 10000 (minus1 minus1 minus1) (05 05 05) (10 10 10) (1 1 1)

5 Three-hump function [25] 2 (1 1) (5 5) (10 10) (15 15)

6 Extended Himmelblaufunction [23] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)

7 Diagonal 2 function [23] 500 1000 5000 10000 (02 02 02) (025 025 025) (05 05 05) (1 1 1)8 NONSCOMP function [24] 500 1000 5000 10000 (1 1 1) (minus1 minus1 minus1) (minus2 minus2 minus2) (minus5 minus5 minus5)

9 Extended DENSCHNBfunction [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)

10 Booth function [25] 2 (1 1) (5 5) (10 10) (15 15)

11 Extended quadratic penaltyfunction QP2 [24] 100 200 300 400 (2 2 2) (5 5 5) (10 10 10) (15 15 15)

12 DIXMAANA function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)13 DIXMAANB function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)14 DIXMAANC function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)15 DIXMAAND function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)16 DIXMAANE function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)17 DIXMAANF function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)18 DIXMAANG function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)19 NONDIA function [24] 500 1000 5000 10000 (minus2 minus2 minus2) (minus1 minus1 minus1) (0 0 0) (1 1 1)

20 Extended Tridiagonal 1function [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)

21 DQDRTIC function [24] 500 1000 5000 10000 (minus1 minus1 minus1) (1 1 1) (2 2 2) (3 3 3)22 Diagonal 4 function [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)23 DQDRTIC function [24] 10 20 30 40 (1 1 1) (minus1 minus1 minus1) (1 1 1) (2 2 2)

24 Extended Cliff function[24] 100 200 300 400 (1 1 1) (5 5 5) (10 10 10) (15 15 15)

Suppose that a parameter 119903119872ge 119903119901119904

for all 119901 119904 is chosen and119903119901119904= 119903119872

if and only if solver 119904 does not solve problem 119901Because we would like to obtain an overall assessment of theperformance of a solver we defined the measure

120588119904(119905) =

1119899119901

size 119901 isin119875 119903119901119904le 119905 (42)

Thus 120588119904(119905) is the probability for solver 119904 isin 119878 that the

performance ratio 119903119901119904

is within a factor 119905 isin 119877 of the bestpossible ratio If we define the function 119901

119904as the cumulative

distribution function for the performance ratio then theperformance measure 119901

119904 119877 rarr [0 1] for a solver is

nondecreasing and piecewise continuous from the right Thevalue of 119901

119904(1) is the probability that the solver has the best

performance of all of the solvers In general a solverwith highvalues of 119901(119905) which would appear in the upper right cornerof the figure is preferable

Based on the left side of Figures 1 and 2 the PRPlowastlowastformula is above the other curves Therefore it is the most

efficient method among related PRP methods in terms ofefficiency and robustness In Figure 2 we see that the curveof PRPlowastlowast is still the best but the efficiency is not goodas the number of iterations since we use the complicatedhybrid algorithm leads to high CPU time Thus using highprocessors computers to find the solution will be moreefficient since the number of iterations decreased rapidlyunder PRPlowastlowast method

5 Conclusion

In this paper we proposed hybrid conjugate gradient methodby using nonnegative PRP and NPRP formulas with the SWPline search which extended the cases of using PRP methodunder mild condition The global convergence property isestablished and it is very simple Our numerical results hadshown that the hybrid method is the best when compared toother related PRP CG methods

Mathematical Problems in Engineering 7

t

e0

e1

e2

e3

Ps(t)

00

02

04

06

08

10

FR

FR

NPRP

VHSNPRPWYL

WYLVHS

PRPlowastlowast

PRPlowastlowast

PRP+

PRP+

Figure 2 Performance profile based on the CPU time

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgments

The authors are grateful to the editor and the anony-mous reviewers for their comments and suggestions whichimproved this paper substantially They would also like tothank The Ministry of Education Malaysia (MOE) for fund-ing this research under The Fundamental Research GrantScheme (Grant no 59256)

References

[1] P Wolfe ldquoConvergence conditions for ascent methodsrdquo SIAMReview vol 11 no 2 pp 226ndash235 1969

[2] PWolfe ldquoConvergence conditions for ascent methods II somecorrectionsrdquo SIAM Review vol 13 no 2 pp 185ndash188 1971

[3] J CaiQ Li L LiH Peng andYYang ldquoA fuzzy adaptive chaoticant swarm optimization for economic dispatchrdquo InternationalJournal of Electrical Power and Energy Systems vol 34 no 1 pp154ndash160 2012

[4] KNatalliaAn Introduction toHeuristic Slgorithms Departmentof Informatics and Telecommunications 2005

[5] L LiH Peng J Kurths Y Yang andH J Schellnhuber ldquoChaos-order transition in foraging behavior of antsrdquo Proceedings of theNational Academy of Sciences of the United States of Americavol 111 no 23 pp 8392ndash8397 2014

[6] M Wan L Li J Xiao C Wang and Y Yang ldquoData clusteringusing bacterial foraging optimizationrdquo Journal of IntelligentInformation Systems vol 38 no 2 pp 321ndash341 2012

[7] M Wan C Wang L Li and Y Yang ldquoChaotic ant swarmapproach for data clusteringrdquo Applied Soft Computing Journalvol 12 no 8 pp 2387ndash2393 2012

[8] M R Hestenes and E Stiefel ldquoMethods of conjugate gradientsfor solving linear systemsrdquo Journal of Research of the NationalBureau of Standards vol 49 no 6 pp 409ndash436 1952

[9] R Fletcher and C M Reeves ldquoFunction minimization byconjugate gradientsrdquo The Computer Journal vol 7 no 2 pp149ndash154 1964

[10] P Elijah and G Ribiere ldquoNote sur la convergence de methodesde directions conjugueesrdquo Revue francaise drsquoinformatique et derecherche operationnelle vol 3 no 1 pp 35ndash43 1969

[11] Z Wei S Yao and L Liu ldquoThe convergence properties of somenew conjugate gradient methodsrdquo Applied Mathematics andComputation vol 183 no 2 pp 1341ndash1350 2006

[12] G Zoutendijk ldquoNonlinear programming computationalmeth-odsrdquo Integer and Nonlinear Programming vol 143 no 1 pp 37ndash86 1970

[13] M Al-Baali ldquoDescent property and global convergence ofthe FletchermdashReeves method with inexact line searchrdquo IMAJournal of Numerical Analysis vol 5 no 1 pp 121ndash124 1985

[14] L Guanghui H Jiye and Y Hongxia ldquoGlobal convergence ofthe fletcher-reeves algorithm with inexact linesearchrdquo AppliedMathematics-A Journal of Chinese Universities vol 10 no 1 pp75ndash82 1995

[15] M J D Powell ldquoNonconvex minimization calculations and theconjugate gradient methodrdquo in Numerical Analysis vol 1066of Lecture Notes in Mathematics pp 122ndash141 Springer BerlinGermany 1984

[16] J C Gilbert and J Nocedal ldquoGlobal convergence properties ofconjugate gradient methods for optimizationrdquo SIAM Journal onOptimization vol 2 no 1 pp 21ndash42 1992

[17] D Touati-Ahmed and C Storey ldquoEfficient hybrid conjugategradient techniquesrdquo Journal of OptimizationTheory and Appli-cations vol 64 no 2 pp 379ndash397 1990

[18] Y Shengwei ZWei andH Huang ldquoA note aboutWYLrsquos conju-gate gradientmethod and its applicationsrdquoAppliedMathematicsand Computation vol 191 no 2 pp 381ndash388 2007

[19] L Zhang ldquoAn improved Wei-Yao-Liu nonlinear conjugategradient method for optimization computationrdquoApplied Math-ematics and Computation vol 215 no 6 pp 2269ndash2274 2009

[20] Z Dai and F Wen ldquoAnother improvedWeindashYaondashLiu nonlinearconjugate gradient method with sufficient descent propertyrdquoAppliedMathematics andComputation vol 218 no 14 pp 7421ndash7430 2012

[21] M Rivaie M Mamat L W June and I Mohd ldquoA new classof nonlinear conjugate gradient coefficients with global conver-gence propertiesrdquo Applied Mathematics and Computation vol218 no 22 pp 11323ndash11332 2012

[22] A AlhawaratMMamatM Rivaie and IMohd ldquoA newmodi-fication of nonlinear conjugate gradient coefficients with globalconvergence propertiesrdquo International Journal of MathematicalComputational Statistical Natural and Physical Engineeringvol 8 no 1 pp 54ndash60 2014

[23] A Neculai ldquoAn unconstrained optimization test functionscollectionrdquo Advanced Modeling and Optimization vol 10 no 1pp 147ndash161 2008

[24] I Bongartz A R Conn N Gould P L Toint and I Bon-gartz Constrained and Unconstrained Testing EnvironmentDepartement de Mathematique 1993

[25] E P Adorio and U Diliman ldquoMvf-multivariate test functionslibrary in c for unconstrained global optimizationrdquo 2005

[26] E D Dolan and J J More ldquoBenchmarking optimization soft-ware with performance profilesrdquo Mathematical Programmingvol 91 no 2 pp 201ndash213 2002

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

Mathematical Problems in Engineering 3

obtain the convergence propertiesThe PRPmethod could besimplified as follows

120573PRP119896=119892119879119896(119892119896minus 119892119896minus1)

1003817100381710038171003817119892119896minus110038171003817100381710038172 =

100381710038171003817100381711989211989610038171003817100381710038172minus 119892119879119896119892119896minus1

1003817100381710038171003817119892119896minus110038171003817100381710038172 (16)

Therefore we have the following cases

Case A If 119892119879119896119892119896minus1 gt 0 then we have the following two

possibilities

Case A1 If 1198921198962 gt 119892119879

119896119892119896minus1 then we have 0 le 120573PRP

119896le 120573FR119896 In

this case PRPmethod is efficient and has global convergenceproperties

Case A2 If 1198921198962 lt 119892119879

119896119892119896minus1 then we have 120573PRP

119896lt 0 In this

case based on [16] we fail to obtain the global convergenceproperties for nonconvex functions

Case B If 119892119879119896119892119896minus1 lt 0 then

120573PRP119896=119892119879119896(119892119896minus 119892119896minus1)

1003817100381710038171003817119892119896minus110038171003817100381710038172 =

100381710038171003817100381711989211989610038171003817100381710038172+10038161003816100381610038161003816119892119879

119896119892119896minus110038161003816100381610038161003816

1003817100381710038171003817119892119896minus110038171003817100381710038172

= 120573FR119896+

10038161003816100381610038161003816119892119879

119896119892119896minus110038161003816100381610038161003816

1003817100381710038171003817119892119896minus110038171003817100381710038172

(17)

In Case B there is no guarantee that this method will satisfythe sufficient descent condition

For the next discussion we will discuss the nonnegativePRP method which is given as follows

120573PRP+119896

= max 120573PRP119896 0 ie

120573PRP+119896

= max Case A1 Case A2 Case B 0 (18)

Therefore we have a problem inCase B To solve this problemGilbert and Nocedal [16] used another line search to satisfythe convergence properties In addition if 120573PRP+

119896= 0 the

CG method returns to the steepest descent method whichis sometimes a weak tool to find the optimum point forfunctions Furthermore we can notice that

120573TS119896=

Case A1

120573FR119896 otherwise

(19)

So from PRP method we can use only Case A1To improve the above ideas we suggest the following

hybrid method

120573PRPlowastlowast

119896=

120573PRP119896=119892119879119896(119892119896minus 119892119896minus1)

1003817100381710038171003817119892119896minus110038171003817100381710038172 if 1003817100381710038171003817119892119896

10038171003817100381710038172gt10038161003816100381610038161003816119892119879

119896119892119896minus110038161003816100381610038161003816

120573NPRP119896

=100381710038171003817100381711989211989610038171003817100381710038172minus (1003817100381710038171003817119892119896

1003817100381710038171003817 1003817100381710038171003817119892119896minus1

1003817100381710038171003817)1003816100381610038161003816119892119896119892119896minus1

10038161003816100381610038161003817100381710038171003817119892119896minus1

10038171003817100381710038172 otherwise

(20)

If 120573PRPlowastlowast

119896= (119892

1198962 minus 119892119879

119896119892119896minus1)119892119896minus1

2 then under thecondition 119892

1198962 gt |119892119879

119896119892119896minus1| we obtain

0 lt 120573PRPlowastlowast

119896=100381710038171003817100381711989211989610038171003817100381710038172minus 119892119879119896119892119896minus1

1003817100381710038171003817119892119896minus110038171003817100381710038172 le

100381710038171003817100381711989211989610038171003817100381710038172+10038161003816100381610038161003816119892119879

119896119892119896minus110038161003816100381610038161003816

1003817100381710038171003817119892119896minus110038171003817100381710038172

lt2 1003817100381710038171003817119892119896

10038171003817100381710038172

1003817100381710038171003817119892119896minus110038171003817100381710038172

(21)

And if

120573PRPlowastlowast

119896= 120573NPRP119896

=

100381710038171003817100381711989211989610038171003817100381710038172minus (1003817100381710038171003817119892119896

1003817100381710038171003817 1003817100381710038171003817119892119896minus1

1003817100381710038171003817)10038161003816100381610038161003816119892119879

119896119892119896minus110038161003816100381610038161003816

1003817100381710038171003817119892119896minus110038171003817100381710038172 (22)

we obtain

0 le 120573PRPlowastlowast

119896lt100381710038171003817100381711989211989610038171003817100381710038172

1003817100381710038171003817119892119896minus110038171003817100381710038172 = 120573

FR119896 (23)

One of the advantages for 120573PRPlowastlowast

119896is that we can use Case A1

and Case B under the condition 1198921198962 gt |119892119879

119896119892119896minus1|

Choosing 120573NPRP119896

as a restart CG formula in (20) if thecondition 119892

1198962 gt |119892119879

119896119892119896minus1| is not satisfied that means

100381710038171003817100381711989211989610038171003817100381710038172le10038161003816100381610038161003816119892119879

119896119892119896minus110038161003816100381610038161003816 (24)

Thus NPRP method is a suitable nonnegative value to useThe following algorithm is an algorithm of CG method

with the new coefficient 120573PRPlowastlowast

119896

Algorithm 1 Consider the followingStep 1 Initialization given 1199091 set 119896 = 1Step 2 Compute 120573

119896based on (20)

Step 3 Compute 119889119896based on (3)

Step 4 Compute 120572119896based on (5) and (6)

Step 5 Update new point based on (2)Step 6 Convergent test and stopping criteria if 119892

119896 le 120576 then

stop otherwise go to Step 2 with 119896 = 119896 + 1

4 Mathematical Problems in Engineering

3 The Global Convergence Properties for120573PRP

lowastlowast

119896with SWP Line Search

The following standard assumptions are necessary for thiswork

Assumption 1 The level set Χ = 119909 isin 119877119899 119891(119909) le 119891(1199091)with 1199091 to the starting point of the iterative method (2) isbounded

Assumption 2 In some open convex neighborhood119873 ofΧ119891is continuous and differentiable and its gradient is Lipschitzcontinuous that is for any 119909 119910 isin 119873 there exists a constant119871 gt 0 such that 119892(119909) minus 119892(119910) le 119871119909 minus 119910

The following lemma is one of the most important lem-maswhich is used to prove the global convergence properties

Lemma 2 (see [12]) Suppose Assumptions 1 and 2 are trueConsider any form of (2) and (3) with 120572

119896computed by WWP

line search direction 119889119896 is descent for all 119896 ge 1 theninfin

sum119896=1

cos2120579119896

100381710038171003817100381711989211989610038171003817100381710038172lt infin (25)

where

cos 120579119896= minus

(119892119879119896119889119896)

1003817100381710038171003817119889119896100381710038171003817100381710038171003817100381710038171198921198961003817100381710038171003817 (26)

Equation (25) is known as Zoutendijk condition

The following discussion will discuss the global conver-gence properties for 120573PRP

lowastlowast

119896with SWP line search

Case 1 (120573PRPlowastlowast

119896= 120573NPRP119896

) Since 0 le 120573NPRP119896

le 120573FR119896

the proof ofthe global convergence properties is similar to 120573FR

119896 We refer

the reader to see [13 14]

Case 2 (120573PRPlowastlowast

119896= 120573PRP119896

where 1198921198962 gt |119892119879

119896119892119896minus1|) In this case

we have

0 le 120573PRPlowastlowast

119896=100381710038171003817100381711989211989610038171003817100381710038172minus 119892119879119896119892119896minus1

1003817100381710038171003817119892119896minus110038171003817100381710038172 le

100381710038171003817100381711989211989610038171003817100381710038172+10038161003816100381610038161003816119892119879

119896119892119896minus110038161003816100381610038161003816

1003817100381710038171003817119892119896minus110038171003817100381710038172

le2 1003817100381710038171003817119892119896

10038171003817100381710038172

1003817100381710038171003817119892119896minus110038171003817100381710038172

(27)

The following theoremdemonstrates120573PRPlowastlowast

119896in Case 2 satisfies

the sufficient descent condition with SWP line search

Theorem 3 Consider the sequences 119892119896and 119889

119896are constructed

by Algorithm 1 and 120572119896is computed by (5) and (6) if 120590 isin

(0 14] Then (15) holds

Proof From (3) for 119896 = 1 it is obvious Suppose it is trueuntil 119896 minus 1 that is 119892119879

119896minus1119889119896minus1 = minus119892119896minus12 Multiply (3) by 119892

119896

we obtain

119892119879119896119889119896= 119892119879119896(minus119892119879119896+120573119896119889119896minus1) = minus

100381710038171003817100381711989211989610038171003817100381710038172+120573119896119892119879119896119889119896minus1 (28)

Dividing both sides by 119892119896minus1

2 and by using Case 2 and (6)then we obtain

minus 1+ 2120590119892119879119896minus1119889119896minus11003817100381710038171003817119892119896minus1

10038171003817100381710038172 le

119892119879119896119889119896

100381710038171003817100381711989211989610038171003817100381710038172 le minus 1minus 2120590

119892119879119896minus1119889119896minus11003817100381710038171003817119892119896minus1

10038171003817100381710038172 (29)

From (29) we obtain

minus119896minus1sum119895=0(2120590)119895 le

119892119879119896119889119896

100381710038171003817100381711989211989610038171003817100381710038172 le minus 2+

119896minus1sum119895=0(2120590)119895 (30)

Since119896minus1sum119895=0(2120590)119895 le 1 minus (2120590)119896

1 minus 2120590 (31)

we have

minus1 minus (2120590)119896

1 minus 2120590le119892119879119896119889119896

100381710038171003817100381711989211989610038171003817100381710038172 le minus 2+

1 minus (2120590)119896

1 minus 2120590 (32)

If 120590 le 14 we get

1 minus (2120590)119896

1 minus 2120590lt 2 (33)

Let

119888 = 2minus 1 minus (120590)119896

1 minus 120590 (34)

then

119888 minus 2 le119892119879119896119889119896

100381710038171003817100381711989211989610038171003817100381710038172 le minus 119888 (35)

Thus

119892119879119896119889119896le minus 119888 1003817100381710038171003817119892119896

10038171003817100381710038172 (36)

where 119888 isin (0 1) The proof is complete

Gilbert andNocedal [16] presented an important theorem(Theorem 4) to find the global convergence properties fornonnegative PRPmethod if the descent condition is satisfiedFurthermore [16] presented nice property called Propertylowast asfollows

119875119903119900119901119890119903119905119910lowast Consider a CG method of forms (2) and (3) andsuppose 0 lt 120574 le 119892

119896 le 120574 we say that the CG method

possesses Propertylowast if there exists constant 119887 gt 1 and 120582 gt 0such that for all 119896 ge 1 we get |120573

119896| le 119887 and if 119909

119896minus119909119896minus1 le 120582

we obtain

10038161003816100381610038161205731198961003816100381610038161003816 le

12119887 (37)

Theorem 4 Consider any CG method of forms (2) and (3)achieves the following properties

(I) 120573119896ge 0

(II) The sufficient descent condition (15) holds

Mathematical Problems in Engineering 5

(III) The Zoutendijk condition (25) is satisfied by the linesearch

(IV) 119875119903119900119901119890119903119905119910lowast holds

(V) Assumptions 1 and 2 hold

Then the iterates are globally convergent

The next lemma shows that if the gradients are boundedaway from zero and Propertylowast holds then a certain fractionof steps cannot be too small The proof is given in [16]However we state it for readability

Lemma 5 Consider a CG algorithm as defined in (2) and (3)with the parameter 120573PRP

lowastlowast

119896 If Assumptions 1 and 2 are satisfied

then 119875119903119900119901119890119903119905119910lowast holds

Proof Let 119887 = 212057421205742 ge 1 and 120582 le 12057422119871120574119887

100381610038161003816100381610038161003816120573PRP

lowastlowast

119896

100381610038161003816100381610038161003816le

1003816100381610038161003816100381610038161003816100381610038161003816

100381710038171003817100381711989211989610038171003817100381710038172minus 119892119879119896119892119896minus1

1003817100381710038171003817119892119896minus110038171003817100381710038172

1003816100381610038161003816100381610038161003816100381610038161003816le

100381710038171003817100381711989211989610038171003817100381710038172+10038161003816100381610038161003816119892119879

119896119892119896minus110038161003816100381610038161003816

1003817100381710038171003817119892119896minus110038171003817100381710038172

le10038171003817100381710038171198921198961003817100381710038171003817 (10038171003817100381710038171198921198961003817100381710038171003817 +1003817100381710038171003817119892119896minus1

1003817100381710038171003817)1003817100381710038171003817119892119896minus1

10038171003817100381710038172 le

21205742

1205742= 119887 gt 1

(38)

Using 119904119896 le 120582 we have

100381610038161003816100381610038161003816120573PRP

lowastlowast

119896

100381610038161003816100381610038161003816le

1003816100381610038161003816100381610038161003816100381610038161003816

119892119879119896(119892119896minus 119892119896minus1)

1003817100381710038171003817119892119896minus110038171003817100381710038172

1003816100381610038161003816100381610038161003816100381610038161003816le100381710038171003817100381711989211989610038171003817100381710038171003817100381710038171003817119892119896 minus 119892119896minus1

10038171003817100381710038171003817100381710038171003817119892119896minus1

10038171003817100381710038172 (39)

By Assumption 2

100381610038161003816100381610038161003816120573PRP

lowastlowast

119896

100381610038161003816100381610038161003816le119871120582120574

1205742le

12119887 (40)

The proof is complete

Lemma 6 The CG formula presented in Case 2 has thefollowing properties

(1) 120573PRPlowastlowast

119896gt 0 since the condition 119892

1198962 gt |119892119879

119896119892119896minus1| forces

the CG formula in (20) to be nonnegative

(2) 120573PRPlowastlowast

119896satisfies 119875119903119900119901119890119903119905119910lowast based on Lemma 5

(3) 120573PRPlowastlowast

119896satisfies the sufficient descent condition based on

Theorem 3 and 119888 isin (0 1)

By using Theorems 3 and 4 and Lemma 6 we havethe following convergence result The proof is similar toTheorem 43 which is presented in [16]

Theorem 7 Suppose that Assumption 1 holds Consider theCG method of forms (2) and (3) and 120573

119896as in Case 2

where 120572119896is computed by (5) and (6) with 120590 le 14 then

lim119896rarrinfin

inf 119892119896 = 0

t

e0

e1

e2

e3

Ps(t)

00

02

04

06

08

10

FR

FR

NPRPWYLVHS

VHSNPRPWYL

PRPlowastlowast

PRPlowastlowast

PRP+

PRP+

Figure 1 Performance profile based on the number of iterations

4 Numerical Results and Discussions

To evaluate the efficiency of the new method we selectedsome of the test functions in Table 1 from CUTEr [24]Neculai [23] and Adorio and Diliman [25] We performed acomparison with other CG methods including VHS NPRPPRP+ FR and PRPlowastlowast formulas The tolerance 120576 is selectedto 10minus6 for all algorithms to investigate the rapidity of theiterationmethods towards the optimal solutionThe gradientvalue is used as the stopping criteria Here the stoppingcriteria considered 119892

119896 le 10minus6 We considered the method

failed if the number of iterations exceeds 1000 timesIn Table 1 we selected different initial points for every

function Thus this demonstrated that this method can beused in several real life functions from other fields such asengineering and medical science as we mentioned before Inaddition different dimensions from 500 until 10000 are usedWe also choose from different group of functions We usedMatlab 79 subroutine program with CPU processor Intel (R)Core (TM) i3 CPU and 2GB DDR2 RAM under SWP linesearch to find the optimumpointThe efficiency comparisonsresults are shown in Figures 1 and 2 respectively using aperformance profile introduced by Dolan and More [26]

This performance measure was introduced to compare aset of solvers 119878 on a set of problems 119875 Assuming 119899

119904solvers

and 119899119901problems in 119878 and 119875 respectively the measure 119905

119901119904

is defined as the computation time (eg the number ofiterations or the CPU time) required to solve problem 119901 bysolver 119904

To create a baseline for comparison the performance ofsolver 119904 on problem119901 is scaled by the best performance of anysolver in 119878 on the problem using the ratio

119903119901119904=

119905119901119904

min 119905119901119904 119904 isin 119878

(41)

6 Mathematical Problems in Engineering

Table 1 A list of problem functions used with the SWP condition with 120575 = 001 and 120590 = 01

Number Function Dimensions Initial points

1 Extended White amp Holstfunction [23] 500 1000 5000 10000 (minus12 1 minus12 1 minus12 1) (5 5 5) (10 10 10) (15 15 15)

2 Extended Rosenbrockfunction [24] 500 1000 5000 10000 (minus12 1 minus12 1 minus12 1) (5 5 5) (10 10 10) (15 15 15)

3 Six-hump function [25] 2 (1 1) (5 5) (10 10) (15 15)

4 Extended Beale function[23] 500 1000 5000 10000 (minus1 minus1 minus1) (05 05 05) (10 10 10) (1 1 1)

5 Three-hump function [25] 2 (1 1) (5 5) (10 10) (15 15)

6 Extended Himmelblaufunction [23] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)

7 Diagonal 2 function [23] 500 1000 5000 10000 (02 02 02) (025 025 025) (05 05 05) (1 1 1)8 NONSCOMP function [24] 500 1000 5000 10000 (1 1 1) (minus1 minus1 minus1) (minus2 minus2 minus2) (minus5 minus5 minus5)

9 Extended DENSCHNBfunction [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)

10 Booth function [25] 2 (1 1) (5 5) (10 10) (15 15)

11 Extended quadratic penaltyfunction QP2 [24] 100 200 300 400 (2 2 2) (5 5 5) (10 10 10) (15 15 15)

12 DIXMAANA function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)13 DIXMAANB function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)14 DIXMAANC function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)15 DIXMAAND function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)16 DIXMAANE function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)17 DIXMAANF function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)18 DIXMAANG function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)19 NONDIA function [24] 500 1000 5000 10000 (minus2 minus2 minus2) (minus1 minus1 minus1) (0 0 0) (1 1 1)

20 Extended Tridiagonal 1function [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)

21 DQDRTIC function [24] 500 1000 5000 10000 (minus1 minus1 minus1) (1 1 1) (2 2 2) (3 3 3)22 Diagonal 4 function [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)23 DQDRTIC function [24] 10 20 30 40 (1 1 1) (minus1 minus1 minus1) (1 1 1) (2 2 2)

24 Extended Cliff function[24] 100 200 300 400 (1 1 1) (5 5 5) (10 10 10) (15 15 15)

Suppose that a parameter 119903119872ge 119903119901119904

for all 119901 119904 is chosen and119903119901119904= 119903119872

if and only if solver 119904 does not solve problem 119901Because we would like to obtain an overall assessment of theperformance of a solver we defined the measure

120588119904(119905) =

1119899119901

size 119901 isin119875 119903119901119904le 119905 (42)

Thus 120588119904(119905) is the probability for solver 119904 isin 119878 that the

performance ratio 119903119901119904

is within a factor 119905 isin 119877 of the bestpossible ratio If we define the function 119901

119904as the cumulative

distribution function for the performance ratio then theperformance measure 119901

119904 119877 rarr [0 1] for a solver is

nondecreasing and piecewise continuous from the right Thevalue of 119901

119904(1) is the probability that the solver has the best

performance of all of the solvers In general a solverwith highvalues of 119901(119905) which would appear in the upper right cornerof the figure is preferable

Based on the left side of Figures 1 and 2 the PRPlowastlowastformula is above the other curves Therefore it is the most

efficient method among related PRP methods in terms ofefficiency and robustness In Figure 2 we see that the curveof PRPlowastlowast is still the best but the efficiency is not goodas the number of iterations since we use the complicatedhybrid algorithm leads to high CPU time Thus using highprocessors computers to find the solution will be moreefficient since the number of iterations decreased rapidlyunder PRPlowastlowast method

5 Conclusion

In this paper we proposed hybrid conjugate gradient methodby using nonnegative PRP and NPRP formulas with the SWPline search which extended the cases of using PRP methodunder mild condition The global convergence property isestablished and it is very simple Our numerical results hadshown that the hybrid method is the best when compared toother related PRP CG methods

Mathematical Problems in Engineering 7

t

e0

e1

e2

e3

Ps(t)

00

02

04

06

08

10

FR

FR

NPRP

VHSNPRPWYL

WYLVHS

PRPlowastlowast

PRPlowastlowast

PRP+

PRP+

Figure 2 Performance profile based on the CPU time

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgments

The authors are grateful to the editor and the anony-mous reviewers for their comments and suggestions whichimproved this paper substantially They would also like tothank The Ministry of Education Malaysia (MOE) for fund-ing this research under The Fundamental Research GrantScheme (Grant no 59256)

References

[1] P Wolfe ldquoConvergence conditions for ascent methodsrdquo SIAMReview vol 11 no 2 pp 226ndash235 1969

[2] PWolfe ldquoConvergence conditions for ascent methods II somecorrectionsrdquo SIAM Review vol 13 no 2 pp 185ndash188 1971

[3] J CaiQ Li L LiH Peng andYYang ldquoA fuzzy adaptive chaoticant swarm optimization for economic dispatchrdquo InternationalJournal of Electrical Power and Energy Systems vol 34 no 1 pp154ndash160 2012

[4] KNatalliaAn Introduction toHeuristic Slgorithms Departmentof Informatics and Telecommunications 2005

[5] L LiH Peng J Kurths Y Yang andH J Schellnhuber ldquoChaos-order transition in foraging behavior of antsrdquo Proceedings of theNational Academy of Sciences of the United States of Americavol 111 no 23 pp 8392ndash8397 2014

[6] M Wan L Li J Xiao C Wang and Y Yang ldquoData clusteringusing bacterial foraging optimizationrdquo Journal of IntelligentInformation Systems vol 38 no 2 pp 321ndash341 2012

[7] M Wan C Wang L Li and Y Yang ldquoChaotic ant swarmapproach for data clusteringrdquo Applied Soft Computing Journalvol 12 no 8 pp 2387ndash2393 2012

[8] M R Hestenes and E Stiefel ldquoMethods of conjugate gradientsfor solving linear systemsrdquo Journal of Research of the NationalBureau of Standards vol 49 no 6 pp 409ndash436 1952

[9] R Fletcher and C M Reeves ldquoFunction minimization byconjugate gradientsrdquo The Computer Journal vol 7 no 2 pp149ndash154 1964

[10] P Elijah and G Ribiere ldquoNote sur la convergence de methodesde directions conjugueesrdquo Revue francaise drsquoinformatique et derecherche operationnelle vol 3 no 1 pp 35ndash43 1969

[11] Z Wei S Yao and L Liu ldquoThe convergence properties of somenew conjugate gradient methodsrdquo Applied Mathematics andComputation vol 183 no 2 pp 1341ndash1350 2006

[12] G Zoutendijk ldquoNonlinear programming computationalmeth-odsrdquo Integer and Nonlinear Programming vol 143 no 1 pp 37ndash86 1970

[13] M Al-Baali ldquoDescent property and global convergence ofthe FletchermdashReeves method with inexact line searchrdquo IMAJournal of Numerical Analysis vol 5 no 1 pp 121ndash124 1985

[14] L Guanghui H Jiye and Y Hongxia ldquoGlobal convergence ofthe fletcher-reeves algorithm with inexact linesearchrdquo AppliedMathematics-A Journal of Chinese Universities vol 10 no 1 pp75ndash82 1995

[15] M J D Powell ldquoNonconvex minimization calculations and theconjugate gradient methodrdquo in Numerical Analysis vol 1066of Lecture Notes in Mathematics pp 122ndash141 Springer BerlinGermany 1984

[16] J C Gilbert and J Nocedal ldquoGlobal convergence properties ofconjugate gradient methods for optimizationrdquo SIAM Journal onOptimization vol 2 no 1 pp 21ndash42 1992

[17] D Touati-Ahmed and C Storey ldquoEfficient hybrid conjugategradient techniquesrdquo Journal of OptimizationTheory and Appli-cations vol 64 no 2 pp 379ndash397 1990

[18] Y Shengwei ZWei andH Huang ldquoA note aboutWYLrsquos conju-gate gradientmethod and its applicationsrdquoAppliedMathematicsand Computation vol 191 no 2 pp 381ndash388 2007

[19] L Zhang ldquoAn improved Wei-Yao-Liu nonlinear conjugategradient method for optimization computationrdquoApplied Math-ematics and Computation vol 215 no 6 pp 2269ndash2274 2009

[20] Z Dai and F Wen ldquoAnother improvedWeindashYaondashLiu nonlinearconjugate gradient method with sufficient descent propertyrdquoAppliedMathematics andComputation vol 218 no 14 pp 7421ndash7430 2012

[21] M Rivaie M Mamat L W June and I Mohd ldquoA new classof nonlinear conjugate gradient coefficients with global conver-gence propertiesrdquo Applied Mathematics and Computation vol218 no 22 pp 11323ndash11332 2012

[22] A AlhawaratMMamatM Rivaie and IMohd ldquoA newmodi-fication of nonlinear conjugate gradient coefficients with globalconvergence propertiesrdquo International Journal of MathematicalComputational Statistical Natural and Physical Engineeringvol 8 no 1 pp 54ndash60 2014

[23] A Neculai ldquoAn unconstrained optimization test functionscollectionrdquo Advanced Modeling and Optimization vol 10 no 1pp 147ndash161 2008

[24] I Bongartz A R Conn N Gould P L Toint and I Bon-gartz Constrained and Unconstrained Testing EnvironmentDepartement de Mathematique 1993

[25] E P Adorio and U Diliman ldquoMvf-multivariate test functionslibrary in c for unconstrained global optimizationrdquo 2005

[26] E D Dolan and J J More ldquoBenchmarking optimization soft-ware with performance profilesrdquo Mathematical Programmingvol 91 no 2 pp 201ndash213 2002

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

4 Mathematical Problems in Engineering

3 The Global Convergence Properties for120573PRP

lowastlowast

119896with SWP Line Search

The following standard assumptions are necessary for thiswork

Assumption 1 The level set Χ = 119909 isin 119877119899 119891(119909) le 119891(1199091)with 1199091 to the starting point of the iterative method (2) isbounded

Assumption 2 In some open convex neighborhood119873 ofΧ119891is continuous and differentiable and its gradient is Lipschitzcontinuous that is for any 119909 119910 isin 119873 there exists a constant119871 gt 0 such that 119892(119909) minus 119892(119910) le 119871119909 minus 119910

The following lemma is one of the most important lem-maswhich is used to prove the global convergence properties

Lemma 2 (see [12]) Suppose Assumptions 1 and 2 are trueConsider any form of (2) and (3) with 120572

119896computed by WWP

line search direction 119889119896 is descent for all 119896 ge 1 theninfin

sum119896=1

cos2120579119896

100381710038171003817100381711989211989610038171003817100381710038172lt infin (25)

where

cos 120579119896= minus

(119892119879119896119889119896)

1003817100381710038171003817119889119896100381710038171003817100381710038171003817100381710038171198921198961003817100381710038171003817 (26)

Equation (25) is known as Zoutendijk condition

The following discussion will discuss the global conver-gence properties for 120573PRP

lowastlowast

119896with SWP line search

Case 1 (120573PRPlowastlowast

119896= 120573NPRP119896

) Since 0 le 120573NPRP119896

le 120573FR119896

the proof ofthe global convergence properties is similar to 120573FR

119896 We refer

the reader to see [13 14]

Case 2 (120573PRPlowastlowast

119896= 120573PRP119896

where 1198921198962 gt |119892119879

119896119892119896minus1|) In this case

we have

0 le 120573PRPlowastlowast

119896=100381710038171003817100381711989211989610038171003817100381710038172minus 119892119879119896119892119896minus1

1003817100381710038171003817119892119896minus110038171003817100381710038172 le

100381710038171003817100381711989211989610038171003817100381710038172+10038161003816100381610038161003816119892119879

119896119892119896minus110038161003816100381610038161003816

1003817100381710038171003817119892119896minus110038171003817100381710038172

le2 1003817100381710038171003817119892119896

10038171003817100381710038172

1003817100381710038171003817119892119896minus110038171003817100381710038172

(27)

The following theoremdemonstrates120573PRPlowastlowast

119896in Case 2 satisfies

the sufficient descent condition with SWP line search

Theorem 3 Consider the sequences 119892119896and 119889

119896are constructed

by Algorithm 1 and 120572119896is computed by (5) and (6) if 120590 isin

(0 14] Then (15) holds

Proof From (3) for 119896 = 1 it is obvious Suppose it is trueuntil 119896 minus 1 that is 119892119879

119896minus1119889119896minus1 = minus119892119896minus12 Multiply (3) by 119892

119896

we obtain

119892119879119896119889119896= 119892119879119896(minus119892119879119896+120573119896119889119896minus1) = minus

100381710038171003817100381711989211989610038171003817100381710038172+120573119896119892119879119896119889119896minus1 (28)

Dividing both sides by 119892119896minus1

2 and by using Case 2 and (6)then we obtain

minus 1+ 2120590119892119879119896minus1119889119896minus11003817100381710038171003817119892119896minus1

10038171003817100381710038172 le

119892119879119896119889119896

100381710038171003817100381711989211989610038171003817100381710038172 le minus 1minus 2120590

119892119879119896minus1119889119896minus11003817100381710038171003817119892119896minus1

10038171003817100381710038172 (29)

From (29) we obtain

minus119896minus1sum119895=0(2120590)119895 le

119892119879119896119889119896

100381710038171003817100381711989211989610038171003817100381710038172 le minus 2+

119896minus1sum119895=0(2120590)119895 (30)

Since119896minus1sum119895=0(2120590)119895 le 1 minus (2120590)119896

1 minus 2120590 (31)

we have

minus1 minus (2120590)119896

1 minus 2120590le119892119879119896119889119896

100381710038171003817100381711989211989610038171003817100381710038172 le minus 2+

1 minus (2120590)119896

1 minus 2120590 (32)

If 120590 le 14 we get

1 minus (2120590)119896

1 minus 2120590lt 2 (33)

Let

119888 = 2minus 1 minus (120590)119896

1 minus 120590 (34)

then

119888 minus 2 le119892119879119896119889119896

100381710038171003817100381711989211989610038171003817100381710038172 le minus 119888 (35)

Thus

119892119879119896119889119896le minus 119888 1003817100381710038171003817119892119896

10038171003817100381710038172 (36)

where 119888 isin (0 1) The proof is complete

Gilbert andNocedal [16] presented an important theorem(Theorem 4) to find the global convergence properties fornonnegative PRPmethod if the descent condition is satisfiedFurthermore [16] presented nice property called Propertylowast asfollows

119875119903119900119901119890119903119905119910lowast Consider a CG method of forms (2) and (3) andsuppose 0 lt 120574 le 119892

119896 le 120574 we say that the CG method

possesses Propertylowast if there exists constant 119887 gt 1 and 120582 gt 0such that for all 119896 ge 1 we get |120573

119896| le 119887 and if 119909

119896minus119909119896minus1 le 120582

we obtain

10038161003816100381610038161205731198961003816100381610038161003816 le

12119887 (37)

Theorem 4 Consider any CG method of forms (2) and (3)achieves the following properties

(I) 120573119896ge 0

(II) The sufficient descent condition (15) holds

Mathematical Problems in Engineering 5

(III) The Zoutendijk condition (25) is satisfied by the linesearch

(IV) 119875119903119900119901119890119903119905119910lowast holds

(V) Assumptions 1 and 2 hold

Then the iterates are globally convergent

The next lemma shows that if the gradients are boundedaway from zero and Propertylowast holds then a certain fractionof steps cannot be too small The proof is given in [16]However we state it for readability

Lemma 5 Consider a CG algorithm as defined in (2) and (3)with the parameter 120573PRP

lowastlowast

119896 If Assumptions 1 and 2 are satisfied

then 119875119903119900119901119890119903119905119910lowast holds

Proof Let 119887 = 212057421205742 ge 1 and 120582 le 12057422119871120574119887

100381610038161003816100381610038161003816120573PRP

lowastlowast

119896

100381610038161003816100381610038161003816le

1003816100381610038161003816100381610038161003816100381610038161003816

100381710038171003817100381711989211989610038171003817100381710038172minus 119892119879119896119892119896minus1

1003817100381710038171003817119892119896minus110038171003817100381710038172

1003816100381610038161003816100381610038161003816100381610038161003816le

100381710038171003817100381711989211989610038171003817100381710038172+10038161003816100381610038161003816119892119879

119896119892119896minus110038161003816100381610038161003816

1003817100381710038171003817119892119896minus110038171003817100381710038172

le10038171003817100381710038171198921198961003817100381710038171003817 (10038171003817100381710038171198921198961003817100381710038171003817 +1003817100381710038171003817119892119896minus1

1003817100381710038171003817)1003817100381710038171003817119892119896minus1

10038171003817100381710038172 le

21205742

1205742= 119887 gt 1

(38)

Using 119904119896 le 120582 we have

100381610038161003816100381610038161003816120573PRP

lowastlowast

119896

100381610038161003816100381610038161003816le

1003816100381610038161003816100381610038161003816100381610038161003816

119892119879119896(119892119896minus 119892119896minus1)

1003817100381710038171003817119892119896minus110038171003817100381710038172

1003816100381610038161003816100381610038161003816100381610038161003816le100381710038171003817100381711989211989610038171003817100381710038171003817100381710038171003817119892119896 minus 119892119896minus1

10038171003817100381710038171003817100381710038171003817119892119896minus1

10038171003817100381710038172 (39)

By Assumption 2

100381610038161003816100381610038161003816120573PRP

lowastlowast

119896

100381610038161003816100381610038161003816le119871120582120574

1205742le

12119887 (40)

The proof is complete

Lemma 6 The CG formula presented in Case 2 has thefollowing properties

(1) 120573PRPlowastlowast

119896gt 0 since the condition 119892

1198962 gt |119892119879

119896119892119896minus1| forces

the CG formula in (20) to be nonnegative

(2) 120573PRPlowastlowast

119896satisfies 119875119903119900119901119890119903119905119910lowast based on Lemma 5

(3) 120573PRPlowastlowast

119896satisfies the sufficient descent condition based on

Theorem 3 and 119888 isin (0 1)

By using Theorems 3 and 4 and Lemma 6 we havethe following convergence result The proof is similar toTheorem 43 which is presented in [16]

Theorem 7 Suppose that Assumption 1 holds Consider theCG method of forms (2) and (3) and 120573

119896as in Case 2

where 120572119896is computed by (5) and (6) with 120590 le 14 then

lim119896rarrinfin

inf 119892119896 = 0

t

e0

e1

e2

e3

Ps(t)

00

02

04

06

08

10

FR

FR

NPRPWYLVHS

VHSNPRPWYL

PRPlowastlowast

PRPlowastlowast

PRP+

PRP+

Figure 1 Performance profile based on the number of iterations

4 Numerical Results and Discussions

To evaluate the efficiency of the new method we selectedsome of the test functions in Table 1 from CUTEr [24]Neculai [23] and Adorio and Diliman [25] We performed acomparison with other CG methods including VHS NPRPPRP+ FR and PRPlowastlowast formulas The tolerance 120576 is selectedto 10minus6 for all algorithms to investigate the rapidity of theiterationmethods towards the optimal solutionThe gradientvalue is used as the stopping criteria Here the stoppingcriteria considered 119892

119896 le 10minus6 We considered the method

failed if the number of iterations exceeds 1000 timesIn Table 1 we selected different initial points for every

function Thus this demonstrated that this method can beused in several real life functions from other fields such asengineering and medical science as we mentioned before Inaddition different dimensions from 500 until 10000 are usedWe also choose from different group of functions We usedMatlab 79 subroutine program with CPU processor Intel (R)Core (TM) i3 CPU and 2GB DDR2 RAM under SWP linesearch to find the optimumpointThe efficiency comparisonsresults are shown in Figures 1 and 2 respectively using aperformance profile introduced by Dolan and More [26]

This performance measure was introduced to compare aset of solvers 119878 on a set of problems 119875 Assuming 119899

119904solvers

and 119899119901problems in 119878 and 119875 respectively the measure 119905

119901119904

is defined as the computation time (eg the number ofiterations or the CPU time) required to solve problem 119901 bysolver 119904

To create a baseline for comparison the performance ofsolver 119904 on problem119901 is scaled by the best performance of anysolver in 119878 on the problem using the ratio

119903119901119904=

119905119901119904

min 119905119901119904 119904 isin 119878

(41)

6 Mathematical Problems in Engineering

Table 1 A list of problem functions used with the SWP condition with 120575 = 001 and 120590 = 01

Number Function Dimensions Initial points

1 Extended White amp Holstfunction [23] 500 1000 5000 10000 (minus12 1 minus12 1 minus12 1) (5 5 5) (10 10 10) (15 15 15)

2 Extended Rosenbrockfunction [24] 500 1000 5000 10000 (minus12 1 minus12 1 minus12 1) (5 5 5) (10 10 10) (15 15 15)

3 Six-hump function [25] 2 (1 1) (5 5) (10 10) (15 15)

4 Extended Beale function[23] 500 1000 5000 10000 (minus1 minus1 minus1) (05 05 05) (10 10 10) (1 1 1)

5 Three-hump function [25] 2 (1 1) (5 5) (10 10) (15 15)

6 Extended Himmelblaufunction [23] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)

7 Diagonal 2 function [23] 500 1000 5000 10000 (02 02 02) (025 025 025) (05 05 05) (1 1 1)8 NONSCOMP function [24] 500 1000 5000 10000 (1 1 1) (minus1 minus1 minus1) (minus2 minus2 minus2) (minus5 minus5 minus5)

9 Extended DENSCHNBfunction [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)

10 Booth function [25] 2 (1 1) (5 5) (10 10) (15 15)

11 Extended quadratic penaltyfunction QP2 [24] 100 200 300 400 (2 2 2) (5 5 5) (10 10 10) (15 15 15)

12 DIXMAANA function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)13 DIXMAANB function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)14 DIXMAANC function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)15 DIXMAAND function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)16 DIXMAANE function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)17 DIXMAANF function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)18 DIXMAANG function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)19 NONDIA function [24] 500 1000 5000 10000 (minus2 minus2 minus2) (minus1 minus1 minus1) (0 0 0) (1 1 1)

20 Extended Tridiagonal 1function [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)

21 DQDRTIC function [24] 500 1000 5000 10000 (minus1 minus1 minus1) (1 1 1) (2 2 2) (3 3 3)22 Diagonal 4 function [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)23 DQDRTIC function [24] 10 20 30 40 (1 1 1) (minus1 minus1 minus1) (1 1 1) (2 2 2)

24 Extended Cliff function[24] 100 200 300 400 (1 1 1) (5 5 5) (10 10 10) (15 15 15)

Suppose that a parameter 119903119872ge 119903119901119904

for all 119901 119904 is chosen and119903119901119904= 119903119872

if and only if solver 119904 does not solve problem 119901Because we would like to obtain an overall assessment of theperformance of a solver we defined the measure

120588119904(119905) =

1119899119901

size 119901 isin119875 119903119901119904le 119905 (42)

Thus 120588119904(119905) is the probability for solver 119904 isin 119878 that the

performance ratio 119903119901119904

is within a factor 119905 isin 119877 of the bestpossible ratio If we define the function 119901

119904as the cumulative

distribution function for the performance ratio then theperformance measure 119901

119904 119877 rarr [0 1] for a solver is

nondecreasing and piecewise continuous from the right Thevalue of 119901

119904(1) is the probability that the solver has the best

performance of all of the solvers In general a solverwith highvalues of 119901(119905) which would appear in the upper right cornerof the figure is preferable

Based on the left side of Figures 1 and 2 the PRPlowastlowastformula is above the other curves Therefore it is the most

efficient method among related PRP methods in terms ofefficiency and robustness In Figure 2 we see that the curveof PRPlowastlowast is still the best but the efficiency is not goodas the number of iterations since we use the complicatedhybrid algorithm leads to high CPU time Thus using highprocessors computers to find the solution will be moreefficient since the number of iterations decreased rapidlyunder PRPlowastlowast method

5 Conclusion

In this paper we proposed hybrid conjugate gradient methodby using nonnegative PRP and NPRP formulas with the SWPline search which extended the cases of using PRP methodunder mild condition The global convergence property isestablished and it is very simple Our numerical results hadshown that the hybrid method is the best when compared toother related PRP CG methods

Mathematical Problems in Engineering 7

t

e0

e1

e2

e3

Ps(t)

00

02

04

06

08

10

FR

FR

NPRP

VHSNPRPWYL

WYLVHS

PRPlowastlowast

PRPlowastlowast

PRP+

PRP+

Figure 2 Performance profile based on the CPU time

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgments

The authors are grateful to the editor and the anony-mous reviewers for their comments and suggestions whichimproved this paper substantially They would also like tothank The Ministry of Education Malaysia (MOE) for fund-ing this research under The Fundamental Research GrantScheme (Grant no 59256)

References

[1] P Wolfe ldquoConvergence conditions for ascent methodsrdquo SIAMReview vol 11 no 2 pp 226ndash235 1969

[2] PWolfe ldquoConvergence conditions for ascent methods II somecorrectionsrdquo SIAM Review vol 13 no 2 pp 185ndash188 1971

[3] J CaiQ Li L LiH Peng andYYang ldquoA fuzzy adaptive chaoticant swarm optimization for economic dispatchrdquo InternationalJournal of Electrical Power and Energy Systems vol 34 no 1 pp154ndash160 2012

[4] KNatalliaAn Introduction toHeuristic Slgorithms Departmentof Informatics and Telecommunications 2005

[5] L LiH Peng J Kurths Y Yang andH J Schellnhuber ldquoChaos-order transition in foraging behavior of antsrdquo Proceedings of theNational Academy of Sciences of the United States of Americavol 111 no 23 pp 8392ndash8397 2014

[6] M Wan L Li J Xiao C Wang and Y Yang ldquoData clusteringusing bacterial foraging optimizationrdquo Journal of IntelligentInformation Systems vol 38 no 2 pp 321ndash341 2012

[7] M Wan C Wang L Li and Y Yang ldquoChaotic ant swarmapproach for data clusteringrdquo Applied Soft Computing Journalvol 12 no 8 pp 2387ndash2393 2012

[8] M R Hestenes and E Stiefel ldquoMethods of conjugate gradientsfor solving linear systemsrdquo Journal of Research of the NationalBureau of Standards vol 49 no 6 pp 409ndash436 1952

[9] R Fletcher and C M Reeves ldquoFunction minimization byconjugate gradientsrdquo The Computer Journal vol 7 no 2 pp149ndash154 1964

[10] P Elijah and G Ribiere ldquoNote sur la convergence de methodesde directions conjugueesrdquo Revue francaise drsquoinformatique et derecherche operationnelle vol 3 no 1 pp 35ndash43 1969

[11] Z Wei S Yao and L Liu ldquoThe convergence properties of somenew conjugate gradient methodsrdquo Applied Mathematics andComputation vol 183 no 2 pp 1341ndash1350 2006

[12] G Zoutendijk ldquoNonlinear programming computationalmeth-odsrdquo Integer and Nonlinear Programming vol 143 no 1 pp 37ndash86 1970

[13] M Al-Baali ldquoDescent property and global convergence ofthe FletchermdashReeves method with inexact line searchrdquo IMAJournal of Numerical Analysis vol 5 no 1 pp 121ndash124 1985

[14] L Guanghui H Jiye and Y Hongxia ldquoGlobal convergence ofthe fletcher-reeves algorithm with inexact linesearchrdquo AppliedMathematics-A Journal of Chinese Universities vol 10 no 1 pp75ndash82 1995

[15] M J D Powell ldquoNonconvex minimization calculations and theconjugate gradient methodrdquo in Numerical Analysis vol 1066of Lecture Notes in Mathematics pp 122ndash141 Springer BerlinGermany 1984

[16] J C Gilbert and J Nocedal ldquoGlobal convergence properties ofconjugate gradient methods for optimizationrdquo SIAM Journal onOptimization vol 2 no 1 pp 21ndash42 1992

[17] D Touati-Ahmed and C Storey ldquoEfficient hybrid conjugategradient techniquesrdquo Journal of OptimizationTheory and Appli-cations vol 64 no 2 pp 379ndash397 1990

[18] Y Shengwei ZWei andH Huang ldquoA note aboutWYLrsquos conju-gate gradientmethod and its applicationsrdquoAppliedMathematicsand Computation vol 191 no 2 pp 381ndash388 2007

[19] L Zhang ldquoAn improved Wei-Yao-Liu nonlinear conjugategradient method for optimization computationrdquoApplied Math-ematics and Computation vol 215 no 6 pp 2269ndash2274 2009

[20] Z Dai and F Wen ldquoAnother improvedWeindashYaondashLiu nonlinearconjugate gradient method with sufficient descent propertyrdquoAppliedMathematics andComputation vol 218 no 14 pp 7421ndash7430 2012

[21] M Rivaie M Mamat L W June and I Mohd ldquoA new classof nonlinear conjugate gradient coefficients with global conver-gence propertiesrdquo Applied Mathematics and Computation vol218 no 22 pp 11323ndash11332 2012

[22] A AlhawaratMMamatM Rivaie and IMohd ldquoA newmodi-fication of nonlinear conjugate gradient coefficients with globalconvergence propertiesrdquo International Journal of MathematicalComputational Statistical Natural and Physical Engineeringvol 8 no 1 pp 54ndash60 2014

[23] A Neculai ldquoAn unconstrained optimization test functionscollectionrdquo Advanced Modeling and Optimization vol 10 no 1pp 147ndash161 2008

[24] I Bongartz A R Conn N Gould P L Toint and I Bon-gartz Constrained and Unconstrained Testing EnvironmentDepartement de Mathematique 1993

[25] E P Adorio and U Diliman ldquoMvf-multivariate test functionslibrary in c for unconstrained global optimizationrdquo 2005

[26] E D Dolan and J J More ldquoBenchmarking optimization soft-ware with performance profilesrdquo Mathematical Programmingvol 91 no 2 pp 201ndash213 2002

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

Mathematical Problems in Engineering 5

(III) The Zoutendijk condition (25) is satisfied by the linesearch

(IV) 119875119903119900119901119890119903119905119910lowast holds

(V) Assumptions 1 and 2 hold

Then the iterates are globally convergent

The next lemma shows that if the gradients are boundedaway from zero and Propertylowast holds then a certain fractionof steps cannot be too small The proof is given in [16]However we state it for readability

Lemma 5 Consider a CG algorithm as defined in (2) and (3)with the parameter 120573PRP

lowastlowast

119896 If Assumptions 1 and 2 are satisfied

then 119875119903119900119901119890119903119905119910lowast holds

Proof Let 119887 = 212057421205742 ge 1 and 120582 le 12057422119871120574119887

100381610038161003816100381610038161003816120573PRP

lowastlowast

119896

100381610038161003816100381610038161003816le

1003816100381610038161003816100381610038161003816100381610038161003816

100381710038171003817100381711989211989610038171003817100381710038172minus 119892119879119896119892119896minus1

1003817100381710038171003817119892119896minus110038171003817100381710038172

1003816100381610038161003816100381610038161003816100381610038161003816le

100381710038171003817100381711989211989610038171003817100381710038172+10038161003816100381610038161003816119892119879

119896119892119896minus110038161003816100381610038161003816

1003817100381710038171003817119892119896minus110038171003817100381710038172

le10038171003817100381710038171198921198961003817100381710038171003817 (10038171003817100381710038171198921198961003817100381710038171003817 +1003817100381710038171003817119892119896minus1

1003817100381710038171003817)1003817100381710038171003817119892119896minus1

10038171003817100381710038172 le

21205742

1205742= 119887 gt 1

(38)

Using 119904119896 le 120582 we have

100381610038161003816100381610038161003816120573PRP

lowastlowast

119896

100381610038161003816100381610038161003816le

1003816100381610038161003816100381610038161003816100381610038161003816

119892119879119896(119892119896minus 119892119896minus1)

1003817100381710038171003817119892119896minus110038171003817100381710038172

1003816100381610038161003816100381610038161003816100381610038161003816le100381710038171003817100381711989211989610038171003817100381710038171003817100381710038171003817119892119896 minus 119892119896minus1

10038171003817100381710038171003817100381710038171003817119892119896minus1

10038171003817100381710038172 (39)

By Assumption 2

100381610038161003816100381610038161003816120573PRP

lowastlowast

119896

100381610038161003816100381610038161003816le119871120582120574

1205742le

12119887 (40)

The proof is complete

Lemma 6 The CG formula presented in Case 2 has thefollowing properties

(1) 120573PRPlowastlowast

119896gt 0 since the condition 119892

1198962 gt |119892119879

119896119892119896minus1| forces

the CG formula in (20) to be nonnegative

(2) 120573PRPlowastlowast

119896satisfies 119875119903119900119901119890119903119905119910lowast based on Lemma 5

(3) 120573PRPlowastlowast

119896satisfies the sufficient descent condition based on

Theorem 3 and 119888 isin (0 1)

By using Theorems 3 and 4 and Lemma 6 we havethe following convergence result The proof is similar toTheorem 43 which is presented in [16]

Theorem 7 Suppose that Assumption 1 holds Consider theCG method of forms (2) and (3) and 120573

119896as in Case 2

where 120572119896is computed by (5) and (6) with 120590 le 14 then

lim119896rarrinfin

inf 119892119896 = 0

t

e0

e1

e2

e3

Ps(t)

00

02

04

06

08

10

FR

FR

NPRPWYLVHS

VHSNPRPWYL

PRPlowastlowast

PRPlowastlowast

PRP+

PRP+

Figure 1 Performance profile based on the number of iterations

4 Numerical Results and Discussions

To evaluate the efficiency of the new method we selectedsome of the test functions in Table 1 from CUTEr [24]Neculai [23] and Adorio and Diliman [25] We performed acomparison with other CG methods including VHS NPRPPRP+ FR and PRPlowastlowast formulas The tolerance 120576 is selectedto 10minus6 for all algorithms to investigate the rapidity of theiterationmethods towards the optimal solutionThe gradientvalue is used as the stopping criteria Here the stoppingcriteria considered 119892

119896 le 10minus6 We considered the method

failed if the number of iterations exceeds 1000 timesIn Table 1 we selected different initial points for every

function Thus this demonstrated that this method can beused in several real life functions from other fields such asengineering and medical science as we mentioned before Inaddition different dimensions from 500 until 10000 are usedWe also choose from different group of functions We usedMatlab 79 subroutine program with CPU processor Intel (R)Core (TM) i3 CPU and 2GB DDR2 RAM under SWP linesearch to find the optimumpointThe efficiency comparisonsresults are shown in Figures 1 and 2 respectively using aperformance profile introduced by Dolan and More [26]

This performance measure was introduced to compare aset of solvers 119878 on a set of problems 119875 Assuming 119899

119904solvers

and 119899119901problems in 119878 and 119875 respectively the measure 119905

119901119904

is defined as the computation time (eg the number ofiterations or the CPU time) required to solve problem 119901 bysolver 119904

To create a baseline for comparison the performance ofsolver 119904 on problem119901 is scaled by the best performance of anysolver in 119878 on the problem using the ratio

119903119901119904=

119905119901119904

min 119905119901119904 119904 isin 119878

(41)

6 Mathematical Problems in Engineering

Table 1 A list of problem functions used with the SWP condition with 120575 = 001 and 120590 = 01

Number Function Dimensions Initial points

1 Extended White amp Holstfunction [23] 500 1000 5000 10000 (minus12 1 minus12 1 minus12 1) (5 5 5) (10 10 10) (15 15 15)

2 Extended Rosenbrockfunction [24] 500 1000 5000 10000 (minus12 1 minus12 1 minus12 1) (5 5 5) (10 10 10) (15 15 15)

3 Six-hump function [25] 2 (1 1) (5 5) (10 10) (15 15)

4 Extended Beale function[23] 500 1000 5000 10000 (minus1 minus1 minus1) (05 05 05) (10 10 10) (1 1 1)

5 Three-hump function [25] 2 (1 1) (5 5) (10 10) (15 15)

6 Extended Himmelblaufunction [23] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)

7 Diagonal 2 function [23] 500 1000 5000 10000 (02 02 02) (025 025 025) (05 05 05) (1 1 1)8 NONSCOMP function [24] 500 1000 5000 10000 (1 1 1) (minus1 minus1 minus1) (minus2 minus2 minus2) (minus5 minus5 minus5)

9 Extended DENSCHNBfunction [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)

10 Booth function [25] 2 (1 1) (5 5) (10 10) (15 15)

11 Extended quadratic penaltyfunction QP2 [24] 100 200 300 400 (2 2 2) (5 5 5) (10 10 10) (15 15 15)

12 DIXMAANA function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)13 DIXMAANB function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)14 DIXMAANC function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)15 DIXMAAND function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)16 DIXMAANE function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)17 DIXMAANF function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)18 DIXMAANG function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)19 NONDIA function [24] 500 1000 5000 10000 (minus2 minus2 minus2) (minus1 minus1 minus1) (0 0 0) (1 1 1)

20 Extended Tridiagonal 1function [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)

21 DQDRTIC function [24] 500 1000 5000 10000 (minus1 minus1 minus1) (1 1 1) (2 2 2) (3 3 3)22 Diagonal 4 function [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)23 DQDRTIC function [24] 10 20 30 40 (1 1 1) (minus1 minus1 minus1) (1 1 1) (2 2 2)

24 Extended Cliff function[24] 100 200 300 400 (1 1 1) (5 5 5) (10 10 10) (15 15 15)

Suppose that a parameter 119903119872ge 119903119901119904

for all 119901 119904 is chosen and119903119901119904= 119903119872

if and only if solver 119904 does not solve problem 119901Because we would like to obtain an overall assessment of theperformance of a solver we defined the measure

120588119904(119905) =

1119899119901

size 119901 isin119875 119903119901119904le 119905 (42)

Thus 120588119904(119905) is the probability for solver 119904 isin 119878 that the

performance ratio 119903119901119904

is within a factor 119905 isin 119877 of the bestpossible ratio If we define the function 119901

119904as the cumulative

distribution function for the performance ratio then theperformance measure 119901

119904 119877 rarr [0 1] for a solver is

nondecreasing and piecewise continuous from the right Thevalue of 119901

119904(1) is the probability that the solver has the best

performance of all of the solvers In general a solverwith highvalues of 119901(119905) which would appear in the upper right cornerof the figure is preferable

Based on the left side of Figures 1 and 2 the PRPlowastlowastformula is above the other curves Therefore it is the most

efficient method among related PRP methods in terms ofefficiency and robustness In Figure 2 we see that the curveof PRPlowastlowast is still the best but the efficiency is not goodas the number of iterations since we use the complicatedhybrid algorithm leads to high CPU time Thus using highprocessors computers to find the solution will be moreefficient since the number of iterations decreased rapidlyunder PRPlowastlowast method

5 Conclusion

In this paper we proposed hybrid conjugate gradient methodby using nonnegative PRP and NPRP formulas with the SWPline search which extended the cases of using PRP methodunder mild condition The global convergence property isestablished and it is very simple Our numerical results hadshown that the hybrid method is the best when compared toother related PRP CG methods

Mathematical Problems in Engineering 7

t

e0

e1

e2

e3

Ps(t)

00

02

04

06

08

10

FR

FR

NPRP

VHSNPRPWYL

WYLVHS

PRPlowastlowast

PRPlowastlowast

PRP+

PRP+

Figure 2 Performance profile based on the CPU time

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgments

The authors are grateful to the editor and the anony-mous reviewers for their comments and suggestions whichimproved this paper substantially They would also like tothank The Ministry of Education Malaysia (MOE) for fund-ing this research under The Fundamental Research GrantScheme (Grant no 59256)

References

[1] P Wolfe ldquoConvergence conditions for ascent methodsrdquo SIAMReview vol 11 no 2 pp 226ndash235 1969

[2] PWolfe ldquoConvergence conditions for ascent methods II somecorrectionsrdquo SIAM Review vol 13 no 2 pp 185ndash188 1971

[3] J CaiQ Li L LiH Peng andYYang ldquoA fuzzy adaptive chaoticant swarm optimization for economic dispatchrdquo InternationalJournal of Electrical Power and Energy Systems vol 34 no 1 pp154ndash160 2012

[4] KNatalliaAn Introduction toHeuristic Slgorithms Departmentof Informatics and Telecommunications 2005

[5] L LiH Peng J Kurths Y Yang andH J Schellnhuber ldquoChaos-order transition in foraging behavior of antsrdquo Proceedings of theNational Academy of Sciences of the United States of Americavol 111 no 23 pp 8392ndash8397 2014

[6] M Wan L Li J Xiao C Wang and Y Yang ldquoData clusteringusing bacterial foraging optimizationrdquo Journal of IntelligentInformation Systems vol 38 no 2 pp 321ndash341 2012

[7] M Wan C Wang L Li and Y Yang ldquoChaotic ant swarmapproach for data clusteringrdquo Applied Soft Computing Journalvol 12 no 8 pp 2387ndash2393 2012

[8] M R Hestenes and E Stiefel ldquoMethods of conjugate gradientsfor solving linear systemsrdquo Journal of Research of the NationalBureau of Standards vol 49 no 6 pp 409ndash436 1952

[9] R Fletcher and C M Reeves ldquoFunction minimization byconjugate gradientsrdquo The Computer Journal vol 7 no 2 pp149ndash154 1964

[10] P Elijah and G Ribiere ldquoNote sur la convergence de methodesde directions conjugueesrdquo Revue francaise drsquoinformatique et derecherche operationnelle vol 3 no 1 pp 35ndash43 1969

[11] Z Wei S Yao and L Liu ldquoThe convergence properties of somenew conjugate gradient methodsrdquo Applied Mathematics andComputation vol 183 no 2 pp 1341ndash1350 2006

[12] G Zoutendijk ldquoNonlinear programming computationalmeth-odsrdquo Integer and Nonlinear Programming vol 143 no 1 pp 37ndash86 1970

[13] M Al-Baali ldquoDescent property and global convergence ofthe FletchermdashReeves method with inexact line searchrdquo IMAJournal of Numerical Analysis vol 5 no 1 pp 121ndash124 1985

[14] L Guanghui H Jiye and Y Hongxia ldquoGlobal convergence ofthe fletcher-reeves algorithm with inexact linesearchrdquo AppliedMathematics-A Journal of Chinese Universities vol 10 no 1 pp75ndash82 1995

[15] M J D Powell ldquoNonconvex minimization calculations and theconjugate gradient methodrdquo in Numerical Analysis vol 1066of Lecture Notes in Mathematics pp 122ndash141 Springer BerlinGermany 1984

[16] J C Gilbert and J Nocedal ldquoGlobal convergence properties ofconjugate gradient methods for optimizationrdquo SIAM Journal onOptimization vol 2 no 1 pp 21ndash42 1992

[17] D Touati-Ahmed and C Storey ldquoEfficient hybrid conjugategradient techniquesrdquo Journal of OptimizationTheory and Appli-cations vol 64 no 2 pp 379ndash397 1990

[18] Y Shengwei ZWei andH Huang ldquoA note aboutWYLrsquos conju-gate gradientmethod and its applicationsrdquoAppliedMathematicsand Computation vol 191 no 2 pp 381ndash388 2007

[19] L Zhang ldquoAn improved Wei-Yao-Liu nonlinear conjugategradient method for optimization computationrdquoApplied Math-ematics and Computation vol 215 no 6 pp 2269ndash2274 2009

[20] Z Dai and F Wen ldquoAnother improvedWeindashYaondashLiu nonlinearconjugate gradient method with sufficient descent propertyrdquoAppliedMathematics andComputation vol 218 no 14 pp 7421ndash7430 2012

[21] M Rivaie M Mamat L W June and I Mohd ldquoA new classof nonlinear conjugate gradient coefficients with global conver-gence propertiesrdquo Applied Mathematics and Computation vol218 no 22 pp 11323ndash11332 2012

[22] A AlhawaratMMamatM Rivaie and IMohd ldquoA newmodi-fication of nonlinear conjugate gradient coefficients with globalconvergence propertiesrdquo International Journal of MathematicalComputational Statistical Natural and Physical Engineeringvol 8 no 1 pp 54ndash60 2014

[23] A Neculai ldquoAn unconstrained optimization test functionscollectionrdquo Advanced Modeling and Optimization vol 10 no 1pp 147ndash161 2008

[24] I Bongartz A R Conn N Gould P L Toint and I Bon-gartz Constrained and Unconstrained Testing EnvironmentDepartement de Mathematique 1993

[25] E P Adorio and U Diliman ldquoMvf-multivariate test functionslibrary in c for unconstrained global optimizationrdquo 2005

[26] E D Dolan and J J More ldquoBenchmarking optimization soft-ware with performance profilesrdquo Mathematical Programmingvol 91 no 2 pp 201ndash213 2002

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

6 Mathematical Problems in Engineering

Table 1 A list of problem functions used with the SWP condition with 120575 = 001 and 120590 = 01

Number Function Dimensions Initial points

1 Extended White amp Holstfunction [23] 500 1000 5000 10000 (minus12 1 minus12 1 minus12 1) (5 5 5) (10 10 10) (15 15 15)

2 Extended Rosenbrockfunction [24] 500 1000 5000 10000 (minus12 1 minus12 1 minus12 1) (5 5 5) (10 10 10) (15 15 15)

3 Six-hump function [25] 2 (1 1) (5 5) (10 10) (15 15)

4 Extended Beale function[23] 500 1000 5000 10000 (minus1 minus1 minus1) (05 05 05) (10 10 10) (1 1 1)

5 Three-hump function [25] 2 (1 1) (5 5) (10 10) (15 15)

6 Extended Himmelblaufunction [23] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)

7 Diagonal 2 function [23] 500 1000 5000 10000 (02 02 02) (025 025 025) (05 05 05) (1 1 1)8 NONSCOMP function [24] 500 1000 5000 10000 (1 1 1) (minus1 minus1 minus1) (minus2 minus2 minus2) (minus5 minus5 minus5)

9 Extended DENSCHNBfunction [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)

10 Booth function [25] 2 (1 1) (5 5) (10 10) (15 15)

11 Extended quadratic penaltyfunction QP2 [24] 100 200 300 400 (2 2 2) (5 5 5) (10 10 10) (15 15 15)

12 DIXMAANA function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)13 DIXMAANB function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)14 DIXMAANC function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)15 DIXMAAND function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)16 DIXMAANE function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)17 DIXMAANF function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)18 DIXMAANG function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)19 NONDIA function [24] 500 1000 5000 10000 (minus2 minus2 minus2) (minus1 minus1 minus1) (0 0 0) (1 1 1)

20 Extended Tridiagonal 1function [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)

21 DQDRTIC function [24] 500 1000 5000 10000 (minus1 minus1 minus1) (1 1 1) (2 2 2) (3 3 3)22 Diagonal 4 function [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)23 DQDRTIC function [24] 10 20 30 40 (1 1 1) (minus1 minus1 minus1) (1 1 1) (2 2 2)

24 Extended Cliff function[24] 100 200 300 400 (1 1 1) (5 5 5) (10 10 10) (15 15 15)

Suppose that a parameter 119903119872ge 119903119901119904

for all 119901 119904 is chosen and119903119901119904= 119903119872

if and only if solver 119904 does not solve problem 119901Because we would like to obtain an overall assessment of theperformance of a solver we defined the measure

120588119904(119905) =

1119899119901

size 119901 isin119875 119903119901119904le 119905 (42)

Thus 120588119904(119905) is the probability for solver 119904 isin 119878 that the

performance ratio 119903119901119904

is within a factor 119905 isin 119877 of the bestpossible ratio If we define the function 119901

119904as the cumulative

distribution function for the performance ratio then theperformance measure 119901

119904 119877 rarr [0 1] for a solver is

nondecreasing and piecewise continuous from the right Thevalue of 119901

119904(1) is the probability that the solver has the best

performance of all of the solvers In general a solverwith highvalues of 119901(119905) which would appear in the upper right cornerof the figure is preferable

Based on the left side of Figures 1 and 2 the PRPlowastlowastformula is above the other curves Therefore it is the most

efficient method among related PRP methods in terms ofefficiency and robustness In Figure 2 we see that the curveof PRPlowastlowast is still the best but the efficiency is not goodas the number of iterations since we use the complicatedhybrid algorithm leads to high CPU time Thus using highprocessors computers to find the solution will be moreefficient since the number of iterations decreased rapidlyunder PRPlowastlowast method

5 Conclusion

In this paper we proposed hybrid conjugate gradient methodby using nonnegative PRP and NPRP formulas with the SWPline search which extended the cases of using PRP methodunder mild condition The global convergence property isestablished and it is very simple Our numerical results hadshown that the hybrid method is the best when compared toother related PRP CG methods

Mathematical Problems in Engineering 7

t

e0

e1

e2

e3

Ps(t)

00

02

04

06

08

10

FR

FR

NPRP

VHSNPRPWYL

WYLVHS

PRPlowastlowast

PRPlowastlowast

PRP+

PRP+

Figure 2 Performance profile based on the CPU time

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgments

The authors are grateful to the editor and the anony-mous reviewers for their comments and suggestions whichimproved this paper substantially They would also like tothank The Ministry of Education Malaysia (MOE) for fund-ing this research under The Fundamental Research GrantScheme (Grant no 59256)

References

[1] P Wolfe ldquoConvergence conditions for ascent methodsrdquo SIAMReview vol 11 no 2 pp 226ndash235 1969

[2] PWolfe ldquoConvergence conditions for ascent methods II somecorrectionsrdquo SIAM Review vol 13 no 2 pp 185ndash188 1971

[3] J CaiQ Li L LiH Peng andYYang ldquoA fuzzy adaptive chaoticant swarm optimization for economic dispatchrdquo InternationalJournal of Electrical Power and Energy Systems vol 34 no 1 pp154ndash160 2012

[4] KNatalliaAn Introduction toHeuristic Slgorithms Departmentof Informatics and Telecommunications 2005

[5] L LiH Peng J Kurths Y Yang andH J Schellnhuber ldquoChaos-order transition in foraging behavior of antsrdquo Proceedings of theNational Academy of Sciences of the United States of Americavol 111 no 23 pp 8392ndash8397 2014

[6] M Wan L Li J Xiao C Wang and Y Yang ldquoData clusteringusing bacterial foraging optimizationrdquo Journal of IntelligentInformation Systems vol 38 no 2 pp 321ndash341 2012

[7] M Wan C Wang L Li and Y Yang ldquoChaotic ant swarmapproach for data clusteringrdquo Applied Soft Computing Journalvol 12 no 8 pp 2387ndash2393 2012

[8] M R Hestenes and E Stiefel ldquoMethods of conjugate gradientsfor solving linear systemsrdquo Journal of Research of the NationalBureau of Standards vol 49 no 6 pp 409ndash436 1952

[9] R Fletcher and C M Reeves ldquoFunction minimization byconjugate gradientsrdquo The Computer Journal vol 7 no 2 pp149ndash154 1964

[10] P Elijah and G Ribiere ldquoNote sur la convergence de methodesde directions conjugueesrdquo Revue francaise drsquoinformatique et derecherche operationnelle vol 3 no 1 pp 35ndash43 1969

[11] Z Wei S Yao and L Liu ldquoThe convergence properties of somenew conjugate gradient methodsrdquo Applied Mathematics andComputation vol 183 no 2 pp 1341ndash1350 2006

[12] G Zoutendijk ldquoNonlinear programming computationalmeth-odsrdquo Integer and Nonlinear Programming vol 143 no 1 pp 37ndash86 1970

[13] M Al-Baali ldquoDescent property and global convergence ofthe FletchermdashReeves method with inexact line searchrdquo IMAJournal of Numerical Analysis vol 5 no 1 pp 121ndash124 1985

[14] L Guanghui H Jiye and Y Hongxia ldquoGlobal convergence ofthe fletcher-reeves algorithm with inexact linesearchrdquo AppliedMathematics-A Journal of Chinese Universities vol 10 no 1 pp75ndash82 1995

[15] M J D Powell ldquoNonconvex minimization calculations and theconjugate gradient methodrdquo in Numerical Analysis vol 1066of Lecture Notes in Mathematics pp 122ndash141 Springer BerlinGermany 1984

[16] J C Gilbert and J Nocedal ldquoGlobal convergence properties ofconjugate gradient methods for optimizationrdquo SIAM Journal onOptimization vol 2 no 1 pp 21ndash42 1992

[17] D Touati-Ahmed and C Storey ldquoEfficient hybrid conjugategradient techniquesrdquo Journal of OptimizationTheory and Appli-cations vol 64 no 2 pp 379ndash397 1990

[18] Y Shengwei ZWei andH Huang ldquoA note aboutWYLrsquos conju-gate gradientmethod and its applicationsrdquoAppliedMathematicsand Computation vol 191 no 2 pp 381ndash388 2007

[19] L Zhang ldquoAn improved Wei-Yao-Liu nonlinear conjugategradient method for optimization computationrdquoApplied Math-ematics and Computation vol 215 no 6 pp 2269ndash2274 2009

[20] Z Dai and F Wen ldquoAnother improvedWeindashYaondashLiu nonlinearconjugate gradient method with sufficient descent propertyrdquoAppliedMathematics andComputation vol 218 no 14 pp 7421ndash7430 2012

[21] M Rivaie M Mamat L W June and I Mohd ldquoA new classof nonlinear conjugate gradient coefficients with global conver-gence propertiesrdquo Applied Mathematics and Computation vol218 no 22 pp 11323ndash11332 2012

[22] A AlhawaratMMamatM Rivaie and IMohd ldquoA newmodi-fication of nonlinear conjugate gradient coefficients with globalconvergence propertiesrdquo International Journal of MathematicalComputational Statistical Natural and Physical Engineeringvol 8 no 1 pp 54ndash60 2014

[23] A Neculai ldquoAn unconstrained optimization test functionscollectionrdquo Advanced Modeling and Optimization vol 10 no 1pp 147ndash161 2008

[24] I Bongartz A R Conn N Gould P L Toint and I Bon-gartz Constrained and Unconstrained Testing EnvironmentDepartement de Mathematique 1993

[25] E P Adorio and U Diliman ldquoMvf-multivariate test functionslibrary in c for unconstrained global optimizationrdquo 2005

[26] E D Dolan and J J More ldquoBenchmarking optimization soft-ware with performance profilesrdquo Mathematical Programmingvol 91 no 2 pp 201ndash213 2002

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

Mathematical Problems in Engineering 7

t

e0

e1

e2

e3

Ps(t)

00

02

04

06

08

10

FR

FR

NPRP

VHSNPRPWYL

WYLVHS

PRPlowastlowast

PRPlowastlowast

PRP+

PRP+

Figure 2 Performance profile based on the CPU time

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgments

The authors are grateful to the editor and the anony-mous reviewers for their comments and suggestions whichimproved this paper substantially They would also like tothank The Ministry of Education Malaysia (MOE) for fund-ing this research under The Fundamental Research GrantScheme (Grant no 59256)

References

[1] P Wolfe ldquoConvergence conditions for ascent methodsrdquo SIAMReview vol 11 no 2 pp 226ndash235 1969

[2] PWolfe ldquoConvergence conditions for ascent methods II somecorrectionsrdquo SIAM Review vol 13 no 2 pp 185ndash188 1971

[3] J CaiQ Li L LiH Peng andYYang ldquoA fuzzy adaptive chaoticant swarm optimization for economic dispatchrdquo InternationalJournal of Electrical Power and Energy Systems vol 34 no 1 pp154ndash160 2012

[4] KNatalliaAn Introduction toHeuristic Slgorithms Departmentof Informatics and Telecommunications 2005

[5] L LiH Peng J Kurths Y Yang andH J Schellnhuber ldquoChaos-order transition in foraging behavior of antsrdquo Proceedings of theNational Academy of Sciences of the United States of Americavol 111 no 23 pp 8392ndash8397 2014

[6] M Wan L Li J Xiao C Wang and Y Yang ldquoData clusteringusing bacterial foraging optimizationrdquo Journal of IntelligentInformation Systems vol 38 no 2 pp 321ndash341 2012

[7] M Wan C Wang L Li and Y Yang ldquoChaotic ant swarmapproach for data clusteringrdquo Applied Soft Computing Journalvol 12 no 8 pp 2387ndash2393 2012

[8] M R Hestenes and E Stiefel ldquoMethods of conjugate gradientsfor solving linear systemsrdquo Journal of Research of the NationalBureau of Standards vol 49 no 6 pp 409ndash436 1952

[9] R Fletcher and C M Reeves ldquoFunction minimization byconjugate gradientsrdquo The Computer Journal vol 7 no 2 pp149ndash154 1964

[10] P Elijah and G Ribiere ldquoNote sur la convergence de methodesde directions conjugueesrdquo Revue francaise drsquoinformatique et derecherche operationnelle vol 3 no 1 pp 35ndash43 1969

[11] Z Wei S Yao and L Liu ldquoThe convergence properties of somenew conjugate gradient methodsrdquo Applied Mathematics andComputation vol 183 no 2 pp 1341ndash1350 2006

[12] G Zoutendijk ldquoNonlinear programming computationalmeth-odsrdquo Integer and Nonlinear Programming vol 143 no 1 pp 37ndash86 1970

[13] M Al-Baali ldquoDescent property and global convergence ofthe FletchermdashReeves method with inexact line searchrdquo IMAJournal of Numerical Analysis vol 5 no 1 pp 121ndash124 1985

[14] L Guanghui H Jiye and Y Hongxia ldquoGlobal convergence ofthe fletcher-reeves algorithm with inexact linesearchrdquo AppliedMathematics-A Journal of Chinese Universities vol 10 no 1 pp75ndash82 1995

[15] M J D Powell ldquoNonconvex minimization calculations and theconjugate gradient methodrdquo in Numerical Analysis vol 1066of Lecture Notes in Mathematics pp 122ndash141 Springer BerlinGermany 1984

[16] J C Gilbert and J Nocedal ldquoGlobal convergence properties ofconjugate gradient methods for optimizationrdquo SIAM Journal onOptimization vol 2 no 1 pp 21ndash42 1992

[17] D Touati-Ahmed and C Storey ldquoEfficient hybrid conjugategradient techniquesrdquo Journal of OptimizationTheory and Appli-cations vol 64 no 2 pp 379ndash397 1990

[18] Y Shengwei ZWei andH Huang ldquoA note aboutWYLrsquos conju-gate gradientmethod and its applicationsrdquoAppliedMathematicsand Computation vol 191 no 2 pp 381ndash388 2007

[19] L Zhang ldquoAn improved Wei-Yao-Liu nonlinear conjugategradient method for optimization computationrdquoApplied Math-ematics and Computation vol 215 no 6 pp 2269ndash2274 2009

[20] Z Dai and F Wen ldquoAnother improvedWeindashYaondashLiu nonlinearconjugate gradient method with sufficient descent propertyrdquoAppliedMathematics andComputation vol 218 no 14 pp 7421ndash7430 2012

[21] M Rivaie M Mamat L W June and I Mohd ldquoA new classof nonlinear conjugate gradient coefficients with global conver-gence propertiesrdquo Applied Mathematics and Computation vol218 no 22 pp 11323ndash11332 2012

[22] A AlhawaratMMamatM Rivaie and IMohd ldquoA newmodi-fication of nonlinear conjugate gradient coefficients with globalconvergence propertiesrdquo International Journal of MathematicalComputational Statistical Natural and Physical Engineeringvol 8 no 1 pp 54ndash60 2014

[23] A Neculai ldquoAn unconstrained optimization test functionscollectionrdquo Advanced Modeling and Optimization vol 10 no 1pp 147ndash161 2008

[24] I Bongartz A R Conn N Gould P L Toint and I Bon-gartz Constrained and Unconstrained Testing EnvironmentDepartement de Mathematique 1993

[25] E P Adorio and U Diliman ldquoMvf-multivariate test functionslibrary in c for unconstrained global optimizationrdquo 2005

[26] E D Dolan and J J More ldquoBenchmarking optimization soft-ware with performance profilesrdquo Mathematical Programmingvol 91 no 2 pp 201ndash213 2002

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of