Neural Network Methods for Neural Network Methods for Boundary Value Problems with Boundary Value Problems with

Irregular BoundariesIrregular Boundaries

I. E. Lagaris, A. Likas, D. G. PapageorgiouI. E. Lagaris, A. Likas, D. G. Papageorgiou

University of IoanninaUniversity of Ioannina

Ioannina - GREECEIoannina - GREECE

Why Neural Networks ?Why Neural Networks ?

• Have already been successfully used on problems with regular boundaries†.

• Analytic, closed form solutionAnalytic, closed form solution.• Highly efficient on parallel

hardware.

†I. E. Lagaris, A. Likas and D. I. Fotiadis, IEEE TNN 9 (1998) pp 987-1000†I. E. Lagaris, A. Likas and D. I. Fotiadis, IEEE TNN 9 (1998) pp 987-1000

What is an What is an Irregular Irregular boundaryboundary ? ?

• A boundary that has not a simple A boundary that has not a simple geometrical shape.geometrical shape.

• A boundary that is described as a set of A boundary that is described as a set of distinct points that belong to it.distinct points that belong to it.

DifficultiesDifficulties• Complex shapes pose severe problems to the Complex shapes pose severe problems to the

existing solution techniques.existing solution techniques.

• Extensions of methods that would apply to Extensions of methods that would apply to problems with simple geometry problems with simple geometry are not trivialare not trivial..

• We here present such an extension, to a We here present such an extension, to a method based on method based on Neural NetworksNeural Networks. .

Statement of the problemStatement of the problem

• Solve the equation: Solve the equation: LL(x) = f(x), x(x) = f(x), xRR(N)(N)

subject to Dirichlet or Neumann BCs.subject to Dirichlet or Neumann BCs.

• L L is a differential non-linear operator is a differential non-linear operator

• The bounding hypersurface may be The bounding hypersurface may be either simple or complex.either simple or complex.

The Case of Simple BoundariesThe Case of Simple Boundaries

• If the boundary is a hypercube then for the If the boundary is a hypercube then for the case of Dirichlet BCs we have developed case of Dirichlet BCs we have developed the following solution model.the following solution model.

mm(x) = B(x) + Z(x)N(x,p)(x) = B(x) + Z(x)N(x,p)

• B(x)B(x) satisfies the boundary conditions.satisfies the boundary conditions.• Z(x)Z(x) is zero is zero onlyonly on the boundary. on the boundary.• N(x,p)N(x,p) is a Neural Network.is a Neural Network.

The The ZZ-function-functionIn the case of an orthogonal hyperboxIn the case of an orthogonal hyperbox

the the ZZ-function is readily constructed as:-function is readily constructed as:

Z(x) = Z(x) = i i (x(xi i -a-ai i )(x)(xi i -b-bi i ))

where where xxi i is the is the iithth component of component of xx that lies in that lies in

the interval [the interval [aai i , b, bi i ].].

In the case of irregular boundaries In the case of irregular boundaries

thethe Z-functionZ-function is not easily constructed.is not easily constructed.

The ProcedureThe Procedure

• Let Let xx(k)(k) be points in the bounded domain. be points in the bounded domain.

• The “Error” is defined as:The “Error” is defined as:

E(E(pp) = ) = kk { {LLmm(x(x(k)(k)) - f(x) - f(x(k)(k)))}}22

and is minimized with respect to the and is minimized with respect to the Neural Network parameters Neural Network parameters pp. .

The resulting modelThe resulting modelmm(x) = B(x) + Z(x)N(x,p) (x) = B(x) + Z(x)N(x,p)

is an approximate solution.is an approximate solution.

Modifications for Irregular Modifications for Irregular Boundaries.Boundaries.

• The Z-function is not easy to construct.The Z-function is not easy to construct.

• The Dirichlet BCs are cast as:The Dirichlet BCs are cast as:

mm(X(X(i)(i)) = b) = bii

wherewhere XX(i)(i) are points are points onon the boundary. the boundary.

There are two options that we examined There are two options that we examined for constructing the solution model.for constructing the solution model.

Constrained OptimizationConstrained Optimization

• The model is written as: The model is written as: mm(x) = N(x,p) (x) = N(x,p)

• The Error to be optimized is taken as:The Error to be optimized is taken as:

2)(2)()( })({)}()({ jj

mj

k

k

km bXxfxL

Where Where μ > 0 μ > 0 is a penalty parameter. is a penalty parameter. XX(j)(j) are boundary points. are boundary points.xx(k)(k) are points in the solution domain are points in the solution domain ..

RBF-CorrectionRBF-Correction

• The model is made up, as a sum of two The model is made up, as a sum of two Networks, a perceptron N(x,p) and a Networks, a perceptron N(x,p) and a Radial Basis Functions (RBF) Network.Radial Basis Functions (RBF) Network.

j

Xxjm

j

eapxNx2)( )(),()(

•ααjj are chosen so as to satisfy the BCs exactly. are chosen so as to satisfy the BCs exactly. •λλ is chosen so as to ease the numerics. is chosen so as to ease the numerics.

Pros and ConsPros and Cons• The constrained optimization approach is The constrained optimization approach is

very efficient compared to the RBF synergy very efficient compared to the RBF synergy approach, since to determine the RBF approach, since to determine the RBF coefficients, a linear system must be solved coefficients, a linear system must be solved every time.every time.

• The RBF correction guarantees exact The RBF correction guarantees exact satisfaction of the BCs, which is not the case satisfaction of the BCs, which is not the case in the constrained optimization approach.in the constrained optimization approach.

ProcedureProcedure• Use the constrained optimization to obtain a Use the constrained optimization to obtain a

solution that satisfies the BCs approximately.solution that satisfies the BCs approximately.

• The obtained solution The obtained solution mm(x) = N(x,p) (x) = N(x,p) may be may be

corrected via the RBF approach to exactly corrected via the RBF approach to exactly satisfy the BCs. satisfy the BCs.

• The correction will be small and local, The correction will be small and local, centered around the boundary points.centered around the boundary points.

j

Xxjm

j

eapxNx2)( )(),()(

Experimental ResultsExperimental Results

We experimented with several domains.We experimented with several domains.

• A star with six corners.A star with six corners.

• A cardioidA cardioid

• A part of a hollow sphere.A part of a hollow sphere.

•Boundary points : 109

•Domain points : 391

•Boundary points: 100

•Domain points: 500

Example IExample I

22222),(2

)1(

41),(

yxyxeyx yx

• The analytic solution is:The analytic solution is:

• We solved this problem with Dirichlet BCs We solved this problem with Dirichlet BCs in the star shaped domain.in the star shaped domain.

• We plot the difference between the model We plot the difference between the model and the analytic solution.and the analytic solution.

)1log( 22 yx

|),(),(| yxyxAccuracy exactm

Example IIExample II

• The same (highly non-linear) example The same (highly non-linear) example inside the cardioid domain.inside the cardioid domain.

• We solved it for both Dirichlet and We solved it for both Dirichlet and Neumann BCs by extending the method Neumann BCs by extending the method appropriately. appropriately.

22222),(2

)1(

41),(

yxyxeyx yx

|),(),(| yxyxAccuracy exactm

DiscussionDiscussion

• Similar results hold for the 3-D problem.Similar results hold for the 3-D problem.

• We tested the generalization of the model We tested the generalization of the model by comparing it to the analytic solution by comparing it to the analytic solution in points other than the training points.in points other than the training points.

• The conclusion is that the deviation is in The conclusion is that the deviation is in the same range as for the training points.the same range as for the training points.

ToolsTools• For the optimization procedure we used the For the optimization procedure we used the

Merlin 3.0 Merlin 3.0 Optimization PackageOptimization Package..

• Special linear solvers may be employed for Special linear solvers may be employed for the calculation of the “error” gradient in the calculation of the “error” gradient in the case of the RBF synergy approach.the case of the RBF synergy approach.

• Implementation in parallel machines or on Implementation in parallel machines or on the so called “neuroprocessors” will greatly the so called “neuroprocessors” will greatly contribute to the acceleration of the method.contribute to the acceleration of the method.

ConclusionsConclusions• The method we presented is suitable for The method we presented is suitable for

handling complex boundaries with little effort.handling complex boundaries with little effort.

• We demonstrated its applicability by solving We demonstrated its applicability by solving highly non-linear PDEs.highly non-linear PDEs.

• Currently we are working on a method to Currently we are working on a method to construct a suitable Z-function for irregular construct a suitable Z-function for irregular boundaries.boundaries.