8/9/2019 Mandani

1/8

A COMPARISON OF MANDANI AND SUGENO INFERENCE SYSTEMS FOR A

SPACE FAULT DETECTION APPLICATION

J. J. Jassbi, Azad University, Science and Research Campus, Iran, jassbi@iaucss.org

Paulo J. A. Serra, UNINOVA, Portugal, pja@uninova.pt

Rita A. Ribeiro, UNINOVA, Portugal, rar@ uninova.pt

Alessandro Donati, ESA/ESOC, Darmstad, Germany, alessandro.donati@esa.int

ABSTRACT

This research provides a comparison between the performances of TSK (Takagi, Sugeno, Kang)-type versus Mamdani-type fuzzy inference systems. The main motivation behind this research

was to assess which approach provides the best performance for a gyroscope fault-detection

application, developed in 2002 for the European Space Agency (ESA) satellite ENVISAT. Due to

the importance of performance in online systems we compare the application, developed with

Mamdani model, with a TSK formulation using three types of tests: processing time for both

systems, robustness in the presence of randomly generated noise; and sensitivity analysis of the

systems behaviors to changes in input data. The results show that the TSK model perform better

in all three tests, hence we may conclude that replacing a Mamdani system with an equivalent

TSK system could be a good option to improve the overall performance of a fuzzy inference

system.

KEYWORDS: Fuzzy Inference Systems, Fault detection, Sensitivity analysis

1. INTRODUCTION

Intelligent monitoring and diagnostic tools are essential for mission control purposes, as for

example assessing and monitoring the health status of spacecraft and satellite components, such as

gyroscopes. In 2002 UNINOVA (Portugal) in cooperation with GTD (Spain), developed a

software system called ENVISAT GYROSCOPE MONITOR for the European Space Agency

(ESA) (see details in [1]). The fuzzy inference system (FIS) developed in the ESA project for

monitoring the gyroscopes of the ENVISAT satellite provides the spacecraft controller's team,

responsible for satellite operations, with a new type of gyroscope health-monitoring tool. This

monitoring tool generates alarms with different degrees of criticality and a severity level of the

alarm itself, instead of a simple presence/absence of an alarm. It also supplies an explanation forthe triggering of the alarm. The tool includes three main components, all developed with a

Mamdani inference mechanism [1], [2]: (1) a fault detection module that monitors the health of

each of the 4 gyroscopes; (2) a data quality system to assess the quality of the data triggering the

alarms; (3) and a generic fault detection module for the overall system.

In this paper we only use the last case, generic system fault detection module, to explain the

comparison between both Mamdani [3] and TSK (Takagi, Sugeno and Kang) FIS [4] [5] and to

discuss the performance tests made. The main objective behind this research was to assess which

approach provides the best performance for the Space Fault Detection application. Due to the

importance of inference time and robustness of results in online systems, we compare the

application, developed with Mamdani model, with a TSK formulation using three tests:

processing time, i.e. operational process time of each model in the same situation; robustness of

8/9/2019 Mandani

2/8

the system in case of noise; and sensitivity analysis of the system behavior when we change the

input variables from its minimum values to its maximum, and from zero to max (variables can be

seen in Table 2).

The paper is organized as follows. The next section presents the motivation for the comparison ofMamdani versus TSK types of inference systems. In section 3 the case study and variables are

introduced. In section 4 the results of comparing the performance of the two inference schemes

are discussed and finally in section 5 we present the conclusions.

2. MOTIVATION FOR COMPARING MANDANI FIS AND TSK FIS

In terms of inference process there are two main types of Fuzzy Inference Systems (FIS): the

Mamdani-type [3] and the TSK-type [4].

In terms of use, the Mamdani FIS is more widely used, mostly because it provides reasonable

results with a relatively simple structure, and also due to the intuitive and interpretable nature of

the rule base. Since the consequents of the rules in a TSK FIS are not fuzzy this interpretability is

lost; however, since the TSK FISs rules consequents can have as many parameters per rule asinput values, this translates into more degrees of freedom in the design than a Mamdani FIS thus

providing the systems designer with more flexibility in the design of the system [5]. However, it

should be noted that the Mamdani FIS can be used directly for either MISO systems (multiple

input single output) as well as for MIMO systems (multiple input multiple output), while the TSK

FIS can only be used in MISO systems (we explain below this issue).

The fact that a Mamdani FIS can be seen as a function that maps the systems input space into its

output space, ensures that there exists a TSK FIS that can approximate any given Mamdani FIS

with an arbitrary level of precision (based on a result known as universal approximation theorem

[6]). It is beyond the scope in this paper to explain in detail the formalisms of this comparison.

For a comprehensive comparison and description on several approximate reasoning methods,

including Mamdani FISs and TSK FISs, see [7].Summarizing, our main motivations for testing the Space monitoring application developed with

Mamdani FIS and with a TSK FIS and to compare the results are:

The TSK FIS is more flexible because it allows more parameters in the output and since

the output is a function of the inputs it expresses a more explicit relation among them;

In computational terms the TSK FIS is more effective because the complex

defuzzification process of the Mamdani FIS is replaced with a weighted average;

Because of the structure of the TSK FIS rule outputs, it is more adequate for functional

analysis than a Mamdani FIS.

From the above, it seems that any TSK FIS is always more efficient than a Mamdani FIS and the

question to ask is why wasnt the Space monitoring application developed from scratch with a

TSK FIS? There are two important reasons for this:

1. For classification problems, where the rules outputs are usually independent of each other,

i.e. MIMO systems, it does not make any sense to aggregate different nature outputs with

a weighted average. However, to select the output with the best match (max-min

inference), as in Mamdani FIS, makes perfect sense. TSK FIS is ONLYsuitable for MISO

problems, i.e. systems with the same output linguistic variable. Of course, any MIMO can

always be divided into several MISOs

2. The monitoring tool developed included two MISO FIS (the gyroscopes fault detection

and the data quality fault detection) but only the generic system level of the alarms is a

MISO system. Hence, we selected the latter to develop all modules with the same type of

FIS.

8/9/2019 Mandani

3/8

In summary, in this research only the generic system level alarms module is considered for the

performance comparison of the two FIS because it is a MISO and the three tests performed with

both FIS are: processing time, robustness with respect to noise and sensitivity analysis to changes

in the inputs.

3. GYROSCOPES GENERIC SYSTEM LEVEL

Fig.1. shows the general system level model (from the ESA monitoring tool) which is used in this

work for comparing the Mamdani and TSK FIS. Please note that some of the input variables, in

the left side, were divided in two (as shown in the Mamdani FIS box- Fig 1), thus resulting in 11

input variables in the generic system level monitoring system.

Theta_X

Theta_Y

Theta_Z

Drift_X

Drift_Y

Drift_Z

Incoherency Index

Max

Max

Max

Max/Max Delta

Max/Max Delta

Max/Max Delta

Max

Count

MANDANI FUZZY SYSTEM

Max_Theta_X

Max_Theta_Y

Max_Theta_Z

Max_Drift_X

Max_Drift_Y

Max_Drift_Z

Max_Incoherency Index

Max_Delta_Drift_X

Max_Delta_Drift_Y

Max_Delta_Drift_Z

Times_Exceds_Incoherency

No Alarm

No-Gyro Soft Alert

Soft Gyro Alert

Gyro Alert

Serious Gyro Alert

Critical Gyro Alert

Figure 1. General System Level Model

The input variables are described in Table 1 and their respective fuzzifications are presented in

Table 2. Due to paper size limitations only one example of the fuzzified variables for each axis is

shown (Table 2), since the others are similar.

Variables Description

Max_Theta_X/Y/Z

(variables 1,2,3)

For each of the three satellite axes X/Y/Z this represents the maximum

value of the estimated Attitude.

Max_Drift_X/Y/Z

(variables 4,5,6)

For each of the three satellite axes X/Y/Z this represents the maximum

value of the estimated Drift.

Max_Delta_Drift_X/Y/Z

(variables 7,8,9)

For each of the three satellite axes X/Y/Z this represents the (absolute

value of the) difference between the maximum and minimum value of theestimated Drift

Max_Coherency_Index

(variable 10)

The maximum value of the coherency index.

Coherency_Index_Trigger

(variable 11)

The number of times when the coherency index exceeds a given (user

configurable) threshold.

Table 1. Brief description of the input variables

8/9/2019 Mandani

4/8

Table 2. The fuzzified input variables of the model.

In this research 73 rules are used and the output variables and their definition are as follows:

Alarm level Mamdani TSK

Name: Function Type: Parameters: Parameters

(singletons):

Nominal Trapezoidal [0 0 20 30] [12.36]

Non-GyroSoftAlert Trapezoidal [10 20 40 50] [30]

SoftGyroAlert Trapezoidal [30 40 60 70] [50]

GyroAlert Trapezoidal [50 60 80 90] [70]

SeriousGyroAlert Trapezoidal [70 80 100 110] [90]

CriticalGyroAlert Trapezoidal [90 100 120 120] [107.6]

Table 3. Output variables definitions for both models

It should be noted, that we slightly modified the original model outputs (they were almost uniform

functions) to highlight the comparison between both FIS. Further, we used the centers of gravity

of the output membership functions as the TSK output parameters.The input variables are identical in both FIS and the difference is in the consequents of the rules

as can be observed in Table 3. The min operator is used for the implication in both FIS. For the

Mamdani FIS, the aggregation was done using the max operator and for defuzzifier the center of

gravity was used. For the TSK FIS the aggregation was done with the classical weighted average,

using the singletons presented in Table 3.

4. COMPARISON TESTS

To compare the performance of the two types of rule base models, we use three kinds of tests, as

mentioned in the introduction. Details about each test and discussion of results are presented in

the next sub-sections.

8/9/2019 Mandani

5/8

4.1 Processing Time test

In this test a set of randomly generated data was used (increasing the number of input vectors

from 0 to 35000). For each measurement the same set of data was used for both systems.

Throughout the procedure, increasingly larger sets (10 times more for each test) of input data weretested to see how the execution time evolved.

As for the actual measurements, they were taken as the mean of ten measurements obtained for

each set of input data and for each system. Figure 2 presents the results obtained for both systems.

0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

0 5000 10000 15000 20000 25000 30000

Number of Input Vectors

ExecutionTime(seconds).

Execution time for Mamdani FIS Execution time for Sugeno FIS

Figure 2. Comparison of process time operation between Mamdani and TSK

On the average, the Mamdani system required 14 times more processing time than the TSK

system. More specifically, on the average, the time taken for the Mamdani system to return one

result was 4.6 x-4

seconds and 3.2 x-5

seconds for the TSK system.

4.2 Robustness to noise

The data used in this test was a set of ten thousand randomly generated input vectors. These

points were changed with randomly generated noise, using a normal distribution with zero mean

and increasing the variance 10 times for each test (Figure 3 shows the variance increases in the xx

axis).

0

20

40

60

80

100

120

0.00000 0.00001 0.00010 0.00100 0.01000 0.10000 1.00000

Variance (Logaritmic scale)

Mean

Squared

Error

Mamdani FIS Sugeno FIS

Figure 3. Comparison of Mamdani with two types of TSK by using sensitive analysis

8/9/2019 Mandani

6/8

Figure 3 shows the comparison for both FIS using the mean squared difference between the

outputs obtained from data corrupted with noise and the results for the data without noise. As can

be observed in Figure 3, the TSK FIS is almost always closer to its noise-free results, meaning

that the noise present in the input data is not affecting the behavior of the system and, hence, the

TSK FIS is quite robust. On the other hand, for very high noise variance the TSK FIS deviates

more than the Mamdani FIS; this means that the data for the system is becoming so different from

its original noise-free values that the system begins producing quite different results. Hence, we

may say that the TSK system is better than the Mamdani FIS with respect to noisy input data.

Further, the TSK FIS is responsive to the fact, that when the noise becomes too high (i.e. when

the input data is drastically changed), the TSK system reacts more strongly and, hence, its

behavior is more realistic.

4.3 Sensitivity analysis

In this test a time series was considered (with 1000 points) when all values for the input variables

are increasing, at a constant rate, from their minimum to their maximum. Figure 4 depicts the

results for both systems.

increase from minimum value to maximumFigure 4. Sensitivity comparison between Mamdani and TSK FIS

As can be observed, Figure 4 shows that there is a decrease in the alarm level, for the TSK

system, between iterations 300 and 500 (area highlighted in gray).

a- MaxTheta b- MaxDrift

Figure 5. Two types of input values between iterations 300 and 500

8/9/2019 Mandani

7/8

Regarding iterations 300 to 500 (approximately), the values for the input variables 1 to 3 and 7 to

11 (see Table 2) are entering a range where the membership degrees for Tolerable and not

Tolerable are close to one another (see for example the variable shown in Fig.5.a highlighted

area). On the other hand, for variables 4 to 6 (Table 2) a high membership value for the term

Tolerable is being reached (see Figure 5.b. highlighted area), as one would intuitively expect,

since the only variables where a clear result is being obtained are the Tolerable, and the other

variables are in a state of limbo between Tolerable and Not Tolerable. The TSK system is

sensitive to this area by decreasing the level of alarm while the Mamdani system is completely

numb and does not respond to these changes. It should also be noted that the TSK FIS has

smoother transitions than the Mamdani FIS.

5. CONCLUSIONS

This research showed that for this case study TSK FIS does not only works better in case of

processing time but also perform better in the other tests, showing that the structure of the TSK

FIS is more robust in the presence of noisy input data (until a certain point, obviously).Furthermore, when we tested the sensitivity of both FIS systems we observe that the TSK FIS is

more sensitive in areas where there is significant imprecision in the input representation, i.e. when

the fuzzy sets overlap.

In summary, we believe that the TSK FIS should be used whenever we have applications with a

single output variable (MISO systems). For MIMO systems (usual in classification problems) the

Mamdani FIS is maybe more appropriate, i.e. it can deal directly with MIMO, while TSK FIS

would require dividing the MIMO into as many MISO systems as the number of output variables;

often a cumbersome and time consuming task. We are presently studying ways to transform

automatically a MISO Mamdani FIS into a TSK FIS and vice versa.

6. ACKNOWLEDGEMENTS

This research was developed in part under the project New Operators for Monitoring and

Diagnostic Intelligent Systems, contract No: 18989/05/NL/MV, of the European Space Agency

(ESA/ESOC). The example used was adapted from the research project "Fuzzy Logic for Mission

Control Processes" financed by European Space Agency, ESA/ESOC No: AO/1-3874/01/D/HK

(2001-2003).

7. REFERENCES

[1] A. Pereira, F. Moura-Pires, R. A. Ribeiro, L. Correia, N. Viana, F. Javier Varas, G.

Mantovani, P.L. Bargellini, R. Perez-Bonilla, A. Donati Fuzzy Expert System for Gyroscope

Fault Detection. 16th

European Simulation Multiconference: Modelling and Simulation 2002,

ISBN 90-77039-07-4 . Darmstadt, Germany June 3-5, 2002.

[2] F. Moura-Pires, R. A. Ribeiro, A. Pereira, F. J. Varas, G. Mantovani, A. Donati Data quality

fuzzy expert system. Proceedings of the 10th Mediterranean Conference on Control and

Automation (MED02), Lisbon, Portugal, July 2002.

[3] E. H. Mamdani and S. Assilian, An experiment in linguistic synthesis with a fuzzy logic

controller,International Journal of Man-Machine Studies, Vol. 7, No.1, 1975, pp. 1-13.

[4] T. Takagi and M. Sugeno, Fuzzy identification of systems and its applications to modeling

and control,IEEE Trans, on Systems, Man and Cybernetics, 15, 1985, pp. 116-132.

[5] J. Mendel, Uncertain rule-based fuzzy inference systems: Introduction and new directions,

Prentice-Hall, 2001.

8/9/2019 Mandani

8/8

[6] D. Tikk, L. T. Kczy, T. D. Gedeon, A survey on universal approximation and its limits in

soft computing techniques. Research Working Paper RWP-IT-012001, School of Information

Technology, Murdoch University, Perth, W.A., 2001.

[7] M. Takcs, Critical Analysis of Various Known Methods for Approximate Reasoning inFuzzy Logic Control. 5th International Symposium of Hungarian Researchers on Computational

Intelligence, Budapest, Hungary 2004.