An Epsilon-Greedy Mutation Operator Based on Prior Knowledge for GAConvergence and Accuracy Improvement: an Application to Networks Inference

Mariana R. MendozaPPGC, Instituto de InformaticaUFRGS, Porto Alegre, RS, Brazil

mrmendoza@inf.ufrgs.br

Adriano V. WerhliCentro de Ciencias Computacionais

FURG, Rio Grande, RS, Brazilwerhli@gmail.com

Ana L. C. BazzanPPGC, Instituto de InformaticaUFRGS, Porto Alegre, RS, Brazil

bazzan@inf.ufrgs.br

Abstract—This paper introduces a new mutation operatorfor networks inference based on the epsilon-greedy strategy.Given some prior knowledge, either provided by a third partymethod or collected from literature, our approach performsmutations by randomly exploring the search space with ε-frequency and by exploiting the available prior knowledgein the remaining cases. The algorithm starts with a highlyexploitative profile and gradually decreases the probabilityof employing prior knowledge in the mutation operator, thusreaching a trade-off between exploration and exploitation. Testsperformed have shown that the proposed approach has greatpotential when compared to the traditional genetic algorithm:it not only outperforms the latter in terms of results accuracy,but also accelerates its convergence and allows user to controlthe evolvability speed by adjusting the rate with which theprobability of using prior knowledge is decreased.

Keywords-Genetic Algorithm; Mutation; Epsilon-Greedy;Prior Knowledge; Mutual Information; Networks Inference

I. INTRODUCTION

Genetic algorithms (GA) [1] are a class of evolutionaryalgorithms that aims at optimizing complex computationalproblems using techniques based on the natural evolutiontheory. In contrast to deterministic optimization methods,which tend to get trapped in local optima, GA perform aglobal stochastic search that parallel evolves a populationof potential solutions by selectively composing subsequentgenerations through bio-inspired mechanisms of crossoverand mutation. Due to their outstanding performance onreal, hard problems, as well as to their great versatilityregarding solutions representation, GA have received greatattention from the scientific community in optimization tasks[2]. Recent applications include, but are not limited to,parameters learning in neural networks [3], RNA secondarystructure prediction [4] and networks inference ([5], [6]).

In the last decades, a wide range of enhancements havebeen proposed and investigated, most of them concen-trated on more effective crossover operators [7]. However,mutation also plays a substantial role in improving GAperformance, thus motivating recent efforts towards thedesign of new mutation operators. Within this context,recent works ([7], [8], [9], [10]) have proposed distinctstrategies to either compute the mutation probability or to

accept a mutation proposal, some of which are based onoptimization mechanisms such as Simulated Annealing (SA)and Softmax. In [8], for instance, SA was combined with GAin order to dynamically change the probability of acceptingsome inferior solutions: after mutation occurs, the generatedsolution is evaluated and SA is applied to decide whether toaccept it or to keep the previous solution.

In contrast, in [10] authors proposed to apply Softmaxto compute the mutation probability of each bit, replacingthe traditional blind, random mutation. In this approach,the top and bottom n solutions are taken as positive andnegative examples, respectively, and the Boltzmann prob-ability distribution is used to determine the probability ofeach bit to assume value 0 or 1 in the next generationaccording to these extreme examples. It has been shownthat the Softmax mutation operator causes a faster evolutionthan the traditional approach, allowing the control of theevolutionary speed by means of the parameter used asbase in the probability formula. Furthermore, in [9], themutation probability was dynamically adapted according toindividuals’ fitness: an exponential decrease was appliedsuch that high-fitness solutions are protected, while solutionswith subaverage fitness are disrupted.

However, to our knowledge, none of the improvementsproposed so far have employed prior knowledge to computemutation probabilities nor applied an epsilon-greedy strategyto control the rate with which the use of prior knowl-edge is alternated with random operations. The epsilon-greedy approach is broadly used in learning and optimizationproblems, such as the multi-armed bandit problem [11], toachieve a trade-off between exploration and exploitation andthus improve results accuracy. In short, this method choosesa random option with ε-frequency, and otherwise choosesthe best available option. Although extremely simple, theepsilon-greedy strategy tends to be hard to beat and signifi-cantly better than other optimization methods [11].

In the current work, we propose to balance among muta-tion decisions made by a traditional blind, random mutationoperator and based on some available prior knowledge bymeans of an epsilon-greedy strategy. Our approach randomlyexplores the search space in ε mutations, and exploits the in-

2012 Brazilian Symposium on Neural Networks

1522-4899/12 $26.00 © 2012 IEEE

DOI 10.1109/SBRN.2012.40

67

formation provided by a third party method in the remainingcases. Here, we follow the tendency of combining multiplelearners and propose the integration of two distinct inferencemethods, namely genetic algorithms and mutual information,neither in a parallel or a sequential way, but rather alternatingand balancing their use through an epsilon-greedy approach.In the scope of this study, we use mutual information asthe source of prior knowledge, albeit any other method,as well as prior knowledge gathered from literature, maybe used instead. We validate our mutation operator withthe problem of networks inference, more specifically withstructure learning in Boolean networks, and show that thecoupling scheme proposed in the current work has greatpotential for inference improvement, outperforming both ofthe individual methods.

In what follows we introduce our methods and brieflyexplain the GA modeling and implementation adopted to testour approach. Next, we present the new mutation operatorproposed in the current paper and the experiments run toassess it. Finally, we conclude this paper with a discussionof the results and findings of our work.

II. MATERIALS AND METHODS

A. Random Boolean Networks

Random Boolean networks (RBNs) are defined by adirected graph G(V, F ), which is composed by a set ofnodes V = {v1, . . . , vN}, and a set of Boolean functionsF = {f1, . . . , fN}. Each node vi, i = 1, . . . , N , is aBoolean device that stands for the state of variable i: itcan assume values 0/1, on/off, etc. An example of a simpleBoolean network structure is depicted in Fig. 1, where thedouble line denotes a node on state 1, while the single dashedline stands for nodes on state 0.

Each node has its value determined by a Boolean functionfi ∈ F , which represents the rules of regulatory interactionsbetween nodes, and Ki specific inputs, denoting its regu-latory factors or predictors. Fig. 2(a) specifies the Booleanfunctions for each node of the example network in Fig. 1.A function fi determines, for each possible combination ofKi input values, the state of the variable vi. Being Ki thenumber of input binary variables regulating a given node,the number of combinations of states of the Ki inputs is

(a) (b)

Figure 1. (a) An example of N = 3 interacting nodes with K = 2

modeled as Boolean devices. Double line nodes assume value 1, whilesingle dashed line nodes are on state 0. (b) The network wiring can be alsodescribed as a string with length N ×K , containing all nodes’ predictors.

A B C(OR) (OR) (NAND)

B C A A C B A B C

0 0 0 0 0 0 0 0 10 1 1 0 1 1 0 1 11 0 1 1 0 1 1 0 11 1 1 1 1 1 1 1 0

(a)

(t) (t+1) (t+2)A B C A B C A B C

0 0 0 0 0 1 1 1 10 0 1 1 1 1 1 1 00 1 0 1 0 1 1 1 10 1 1 1 1 1 1 1 01 0 0 0 1 1 1 1 11 0 1 1 1 1 1 1 01 1 0 1 1 0 1 1 01 1 1 1 1 0 1 1 0

(b)

Figure 2. (a) Boolean functions and (b) a three-step state transition tablefor the network example of Fig. 1(a).

2Ki . Furthermore, for each of these combinations, a specificBoolean function must output either 1 or 0 such that the totalnumber of Boolean functions over Ki inputs is 22

Ki . WhenKi = 2, some of these functions are well-known (AND,OR, XOR, NAND, etc.), but in the general case functionshave no obvious semantics. Given the values of nodes V

at time t, the Boolean functions are used to synchronouslyupdate the values of nodes at time t + 1 by mappingvi(t+ 1) = fi(vKi(t)), as shown in Fig. 2(b).

B. Mutual Information as Prior Knowledge

In the context of inferring networks structure, information-theoretic approaches such as entropy and mutual information(MI) have been frequently explored as criterion functions([12], [13], [14]), i.e., a function that evaluates the suitabilityof a subset of predictors for predicting a target node. In short,this strategy aims at generalizing the pairwise correlationcoefficient to measure the degree of independence betweentwo variables. Thus, for each pair of nodes i and j, themutual information MIij is computed as [14]:

Mij = Hi +Hj −Hij (1)

where Hx is the entropy of an arbitrary discrete variable x:

Hx = −∑

k=1

p(xk)log(p(xk)) (2)

In (2), p(xk) = Prob(x = xk) is the probability ofeach discrete state (value) of the variable x. It is assumedthat a non-zero MI indicates the existence of a relationshipamong nodes [13]. However, it is important to note thatthis criterion does not imply a direct causal interactionbetween these nodes in the real network, but rather that theyhave a statistical dependence among them, i.e., they are notrandomly associated to each other. A common approach fornetworks inference based on information-theoretic methodsis to compute the pairwise MI and apply a threshold suchthat only those nodes that were linked to others with a MI

68

higher than the threshold are included in the model ([13],[14]). Since MI is symmetric, this strategy generates anundirected graph G.

Yet, differently from previous works ([13], [14]), we adoptMI as a prior information concerning the target networkand use it to support the inference process and improveconvergence. The normalized MI matrix obtained from thedata is thus interpreted as a degree of belief regarding arelationship among nodes i and j: the higher the value ofMIij , the more likely the nodes i and j are connected in thetarget network, and hence the greater the probability that ourmodel will englobe interactions i→ j or i← j. We computethe MI matrix based on the software implementation by [15].

It is important to notice that the MI matrix may containerroneous information, since the same is extracted from adata set that is characteristically noisy. In addition, as thismatrix is symmetric, many false positive connections maybe inferred when the undirected graph is transformed into adirected one: a high MIij value will enforce both i→ j andi← j connections. Having said that, it is important that themethod to be combined with MI uses this prior informationsolely as a support for its search, rather than as a pictureof the true network structure. Applying random searches bymeans of GA over a network somehow based on MI hasthe benefit of guiding the exploration of the search spacewithout restricting the stochastic nature of GA.

III. MODEL

As the goal of the present paper is to study the effectsof an epsilon-greedy strategy to balance between randomexploration and exploitation of prior knowledge in themutation operator, we resort to the network modeling andGA implementation published by Mendoza and Bazzan[16] to test the proposed approach. In what follows webriefly explain the combination scheme between GA andRBNs, which was previously applied to the problem ofreverse engineering gene regulatory networks (GRNs) [16],emphasizing some algorithmic changes introduced in thecurrent work.

A. Representation

The structure and dynamics of GRNs were described byRBNs, presented in Section II-A. In this context, nodes rep-resent genes and the edges depict the regulatory interactionsthat exist among them. Furthermore, each GA individualrepresents a candidate network codified into a string. In [16],authors implemented a binary encoding including the nodes’state, predictors and Boolean function. However, since weare interested in analyzing the improvements in terms ofstructure inference accuracy, we do not encode the nodes’states or their Boolean functions. Instead, we adopt a distinctapproach and encode solely the network wiring as an integerstring, which contains the set of predictors for each node inthe network. An example of this representation is depicted

in Fig. 1(b), in which nodes are denoted as A, B and C forthe sake of simplicity.

B. Fitness function

Inspired by the work of Nam and colleagues [17], inwhich a sub-exhaustive search was conducted towards net-work topologies consistent with a temporal expression sig-nal, Mendoza and Bazzan proposed the application of GAto explore the solutions space of consistent models. Thus,instead of entailing a search for consistent structures, theauthors evolve a set of candidate solutions encoded as GAindividuals and evaluate them based on an inconsistencyratio (IR) computed in relation to their network structureand the input temporal expression signals, as follows:

IRi = w−1

2Ki∑

k=1

min(wk(0), wk(1)) (3)

in which w represents the weight of each measurement,and variables wk(0) and wk(1) denote the total weight ofmeasurements whose output value is 0 and 1, respectively,for each k = 1, . . . , 2Ki possible input combination of anode i. While the predictors’ states are observed at time t,the target node’s state is observed at time t + 1. Once theIR for each node is calculated, the network inconsistency(IRn) is determined by the sum of all nodes’ IR.

The goal of the GA is to minimize the IRn regardingthe input temporal expression signal. The fitness function isthus defined as in (4), such that individuals with lower IRare more likely to remain in subsequent generations. Theterm NI denotes the number of actual inferred interactionsand N2 is the maximum number of possible interactions.

f =1

1 + IRn

0.5×N+ NI

N2

(4)

C. Crossover and mutation

In [16], authors applied purely random crossover and mu-tation operators. Given the binary strings of two individuals,crossover was performed by randomly selecting two pointsin the interval [1, L], where L denotes the length of thestrings, and swapping all the genetic material among bothpoints between the pair of individuals. Furthermore, pointmutations were applied as bit inversion, in which a set ofrandomly selected bits had their value flipped.

In the present paper we implement some changes overthese operators following the direction discussed in [18], inwhich crossover and mutation were performed over networkwiring. This strategy aims at finding the best network struc-ture by varying the connections between nodes and lookingfor the combination that maximizes the fitness function.Therefore, crossover is implemented as a connections swapamong a subset of nodes of two GA individuals. As anexample, consider the situation exposed in [18]: given twoindividuals of length 10, referring to networks of N = 5

69

Algorithm 1 The epsilon-greedy mutation operator1: for each individual in population do2: if random ≤ Pmut then3: randomly choose a pair of nodes i and j;4: extract the belief MIij from MI matrix;5: if random ≤ Pprior then6: mutate link (i, j) by exploiting prior knowledge MIij ;7: else8: mutate link (i, j) by randomly exploring search space;9: end if

10: end if11: end for12: Pprior = Pprior ×Δ;

nodes and K = 2, 0545120300 and 2534102340, ifthe random choice of the operator is to start the crossoverat point 3, all connections regarding nodes 3 to N willbe exchanged between the pair of mates to generate theoffspring, which in this case will be 0545122340 and2534100300. In the next section we present our epsilon-greedy mutation operator.

IV. AN EPSILON-GREEDY MUTATION OPERATOR

In the current work, we propose an epsilon-greedy mu-tation operator to replace the traditional GA blind, ran-dom mutation. This operator is based on an epsilon-greedystrategy and aims at alternating between random operationsand decisions made upon available prior knowledge. Theepsilon-greedy approach has been frequently used to achievea trade-off between exploration and exploitation in otherscenarios [11] – a phenomenon that we intend to reproducein the context of networks inference via GA.

The basic functioning of the proposed epsilon-greedymutation operator is described in Algorithm 1. Our operatorworks with two probabilities: Pmut and Pprior. The trade-off between exploration and exploitation is controlled byPprior = 1 − ε, which is the probability of using priorknowledge when performing a mutation. When ε = 0,Pprior will be equal to 1 and thus our mutation operatorwill follow an exploitative policy (Algorithm 1, line 6). Inthis case, the probability of performing a mutation overa randomly selected link will be determined by the priorknowledge. In the present paper, we have used a normalizedMI matrix as the prior information such that the higherthe belief MIij attached to a link between genes i and j

(MIij � 0.5), the more likely this link will be added to ourmodel during a mutation. Conversely, the smaller this belief(MIij � 0.5), the more probable a mutation will removethis link. In contrast, when ε = 1, Pprior = 0 and henceour operator reproduces the traditional GA blind, randommutation (Algorithm 1, line 8).

Simulations start with ε = 0, thus allowing GA to behighly exploitative during the first generations. This meansthat the networks encoded by early GA populations will bevery close to the network structure inferred by MI. The

probability Pprior is then gradually decreased throughoutgenerations by a multiplicative factor Δ (Algorithm 1,line 12), the annealing, whose value is the parameter thatcontrols convergence speed. As Pprior decreases, morerandom searches will be performed over the MI-based initialnetworks. The inferior limit for Pprior is zero, which refersto the case where ε = 1 and hence, that GA will follow apurely explorative approach.

The decision about attempting a mutation over an individ-ual follows the traditional approach: a probability Pmut isapplied to decide whether the GA will propose to mutate thegenetic material of a candidate solution. However, the actualoccurrence of a mutation depends on other factors, such asthe belief associated with the link to be mutated in the caseof using prior knowledge, or if the operations proposed arefree of redundancy and valid from the viewpoint of networksyntax, i.e., they satisfy some constraints like the maximumconnectivity allowed for each node.

V. EXPERIMENTS AND RESULTS

We run our experiments with a benchmark network ob-tained with an Artificial Gene Network (AGN) validationand simulation model [19]. As our study aims at investi-gating the effects of an epsilon-greedy mutation operationover inference accuracy, rather than the efficiency of aspecific network inference method, we test the enhancementsproposed with a single deterministic 50-node target network,which is generated according to a scale-free Boolean net-work model. The scale-free topology [20] is currently knownto be one of the most similar models to real biologicalnetworks [21], which is the study case in [16]. The upperbound limit of nodes’ average connectivity was configured as〈k〉 = 3 [22]. We simulated 10 temporal expression signalsof length 10, each one starting from a randomly choseninitial state and concatenated them into a single time seriesused for network inference.

GA parameters were configured as described in [16],except for Pmut. We evolved a population of 50 individualsover 1000 generations, applying crossover with probabilityPcross = 1 and elitism with an elite size E = 4. The proba-bility of attempting a mutation Pmut was varied between 0.1and 0.5 in steps of 0.1. We tested and analyzed the effectsof the proposed epsilon-greedy mutation operator applyingthe annealing factors Δ = {0.9, 0.99, 0.999} over Pprior.We compare this results with the two extreme situations,i.e., a purely exploitative (ε = 0, no decay) and a purelyexplorative (ε = 1, no decay) GA algorithm, where the lattermimics the traditional blind, random GA mutation. For eachof these scenarios, we performed 30 simulations and buildan ensemble prediction from each of which by letting thebest individuals of the final generation to perform a majorityvoting on the network structure. Furthermore, we combineall ensemble predictions into a single consensus network anduse it for results assessment in the current work.

70

Table IEFFECTIVE MUTATION RATES FOR SIMULATIONS WITH NO DECAY.

Pmut

0.1 0.2 0.3 0.4 0.5

ε = 0, no decay 0.32% 0.61% 0.91% 1.17% 1.49%ε = 1, no decay 3.47% 5.81% 8.05% 9.61% 11.39%

The effective mutation rates for simulations with noPprior decay appear in Table I. Readers may notice thatalbeit Pmut is configured with relatively large values consid-ering traditional GA setups, the number of actual mutationsperformed is lower due to the set of conditions that must besatisfied in order to a mutation in fact occur. When a decayrate Δ is applied, the effective mutation rates range from thevalues observed in the extreme cases, i.e, ε = 0 and ε = 1,being very close to the latter when Δ = 0.9.

A comparison in terms of the average fitness among thescenarios tested is shown in Fig. 3. The graphs behaviorevidences the better results obtained when prior knowledgeis applied (ε = 0), especially for no decay and decay rateΔ = 0.999: the average fitness after 1000 generations ishigher than values for the traditional GA. Furthermore, theuse of prior knowledge combined to low decay rates seemsto originate populations with greater internal variability. Thisimprovement comes at a cost though: the convergence speedfor these cases is slightly slower than for the remainder.As this tendency was observed for all mutation rates, wegive here solely the results for Pmut = 0.3. A trade-offbetween performance and speed can be achieve by tuningthe method’s parameters, such as the decay rate Δ.

Furthermore, we compare the results based on the AUCscore of the inferred networks, which is computed as thearea under the ROC curve (Table II). The highest scores foreach Pmut value are emphasized in boldface style. Exceptfor Pmut = 0.4, in which the purely random explorationapproach (ε = 1) has yielded the best results, the epsilongreedy mutation operator has introduced improvements up

0 100 200 300 400 500 600 700 800 900 10000.65

0.7

0.75

0.8

0.85

0.9

Avg

. Fitn

ess

0 100 200 300 400 500 600 700 800 900 10000

0.005

0.01

0.015

0.02

0.025

Std

. Fitn

ess

Generations

ε=1, no decay

ε=0, no decay

ε=0, Δ=0.999

ε=0, Δ=0.99

ε=0, Δ=0.9

Figure 3. Evolution of average fitness values when Pmut = 0.3.

0 0.2 0.4 0.6 0.8 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Tru

e po

sitiv

e ra

te

False positive rate

MIGAGA + MI

Figure 4. A comparison between the performance of the proposed couplingscheme (GA + MI) and the individual methods when Pmut = 0.3.

to 11% higher than the traditional GA, providing the mostaccurate inferences in most of the scenarios.

To stress even more the benefits of combining GA and MIvia our epsilon-greedy mutation operator, we have assessedthe network inferred solely from the MI matrix as describedin Section II-B, using as threshold every MI value in thematrix in an ascending order, and have found an AUCscore of 0.5345. This means that the coupling schemebetween GA and MI embedded in the proposed epsilon-greedy mutation operator outperforms both methods whenindividually applied, as shown in Fig. 4. Again, only thePmut = 0.3 case is depicted as this behavior is general.However, there is no consensus regarding the best annealingvalue (Δ), since enhancements were observed for all Pprior

decay rates.

VI. CONCLUSION

In the present paper we introduced an epsilon-greedymutation operator as a means of improving accuracy of net-works inference by GA through the use of prior knowledge.As far as we are concerned, up to now, this strategy hasnot been applied to this purpose. Our operator balancesbetween random mutations and mutations decided uponsome available prior knowledge, which in the scope of thiswork was obtained from a MI matrix. Experiments haveshown that the proposed coupling scheme provides a fasterand better convergence in relation to the traditional GA.In addition, the AUC scores of inferred networks wereboosted, achieving superior marks than those provided byboth methods when individually applied. Furthermore, thismutation operator allows user to control GA evolvabilityand the trade-off between exploration and exploitation byadjusting the rate with which the probability of using priorknowledge is decreased. For future works, we intend totest this approach with distinct prior knowledge sources andcompare the respective sensibility and performance.

ACKNOWLEDGMENT

We thank the Brazilian National Council for Research(CNPq) for its support to the authors.

71

Table IIRESULTS IN TERMS OF AUC SCORE OBTAINED FROM THE CONSENSUS NETWORK BUILT AFTER 30 SIMULATIONS.

Pmut ε = 0, no decay ε = 1, no decay ε = 0,Δ = 0.999 ε = 0,Δ = 0.99 ε = 0,Δ = 0.9

0.1 0.5848 0.6567 0.6474 0.6738 0.66240.2 0.6047 0.6667 0.6793 0.7034 0.67640.3 0.5964 0.6414 0.6487 0.6979 0.71330.4 0.5823 0.6726 0.6482 0.6538 0.62000.5 0.5542 0.6568 0.6955 0.6847 0.6925

REFERENCES

[1] J. Holland, Adaptation in Natural and Artificial Systems. AnnArbor: Univ. of Michigan Press, 1975.

[2] D. E. Goldberg, Genetic Algorithms in Search, Optimizationand Machine Learning. Addison-Wesley Longman Publish-ing Co., Inc., 1989.

[3] F. Leung, H. Lam, S. Ling, and P. Tam, “Tuning of the struc-ture and parameters of a neural network using an improvedgenetic algorithm,” Neural Networks, IEEE Transactions on,vol. 14, no. 1, pp. 79–88, jan 2003.

[4] F. H. van Batenburg, A. P. Gultyaev, and C. W. Pleij, “AnAPL-programmed genetic algorithm for the prediction ofRNA secondary structure.” J Theor Biol, vol. 174, no. 3, pp.269–280, Jun. 1995.

[5] C. Cotta and J. M. Troya, “Reverse engineering of temporalBoolean networks from noisy data using evolutionary algo-rithms,” Neurocomputing, vol. 62, pp. 111–129, 2004.

[6] A. Carvalho, “A cooperative coevolutionary genetic algorithmfor learning Bayesian network structures,” in Proceedingsof the 13th annual conference on Genetic and evolutionarycomputation, ser. GECCO ’11. New York, NY, USA: ACM,2011, pp. 1131–1138.

[7] K. Deep and M. Thakur, “A new mutation operator forreal coded genetic algorithms,” Applied Mathematics andComputation, vol. 193, no. 1, pp. 211–230, 2007.

[8] D. Adler, “Genetic algorithms and simulated annealing: amarriage proposal,” in IEEE International Conference onNeural Networks, vol. 2, 1993, pp. 1104–1109.

[9] M. Srinivas and L. M. Patnaik, “Adaptive probabilities ofcrossover and mutation in genetic algorithms,” IEEE Trans-actions on Systems, Man, and Cybernetics, vol. 24, no. 4, pp.656–667, 1994.

[10] Y. Sasaki and H. de Garis, “Faster evolution and evolva-bility control of genetic algorithms using a softmax mu-tation method,” in Proceedings of the 2003 Congress onEvolutionary Computation CEC2003, R. Sarker, R. Reynolds,H. Abbass, K. C. Tan, B. McKay, D. Essam, and T. Gedeon,Eds. Canberra: IEEE Press, 2003, pp. 886–891.

[11] J. Vermorel and M. Mohri, “Multi-armed bandit algorithmsand empirical evaluation,” in Machine Learning: ECML 2005,ser. Lecture Notes in Computer Science, J. Gama, R. Cama-cho, P. Brazdil, A. Jorge, and L. Torgo, Eds. Springer Berlin/ Heidelberg, 2005, vol. 3720, pp. 437–448.

[12] S. Liang, S. Fuhrman, and R. Somogyi, “REVEAL, A GeneralReverse Engineering Algorithm for Inference of Genetic Net-work Architectures,” in Proceedings of the Pacific Symposiumon Biocomputing, vol. 3, 1998, pp. 18–29.

[13] A. J. Butte, I. S. Kohane, and I. S. Kohane, “Mutual in-formation relevance networks: Functional genomic clusteringusing pairwise entropy measurements,” Pacific Symposium onBiocomputing, vol. 5, pp. 415–426, 2000.

[14] A. Margolin, K. N. Basso, C. Wiggins, G. Stolovitzky,R. Favera, and A. Califano, “ARACNE: An algorithm for thereconstruction of gene regulatory networks in a mammaliancellular context,” BMC Bioinformatics, vol. 7, no. Suppl 1,p. S7, 2006.

[15] P. Qiu, A. J. Gentles, and S. K. Plevritis, “Fast calcu-lation of pairwise mutual information for gene regulatorynetwork reconstruction,” Computer Methods and Programsin Biomedicine, vol. 94, no. 2, pp. 177–180, 2009.

[16] M. R. Mendoza and A. L. C. Bazzan, “Evolving RandomBoolean Networks with Genetic Algorithms for RegulatoryNetworks Reconstruction,” in Proceedings of the 13th AnnualConference on Genetic and Evolutionary Computation, ser.GECCO ’11. New York, NY, USA: ACM, July 2011, pp.291–298.

[17] D. Nam, S. Seo, and S. Kim, “An efficient top-down searchalgorithm for learning boolean networks of gene expression,”in Machine Learning. Springer Science, 2006, vol. 65, pp.229–245.

[18] M. R. Mendoza, F. M. Lopes, and A. L. C. Bazzan, “ReverseEngineering of GRNs: an Evolutionary Approach based onthe Tsallis Entropy,” in Proceedings of the 14th AnnualConference on Genetic and Evolutionary Computation, 2012.

[19] F. M. Lopes, R. M. Cesar-Jr, and L. da F. Costa, “ AGN Si-mulation and Validation model,” in Proceedings of Advancesin Bioinformatics and Computational Biology, vol. 5167 ofLecture Notes in Bioinformatics. Springer-Verlag Berlin,2008, pp. 169–173.

[20] A. L. Barabasi and R. Albert, “Emergence of scaling inrandom networks,” Science, vol. 286, no. 5439, pp. 509–512,1999.

[21] R. Albert, “Scale-free networks in cell biology,” J Cell Sci,vol. 118, no. 21, pp. 4947–4957, 2005.

[22] S. A. Kauffman, “Metabolic stability and epigenesis in ran-domly constructed genetic nets.” J Theor Biol, vol. 22, no. 3,pp. 437–467, March 1969.

72