Upload
doantuyen
View
220
Download
0
Embed Size (px)
Citation preview
Bio-Inspired Dynamic Radio Access in Cognitive
Networks based on Social Foraging Swarms
Paolo Di Lorenzo
Universita degli studi di Roma “Sapienza”
Dottorato di Ricerca in Ingegneria dell’Informazione e della Comunicazione
24 Ciclo
Advisor: Professor Sergio Barbarossa
May 8, 2012
ai miei genitori,
2
Abstract
There is strong trend, in current research on communication and sensor net-
works, to study selforganizing, self-healing systems. This poses great challenges
to the research on decentralized systems, but at the same offers great potentials
for future developments, especially in view of the current trend towards minia-
turized systems. Even if the development of self-organizing systems is probably
at the beginning, biological systems offers many examples of self-organization
and selfhealing. This is as testified, for example, by swarming behaviors, brain
activity, and so on. It is then of great interest to derive mathematical models of
biological systems and see how they can suggest novel design tools for engineers.
Signal Processing can play a big role in this cross-fertilization, as it can help
to find out manageable mathematical problems, study their behavior and test
the performance in the presence of disturbances. The challenge is to establish a
cross-fertilization of ideas from biological to artificial systems, as well as to help
understanding biological systems as such.
This dissertation considers the problem of dynamic radio access based on sensing
in cognitive radio networks. In particular, we follow a rather alternative path
with respect to more conventional approaches and, inspired by biological models,
we formulate the search for radio resources, i.e. time and frequency slots, as the
search for food by a flock of birds swarming in a cooperative manner, but with-
out any centralized control. The interference distribution in the time-frequency
plane takes the role of the food spatial distribution: The birds (radio nodes) fly
3
(allocate their resources) over the regions (time-frequency domain) where there
is more food (less interference). During the flight, the birds move (choose their
time-frequency slots) in a coordinated way, even in the absence of any central
control, in order to avoid collisions (conflicts over common radio resources), yet
maintaining the swarm cohesion (i.e., avoiding unnecessary spread in the occu-
pancy of the time-frequency plane). This procedure is applied to the dynamic
resource allocation in the frequency domain and in the time-frequency domain,
where the primary users in a cognitive radio system are modeled as statistically
independent homogeneous continuous-time Markov processes.
A rigorous mathematical analysis of the proposed algorithm is also derived. First,
we study the stability and the cohesiveness of the swarm in case of local inter-
actions among the nodes, providing closed form expressions for the upper and
lower bounds of the swarm size as a function of the network connectivity. Then,
using stochastic approximation arguments, we derive the convergence properties
of the swarming algorithm in the presence of random disturbances introduced by
realistic channels, i.e., link failures, quantization, noise and estimation errors.
Spectrum sensing is a critical prerequisite in envisioned applications of wire-
less cognitive radio networks which promise to resolve the perceived bandwidth
scarcity versus under-utilization dilemma. Creating an interference map of the
operational region plays an instrumental role in enabling spatial frequency reuse
and allowing for dynamic spectrum allocation in a hierarchical access model com-
prising primary and secondary users. For such purpose, a distributed technique
for cooperative spectrum estimation in cognitive radio systems is proposed based
on a basis expansion model of the power spectral density map in frequency. The
proposed method, based on diffusion adaptation algorithms, estimates and learns
the interference profile through local cooperation and without the need for a
central processor. Convergence and mean square analysis of the diffusion filter
applied to the distributed cooperative sensing problem is also derived.
4
Finally, it is proposed a dynamic resource allocation technique combining a dis-
tributed diffusion algorithm, for implementing cooperative sensing, with a swarm-
ing technique, for allocating resources in a parsimonious way (i.e., avoiding un-
necessary spread in the frequency domain), yet avoiding collisions. In particular,
the procedure is applied to the dynamic resource allocation problem in the fre-
quency domain. Numerical results show the improvement that results in the
resource allocation performance due to the cooperative estimation of the spec-
trum. Furthermore, it is shown how the proposed technique endows the resulting
bio-inspired network with powerful learning and adaptation capabilities.
5
6
Acknowledgements
Mi ritengo fortunato di aver conosciuto tutte le persone che mi hanno accom-
pagnato in questa splendida avventura del dottorato di ricerca.
In primis, vorrei ringraziare il mio mentore, il Professor Sergio Barbarossa,
per la sua guida ed il suo continuo insegnamento durante questi anni. Mi sento
estremamente onorato della fiducia e della stima che ha sempre riposto in me.
Non e mai esistito un giorno in cui non mi abbia dedicato tempo ed attenzione,
indipendentemente da quanto occupato fosse. La sua straordinaria motivazione,
il suo intuito e la sua profonda conoscenza tecnica sono state per me fonte di
grande ispirazione. Spero che le mie capacita siano state il piu possibile plasmate
da tali eccezionali qualita.
I wish to thank the supervisor of my work at UCLA, Professor Ali H. Sayed.
He welcomed me in his research group and treated me like another member of the
group with no distinction, making my stay at Los Angeles a very pleasant and
rewarding experience. And of course, I would like to thank all the guys of the
Adaptive System Laboratory at UCLA: Zaid, Alex, Xiaochuan, Jianshu, Shine,
Shang Kee, Victor and Jae-Woo. I will never forget the friendship with which
they honored me during my stay.
I miei ringraziamenti vanno anche i miei colleghi Marco, Stefania, Alessandro
e Pasquale con cui ho condiviso le esperienze ed il lavoro di tutti i giorni in
un clima sereno e piacevole. Desidero anche ringraziare in modo particolare
7
Gesualdo Scutari per avermi sempre incoraggiato ed entusiasmato nel perseguire
l’eccellenza del mio lavoro fin dall’inizio del mio dottorato.
Di certo in questa lista di ringraziamenti non posso scordare i miei amici
di una vita con i quali da sempre condivido tutte le gioie e i dolori. Marco,
Roberto, Marco, Christian, Luca, Daniele, Valerio, Simone, Marco, Massimiliano.
Ognuno di voi ha contribuito ogni giorno a rendermi un uomo migliore e mi ritengo
estremamente fortunato di aver potuto sempre contare su amici veri come voi.
Un ringraziamento speciale lo dedico al maestro Giampiero che mi ha ritenuto
degno di poter prendere il suo posto. Mi ha sempre insegnato tutto quello che
sapeva al meglio che poteva e, di questo, gliene saro sempre grato. Vorrei anche
includere Francesco, Ciro e Matteo, per ringraziarli dell’impegno che ogni giorno
dedicano a cio che e parte fondamentale della mia vita.
Il cammino della vita d’improvviso mi ha fatto incontrare te, Anna. La tua
bellezza, intelligenza, forza e dolcezza hanno completato la mia vita come non
credevo possibile. Grazie alla nostra splendida sinergia, niente mi spaventa se
sono con te ad affrontarlo. Oggi, cosı come domani, ti ringraziero sempre di
camminare con me sul sentiero della vita e di essere la mia felicita.
Infine, ringrazio mio padre Bernardo, mia madre Rosalba e mio fratello An-
drea, le persone da cui ho imparato di piu. Il loro incondizionato amore, inseg-
namento e supporto sono stati ineguagliabili durante tutta la mia vita.
8
Contents
Abstract 3
1 Introduction and Overview 17
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
1.2 Network Design Inspired by Biological Models . . . . . . . . . . . . 26
1.2.1 Modeling Approaches . . . . . . . . . . . . . . . . . . . . . 27
1.2.2 Classification and Categorization . . . . . . . . . . . . . . . 28
1.3 Bio-Inspired Networking . . . . . . . . . . . . . . . . . . . . . . . . 29
1.3.1 Swarm Intelligence . . . . . . . . . . . . . . . . . . . . . . . 29
1.3.2 Firefly Synchronization . . . . . . . . . . . . . . . . . . . . 32
1.3.3 Activator-Inhibitor Systems . . . . . . . . . . . . . . . . . . 33
1.3.4 Artificial Immune System . . . . . . . . . . . . . . . . . . . 35
1.3.5 Epidemic Spreading . . . . . . . . . . . . . . . . . . . . . . 37
1.3.6 Nano-scale and Molecular Communication . . . . . . . . . . 39
1.4 Outline of Dissertation . . . . . . . . . . . . . . . . . . . . . . . . . 43
1.5 Research Contributions . . . . . . . . . . . . . . . . . . . . . . . . 45
2 Mathematical Background 49
2.1 Dynamical Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
2.1.1 Linear Systems . . . . . . . . . . . . . . . . . . . . . . . . . 51
2.1.2 Nonlinear Systems . . . . . . . . . . . . . . . . . . . . . . . 54
9
CONTENTS
2.2 Distributed Optimization . . . . . . . . . . . . . . . . . . . . . . . 60
2.2.1 Unconstrained Optimization . . . . . . . . . . . . . . . . . . 60
2.2.2 Constrained Optimization . . . . . . . . . . . . . . . . . . . 62
2.2.3 Convex Constrained Optimization Problems . . . . . . . . . 63
2.3 Stochastic Approximation . . . . . . . . . . . . . . . . . . . . . . . 66
2.3.1 Robbins-Monro procedure . . . . . . . . . . . . . . . . . . . 67
2.3.2 Kiefer-Wolfowitz procedure . . . . . . . . . . . . . . . . . . 71
2.4 Graph Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
2.4.1 Directed Graphs: The Basic Mathematical Tool to De-
scribe Interactions . . . . . . . . . . . . . . . . . . . . . . . 74
2.4.2 Algebraic Graph Theory . . . . . . . . . . . . . . . . . . . . 78
3 Distributed Resource Allocation Based on Swarming Mecha-
nisms 83
3.1 Introduction on Cognitive Radio and Dynamic Radio Access . . . 84
3.2 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
3.3 Continuous-Time Distributed Optimization . . . . . . . . . . . . . 94
3.4 Stability and Cohesion Analysis . . . . . . . . . . . . . . . . . . . . 97
3.4.1 Profiles with Bounded Gradient . . . . . . . . . . . . . . . . 98
3.4.2 Quadratic Profile . . . . . . . . . . . . . . . . . . . . . . . . 107
3.5 Swarming in the Frequency Domain . . . . . . . . . . . . . . . . . 110
3.5.1 Local Stability Analysis . . . . . . . . . . . . . . . . . . . . 111
3.5.2 Discrete-Time Implementation . . . . . . . . . . . . . . . . 112
3.5.3 Fast Swarming Algorithms . . . . . . . . . . . . . . . . . . 114
3.5.4 Numerical Examples . . . . . . . . . . . . . . . . . . . . . . 116
3.6 Swarming in the Time-Frequency Domain . . . . . . . . . . . . . . 126
3.6.1 Swarming in a Static Interference Environment . . . . . . . 127
3.6.2 Swarming in the Presence of Markovian Interference . . . . 130
3.7 Discrete-Time Distributed Optimization . . . . . . . . . . . . . . . 132
3.7.1 Projected Swarming Algorithms . . . . . . . . . . . . . . . 135
10
CONTENTS
3.8 The Effect of Noise and Realistic Channels . . . . . . . . . . . . . 137
3.8.1 Random Link Failures . . . . . . . . . . . . . . . . . . . . . 139
3.8.2 Dithered Quantization . . . . . . . . . . . . . . . . . . . . . 139
3.8.3 Stochastic Convergence . . . . . . . . . . . . . . . . . . . . 140
3.9 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
4 Distributed Cooperative Spectrum Sensing Based on Diffusion
Adaptation 171
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
4.2 Diffusion Adaptation . . . . . . . . . . . . . . . . . . . . . . . . . . 174
4.3 Sparse Diffusion Adaptation . . . . . . . . . . . . . . . . . . . . . . 176
4.3.1 Sparse ATC Diffusion . . . . . . . . . . . . . . . . . . . . . 177
4.3.2 Performance Analysis . . . . . . . . . . . . . . . . . . . . . 179
4.3.3 Numerical Results . . . . . . . . . . . . . . . . . . . . . . . 183
4.4 Basis Expansion Model of the Spectrum . . . . . . . . . . . . . . . 187
4.5 ATC Diffusion for Adaptive Spectrum Estimation . . . . . . . . . . 190
4.5.1 Performance Analysis . . . . . . . . . . . . . . . . . . . . . 191
4.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196
5 Swarming for Dynamic Radio Access Based on Diffusion Adap-
tation 199
5.1 Swarm Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200
5.2 Diffusion Adaptation for Cooperative Spectrum Sensing . . . . . . 202
5.3 Adaptive Swarming . . . . . . . . . . . . . . . . . . . . . . . . . . . 204
5.4 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
5.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
6 Concluding Remarks 215
6.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216
Bibliography 221
11
12
List of Figures
2.1 Fixed points with three different types of stability. The fixed point
on the left is stable. The fixed point in the center is marginally
stable. The fixed point on the right is unstable. . . . . . . . . . . . 57
2.2 Examples of graphs: (a) Strongly connected graph. (b) Quasi
strongly connected graph with one root strongly connected com-
ponent and two strongly connected components. (c) WC graph
containing a two-tree forest. . . . . . . . . . . . . . . . . . . . . . . 78
3.1 Magnitude of the coupling function g(·) in (3.4) with linear attrac-
tion (3.7) and unbounded repulsion (3.8), using the values cA = 1
and cR = 2. The distance between the red points and zero is the
equilibrium distance between the swarm agents. . . . . . . . . . . . 96
3.2 Magnitude of the coupling function g(·) in (3.4) with linear attrac-
tion (3.7) and bounded repulsion (3.10), using the values cA = 1,
cR = 10 and cG = 2. The distance between the red points and
zero is the equilibrium distance between the swarm agents. . . . . 97
3.3 Upper and lower bounds of the potential function time derivative. 104
3.4 Swarm size parameter versus the node covering radius. . . . . . . . 116
3.5 Swarm size parameter versus the attraction parameter cA. . . . . . 117
3.6 Network topology and allocation example. . . . . . . . . . . . . . . 118
3.7 Interference profile and allocation example. . . . . . . . . . . . . . 119
13
LIST OF FIGURES
3.8 Frequency reuse parameter versus covering radius. . . . . . . . . . 120
3.9 Normalized system potential function vs. time index, for different
coverage radii. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
3.10 Normalized system potential function vs. time index, for different
descent directions of the algorithm : gradient descent (dashed) and
Newton approximation (solid). . . . . . . . . . . . . . . . . . . . . 122
3.11 Dynamic resource allocation by swarming: Reaction time to PU’s
activations, for basic swarming. . . . . . . . . . . . . . . . . . . . . 123
3.12 Dynamic resource allocation by swarming: Reaction time to PU’s
activations, for adaptive scaling. . . . . . . . . . . . . . . . . . . . 124
3.13 Average number of iterations to obtain convergence versus number
of nodes, for different degrees of network connectivity. . . . . . . . 125
3.14 Example of 2D allocation, considering a quadratic profile. . . . . . 127
3.15 Swarm size versus the magnitude of the quadratic profile Aσ. . . . 128
3.16 Swarm size versus the magnitude of the quadratic profile Aσ. . . . 129
3.17 Example of time-frequency allocation. . . . . . . . . . . . . . . . . 130
3.18 Example of time-frequency allocation with Markovian interference. 132
3.19 Secondary network. The square nodes denote primary users and
the circle nodes denote secondary users. . . . . . . . . . . . . . . . 158
3.20 Examples of resource allocation by swarming. . . . . . . . . . . . . 159
3.21 Average interference perceived by the swarm vs. time index, for
different probabilities of correct packet reception. . . . . . . . . . . 160
3.22 Average interference perceived by the swarm at convergence, ver-
sus the probability to establish a communication link, for different
values of the swarm attraction parameter cA. . . . . . . . . . . . . 161
3.23 Average convergence time versus number of nodes, for different
number of bits used for quantization. . . . . . . . . . . . . . . . . . 162
3.24 Average convergence time versus number of nodes, for different
degrees of network connectivity. . . . . . . . . . . . . . . . . . . . . 163
14
LIST OF FIGURES
3.25 Average interference perceived by the swarm vs. time index, for
different algorithms and probabilities of correct packet reception. . 164
3.26 Average interference perceived by the swarm at convergence, ver-
sus the slope parameter of the linear scaling functions, for different
probabilities of correct packet reception and different values of the
attraction parameter cA. . . . . . . . . . . . . . . . . . . . . . . . . 165
3.27 Example of resource allocation by swarming. . . . . . . . . . . . . 167
4.1 Transient network MSD for the non-cooperative approaches LMS
[190], ZA-LMS [166], RZA-LMS [166], and the diffusion techniques
ATC [145], ZA-ATC (eq.(4.3)-(4.9)), RZA-ATC (eq.(4.3)-(4.11)). . 184
4.2 Differential MSD versus sparsity parameter ρ for ZA-ATC Diffu-
sion LMS, for different degrees of system sparsity. . . . . . . . . . . 185
4.3 Differential MSD versus sparsity parameter ρ for RZA-ATC Dif-
fusion LMS, for different degrees of system sparsity. . . . . . . . . 186
4.4 Example of basis expansion using Gaussian pulses. The dotted
curves represent the Gaussian basis functions, whereas the contin-
uous curve denotes the behavior of a generic interference profile
described by 6 Gaussian pulses. . . . . . . . . . . . . . . . . . . . . 188
5.1 Secondary network. The square nodes denote primary users and
the dot nodes denote secondary users. . . . . . . . . . . . . . . . . 206
5.2 Comparison of the result of spectrum estimation through coop-
erative diffusion adaptation and without cooperation among the
users. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
5.3 Steady-state MSD versus node index. . . . . . . . . . . . . . . . . . 208
5.4 Average interference perceived by the swarm at convergence, for
the non cooperative estimation case and for adaptive diffusion. . . 209
5.5 Different resource assignments in dynamic environment. . . . . . . 210
5.6 ATC diffusion learning curve, in terms of MSD. . . . . . . . . . . . 211
5.7 Average perceived interference versus iteration index. . . . . . . . . 212
15
16
Chapter 1
Introduction and Overview
1.1 Motivation
The last few decades have witnessed striking developments in communication
and networking technologies, yielding many information network architectures.
These next generation information networks are envisioned to be characterized
by an invisible and ubiquitous halo of information and communication services,
which should be easily accessible by users in a transparent, location-independent,
and seamless fashion. Therefore, the result will be a pervasive and, in fact, liv-
ing network. This ubiquitous networking space will include, in addition to the
traditional Internet-connected devices, networked entities that are in much closer
interaction with us such as wearable networks [6], in-body molecular communi-
cation networks [7], unattended ground, air, and underwater sensor networks [8],
self-organizing sensor and actor networks [9, 10] and locally intelligent and self-
cognitive devices exploiting the communication resources with the help of cogni-
tive capabilities, e.g., cognitive radio networks [92]. Clearly, this vision implies
that almost every object will be able to effectively and collaboratively communi-
cate, thus becoming, to some extent, a node of the future pervasive global net-
work. The evolution in communication and networking technologies brings many
17
1.1 Motivation
potential advantages to our daily lives. At the same time, the complexity of the
existing and envisioned networked information systems is rapidly going beyond
what conventional networking paradigms can do. Thus, self-organization tech-
niques are demanded to overcome current technical limitations [11]. In fact, there
exist many common significant challenges that need to be addressed for practi-
cal realization of existing and next generation networking architectures, such as
increased complexity with large scale networks, their dynamic nature, resource
constraints, heterogeneous architectures, absence or impracticality of centralized
control and infrastructure, need for survivability, and unattended resolution of
potential failures.
Most of the existing and next generation communication systems, handled ac-
cording to the conventional networking paradigms, do not totally accommodate
the scale, heterogeneity and complexity of such scenarios. Then, novel paradigms
are needed for designing, engineering and managing these communication sys-
tems. While the challenges outlined above such as scalability, heterogeneity and
complexity are somehow new byproducts of the evolution in the communication
technologies in the last few decades, they have been successfully dealt with by Na-
ture for quite some time. Unlike the evolution in the communication technologies
that have brought these challenges, the evolution in Nature has shown how bio-
logical systems can actually handle many of these challenges with an elegance and
efficiency still far beyond current techniques. In fact, when we look carefully into
Nature, it is clearly observed that the dynamics of many biological systems and
laws governing them are based on a surprisingly small number of simple generic
rules, which yield collaborative yet effective patterns for resource management
and task allocation, social differentiation, synchronization (or de-synchronization)
without the need for any externally controlling entity. For example, by means
of these capabilities, billions of blood cells that constitute the immune system
can protect the organism from the pathogens without the central control of the
brain [20]. Similarly, the biological homeostasis autonomously maintains the op-
eration of vital functions of an entire organism without any need for a central
18
1.1 Motivation
biological controller [21]. Furthermore, the task allocation process in the insect
colonies is collaboratively decided and performed such that the overall task is op-
timized with a global intelligence comprised of simple individual responses [22].
These examples and, in general, as a result of millions of years of evolution, bi-
ological systems and processes have intrinsic appealing characteristics. Among
others, they are:
• adaptive to the varying environmental circumstances,
• robust and resilient to the failures caused by internal or external factors,
• able to achieve complex behaviors on the basis of a usually limited set of
basic rules,
• able to learn and evolve itself when new conditions are applied,
• effective in managing constrained resources with an apparently global in-
telligence larger than the superposition of individuals,
• able to self-organize in a fully distributed fashion, collaboratively achieving
an efficient equilibrium,
• survivable despite harsh environmental conditions due to its inherent and
sufficient redundancy.
These characteristics lead to different levels of inspiration from biological sys-
tems towards the derivation of different algorithm designs for efficient, robust
and resilient communication networks. Therefore, in order to keep pace with
the evolution in-networking technologies, many researchers are currently engaged
in developing innovative design paradigms inspired by biology in order to ad-
dress the networking challenges of existing and envisioned information systems.
The common rational behind this effort is to capture the governing dynamics
and understand the fundamentals of biological systems in order to devise new
methodologies and tools for designing and managing communication systems
19
1.1 Motivation
and information networks that are inherently adaptive to dynamic environments,
heterogeneous, scalable, self-organizing, and evolvable. Besides bio-inspired net-
working solutions, communication on the nano-scale is being investigated with
two important but conceptually different goals. On the one hand, bio-inspired
nano machinery is investigated in order to build machines on the nano level using
communication and actuation capabilities derived from biological counterparts.
More specifically, the most promising communication mechanism between nano-
machines forming nano-scale networks is currently envisioned to be molecular
communication, i.e., coding and transfer of information in terms of molecules,
which is also mainly inspired by the cellular signaling networks observed in living
organisms. On the other hand, such nano-machines can also be used in the main
field of molecular biology to study biological systems. Clearly, there exist many
challenges for the realization of the existing and the envisioned next generation
network architectures. At the same time, we would like to stress that several
biological approaches may be used as a solution of these networking paradigms.
In this section, we review some of the most challenging issues for networking and
highlight the analogies with their counterparts and corresponding solutions that
already biological systems offer.
Large Scale Networking
One of the main challenges is related to the increasing size exhibited by the
networking systems, which connect a huge numbers of users and devices in a sin-
gle, omni-comprehensive, preferably always-on network. The size of this network,
in terms of both number of constituent nodes and running services, is expected to
exceed by several orders of magnitude that of the current Internet. For example,
Wireless Sensor Networks (WSNs), having a broad range of current and future
applications, are generally envisioned to be composed of a large number, e.g., in
numbers ranging between few hundreds to several hundred thousands, of sensor
nodes [12]. The first direct consequence of such large scales is the huge amount
20
1.1 Motivation
of traffic load to be incurred over the network, which could easily exceed the
network capacity, hampering the communication reliability due to packet losses
by both collisions in the local wireless channel as well as congestion along the
network path [14]. Similarly, it becomes more important to find the optimal
routes, in order to keep the communication overhead at acceptable levels during
the dissemination of a large amount of information over a large scale network. As
the network scale expands, the number of possible paths, and hence, the search
space for the optimal route in terms of a preset criteria, also drastically enlarges.
Hence, networking mechanisms must be scalable and adaptive to variations in the
network size. Actually, there exist many biological systems that inspire the design
of effective communication solutions for large scale networks. For example, Ant
Colony Optimization (ACO) techniques [23] provide efficient routing mechanisms
for large-scale mobile ad hoc networks and information dissemination over large
scales can be handled using epidemic spreading [24], which is the transmission
mechanism of viruses.
Dynamic Nature
Unlike the early communication systems composed of a transmitter/receiver
pair and a communication channel, which are all static, the existing and future
networking architectures are highly dynamic in terms of node behaviors, traffic
and bandwidth demand patterns, channel and network conditions. According to
the mobility of the nodes, network dimensions, and radio ranges, communica-
tion links may frequently be established and become obsolete in mobile ad hoc
networks [16]. Furthermore, due to mobility of the nodes, and environmental
variations as a result of movement, the channel conditions and hence link qual-
ities may be highly dynamic. Dynamic spectrum access and its management in
cognitive radio networks is an important case where the dynamic nature of the
user behaviors poses significant challenges on the network design [92]. The objec-
tive of cognitive radio networks itself is to exploit the dynamic usage of spectrum
21
1.1 Motivation
resources in order to maximize the overall spectrum utilization.
To this end, biological systems are known to be capable of adapting them-
selves to varying circumstances. For example, Artificial Immune System (AIS),
inspired by the principles and processes of the mammalian immune system [25],
efficiently detects variations in the dynamic environment or deviations from the
expected system patterns. Similarly, activator-inhibitor systems and the analysis
of reaction-diffusion mechanisms in biological systems [26] also capture dynam-
ics of highly interacting systems through differential equations. These specific
models can be exploited to develop communication techniques that can adapt to
varying environmental conditions.
Resource Constraints
As the communication technologies evolve, network demands also increase in
terms of available services, service quality and lifetime. For example, the cur-
rent Internet can no longer respond to every demand as its capacity is almost
exceeded by the total traffic created, which lays a basis for the development of
next generation Internet [17]. At the same time, with the increased demand
from wireless networking, fixed spectrum assignment-based traditional wireless
communications has become insufficient in accommodating a wide range of radio
communication requests. Consequently, cognitive radio networks with dynamic
spectrum management and access has been proposed and is currently being de-
signed in order to improve utilization of spectrum resources [92]. More specif-
ically, for the networks composed of nodes that are inherently constrained in
terms of energy and communication resources, e.g., WSNs [12], Mobile Ad Hoc
Networks (MANETs) [16], nano-scale and molecular communication networks [7],
these limitations directly bound their performance and mandate for intelligent re-
source allocation mechanisms. The biological systems yet again help researchers
by providing potential solutions that address the trade-off between the high de-
mand and limited supply of resources. For example, in the foraging process [23],
22
1.1 Motivation
ants use their individual limited resources towards optimizing the global behavior
of colonies in order to find food source in a cost-effective way. The behavior of ant
colonies in the foraging process may inspire many resource-efficient networking
techniques. Furthermore, cellular signaling networks represent and capture the
dynamics of interactions contributing to the main function of a living cell. Hence,
they might also be exploited in order to obtain efficient communication techniques
for resource constrained nano-scale and molecular communication networks.
Need for Infrastructure-less and Autonomous Operation
The significant increase in network dimensions, both spatially and in the
number of nodes, makes centralized control of the network very unpractical.
On the other hand, some networks are by definition free from infrastructure
such as wireless ad hoc networks [16], Delay Tolerant Networks (DTNs) [17],
WSNs [12], and some have a heterogeneous, mostly distributed and non-unified
system architecture such as cognitive radio networks [92], wireless mesh networks
and WiMAX [13]. These networking environments mandate for distributed com-
munication and networking algorithms which can effectively work without any
help from a centralized unit. At the same time, communication networks are
subject to failure either by device malfunction, e.g., nodes in a certain area may
run out of battery in sensor networks, or misuse of their capacity, e.g., overloading
the network may cause heavy congestion blocking the connections. In most cases,
networks are expected to continue their operation without any interruption due to
these potential failures. Considering the dynamic nature, lack of infrastructure,
and impracticality of centralized communication control, it is clear that networks
must be capable of self-organization and self-healing in order to be able to resume
their operation. Hence, the existing and next generation information networks
must have the capabilities of self-organization, self-evolution and survivability.
In order to address all these needs, networks could exploit some intelligent algo-
rithms and processes that were largely observed in biological systems. In fact,
23
1.1 Motivation
inherent features of many biological systems stand as promising solutions for these
challenges. For example, an epidemic spreading mechanism could be modified for
efficient information dissemination in highly partitioned networks and for oppor-
tunistic routing in delay tolerant networking environments [24]. Ant colonies, and
in general insect colonies, which perform global tasks without the control of any
centralized entity, could also inspire the design of communication techniques for
infrastructureless networking environments [76]. Furthermore, synchronization
principles of fireflies [27] could be applied to the design of time synchronization
protocols as well as communication protocols requiring precise time synchroniza-
tion. Activator-inhibitor systems may be exploited for distributed control of
sensing periods and duty cycle of target tracking sensor networks [28]. The au-
tonomous behavior of artificial immune systems may be a good model for the
design of effective algorithms for unattended and autonomous communication in
sensor networks. Thus, the lack of infrastructure and autonomous communica-
tion requirements in various networking environments could be addressed through
careful exploration of self organization capabilities of biological systems.
Heterogeneous Architectures
The other critical aspect of many of the existing and envisioned communica-
tion networks is linked to their heterogeneity and its resultant extremely complex
global behavior, emerging from the diverse range of network elements and large
number of possible interactions among them. Next generation communication
systems are generally envisioned to be composed of a vast class of communi-
cating devices differing in their communication/storage/processing capabilities,
ranging from Radio Frequency Identification (RFID) devices and simple sensors
to mobile vehicles equipped with broadband wireless access devices. Similarly,
cognitive radio networks involve the design of new communication techniques
to realize the co-existence of different wireless systems communicating on over-
lapping spectrum bands with an ultimate objective of maximizing the spectrum
24
1.1 Motivation
utilization. Wireless mesh networks and WiMAX are also expected to be com-
posed of heterogeneous communication devices and algorithms [13]. Sensor and
Actor Networks (SANETs) architecturally incorporate both heterogeneous low-
end sensor nodes and highly capable actor nodes [11]; and Vehicular Ad Hoc
Networks (VANETs) [29] exhibit significant levels of heterogeneity in terms of
wireless communication technologies in use and mobility patterns of ad hoc vehi-
cles. Such heterogeneity and asymmetry in terms of capabilities, communication
devices and techniques need to be understood, modeled and effectively managed,
in order to allow the realization of heterogeneous novel communication networks.
Different levels of heterogeneity are also observed in biological systems. For
example, in many biological organisms, despite external disturbances, a stable
internal state is maintained through collaborative effort of heterogeneous set of
subsystems and mechanisms, e.g., nervous system, endocrine system, immune
system. This functionality is called homeostasis, and the collective homeostatic
behavior [78] can be applied towards designing communication techniques for
networks with heterogeneous architectures. On the other hand, insect colonies
are composed of individuals with different capabilities and abilities to respond to
a certain environmental stimuli. Despite this inherent heterogeneity, colonies can
globally optimize the task allocation via their collective intelligence [23]. Similar
approaches can be adopted to address task assignment and selection in SANETs,
for spectrum sharing in heterogeneous cognitive radio networks [76], as well as
multi-path routing in overlay networks [30].
Communication on the Micro Level
With the advances in micro- and nano-technologies, electro-mechanical de-
vices have been downscaled to micro and nano-levels. Consequently, there ex-
ist many micro (MEMS) and nano-electro-mechanical systems (NEMS) with a
large spectrum of applications. Clearly, capabilities for communication and net-
working at micro and even at nano-scales become imperative in order to enable
25
1.2 Network Design Inspired by Biological Models
micro and nano-devices to cooperate and, hence, collaboratively realize certain
common complex task that cannot be handled individually. In this regard, nano-
networks could be defined as a network composed of nano-scale machines, i.e.,
nano-machines, cooperatively communicating with each other and sharing infor-
mation in order to fulfill a common objective [79]. The dimensions of nano-
machines render conventional communication technologies, such as electromag-
netic or acoustic waves, inapplicable at these scales due to antenna size and
channel limitations. Furthermore, the communication medium and channel char-
acteristics also show important deviations from the traditional cases due to the
rules of physics governing these scales. The main idea of nano-machines and nano-
scale communications and networks have also been motivated and inspired by the
biological systems and processes. Hence, it is conceivable that the solutions for
the challenges in communication and networking at micro and nano-scales could
also be developed through inspiration from the existing biological structures and
communication mechanisms. In fact, many biological entities in organisms have
similar structures with nano-machines. For example, every living cell has the
capability of sensing the environment, receiving external signals, performing cer-
tain tasks at nano-scales. More importantly, based on transmission and reception
of molecules, cells in a biological organism may establish cellular signaling net-
works [80], through which they can communicate in order to realize more complex
and vital tasks, e.g., immune system responses. Therefore, the inspiration from
cellular signaling networks, and hence, molecular communication, provide impor-
tant research directions and promising design approaches for communication and
networking solutions at micro- and nano-scales.
1.2 Network Design Inspired by Biological Models
The main intention of this first chapter is to introduce and to overview the
emerging area of bio-inspired networking. Therefore, the scope of this section is
first to introduce the general approach to bio-inspired networking by discussing
26
1.2 Network Design Inspired by Biological Models
the identification of biological structures and techniques relevant to communica-
tion networks, modeling the systems and system properties, and finally deriving
optimized technical solutions. Secondly, we try to classify the field of biologi-
cally inspired approaches to networking. Bio-inspired algorithms can effectively
be used for optimization problems, exploration and mapping, and pattern recog-
nition. Based on several examples, we will see that bio-inspired approaches have
outstanding capabilities and potential applications that motivate our interest.
1.2.1 Modeling Approaches
Before introducing the specific biological models that have been exploited to-
wards the development and realization of bio-inspired networking solutions, we
briefly illustrate the general modeling approach. First modeling approaches date
back to the early 1970ies [31, 32], since, that time, quite a number of technical
solutions mimicking biological counterparts have been developed and published.
Typical bio-networking architectures showing the complete modeling approach
are described in [33, 34]. This bio-networking architecture can be seen as a cat-
alyzer or promoter for many other investigations in the last decade. Looking at
many papers that have been published in recent years, the main effort was focused
on presenting technical solutions with some similarities to biological counterparts
without really investigating the key advantages or objectives of the biological sys-
tems. Obviously, many methods and techniques are really bio-inspired as they
follow principles that have been studied in Nature and that promise positive ef-
fects if applied to technical systems. Three steps can be identified that are always
necessary for developing bio-inspired methods that have a remarkable impact in
the domain under investigation:
1. Identification of analogies - similar structures and methods,
2. Understanding - detailed modeling of realistic biological behavior,
3. Engineering - model simplification and tuning for technical applications.
27
1.2 Network Design Inspired by Biological Models
First, analogies between biological and technical systems, such as computing and
networking systems, must be identified. It is especially necessary that all the
biological principles are understood properly, which is often not yet the case in
biology. Secondly, models must be created for the biological behavior. These
models will later be used to develop the technical solution. The translation
from biological models to the model describing bio-inspired technical systems is
a pure engineering step. Finally, the model must be simplified and tuned for the
technical application. As a remark, it should be mentioned that biologists already
started looking at bio-inspired systems to learn more about the behavioral pattern
in nature [36]. Thus, the loop closes from technical applications to biological
systems.
1.2.2 Classification and Categorization
Basically, the following application domains of bio-inspired solutions to prob-
lems related to computing and communications can be distinguished:
• Bio-inspired computing represents a class of algorithms focusing on efficient
computing, e.g. for optimization processes and pattern recognition.
• Bio-inspired systems constitute a class of system architectures for massively
distributed and collaborative systems, e.g. for distributed sensing and ex-
ploration.
• Bio-inspired networking is a class of strategies for efficient and scalable
networking under uncertain conditions, e.g. for autonomic organization in
massively distributed systems.
Looking from biological principles, several application domains in networking
can be distinguished. Besides these specific algorithms that are mimicking biolog-
ical mechanisms and behavior, the general organization of biological systems, i.e.
the structure of bodies down to organs and cells, can be used as an inspiration
to develop scalable and self-organizing technical systems.
28
1.3 Bio-Inspired Networking
1.3 Bio-Inspired Networking
In this section, we introduce some examples of bio-inspired networking devel-
oped in last years. Of course, the following list is not meant to be comprehensive
and to completely represent all approaches in the domain of bio-inspired network-
ing. However, we selected a number of techniques and methods for more detailed
presentation that clearly show advantages in fields of communication networks.
In the discussion, we try to highlight the necessary modeling of biological phe-
nomena or principles and their application in networking.
1.3.1 Swarm Intelligence
Coordination principles studied in the fields of swarm intelligence [22] and es-
pecially those related to social insects give insights into principles of distributed
coordination in Nature. In many cases, direct communication among individ-
ual insects is exploited, e.g., in the case of dancing bees [39] or social foraging
swarms [118]. However, especially the stigmergic communication via changes in
the environment is as fascinating as helpful to coordinate massively distributed
systems. For example, Ma and Krings studied the chemosensory communication
systems in many of the moth, ant and beetle populations [38]. The difference
between the wireless network of an insect population and an engineered wireless
sensor network is that insects encode messages with semiochemicals (also known
as infochemicals) rather than with radio frequencies. Application examples of the
bees’ dance range from routing to intruder detection [39]. Another typical ex-
ample is the communication between ants for collaborative foraging. Ant Colony
Optimization (ACO) is perhaps the best analyzed branch of swarm intelligence
based algorithms. In general, swarm intelligence is based on the observation of
the collective behavior of decentralized and self-organized systems such as ant
colonies, flocks of fish, or swarms of bees or birds [22]. Such systems are typically
made up of a population of simple agents interacting locally with one another
and with their environment. In most cases, swarm intelligence based algorithms
29
1.3 Bio-Inspired Networking
are inspired by the behavior of foraging ants [22]. Ants are able to solve complex
tasks by simple local means. There is only indirect interaction between individ-
uals through modifications of the environment, e.g. pheromone trails are used
for efficient foraging. ACO is based on the principles of the foraging process of
ants. Ants perform a random search (random walk) for food. The way back to
the nest is marked with a pheromone trail. If successful, the ants return to the
nest (following their own trail). While returning, an extensive pheromone trail
is produced pointing towards the food source. Thus, further ants will follow and
reinforce the trail on the shortest path towards the food. The ants therefore
communicate based on environmental changes (pheromone trail), i.e. they use
stigmergic communication techniques for communication and collaboration. The
complete ACO algorithm is described in [40]. The most important aspect in this
algorithm is the transition probability pkij for an ant k to move from i to j. This
probability represents the routing information for the exploring process
pkij =
(
ταij(t)× ηβij
)
/(
∑
l∈Jkiταil (t)η
βil
)
, if j ∈ Jki ;
0, otherwise.(1.1)
Each move depends on the following parameters:
• Jki is the “tabu” list of not yet visited nodes, i.e., by exploiting Jk
i , an ant
k can avoid visiting a node i more than once.
• ηij is the visibility of j when standing at i, i.e., the inverse of the distance.
• τij is the pheromone level of edge (i; j), i.e., the learned desirability of
choosing node j and currently at node i.
• α and β are adjustable parameters that control the relative weight of the
trail intensity τij and the visibility ηij , respectively.
After completing a tour, each ant k lays a quantity of pheromone ∆τkij on each
edge (i; j) according to the following rule, where T k(t) is the tour done by ant
30
1.3 Bio-Inspired Networking
k at iteration t, Lk(t) is its length, and Q is a parameter (which only weakly
influences the final result)
∆τkij =
Q/Lk(t), if (i; j) ∈ T k(t);
0, otherwise.(1.2)
Dynamics in the environment are explicitly considered by the ant foraging scheme.
The pheromone slowly evaporates. Thus, if foraging ants are no longer successful,
the pheromone trail will dissolve and the ants continue with their search process.
Additionally, randomness is also a strong factor during successful foraging. A
number of ants will continue the random search for food. This adaptive behavior
leads to an optimal search and exploration strategy. This effect is provided by
the pheromone update rule, where ∆τij =∑m
k=1∆τkij. The decay is implemented
in form of a coefficient ρ with 0 < ρ < 1.
τij(t)← (1− ρ)× τij(t) + ∆τij(t) (1.3)
According to [40], the total number of ants m is an important parameter of the
algorithm. Too many ants would quickly reinforce suboptimal tracks and lead
to early convergence to bad solutions, whereas too few ants would not produce
enough decaying pheromone to achieve the desired cooperative behavior. Thus,
the decay rate needs to be carefully controlled.
The main applications in networking that are based on the main concepts of
ACO are briefly reported in the following.
1. Routing : The best known examples of ACO in networking are the AntNet
[41] and AntHocNet [40] routing protocols. Both protocols follow the con-
cepts of ant routing. In particular, so called agents are used to concurrently
explore the network and exchange collected information in the same way
as ants explore the environment. The communication among the agents is
indirect, following the stigmergy approach, and mediated by the network
itself.
31
1.3 Bio-Inspired Networking
2. Task Allocation : Based on the same concepts, integrated task allocation
and routing in SANETs has been investigated [81]. The proposed architec-
ture is completely based on probabilistic decisions. During the lifetime of
the SANET, all nodes maintain and adapt a probability P (i) to execute a
task out i of a given set. Reinforcement strategies are exploited to optimize
the overall system behavior. It needs to be mentioned that the integrated
task allocation and routing approach represents a typical cross-layer solu-
tion. Application layer and network layer are both responsible for operating
the entire SANET.
3. Search in Peer-2-Peer networks : Search in Peer-2-Peer (P2P) net-
works is usually provided by centralized or decentralized lookup tables.
However, the effort to find data in unstructured decentralized P2P net-
works can easily become the dominating factor. Also in this case, it is
expected the use of ant-based approaches [42].
1.3.2 Firefly Synchronization
Precise synchronization in massively distributed systems is a complex issue
and hard to achieve. Recently, new models for clock synchronization have been
proposed based on the synchronization principles of fireflies. In this context, early
biological experiments have been conducted by Richmond, who also discovered
the underlying mathematical synchronization model [27]. Basically, the firefly
synchronization is based on pulse-coupled oscillators [43]. The simple model for
synchronous firing of biological oscillators consists of a population of identical
integrate-and-fire oscillators. A local variable xi is integrated from zero to one
and the oscillator fires when xi = 1. Then, the xi jumps back to zero.
dxidt
= S0 − γxi. (1.4)
Multiple oscillators are assumed to interact in form of simple pulse coupling:
when a given oscillator fires, it pulls the others up by a fixed amount ǫ, bringing
32
1.3 Bio-Inspired Networking
them toward the firing threshold.
xi(t) = 1 ∀j 6= i : xj(t+) = min(1;xj(t) + ǫ) (1.5)
As a result, for almost all initial conditions the population evolves to a state in
which all the oscillators are firing synchronously. The presented concept of self-
organized clock synchronization has been successfully applied to synchronization
in ad hoc networks [58, 59]. Using a linearly incrementing phase function φi, the
local pulse of a node is controlled: when φi reaches a threshold φth, the local
oscillator fires. For a period of T , this can be described as follows:
dφidt
=φthT. (1.6)
When coupling identical oscillators, the phase can be controlled according to
Equation (1.5). Additional effort is needed to compensate the transmission de-
lays in ad hoc and sensor networks. This can be done by selecting appropriate
values for ǫ. In particular, the phase shift is dynamically updated according to
the estimated transmission delay. The general application of this clock synchro-
nization technique for wireless networks is discussed in [44].
1.3.3 Activator-Inhibitor Systems
The basis for exploiting the characteristics of activator-inhibitor systems in
technical systems is the analysis of reaction-diffusion mechanisms. In the 1950ies,
the chemical basis of morphogenesis has been analyzed [45]. The underlying
reaction and diffusion in a ring of cells has been successfully described in form
of differential equations. Assuming that for concentrations of X and Y chemical
reactions are tending to increase X at the rate f(X;Y ) and Y at the rate of
g(X;Y ), the changes of X and Y due to diffusion also take into account the
behavior of the entire system, i.e. all the neighboring N cells. Thus, the rate
of such chemical reactions can be described by the 2N differential equations [45]
(where r = 1, . . . , N , µ is the diffusion constant for X and ν is the diffusion
33
1.3 Bio-Inspired Networking
constant for Y ):
dXr
dt= f(Xr;Yr) + µ(Xr+1 −X2r +Xr−1) (1.7)
dYrdt
= g(Xr;Yr) + ν(Xr+1 −X2r +Xr−1) (1.8)
For general application (independent of the shape of the generated pattern or the
structure of interacting systems), this set of differential equations can be written
as (with F and G being nonlinear functions for (chemical) reactions, Du and Dv
describe the diffusion rates of activator and inhibitor, and ∇2 is the Laplacian
operator):
du
dt= f(u; v)−Du∇
2u (1.9)
dv
dt= g(u; v) −Dv∇
2v (1.10)
Reaction-diffusion pattern formation is used to support high-level tasks in smart
sensor networks. In particular, on-off patterns in large-scale deployments for for-
est fire scenarios have been investigated. As a key result, different shapes have
been detected such as stripes, spots, and ring patterns, that can be exploited for
high-level activities such as navigating robots to the source of the fire. Further
experiments and considerations on reaction-diffusion based pattern generation
in sensor networks are described in [46]. Again, reaction-diffusion based con-
trol mechanisms have been investigated. Similarly, cooperative control can be
achieved based on a reaction-diffusion equation for surveillance system [49]. As
can be seen from the mentioned approaches, sensor coordination is one of the
primary application fields for employing activator-inhibitor mechanisms. In the
following, two further solutions are depicted that coordinate sensing activities in
WSNs to achieve improved energy performance, i.e. to maximize the network
lifetime [48]. In [47], pattern formation models are used to coordinate the on-off
cycles of sensor nodes. In particular, sensors are allowed to control their sen-
sory and their radio transceiver while, at the same time, the network needs to
be able to transmit sensor data over a multi-hop network to one or more data
34
1.3 Bio-Inspired Networking
sinks. Each sensor stores its own activator and inhibitor values and broadcasts
them every τ seconds. Using the received data, the neighboring nodes re-evaluate
the reaction-diffusion equations. Sensors with a activator value exceeding some
(given) threshold become active by turning on their sensing circuitry. As shown
in [47], the performance of the system achieves astonishingly good results. Sim-
ilarly, the distributed control of processing periods is investigated in [28]. Using
the programming system Rule-based Sensor Network (RSN), a sensor network is
configured for target tracking. In this example, the duty cycle is controlled by a
promoter/inhibitor system that takes into account the efficiency of the local ob-
servations and the results from neighboring nodes. By exploiting the information
transmitted towards a sink node, each node can estimate the need for further
local measurements and adequately update the local sampling period.
1.3.4 Artificial Immune System
The term Artificial Immune System (AIS) refers to adaptive systems inspired
by theoretical and experimental immunology with the goal of problem solving [50].
The primary goal of an AIS, which is inspired by the principles and processes
of the mammalian immune system [25], is to efficiently detect changes in the
environment or deviations from the normal system behavior in complex problems
domains. The role of the mammalian immune system can be summarized as
follows: It protects the body from infections by continuously scanning for invading
pathogens, e.g. exogenous (non-self) proteins. AIS based algorithms typically
exploit the immune system’s characteristics of self-learning and memorization.
The immune system is, in its simplest form, a cascade of detection and adaptation,
culminating in a system that is remarkably effective. In Nature, two immune
responses were identified. The primary one is to launch a response to invading
pathogens leading to an unspecific response (using Leucoytes). In contrast, the
secondary immune response remembers past encounters, i.e. it represents the
immunologic memory. It allows a faster response the second time by showing a
35
1.3 Bio-Inspired Networking
very specific response (using B-cells and T-cells). An AIS basically consists of
three parts, which have to be worked out in the immune engineering process [50]:
• Representations of the system components, i.e. the mapping of technical
components to antigens and antibodies.
• Affinity measures, i.e. mechanisms to evaluate interactions (e.g., stimula-
tion pattern and fitness functions) and the matching of antigens and anti-
bodies.
• Adaptation procedures to incorporate the system’s dynamics, i.e. genetic
selection.
A first AIS has been developed by Kephart [51], and early approaches showing
the successful application of such AISs in computer and communication systems
have been presented in [25]. Meanwhile, a number of frameworks are available.
Focusing on the design phase of an AIS, de Castro and Timmis [50] proposed an
immune engineering framework. A similar conceptual frameworks for Artificial
Immune Systems for generic application in networking has been presented in [52].
Again, three steps for designing the framework have been emphasized: represen-
tation, selection of appropriate affinity measures, and development of immune
algorithms. In this framework, Markov chains are used to describe the system’s
dynamics. Data analysis and anomaly detection represent typical application
domains [50]. The complete scope of AISs is widespread. Sample applications
have been developed for fault and anomaly detection, data mining (e.g., ma-
chine learning, pattern recognition), agent based systems, control, and robotics.
Pioneering work by Timmis and coworkers conceptually analyzed the AIS and ap-
plied it to several problem domains [20,52]. An application of an immune system
based distributed node and rate selection in sensor networks has been proposed
in [15]. Sensor networks and their capabilities, in particular their transmission
rate, are modeled as antigens and antibodies. The distributed node and rate
selection (DNRS) algorithm for event monitoring and reporting is achieved by B-
36
1.3 Bio-Inspired Networking
cell stimulation, i.e. appropriate node selection. This stimulation depends on the
following influences: (1) the affinity between the sensor node (B-cell) and event
source (pathogen), (2) the affinity between the sensor node and its uncorrelated
neighbor nodes (stimulating B-cells), and (3) the affinity between the sensor node
and its correlated neighbor nodes (suppressing B-cells).
1.3.5 Epidemic Spreading
Epidemic Spreading is frequently used as an analogy to understand the in-
formation dissemination in wireless ad hoc networks. Information dissemination
in this context can refer to the distribution of information particles (as usually
provided by ad hoc routing techniques) [24] or to the spread of viruses in the
Internet [53] or on mobile devices [55]. Biological models of virus transmission
provide means for assessing such emerging threats and to understand epidemics
as a general purpose communication mechanism. A number of mathematical
models of different networks have been investigated, which lie at various points
on a broad conceptual spectrum. At one end there are network models reflect-
ing strong spatial effects, with nodes at fixed positions in two dimensions, each
connected to a small number of neighbors. At the other end there are scale-free
networks, which are essentially unconstrained by physical proximity, and in which
the number of contacts per node are widely spread. The main difference is in the
epidemic spread. In scale-free networks, epidemics can persist at arbitrarily low
levels, whereas in simple two-dimensional models a minimum level of virulence
is needed to prevent them from dying out quickly [55]. The system model for
epidemic communication relies on a population, i.e. a number of nodes that rep-
resent the network. Information entities are exchanged among the nodes using
a diffusion algorithm. All transmissions are usually assumed to be atomic, i.e.
there will be no split during diffusion. Then, all the nodes can be distinguished
into two groups: susceptible nodes, S(t) describes this set at a certain time t, and
infective nodes I(t) [56]. The diffusion algorithm is then a process that converts
37
1.3 Bio-Inspired Networking
susceptible nodes into infective nodes with a rate α = βxN I(t), where β is the
probability of information transmission, i.e. the infection probability, x describes
the number of contacts among susceptible nodes, and N is the total number of
nodes. The infection rate can then be described as:
dI
dt= α× S(t) =
βx
NI(t)× S(t) (1.11)
A measure for the connectedness of the nodes is termed eigenvector centrality.
Let us consider a graph model of the network topology and denote by A the
corresponding adjacency matrix. The eigenvector centrality of a node i is defined
being proportional to the sum of the eigenvector centralities of i’s neighbors,
where e represents the vector of nodes’ centrality scores. Otherwise stated, e is
the eigenvector of A relative to the eigenvalue λ:
ei = Ae/λ. (1.12)
Depending on the particular application scenario, the healing rate, i.e. the non-
negative rate of converting infective nodes, also needs to be considered in this
equation. There is a wide application range for epidemic communication in com-
puter networks. Primarily, the focus is on routing in mobile ad hoc networks with
growing interest in opportunistic routing, in which messages are passed between
devices that come into physical proximity, with the goal of eventually reaching
a specified recipient. For example, the understanding of the spread of epidemics
in highly partitioned mobile networks has been studied in [57]. The main appli-
cation field in this work was the use of epidemic communication in DTNs. As a
conclusion, the paper outlines the possibility to roughly measure the importance
of a node to the process of epidemic spreading by the node’s eigenvector central-
ity. Regions, as defined by the steepest-ascent rule, are clusters of the network in
which spreading is expected to be relatively rapid and predictable. Furthermore,
nodes whose links connect distinct regions play an important role in the (less
rapid, and less predictable) spreading from one region to another. The charac-
teristics of epidemic information dissemination have been carefully modeled to
38
1.3 Bio-Inspired Networking
investigate the inherent characteristics [58]. For example, the buffer management
plays and important role and a stepwise probabilistic buffering has been proposed
as a solution [59]. Detailed models have been built to study the performance im-
pact of epidemic spreading [60]. Whereas Markov models lead to quite accurate
performance predictions, the numerical solution becomes impractical if the num-
ber of nodes is large. In [60], an unified framework based on ordinary differential
equations is presented that provides appropriate scaling as the number of nodes
increases. This approach allows the derivation of closed-form formulas for the
performance metrics while obtaining matching results compared to the Markov
models. The interesting result is that the network topology plays an important
role whether epidemics can be applied for improved robustness and efficiency. In
particular, the scale-free property must be ensured in order to overcome possible
problems with transmissions that quickly die out. A slightly different problem
(and solution) has been addressed in [61]. The targeted question is that the prob-
lem of determining the right information collection infrastructure can be viewed
as a variation of the network design problem including additional constraints such
as energy efficiency and redundancy. As the general problem is NP-hard, the au-
thors propose a heuristic based on the mammalian circulatory system, which
results in a better solution to the design problem than the state-of-the-art alter-
natives. The resulting system for wireless sensor networks is quite similar to the
epidemics approach even though only the communication within an organism is
used as an analogy. Besides efficient routing solutions, the application to network
security is maybe the most important aspect of epidemic models.
1.3.6 Nano-scale and Molecular Communication
Incredible improvements in the field of nano-technologies have enabled nano-
scale machines that promise new solutions for several applications in biomedical,
industry and military fields. Some of these applications might exploit the po-
tential advantages of communication and hence cooperative behavior of these
39
1.3 Bio-Inspired Networking
nano-scale machines to achieve a common and challenging objective that exceeds
the capabilities of a single device. At this point, the term nano-networks is de-
fined as a set of nano-scale devices, i.e., nano-machines, communicating with each
other and sharing information to realize a common objective. Nano-networks
allow nano-machines to cooperatively communicate and share any kind of in-
formation such as odor, flavor, light, or any chemical state in order to achieve
specific tasks required by wide range of applications including biomedical en-
gineering, nuclear, biological, and chemical defense technologies, environmental
monitoring. Despite the similarity between communication and network func-
tional requirements of traditional and nano-scale networks, nano-networks bring
a set of unique challenges. In general, nano-machines can be categorized into
two types: one type mimics the existing electro-mechanical machines and the
other type mimics nature-made nano-machines, e.g., molecular motors and re-
ceptors. In both types, the dimensions of nano-machines render conventional
communication technologies such as electromagnetic wave, acoustic, inapplicable
at these scales due to antenna size and channel limitations. In addition, the avail-
able memory and processing capabilities are extremely limited, which makes the
use of complex communication algorithms and protocols impractical in the nano
regime. Furthermore, the communication medium and the channel characteris-
tics also show important deviations from the traditional cases due to the rules of
physics governing these scales. For example, due to size and capabilities of nano-
machines, traditional wireless communication with radio waves cannot be used to
communicate between nano-machines, which may constitute of just several moles
of atoms or molecules and scale on the orders of a few nanometers. Hence, these
unique challenges need to be addressed in order to effectively realize the nano-
scale communication and nano-networks in many applications from nano-scale
body area networks to nano-scale molecular computers. The motivation behind
nano-machines and nano-scale communications and networks have also originated
and been inspired by the biological systems and processes. In fact, nano-networks
are significant and novel artifacts of bio-inspiration in terms of both their archi-
40
1.3 Bio-Inspired Networking
tectural elements, e.g., nano-machines, and their communication mechanism, i.e.,
molecular communication. Indeed, many biological entities in organisms have
similar structures with nano-machines, i.e., cells, and similar interaction mech-
anism and vital processes, cellular signaling [80], with nano-networks. Within
cells of living organisms, nano-machines called molecular motors, such as dynein,
myosin [62], realize intracellular communication through chemical energy trans-
formation. Similarly, within a tissue, cells communicate with each other through
the release over the surface and the diffusion of certain soluble molecules, which
are received by a specific receptor molecule on another cell. Apparently, cellu-
lar signaling networks are the fundamental source of inspiration for the design
of nano-networks. Therefore, the communication and networking problems in
nano-networks may also be inspired by the similar biological processes. The
main communication mechanism of cellular signaling is based on transmission
and reception of certain type of molecules, i.e., molecular communication, which
is, indeed, the most promising and explored communication mechanism for nano-
networks. In Nature, molecular communication between biological entities takes
place according to the ligand receptor binding mechanism. Ligand molecules are
emitted by one biological phenomenon; then, the emitted ligand molecules dif-
fuse in the environment and bind the receptors of another biological phenomenon.
This binding enables the biological phenomenon to receive the bound molecules
by means of the diffusion on cell membrane. The received ligand molecules allow
the biological phenomenon to understand the biological information. For exam-
ple, in a biological endocrine system, gland cells emit hormones to intercellular
environment; then, hormone molecules diffuse and are received by correspond-
ing cells. According to the type of emitted hormone, the corresponding cells
convert the hormone molecule to biologically meaningful information. This nat-
ural mechanism provides the molecular communication for almost all biological
phenomena. Following the main principles of this mechanism, a number of stud-
ies have been performed on the design of nano-scale communication. Molecular
communication and some design approaches are introduced [64], and its funda-
41
1.3 Bio-Inspired Networking
mental research challenges are first manifested in [65]. Different mechanisms are
proposed for molecular communication including a molecular motor communi-
cation system [66], intercellular calcium signaling networks [63], an autonomous
molecular propagation system to transport information molecules using DNA hy-
bridization and bio-molecular linear motors. An information theoretical analysis
of a single molecular communication channel is performed in [7]. An adaptive er-
ror compensation mechanism is devised for improving molecular communication
channel capacity in [67]. In [68], molecular multiple access, relay and broadcast
channels are modeled and analyzed in terms of capacity limits and the effects of
molecular kinetics and environment on the communication performance are inves-
tigated. Based on the use of vesicles embedded with channel forming proteins,
a communication interface mechanism is introduced for molecular communica-
tion in [69, 70]. In addition, a wide range of application domains of molecular
communication based nano-networks are introduced from nano-robotics to future
health-care systems [71]. Clearly, inspired by biological systems, molecular com-
munication, which enables nano-machines to communicate with each other using
molecules as information carrier, stands as the most promising communication
paradigm for nano-networks. While some research efforts and initial set of results
exist in the literature, many open research issues remain to be addressed for the
realization of nano-networks. Among these, first is the thorough exploration of
biological systems, communications and processes, in order to identify different
efficient and practical communication techniques to be exploited by innovative
nano-network designs. The clear set of challenges for networked communication
in nano-scale environments must be precisely determined for these different po-
tential bio-inspired solution avenues. Applicability of the traditional definitions,
performance metrics and well-known basic techniques, e.g., Time Division Mul-
tiple Access (TDMA), random access, minimum cost routing, retransmission,
error control, congestion, must be studied. Furthermore, potential problems for
the fundamental functionalities of nano-networks, such as modulation, channel
coding, medium access control, routing, congestion control, reliability, must be
42
1.4 Outline of Dissertation
investigated without losing the sight of the bio-inspired perspective in order to
develop efficient, practical and reliable nano-network communication techniques
through inspiration from the existing biological structures and communication
mechanisms.
1.4 Outline of Dissertation
The outline of each chapter is as follows.
Chapter 1, the present chapter, gives the motivation, outline and contributions
of this dissertation. A brief survey on bio-inspired networking is also presented
to enlighten the potential benefits offered by bio-inspired solutions.
Chapter 2 introduces several important theories on which many results of this
dissertation are based. In particular, section 2.1 recalls some basic results from
dynamical systems theory, focusing on the stability analysis of linear and nonlin-
ear systems. In section 2.2 we analyze the main descent methods used to optimize
a cost function in a distributed manner, reporting classical convergence results.
Then, in section 2.3, we recall stochastic approximation methods aimed at finding
zeros or maxima of an objective function, whose value measurable at each time
instant is corrupted by additive noise, providing basic convergence results. The
last section is devoted to graph theory, reviewing the notation and basic results
that will be largely used in this dissertation.
Chapter 3 proposes a bio-inspired radio access mechanism for cognitive networks
mimicking the behavior of a flock of birds swarming in search for food in a cohe-
sive fashion without colliding with each other. The equivalence between swarming
and radio resource allocation is established by modeling the interference distribu-
tion in the resource domain, e.g. frequency and time, as the spatial distribution
of food, while the position of the single bird represents the radio resource chosen
by each radio node. The solution is given as the distributed minimization of a
43
1.4 Outline of Dissertation
functional, borrowed from social foraging swarming models, containing the aver-
age interference plus repulsion and attraction terms that help to avoid conflicts
and maintain cohesiveness, respectively. A stability and cohesion analysis is de-
rived under different assumptions on the attraction/repulsion terms, showing the
effect played by the swarm parameters and connectivity on the final swarm size.
Several examples illustrate how the proposed method can be applied to dynamic
resource allocation on the frequency domain and the time-frequency domain, pro-
viding an intrinsic capability of the system to provide spatial reuse of frequency,
through a purely decentralized mechanism. In the last part of the chapter, we
also consider the swarming algorithm in the presence of channel imperfections,
such as link failures, estimation errors, and quantization noise. Thus, we derive
the almost sure convergence of the swarming procedure to an equilibrium con-
figuration dependent on the mean graph of the network, even in the presence of
such random disturbances.
Chapter 4 develops an adaptive algorithm for spectrum estimation in cogni-
tive radio networks based on diffusion adaptation algorithms. In particular, we
address this task through a parsimonious basis expansion model of the PSD in
frequency. This model reduces the sensing task to estimating a common vector of
unknown parameters. The resulting estimator relies on diffusion adaptation al-
gorithms, where the cognitive radios exchange information locally only with their
one-hop neighbors, eliminating the need for a fusion center. First, we describe
the basic diffusion algorithm, then we introduce novel regularized diffusion LMS
techniques for distributed estimation over adaptive networks, which are able to
exploit sparsity in the underlying system model. Convergence and mean square
analysis of the sparse adaptive diffusion filter show under what conditions we
have dominance of the proposed method with respect to its unregularized coun-
terpart in terms of steady-state performance. Simulation results also confirm the
potential benefits of the proposed filter under the sparsity assumption on the
true coefficient vector. Exploiting these estimation schemes, we illustrate the
44
1.5 Research Contributions
proposed distributed spectrum estimation technique based on diffusion adapta-
tion. We first introduce a basis expansion model, which is useful to model the
PU’s transmission, allowing distributed cooperative sensing. Then, we propose a
normalized version of the Adapt then Combine (ATC) diffusion algorithm, which
enables the network to learn and track the time-varying interference profile. Con-
vergence and mean-square performance analysis of the proposed normalized ATC
diffusion filter, applied to the spectrum estimation problem, is also derived.
Chapter 5 studies the learning abilities of adaptive networks in the context of
cognitive radio networks and investigates how well they assist in allocating power
and communications resources in the frequency domain. The allocation mech-
anism is based on a social foraging swarm model that lets every node allocate
its resources (power/bits) in the frequency regions where the interference is at a
minimum while avoiding collisions with other nodes. We employ adaptive diffu-
sion techniques to estimate the interference profile in a cooperative manner and
to guide the motion of the swarm individuals in the resource domain. The result-
ing bio-inspired network cooperatively estimates the interference profile in the
resource domain of a cognitive network and allocates resources through purely
decentralized mechanisms. Finally, the resulting procedure is applied to the dy-
namic resource allocation problem in the frequency domain. Numerical results
show the improvement that results in the resource allocation performance due
to the cooperative estimation of the spectrum. Furthermore, it is shown how
the proposed technique endows the resulting bio-inspired network with powerful
learning and adaptation capabilities.
Chapter 6 concludes the dissertation summarizing the main obtained results.
1.5 Research Contributions
The main contribution of this dissertation is the development of a bio-inspired
resource allocation technique for dynamic radio access in cognitive radio systems.
45
1.5 Research Contributions
Details of the research contributions in each chapter are as follows.
Chapter 3
The main result in this chapter is regarding the bio-inspired formulation of
the resource allocation problem published (or to be published) in two journal
papers and three conference papers:
• P. Di Lorenzo and S. Barbarossa, “A bio-inspired swarming algorithm for
decentralized access in cognitive radio,” IEEE Transactions on Signal Pro-
cessing, vol. 59, no. 12, December 2011.
• P. Di Lorenzo and S. Barbarossa, “Decentralized resource assignment in
cognitive networks based on swarming mechanisms over random graphs,”
accepted for publication in IEEE Transactions on Signal Processing.
• P. Di Lorenzo and S. Barbarossa, “Distributed resource allocation in cog-
nitive radio systems based on social foraging swarms,” in Proceedings of
the 11th IEEE International Workshop on Signal Processing Advances in
Wireless Communications, Marrakech, pp. 1-5, June 2010. (IEEE best
student paper award)
• P. Di Lorenzo and S. Barbarossa, “Bio-inspired swarming models for de-
centralized radio access incorporating random links and quantized commu-
nications,” Proceedings of the 36th International Conference on Acoustics,
Speech and Signal Processing , pp. 5780-5783, Prague, May 2011. (Invited
to the special session: Bio-inspired Information Processing and Networks).
• P. Di Lorenzo, S. Barbarossa, and Ali H. Sayed “A bio-inspired fast swarm-
ing algorithm for dynamic radio access,” in Proceedings of the 17th Interna-
tional Conference on Digital Signal Processing , Corfu, Greece, July 2011,
pp. 1-6. (Invited to the special session: Signal Processing for Cognitive
Radio).
46
1.5 Research Contributions
Chapters 4 and 5
The main results in these chapters regard the development of a totally adap-
tive bio-inspired network, based on swarming and diffusion adaptation mecha-
nisms, that senses, learns and reacts to changes in the environment. The research
contributions are published (or will be published) in two journal papers and two
conference papers:
• P. Di Lorenzo, Ali H. Sayed, and S. Barbarossa “Bio-inspired dynamic radio
access based on swarming mechanisms over adaptive networks,” submitted
to IEEE Transactions on Signal Processing.
• P. Di Lorenzo, Ali H. Sayed, and S. Barbarossa “Sparse distributed estima-
tion over adaptive networks,” submitted to IEEE Transactions on Signal
Processing.
• P. Di Lorenzo, S. Barbarossa, and Ali H. Sayed “Bio-inspired swarming for
dynamic radio access based on diffusion adaptation,” in Proceedings of the
19-th European Signal Processing Conference, Barcelona, Spain, August-
September 2011, pp. 402-406. (EURASIP best student paper award)
• P. Di Lorenzo, S. Barbarossa, and Ali H. Sayed “Sparse diffusion LMS for
distributed adaptive estimation,” in Proc. of the International Conference
on Acoustics, Speech and Signal Processing, Kyoto, Japan, March 2012.
Other contributions not presented in this dissertation
During the author’s Ph.D. period, optimal beamforming techniques were de-
veloped for ambiguity suppression in squinted synthetic aperture radar systems.
The results are published (or will be published) in one journal paper and one
conference paper:
• P. Di Lorenzo, S. Barbarossa, and Leonardo Borgarelli “Optimal beamform-
ing for range-doppler ambiguity minimization in squinted SAR,” accepted
47
1.5 Research Contributions
for publication in IEEE Transactions on Aerospace and Electronic Systems.
• P. Di Lorenzo and S. Barbarossa “Optimal beamforming for range-doppler
ambiguity suppression in squinted SAR systems,” in Proceedings of the 4th
IEEE International Workshop on Computational Advances in Multi-Sensor
Adaptive Processing, San Juan, Puerto Rico, pp. 169-172, December 2011.
(IEEE best student paper award)
Some work was also done in the area of distributed resource allocation in
femtocell networks systems with results published (or to be published) in one
journal paper and one conference paper:
• P. Di Lorenzo, S. Barbarossa and Marco Omilipo, “Distributed sum-rate
maximization over finite rate coordination links affected by random fail-
ures,” submitted to IEEE Transactions on Signal Processing.
• P. Di Lorenzo, Marco Omilipo and S. Barbarossa “Distributed stochas-
tic pricing for sum-rate maximization in femtocell networks with random
graph and quantized communications,” in Proceedings of the 4th IEEE In-
ternational Workshop on Computational Advances in Multi-Sensor Adap-
tive Processing, San Juan, Puerto Rico, pp. 165-168, December 2011.
Furthermore, some work was done in the field of signal processing for wireless
sensor networks with results published (or to be published) in one journal paper
and one conference paper:
• Sergio Barbarossa, Stefania Sardellitti and Paolo Di Lorenzo, “Distributed
estimation and detection with applications in Wireless Sensor Networks,”
submitted to Elsevier Signal Processing e-reference.
• P. Di Lorenzo and S. Barbarossa, “Wireless Sensor Networks With Dis-
tributed Decision Capabilities Based On Self-Synchronization Of Relax-
ation Oscillators,” in Proc. IEEE International Wireless Communications
and Mobile Computing Conference 2008, Creta, August 2008, pp. 45-49.
48
Chapter 2
Mathematical Background
In this chapter we recall some basic mathematical tools that will be largely
used throughout this work. First, in section 2.1, we consider dynamical systems,
both in continuous and discrete time, giving the basic analytic tools to prove
stability in the linear and nonlinear case. Section 2.2 introduces iterative methods
for the solution of nonlinear problems, focusing on descent methods and reporting
several convergence results in the unconstrained and constrained case. Then, in
section 2.3, we recall stochastic approximation methods aimed at finding zeros or
maxima of an objective function, whose value measurable at each time instant is
corrupted by additive noise, providing basic convergence results. The last section
is devoted to graph theory, reviewing the notation and basic results that will be
largely used in this work.
2.1 Dynamical Systems
In this section we introduce and illustrate some important concepts of dy-
namical system theory. A dynamical system consists of a set of variables that
describe its state and a law that describes the evolution of the state variables
with time, i.e., how the state of the system in the next moment of time depends
on the input and its state in the previous moment of time. The evolution law is
49
2.1 Dynamical Systems
given by a system of ordinary differential equations. Mathematically, a dynam-
ical system is specified by a state vector x ∈ Rn, (a list of numbers which may
change as time progresses) and a function f : Rn → Rn, which describes how the
system evolves over time. There are two kinds of dynamical systems: discrete
time and continuous time. For a discrete time dynamical system, we denote time
by k, and the system is specified by the equations
x[k + 1] = f(x[k]), x[0] = x0. (2.1)
It thus follows that x[k] = fk(x0), where fk denotes a k-fold application of f
to x0. For a continuous time dynamical system, we denote time by t, and the
following equations specify the system:
x(t) = f(x(t)), x(0) = x0. (2.2)
where x is a time-dependent vector variable denoting the current state of the
system, x(t) is its derivative with respect to time t, f is a scalar function that
determines the evolution of the system, x0 ∈ Rn is an initial condition. In this
case, Rn is called phase space or state space to stress the fact that each state of the
system corresponds to a certain point in Rn. When all parameters are constant,
the dynamical system is called autonomous. When at least one of the parameters
is time-dependent, the system is non-autonomous. To solve (2.2) means to find
a function x(t) whose initial value is x[0] = x0 and whose derivative is f(x(t))
at each moment t ≥ 0. Finding explicit solutions is often impossible even for
such simple systems, so quantitative analysis is carried out mostly via numerical
simulations. The simplest procedure to solve numerically, known as first-order
Euler method, replaces (2.2) by the discretized system
x(t+ h)− x(t)
h= f(x(t)), (2.3)
where t = 0, h, 2h, 3h... is the discrete time and h is a small time step. Knowing
the current state x(t), we can find the next state point via
x(t+ h) = x(t) + hf (x(t)). (2.4)
50
2.1 Dynamical Systems
Iterating this difference equation starting with x[0] = x0, we can approximate the
analytical solution of (2.2). The approximation has a noticeable error of order h,
so scientific software packages, such as MATLAB, use more sophisticated high-
precision numerical methods. In many cases, however, we do not need exact
solutions, but rather qualitative understanding of the behavior of (2.2) and how
it depends on parameters and the initial state x0. For example, we might be
interested in the number of equilibrium (rest) points the system could have,
whether the equilibria are stable, their attraction domains, etc.
In the following we present some classical results from dynamical system the-
ory that will be used throughout the paper. First, we consider linear systems,
extending then our attention to nonlinear systems. The following sections deal
with fixed points and their stability. In particular, we consider two methods for
assessing stability: linearization and Lyapunov functions.
2.1.1 Linear Systems
Previously, we introduced discrete and continuous time dynamical systems.
The function f : Rn → Rn might be quite simple or terribly complicated. In
this section we study dynamical systems in which the function f is particularly
nice: we assume f is linear. Then, considering the general vector case, we have
f(x) = Ax + b, where A ∈ Rn × R
n is an n × n matrix and b ∈ Rn is a fixed
n-dimensional vector.
Discrete-Time
In this section we consider linear discrete time dynamical systems of the form:
x[k + 1] = Ax[k] + b, x[0] = x0. (2.5)
In our analysis, we begin by dropping the ”+b” term in (2.5) and concentrating
on the system x[k + 1] = Ax[k]. Now, it is very easy to check that
x[k] = Akx0. (2.6)
51
2.1 Dynamical Systems
To go further, we consider the case in which the matrix A diagonalizes. We
assume thatA has n linearly independent eigenvectors u1, . . . ,un with associated
eigenvalues λ1, . . . , λn . Let Λ be the diagonal matrix with diagonal entries
λ1, . . . , λn , and let U ∈ Rn × R
n be the n× n matrix whose i-th column is ui .
Thus, we may write A = UΛU−1. Notice that
Ak = (UΛU−1)(UΛU−1) . . . (UΛU−1). (2.7)
Now, since matrix multiplication is associative and the terms U−1U evaluate to
I, we have
Ak = UΛkU−1. (2.8)
Now, Λ is a diagonal matrix whose diagonal entries are the A’s eigenvalues:
λ1, . . . , λn. Raising a diagonal matrix to a power is an easy task, achieving
Λk = diag[λk1 , . . . , λkn]. Thus, to understand the behavior of the system, we
need to study the eigenvalues of the iteration matrix A. In particular, if all the
eigenvalues of A have absolute value less than 1, thenAk tends to the zero matrix
as k → ∞. Thus x[k] = Akx0 → 0. On the other hand, if some eigenvalue of
A has absolute value greater than 1, entries in Ak are diverging to ∞. Let’s
examine how this affects the values x[k]. We are assuming that the eigenvectors
u1, . . . ,un are linearly independent. Any family of n linearly independent vectors
in Rn forms a basis, hence every vector (x0 in particular) can be written uniquely
as a linear combination of the ui’s. Thus we may write
x0 = c1u1 + . . .+ cnun, (2.9)
where the ci’s are scalar numbers. Now, since each ui is an eigenvector of A, we
have Aui = λiui, thus Akui = λki ui. The iterative process in (2.6) can then be
written in terms of eigenvalues and eigenvectors of the matrix A as
x[k] = Akx0 = c1λk1u1 + . . .+ cnλ
knun. (2.10)
Then, if all the eigenvalues of A all have absolute value less than 1, then x[k]→ 0
as k → ∞. If some eigenvalue has absolute value bigger than 1, then typically
52
2.1 Dynamical Systems
|x[k]| → ∞. For some very special x0 (those with ci = 0 if |λi| > 1), x[k] does
not explode. Finally, if some eigenvalues have absolute value equal to 1, and
the rest have absolute value less than 1, then typically x[k] neither explodes nor
vanishes but dances about at a modest distance from 0. The situation for the
more general case x[k+1] = Ax[k]+b is quite similar. As before, it is possible to
determine the iterative pattern of the dynamical system in (2.5). In particular,
provided that the matrix I−A is invertible (A does not have an eigenvalue equal
to one), we have
x[k] = Akx0 + (I −Ak)(I −A)−1b. (2.11)
Then, if the absolute values of A’s eigenvalues are all less than 1 (hence I −A
is invertible), then Ak tends to the zero matrix, hence x[k] → x = (I −A)−1b.
Alternatively, if some eigenvalues have absolute value bigger than 1, then Ak
blows up, and for most x0 we have |x[k]| → ∞. (There are exceptional x0 ’s, of
course. For example, if 1 is not an eigenvalue of A and if x0 = x = (I −A)−1b,
then x[k] = x for all k. Finally, if some eigenvalues have absolute value equal
to 1 and the other eigenvalues have absolute value less than 1, we see a range of
behaviors and the system might stay near x, or it might blow up.
Continuous-Time
Now we turn to continuous-time multidimensional linear systems, that is,
systems of the form
x(t) = Ax(t) + b, x(0) = x0. (2.12)
As before, we begin our study of multidimensional continuous-time linear systems
with a simplification, namely, that b = 0. The system becomes x(t) = Ax(t).
This is a system of n differential equations in n variables. These equations are
difficult to solve because each is dependent on the other. However, it is possible
to show that the system x(t) = Ax(t) has nearly as simple a solution, which is
given by
x(t) = exp(At)x0. (2.13)
53
2.1 Dynamical Systems
Then, to understand the behavior of the system, we need to study how exp(At)
behaves. Assuming that A diagonalizes as A = UΛU−1, after some algebra it is
possible to show that
exp(At) = U exp(Λt)U−1, (2.14)
where
exp(Λt) =
eλ1t 0 . . . 0
0 eλ2t . . . 0...
.... . .
...
0 0 . . . eλnt
. (2.15)
Also in this case, the behavior of the system depends on the eigenvalues of the
matrix A. It follows, therefore, that the individual components of x (i.e., x1(t)
through xn(t)) are linear combinations of eλ1t, . . . , eλnt.
Considering also the possibility to have complex eigenvalues, we report the
following general principle: we have that x(t) → 0 as t → ∞ if the real parts
of all of A’s eigenvalues are negative; if some eigenvalue has positive real part,
then typically |x(t)| → ∞; and if Reλ ≤ 0 for all λ, but Reλ ≥ 0 for some λ,
then x(t) neither settles down to any specific value nor does it blow up. For the
more general case x(t) = Ax(t) + b, we have similar results. In particular, we
have that x(t)→ x = −A−1b as t→∞ if the real parts of all of A’s eigenvalues
are negative; if some eigenvalue has positive real part, then typically |x(t)| → ∞;
and if Reλ ≤ 0 for all λ, but Reλ ≥ 0 for some λ, then x(t) stays near x but
does not approach it, or does it blow up.
2.1.2 Nonlinear Systems
The general forms for dynamical systems are
x(t) = f(x(t)) for continuous-time, and
x[k + 1] = f(x[k]) for discrete-time.
We have previously considered the case when f(·) is linear. In that case, we can
answer nearly any question we might consider. We can work out exact formulas
54
2.1 Dynamical Systems
for the behavior of x(t) (or x[k]) and deduce from them the long-term behavior
of the system. There are two main behaviors: (1) the system gravitates toward
a fixed point, or (2) the system blows up. There are some marginal behaviors
as well. Now we begin our study of more general systems in which f(·) can
be virtually any function. However, in this section, we do make the following
assumption: f(·) is differentiable with continuous derivative. Nonlinear systems
are more complicated; we seek qualitative descriptions in place of exact formulas.
Indeed, typically, it is impossible to find exact formulas for x. Further, the range
of behaviors available to nonlinear systems is much greater than that for linear
systems. Because it can be terribly difficult to find exact solutions to nonlinear
systems, our goal reduces in determining the long-term behavior of the system.
This is often feasible even when finding an exact solution is not. In the following
we focus on the notion of a fixed point (sometimes called an equilibrium point)
of a dynamical system. We also discuss how to to determine if they are stable or
unstable. Often, understanding the fixed points of a dynamical system can tell
us much about the global behavior of the system.
Fixed Points
The vector x is the state of the dynamical system, and the function f(·) tells
us how the system moves. In special circumstances, however, the system does
not move. The system can be stuck (we will say fixed) in a special state; we
call these states fixed points of the dynamical system. Thus a fixed point of a
dynamical system is a state vector x with the property that if the system is ever
in the state x, it will remain in that state for all time. Finding fixed points of
dynamical systems does not require us to find exact formulas for x[k] or x(t).
All we have to do is solve a system of equations. Of course, solving systems of
equations can be difficult, but it is at least comforting to know that this is the
only issue involved. The equations we solve depend on f(·) and whether the
system is in discrete or continuous time. In discrete time we solve x = f(x), and
55
2.1 Dynamical Systems
in continuous time we solve f(x) = 0.
Not all fixed points are the same. Let us now describe three types of fixed
points a system may possess:
• A fixed point x is called stable provided the following is true: For all starting
values x0 near x, the system not only stays near x but also x(t) → x as
t→∞ [or x[k]→ x as k →∞ in discrete time];
• A fixed point x is called marginally stable or neutral provided the following:
For all starting values x0 near x, the system stays near x but does not
converge to x;
• A fixed point x is called unstable if it is neither stable nor marginally stable.
In other words, there are starting values x0 very near x so that the system
moves far away from x.
Figure 2.1 illustrates each of these possibilities. The fixed point on the left of
the figure is stable; all trajectories which begin near x remain near, and converge
to, x. The fixed point in the center of the figure is marginally stable (neutral).
Trajectories which begin near x stay nearby but never converge to x. Finally,
the fixed point on the right of the figure is unstable. There are trajectories which
start near x and move far away from x.
A question is still open. Once we have found fixed points, how can we tell
whether they are stable or unstable? The next sections will provide some tools
for making this determination. First, we consider linearization methods, where
we approximate our system near its fixed points by linear functions. Then, we
introduce also the general method of using Lyapunov functions to determine the
stability of dynamical systems.
Linearization
The purpose of this section is to provide a method to tell whether a fixed
point x of a dynamical system [either discrete or continuous time] is stable or
56
2.1 Dynamical Systems
Figure 2.1: Fixed points with three different types of stability. The fixed point
on the left is stable. The fixed point in the center is marginally stable. The fixed
point on the right is unstable.
unstable. If the function f(·) is linear, i.e., of the form f(x) = Ax + b, the
answer is relatively easy: We check the eigenvalues of A (either their absolute
values or their real parts, depending on the nature of time). Then, the method
of linearization is based on the approximation of f(·) near its fixed point x by
using a linear function. Since f(·) : Rn → Rn, the approximation we seek is of
the form
f(x) = J(x)(x− x) + f(x), (2.16)
where J(x) : Rn×Rn is an n×n matrix which gives the best approximation. In
particular, if we write
f(x) = J(x)(x− x) + f(x) + error(x− x), (2.17)
then we want|error(x− x)|
|x− x|→ 0 (2.18)
as x→ x. The matrix J =D(f(x)) is the Jacobian matrix of f(·), which is the
matrix of its partial derivatives. Now, we inspect the eigenvalues of the Jacobian
matrix J(x) and apply the results achieved for linear systems, thus obtaining the
following results.
57
2.1 Dynamical Systems
Continuous Time : Let x be a fixed point of a continuous time dynamical
system x(t) = f(x(t)), that is, f(x) = 0. The stability of a fixed point x can
be judged by the signs of the real parts of the eigenvalues of the Jacobian J(x).
If the eigenvalues of the Jacobian J(x) all have negative real part, then x is a
stable fixed point. If some eigenvalues of J(x) have positive real part, then x
is an unstable fixed point. Otherwise (all eigenvalues have nonpositive real part
and some have zero real part) we cannot judge the stability of the fixed point.
Discrete Time : Let x be a fixed point of the system x[k + 1] = f(x[k]), that
is, f(x) = x. Compute the Jacobian evaluated at x, i.e., find J(x). If the
eigenvalues of the Jacobian all have absolute value less than 1, then x is a stable
fixed point. If some eigenvalue of the Jacobian has absolute value greater than
1, then x is an unstable fixed point. Otherwise, we cannot judge the stability of
the fixed point.
Lyapunov’s Method
Linearization is a great tool for determining the stability of fixed points of
dynamical systems. Unfortunately, not always this method gives answer about
the stability of a fixed point. In the following, we introduce the Lyapunov’s
method to prove stability of dynamical systems. This method is based on the
concept of system energy. If a dynamical system models a mechanical system,
then consideration of energy is appropriate. To prove the stability of nonphysical
dynamical systems, the idea is to make up a function which behaves like the
energy, i.e., the Lyapunov function, and show that the system reduces the value
of this function (“energy”) as the time increases. Then, suppose we have a
continuous time dynamical system with state vector x, which has a fixed point
x. Let V be a function defined on the states of the space, i.e., to each state x we
assign a number V (x). Now, suppose V satisfies the following conditions:
• V is a continuously differentiable function with V (x) > 0 for all x 6= x,
and V (x) = 0.
58
2.1 Dynamical Systems
• dV/dt ≤ 0 at all states x. Further, at any state x 6= x where dV/dt = 0,
the system immediately moves to a state where dV/dt < 0.
If we can find such a function V (and this can be difficult), then it must be the case
that x is a stable fixed point of the dynamical system. We know that x is stable
because as time progresses “energy” continually decreases until it bottoms out at
the fixed point. A special class of dynamical system is particularly well suited to
the Lyapunov method. These systems arise from the gradient of a function. In
particular, suppose we are given a continuous time dynamical system of the form
x(t) = f(x(t)) with fixed point x. We seek a function h(x) for which:
• f(x) = −∇h(x),
• h(x) = 0,
• h(x) > 0 for all x 6= x.
If such a function exists, then h(x) is a Lyapunov function and we may conclude
that x is stable. Indeed, as the state vector x(t) changes, the value of h(x)
decreases. The fixed points x of the system are the points where ∇h(x) = 0.
These points include the local minima of h(·), and these are precisely the stable
fixed points of the system.
Lasalle’s Invariance Principle
To conclude this section, we introduce the Lasalle’s invariance principle, which
enables one to prove asymptotic stability of an equilibrium point and will be used
in this work. We denote the solution trajectories of the system x(t) = f(x(t)) as
x(t,x0, t0), which is the solution at time t starting from x0 at t0. To introduce
Lasalle’s invariance principle, we need the definition of invariant set, which we
report in the following.
Invariant set : The set M ⊂ Rn is said to be an (positively) invariant set if
for all y ∈ M and t0 > 0, we have x(t,y, t0) ∈ M ∀t ≥ t0. It may be proved
59
2.2 Distributed Optimization
that the limit set of every trajectory is closed and invariant. We may now state
Lasalle’s Invariance Principle.
Lasalle’s Invariance Principle : Let V : Rn → R be a locally positive definite
function such that on the compact set ωc = x ∈ Rn : V (x) ≤ c we have
V (x) ≤ 0. Define
S = x ∈ ωc : V (x) = 0. (2.19)
As t→∞, the trajectory tends to the largest invariant set inside S. In particular,
if S contains no invariant sets other than x = 0, then 0 is asymptotically stable.
2.2 Distributed Optimization
In this section, we consider iterative methods for the solution of a variety of
nonlinear problems. Nonlinear problems are typically solved by iterative meth-
ods, and the convergence analysis of these methods is one of the focal points
of this section. We consider descent methods, e.g., methods based on the it-
erative reduction of the cost function of an underlying optimization problem.
Throughout the section, we emphasize algorithms that are well suited for paral-
lel implementation such as Jacobi or Gauss-Seidel methods. If the optimization
problem is constrained, we also discuss gradient projection methods, which can
be naturally parallelized if the constraint set is given by the Cartesian product
of smaller sets. Then, we focus on convex constrained optimization problems,
which can be transformed into dual problems that in many cases are easier to
solve and more amenable to parallel implementations.
2.2.1 Unconstrained Optimization
We consider algorithms for minimizing in a distributed manner a continuously
differentiable cost function F (x) : Rn → R, where x = (x1, . . . ,xM ) ∈ Rn, with
xi ∈ Rni . Since the function F (·) is not supposed to be convex, iterative opti-
mization algorithms aimed at finding a solution of ∇F (x) = 0 do not guarantee
60
2.2 Distributed Optimization
that such a solution is a global minimizer of F . We now recall some descent
iterative methods:
1. Jacobi algorithm : This algorithm is achieved through a simultaneous
update of the vector components of x = (x1, . . . ,xM ) as:
xi[k + 1] = xi[k]− α[Bii(x[k])]−1∇xiF (x[k]), (2.20)
i = 1, . . . ,M
where α is a positive step-size, and Bii(x[k]) is the i-th matrix element of
a positive definite block-diagonal scaling matrix. The choice Bii(x[k]) = I,
∀i, leads to the common gradient descent update, whereas Bii(x[k]) =
∇2xiF (x[k]) determines a scaled gradient update that approximates a New-
ton recursion, thus enhancing the convergence speed of the algorithm.
2. Gauss-Seidel : This algorithm sequentially updates the vector components
of x = (x1, . . . ,xM ) as:
xi[k + 1] = xi[k]− α[Bii(zi[k])]−1∇xiF (zi[k]), (2.21)
i = 1, . . . ,M
where zi[k] = (x1[k + 1], . . . ,xi−1[k + 1],xi[k], . . . ,xM [k]). Also in this
case the choice of the scaling Bii(zi[k]) leads to different Gauss-Seidel im-
plementations.
These algorithms fall in the framework of descent methods and their convergence
properties have been deeply studied in the literature [189]. Classical convergence
analysis shows that each update reduces the value of the cost function by an
amount that is bounded away from zero if the magnitude of the update is bounded
away from zero. Given a cost function that is bounded from below, it follows
that the magnitude of the update converges to zero. Then, according to this
argument, it is possible to show that∇F (x(t)) converges to zero. In the following,
we report a convergence result from [189] regarding the convergence of generic
descent methods.
61
2.2 Distributed Optimization
The following assumption on the function F (·) is needed:
Assumption 1 :
a) There holds F (x) ≥ 0 (or bounded from below) for every x ∈ Rn;
b) (Lipschitz Continuity of ∇F ) The function F is continuously differentiable
and there exists a constant K such that
‖∇F (x)−∇F (y)‖2 ≤ K‖x− y‖2, ∀x,y ∈ Rn. (2.22)
Theorem 1 Suppose the Assumption 1 holds and let K1 and K2 be positive
constants. Consider the sequence generated by an algorithm of the form
x[k + 1] = x[k]− αs[k], (2.23)
where s[k] satisfies
‖s[k]‖2 ≥ K1‖∇F (x[k])‖2, ∀k, (2.24)
and
s[k]T∇F (x[k]) ≤ −K2‖s[k]‖22, ∀k. (2.25)
If 0 < α < 2K2/K, then
limk→∞
∇F (x[k]) = 0. (2.26)
Proof. The proof can be found in [189]. The proof for algorithms of the Gauss-
Siedel type follows similar arguments.
2.2.2 Constrained Optimization
Now, we consider the problem of minimizing a continuously differentiable
cost function F (x) : RL → R, in the case x = (x1, . . . ,xN ) ∈ RL lies inside a
nonempty, closed and convex set X ⊂ RL. The most used approach to solve this
problem relies on the gradient projection algorithm, which reads as follows:
x[k + 1] = ΠX [x[k]− α∇xF (x[k])] = T (x[k]), (2.27)
62
2.2 Distributed Optimization
where ΠX [x] denotes the orthogonal projection of the vector x on the set X, and
T : X → X is the nonlinear mapping correspondent to the gradient projection
algorithm. Using descent arguments from [189], also in this case it is possible
to prove convergence of such algorithm to a local minimum of the constrained
problem we aim to solve. In the following, we report the convergence result.
Theorem 2 Suppose F (·) satisfies Assumption 1. If 0 < α < 2/K and if x∗ is a
limit point of the sequence x[k] generated by the gradient projection algorithm
in (2.27), then (y − x∗)T∇F (x∗) ≥ 0 for all y ∈ X. In particular, if F (·) is
convex on the set X, then x∗ minimizes F (·) over the set X.
Proof. The proof can be found in [189]. Similar results can be achieved for
scaled gradient projection algorithms.
The gradient projection algorithm is not, in general, amenable to parallel
implementation. Indeed, even if x − α∇xF (x) can be obviously parallelized as
before, the computation of the projection is, in general, a nontrivial operation
that involves all the components of x. However, in the important special case
where the constraint set X is a Cartesian product of sets Xi, where Xi is a closed
and convex set of Rni , it can be easily seen that the projection computation
can be parallelized as ΠX [x] = (ΠX1[x1] . . . ,ΠXM
[xM ]), thus leading to possible
Jacobi or Gauss-Seidel distributed versions of this algorithm.
2.2.3 Convex Constrained Optimization Problems
In the previous subsections we have analyzed some important optimization
methods that are well suited for parallel implementations, e.g., Jacobi and Gauss-
Siedel algorithms. These methods are not always applicable, e.g., where the
constraint set is not given by the Cartesian product of simpler sets. Then, we
now recall some of the basic techniques that exploit structural problem features,
enhancing in this way parallelization through suitable problem transformations.
The basic idea of these approaches is to consider a dual problem that may be
63
2.2 Distributed Optimization
more suitable for parallel implementation than the original. We consider two
main cases. First, we consider a problem where the primal cost function is sepa-
rable and strictly convex. The strict differentiability of the cost function is very
important because it allows the parallelization of the dual problem. Then, we re-
call the augmented Lagrangian method, which aims to deal with the lack of strict
differentiability of the primal cost, and the consequent lack of differentiability of
the dual.
1) Separable Strictly Convex Problems : Suppose that the space RL is given
by the Cartesian product of sets RLi , where L = L1 + . . .+ LN , and consider
minN∑
i=1
Fi(xi) (2.28)
s.t. eTj x = sj, j = 1, . . . , r,
xi ∈ Pi, i = 1, . . . , N.
where Fi(xi) are strictly convex functions, xi are the components of x, ej are
given vectors in RL, sj are given scalars, and Pi are given polyhedral subsets
of RLi . The constraints eTj x = sj do not allow direct decomposition of this
problem into independent subproblems. However, a possible way to parallelize
this problem is to consider the dual problem that involves Lagrange multipliers
for these constraints. The dual problem has the form:
max q(p) (2.29)
s.t. p ∈ Rr.
where the dual function q(p) is given by
q(p) = minxi∈Pi
N∑
i=1
Fi(xi) +r∑
j=1
pj(eTj x− sj)
=N∑
i=1
qi(p)− pTs, (2.30)
where pTs =∑r
j=1 pjsj , and
qi(p) = minxi∈Pi
Fi(xi) +
r∑
j=1
pjeTjixi
, i = 1, . . . , N, (2.31)
64
2.2 Distributed Optimization
with eji denoting the subvector of ej that corresponds to xi. Now, we notice
how the structure of the dual problem (2.29) is amenable to parallel computation
with separate agents computing each component qi(p) of q(p). Since the primal
cost function is strictly convex, the dual cost is continuously differentiable and
we can apply descent methods such as Jacobi or Gauss-Seidel to get the solution
in a distributed fashion. This is possible because, in contrast with the primal
problem (2.28), the dual problem (2.29) is unconstrained. Then, if the minimum
of equation (2.31) is attained at a point xi(p), the partial derivative of q with
respect to pj is given by
∂q(p)
pj= eTj x(p)− sj. (2.32)
In a distributed system where each node computes the i-th component of x, the
calculation of the dual cost gradient via (2.32) requires a single or multinode
accumulation, so that the gradient ∇q(p) can be distributed to all nodes by
means of a single or multinode broadcasting.
2) Augmented Lagrangian Method : We now recall one of the basic methods
based on a dual approach for overcoming possible lack of strict monotonicity of the
primal cost function. Consider the following constrained optimization problem:
min F (x) (2.33)
s.t. eTj x = sj, j = 1, . . . , r,
x ∈ P.
where F (x) is a convex function, ej are given vectors in RL, sj are given scalars,
and P is a given a nonempty and bounded polyhedral subset of RL. We can now
consider in place of the original problem (2.33), the following problem:
min F (x) +c
2‖Ex− s‖22 (2.34)
s.t. Ex = s,
x ∈ P.
65
2.3 Stochastic Approximation
where c is a positive scalar parameter, and Ex = s is a compact notation for the
constraints eTj x = sj, j = 1, . . . , r,. The dual problem is then
max qc(p) = infx∈P
Lc(x,p) (2.35)
s.t. p ∈ Rr.
where Lc(x,p) is the Augmented Lagrangian function:
Lc(x,p) = F (x) + pT (Ex− s) +c
2‖Ex− s‖22 (2.36)
An important method involving the augmented Lagrangian is the alternating
direction method of multipliers to minimize (2.36). It consists of successive min-
imizations of the form:
x[k + 1] = arg minx∈P
Lc[k](x,p[k]), (2.37)
followed by the updates of the vector p[k] according to
p[k + 1] = p[k] + c[k](Ex[k + 1]− s), (2.38)
where p[0] is arbitrary and c[k] is a nondecreasing sequence of positive numbers.
In [189] it is proved every limit point of the sequence x[k] is a solution of the
primal problem (2.33). One problem with the alternating method of multipliers
is that, even if the cost function F (x) is separable, the Augmented Lagrangian
is typically non separable because of the quadratic term ‖Ex − s‖22. However,
some reformulations are possible to allow parallelization of this algorithm.
2.3 Stochastic Approximation
Stochastic approximation methods are a family of iterative stochastic opti-
mization algorithms that attempt to find zeroes or extrema of functions that
cannot be computed directly, but only estimated via noisy observations. Mathe-
matically, this refers to solving:
minx∈E
f(x) = E[F (x, ξ)] (2.39)
66
2.3 Stochastic Approximation
where the objective is to find the parameter x ∈ E, which minimizes f(x) for some
unknown random variable, ξ. Denoting n as the dimension of the parameter x, we
can assume that while the domain E ⊂ Rn is known, the objective function, f(x),
cannot be computed exactly, but instead only known with an approximation.
This can be intuitively explained as follows. f(x) is the original function we
want to minimize. However, due to noise, f(x) can not be evaluated exactly. This
situation is modeled by the function F (x, ξ), where ξ represents the noise and is
a random variable. Since ξ is a random variable, so is the value of F (x, ξ). The
objective is then to minimize f(x), but through measuring F (x, ξ). A reasonable
way to do this is to minimize the expectation of F (x, ξ), i.e., E[F (x, ξ)].
The first, and prototypical, algorithms of this kind are the Robbins-Monro
[180] and Kiefer-Wolfowitz [181] algorithms, which we will report in the following
together with convergence results.
2.3.1 Robbins-Monro procedure
Let R(x) : Rn → Rn be some unknown function whose values may be mea-
sured at any point x ∈ Rn. The only information available aboutR(x) is general,
concerning, for example, continuity, monotonicity, and so on. In particular, it is
known that the equation
R(x) = 0 (2.40)
has a unique solution x∗. The problem is to determine x∗ by suitable measure-
ments of R(x). More precisely, we wish to draw up a plan for an appropriate
experiment, i.e., to specify the points x[k] ∈ Rn at which R(x) has to be mea-
sured at times k = 1, 2, . . ., in such a way x[k]→ x∗ as k →∞. The methods for
solving this problem depend on the presence of measurement errors on R(x). If
there are no errors, there are several rapidly converging methods for finding x∗,
such as Newton tangent method. If the effect of the measurement is significant,
it is in principle impossible to devise such rapidly convergent methods, and one
must employ slower procedures that we now proceed to construct. It is natural
67
2.3 Stochastic Approximation
to confine our attention to independent (in time) observations of R(x) and to
assume that the measurement error has zero mean and may depend on the point
at which the measurement is made. Then, if Y (k + 1,x[k], ω) is the result of a
measurement at a point x[k] at time k + 1, in the simplest case we have
Y (k + 1,x[k], ω) = R(x[k]) + Γ(k,x[k], ω) (2.41)
where Γ(k,x[k], ω) is a family of unknown zero-mean random vectors in Rn,
defined on some probability space (Ω,F ,P). The problem we have formulated
thus reduces to the determination of x∗ from the observations (2.41). In a pio-
neering paper, published in 1951, Robbins and Monro [180] proposed a recursive
procedure aimed at finding the root of R(x), given by
x[k + 1] = x[k]− α[k]Y (k + 1,x[k], ω), x[0] = x0, (2.42)
where α[k] is a sequence of iteration dependent positive step-sizes satisfying
∞∑
k=0
α[k] =∞,∞∑
k=0
α2[k] <∞. (2.43)
In Robbins and Monro [180], it was shown that if R(x) is a monotone decreas-
ing continuous bounded function and the expectations of the random variables
Γ2(k,x[k], ω) are bounded uniformly in k and x, then E‖x[k] − x∗‖ → 0, as
k → ∞, for any initial point x0 ∈ Rn. A brief explanation of the conditions in
(2.43) is given in the following. The first condition assures the numbers α[k] are
not ”too small” and is a necessary condition for the convergence of x[k] to x∗
even when there are no random errors. Indeed, if Γ(k,x[k], ω) = 0, for all k, and
the series∑∞
k=0 α[k] is convergent, then, as follows from (2.42),
∞∑
k=0
‖x[k + 1]− x[k]‖ ≤∞∑
k=0
α[k]‖R(x[k])‖ < c, (2.44)
where the constant c is independent of the initial point x0. As a consequence,
the sum of increments x[k+1]−x[k] is finite and the value of x[k] does not reach
68
2.3 Stochastic Approximation
x∗ as k → ∞ if, for example, the initial point x0 is sufficiently far away from
x∗. On the other hand, neither should the numbers α[k] be too large, otherwise
the random errors will prevent the convergence. It turns out that the condition∑∞
k=0 α2[k] < ∞ asymptotically damps the effect of random errors, since when
it holds we have
‖α[k]Γ(k,x[k], ω)‖ → 0, as k →∞ (2.45)
with probability one, because
E
(
∞∑
k=0
α[k]Γ(k,x[k], ω)
)2
= E
∞∑
k=0
α2[k]Γ2(k,x[k], ω) <∞.
Conditions (2.43) ensure that the step-size decays to zero, but not too fast. An
example of step-size sequence that satisfies (2.43) is
α[k] =α0
(k + 1)β, α0 > 0, 0.5 < β ≤ 1. (2.46)
Note, moreover, that according to the procedure (2.42), computation of each
consecutive point x[k + 1] requires knowledge of the preceding point x[k] only,
and there is no need for the previous experimental points (This also exemplifies
the practical value of this procedure since it does not impose exaggerated memory
capacity in computer implementation). From the mathematical standpoint, this
means that the process x[k] defined by (2.42) is a Markov process.
In the following, we will provide useful convergence results on the stochastic
procedure (2.42) from [183], first considering the case where the function R(x)
has a single zero in x∗ and then extending the result to the case of multiple
zeros. Convergence conditions are usually formulated in terms of existence of
a Lyapunov function V (x) ∈ C02 , i.e. doubly continuously differentiable with
bounded partial derivatives. We are now ready to state the first convergence
theorem on the RM stochastic approximation procedure (2.42).
Theorem 3 Let x[k]k≥0 be a Markov process defined by the RM difference
equation (2.42). Assume that there exists a function V (x) ∈ C02 satisfying the
69
2.3 Stochastic Approximation
conditions
V (x) > 0, V (x0) = 0, lim‖x‖→∞
V (x) = ∞ (2.47)
supǫ<|x−x0|<1/ǫ
< R(x),∇xV (x) > < 0 for ǫ > 0 (2.48)
where < ·, · > denotes the inner product operator. In addition, if there exist two
positive constants k1 and k2, such that
‖R(x)‖2 + E‖Γ(k,x, ω)‖2 ≤ k1(1 + V (x))− k2 < R(x),∇xV (x) >, (2.49)
then, starting from an arbitrary initial condition x0, the process x[k]k≥0 con-
verges almost surely (a.s.) to x∗ as k →∞, provided the conditions (2.43) hold.
Proof. The proof can be found in [183] (Theorem 4.4.4).
Up to now, we have considered the case in which the equation R(x) = 0 has a
unique root. It is not difficult to imagine a situation in which the observer has no
such information at this disposal. Then, in the following, we report a convergence
result that generalizes the results of the previous theorem to the case where the
function R(x) has multiple zeros.
Theorem 4 Let x[k]k≥0 be a Markov process defined by the RM difference
equation (2.42). Assume that there exists a function V (x) ∈ C02 satisfying the
conditions
V (x) > 0, lim‖x‖→∞
V (x) = ∞ (2.50)
supx∈Uǫ,1/ǫ(S)
< R(x),∇xV (x) > < 0 for ǫ > 0 (2.51)
where < ·, · > denotes the inner product operator, S = x : R(x) = 0 is the
solution set and Uǫ,1/ǫ(S) = x ∈ Rn : ǫ < ‖x − xs‖ < 1/ǫ,xs ∈ S, ǫ > 0. In
addition, if there exist two positive constants k, such that
‖R(x)‖2 + E‖Γ(k,x, ω)‖2 ≤ k(1 + V (x)), (2.52)
70
2.3 Stochastic Approximation
then, starting from an arbitrary initial condition x0, the process x[k]k≥0 con-
verges almost surely (a.s.), as k → ∞, either to a point of the solution set
S = x : R(x) = 0, or to the boundary of its connected components, provided
that the conditions (2.43) hold.
Proof. The proof can be found in [183] (Theorem 5.2.3).
2.3.2 Kiefer-Wolfowitz procedure
In 1952 Kiefer andWolfowitz [181], taking the Robbins-Monro method as their
point of departure, considered the following problem, involving the determination
of the maximum of an unknown function. Let f(x) : Rn → R, x ∈ Rn, be a
continuously differentiable function with a unique maximum at the point x = x∗.
Suppose that the observer may conduct independent measurements of f(x) with
a certain error, so that the measurement result Y (k + 1,x[k], ω) at a point x[k]
at the time k + 1 has the form
Y (k + 1,x[k], ω) = f(x[k]) + φ(k,x[k], ω). (2.53)
It is required to find the maximum point x∗ of f(x) or, equivalently, to solve the
system of equations ∇f(x) = 0. If the values of f(x) had been measured with
no errors, the maximum of f(x) could have been determined via the gradient
method, described by the recurrence relation
x[k + 1] = x[k] + α∇f(x[k]), (2.54)
where α is a positive constant. Nevertheless, if the errors in the measurements
of f(x) at different points are independent, the error involved in calculating
∇f(x[k]) according to these measurements will become infinitely large. The idea
of the Kiefer-Wolfowitz method is to evaluate approximate values of the gradient
in (2.54) as the quotient of the increment of the function and the increment ∆x,
setting ∆x = 2c[k]→ 0 as k →∞, at the same time “decelerating” the motion of
x[k] toward x∗, making the parameter α time dependent. The function α = α[k]
71
2.3 Stochastic Approximation
may be chosen in such a way that, first, the sequence x[k] does not stop too soon,
and, second, the effect of random noise is damped. It is readily seen that even
without noise the condition∞∑
k=0
α[k] =∞ (2.55)
is necessary for the solution of (2.54) to converge to x∗. To meet the second
requirement, it is frequently enough to demand that
∞∑
k=0
(
α[k]
c[k]
)2
<∞. (2.56)
According to Kiefer and Wolfowitz, at each instant k we conduct measurements
at 2l points x±i = x ± c[k]ei, where ei ∈ R
n with coordinates δij , i, j = 1, . . . , l,
and c[k] is some positive function. The approximation of ∇f(x) will then be
given by
∇f(x) ≈ ∇cf(x) =f+(c[k],x)− f−(c[k],x)
2c[k](2.57)
where f±(c[k],x) is the vector with coordinates f(x±i ), i = 1, . . . , l. Measure-
ments of the ith coordinate of the vector [f+(c[k],x)−f−(c[k],x)]/2 will involve
an error of
Γi[k + 1,x[k], ω] = [φ(k + 1,x+i , ω)− φ(k + 1,x−
i , ω)]/2. (2.58)
Thus, the Kiefer-Wolfowitz procedure for locating the maximum of f(x) is de-
scribed by the following difference equation:
x[k + 1] = x[k] +α[k]
c[k][∇cf(x) + Γ[k + 1,x[k], ω]] (2.59)
where α[k] and c[k] are certain sequence of positive numbers, and Γ[k+1,x[k], ω]
is the vector with coordinates (2.58), such that EΓ[k,x, ω] = 0. Convergence
conditions for the procedure (2.59) will be reported below.
Theorem 5 Let x[k]k≥0 be a Markov process defined by the KW difference
equation (2.59). Assume that there exists a function V (x) ∈ C02 satisfying the
72
2.3 Stochastic Approximation
conditions
V (x) > 0, V (x0) = 0, lim‖x‖→∞
V (x) =∞, (2.60)
< ∇xf(x),∇xV (x) > < 0 for x 6= x∗, (2.61)
where < ·, · > denotes the inner product operator. In addition, if the function
f(x) has continuous partial derivatives satisfying a global Lypschitz condition,
and if there exist two positive constants k1 and k2, such that
‖∇xV (x)‖+ ‖∇xf(x)‖2 + E‖Γ(k,x, ω)‖2 ≤ k1(1 + V (x)), (2.62)
then, starting from an arbitrary initial condition x0, the process x[k]k≥0 con-
verges almost surely (a.s.) to x∗ as k →∞, provided these conditions hold:
∞∑
k=0
α[k]c[k] <∞,∞∑
k=0
α[k] =∞,∞∑
k=0
(
α[k]
c[k]
)2
<∞, c[k] < k2. (2.63)
Proof. The proof can be found in [183].
Up to now, we have considered the case in which the function f(x) = 0 has a
unique maximum. It is not difficult to imagine a situation in which the observer
has no such information at this disposal. Then, in the following, we report a
convergence result that generalizes the results of the previous theorem to the
case where the function f(x) has multiple maxima.
Theorem 6 Let x[k]k≥0 be a Markov process defined by the KW difference
equation (2.42). Assume that there exists a function V (x) ∈ C02 satisfying the
conditions
V (x) > 0, lim‖x‖→∞
V (x) = ∞ (2.64)
supx∈Uǫ,1/ǫ(S)
< ∇xf(x),∇xV (x) > < 0 for ǫ > 0 (2.65)
a[k] [< ∇cf(x)−∇xf(x),∇xV (x) >] < g[k](1 + V (x)), (2.66)
73
2.4 Graph Theory
∞∑
k=0
g[k] < ∞ (2.67)
∞∑
k=0
α[k]maxx‖∇cf(x)−∇xf(x)‖ < ∞ (2.68)
where < ·, · > denotes the inner product operator, S = x : ∇xf(x) = 0 is the
solution set and Uǫ,1/ǫ(S2) = x ∈ Rn : ǫ < ‖x − xs‖ < 1/ǫ,xs ∈ S2, ǫ > 0. In
addition, if there exist a positive constants k, such that
‖∇cf(x)‖2 + E‖Γ(k,x, ω)‖2 ≤ k(1 + V (x)), (2.69)
then, starting from an arbitrary initial condition x0, the process x[k]k≥0 con-
verges almost surely (a.s.), as k → ∞, either to a point of the solution set
S2 = x : ∇xf(x) = 0, or to the boundary of its connected components, provided
that the following conditions hold:
∞∑
k=0
α[k] =∞,∞∑
k=0
(
α[k]
c[k]
)2
<∞, 0 < c[k] < K. (2.70)
Proof. The proof can be found in [183].
2.4 Graph Theory
The interaction among the network nodes is properly described by a graph.
For the reader’s convenience, in this section, we briefly review the notation and
basic results of graph theory that will be used throughout this work. For the
reader interested in a more in-depth study of this field, we recommend, for ex-
ample, [82]- [86].
2.4.1 Directed Graphs: The Basic Mathematical Tool to De-
scribe Interactions
To take explicitly into account the possibility of unidirectional links among
the network nodes, we represent the information topology among the nodes by
74
2.4 Graph Theory
their (weighted) directed graph. A weighted directed graph (or digraph, for
short) G = V, E is defined as a set of nodes (or vertices) V and a set of edges
E (i.e., ordered pairs of nodes), with the convention that eij , (vi, vj) ∈ E (i.e.,
vi and vj are the head and the tail of the edge eij , respectively) means that the
information flows from vj to vi. A digraph is weighted if a positive weight aij
is associated to each edge. In our case, there are no loops, so that aii = 0. If
(vi, vj) ∈ E ⇔ (vj , vi) ∈ E , then the graph is said to be (weighted) undirected.
For any node vi ∈ V, we define the information neighbor of vi as
Ni = j = 1, . . . , N : (eij) = (vi, vj) ∈ E (2.71)
The set Ni represents the set of nodes sending data to node vi. The in-degree
and out-degree of node vi ∈ V are, respectively, defined as:
degin(vi) =
N∑
j=1
aij and degout(vi) =
N∑
j=1
aji (2.72)
Observe that for undirected graphs, degin(vi) = degout(vi). We may have the
following class of digraphs.
Balanced digraph: The node vi of a digraph G = V, E is said to be balanced
if and only if its in-degree and out-degree coincide, i.e., degin(vi) = degout(vi). A
digraph G = V, E is balanced if and only if all its nodes are balanced, i.e.,
N∑
j=1
aij =
N∑
j=1
aji, ∀i = 1, . . . , N. (2.73)
Path/Cicle: A strong path in a digraph G is a sequence of distinct nodes
v1, v2, ..., vq ∈ V such that (vi−1, vi) ∈ E , for i = 2, . . . , q. If v1 = vq, the path is
said to be closed. A weak path is a sequence of distinct nodes v1, v2, . . . , vq ∈ V
such that either (vi−1, vi) ∈ E or (vi, vi−1) ∈ E , for i = 2, . . . , q. A strong cycle
(or circuit) is a closed strong path.
Directed Tree/Forest: A digraph with N nodes is a (rooted) directed tree if
it has N − 1 edges and there exists a distinguished node, called the root node,
75
2.4 Graph Theory
which can reach all the other nodes through a (unique) strong path. Hence, a
directed tree cannot have cycles and every node, except the root, has one and
only one incoming edge. A digraph is a (directed) forest if it consists of one or
more directed trees. A subgraph Gs = Vs, Es of a digraph G, with Vs ⊆ V and
Es ⊆ E , is a directed spanning tree (or a spanning forest) if it is a directed tree
(or a directed forest) and it has the same node set as G; i.e., Vs = V. We say
that a digraph G contains a spanning tree (or a spanning forest) if there exists a
subgraph of G that is a directed spanning tree (or a spanning forest).
Connectivity: In a digraph there are many degrees of connectedness:
1) a digraph is strongly connected (SC) if any ordered pair of distinct nodes can
be joined by a strong path;
2) a digraph is quasi strongly connected (QSC) if, for every ordered pair of nodes
vi and vj, there exists a node r that can reach either vi or vj via a strong path;
3) a digraph is weakly connected (WC) if any ordered pair of distinct nodes can
be joined by a weak path;
4) a digraph is disconnected if it is not weakly connected.
According to the above definitions, it is straightforward to see that strong connec-
tivity implies quasi strong connectivity and that quasi strong connectivity implies
weak connectivity, but the converse, in general, does not hold. For undirected
graphs, instead, the above notions of connectivity are equivalent: An undirected
graph is connected if any two distinct nodes can be joined by a path. Moreover,
it is easy to check that the quasi strong connectivity of a digraph is equivalent to
the existence of a directed spanning tree in the graph.
Condensation Digraph: When a digraph G is WC, it may still contain strongly
connected subgraphs. A maximal subgraph of G, which is also SC, is called a
strongly connected component (SCC) of G. Since a node is SC, it follows that
76
2.4 Graph Theory
every node lies in an SCC. Using this concept, any digraph G can be partitioned
into SCCs, let us say G1 = V1, E1, . . . , GK = VK , EK, where Vk ⊆ V and
Ek ⊆ E , k = 1, . . . ,K, denote the set of nodes and edges lying in the k-th SCC,
respectively.
The connectivity properties of a digraph may be better understood by referring
to its corresponding condensation digraph. We may reduce the original digraph
G to the condensation digraph G∗ = V∗, E∗ by associating the node set Vk of
each SCC Gk of G to a single distinct node v∗k ∈ V∗k of G∗ and introducing an
edge in G∗ from v∗i to v∗j if and only if there exists some edges from the SCC Gi
and the SCC Gj of the original graph. An SCC that is reduced to the root of a
directed spanning tree of the condensation digraph is called root SCC (RSCC).
Observe that, by definition, the condensation digraph has no cycles. Building on
this property, we have the following.
Lemma 1 Let G∗ = V∗, E∗ be the condensation digraph of G , composed by K
nodes. Then, the nodes of G∗ can always be ordered as v∗1, . . . , v∗K ∈ V
∗, so that
the existing edges in G∗ are in the form
(v∗i , v∗j ) ∈ E
∗, with 1 ≤ j < i ≤ K, (2.74)
where v∗1 has zero in-degree.
The ordering v∗1 , . . . , v∗K satisfying (2.74) can be obtained by the following iter-
ative procedure. Starting from v∗1 , remove v∗1 and all its out-coming edges from
G∗. Since the reduced digraph with K − 1 nodes has no (strong) cycles by con-
struction, there must exist a node with zero in-degree in it. Let us denote such
a node by v∗2 . Then, no edges in the form (v∗2 , v∗j ), with j > 2, can exist in the
reduced digraph (and thus in G∗). This justifies (2.74) for i = 2 and j = 1, 2.
The rest of (2.74), for j > 2, is obtained by repeating the same procedure on
the remaining nodes. The connectivity properties of a digraph are related to the
structure of its condensation digraph, as given in the following Lemma (we omit
the proof because of space limitations).
77
2.4 Graph Theory
Figure 2.2: Examples of graphs: (a) Strongly connected graph. (b) Quasi strongly
connected graph with one root strongly connected component and two strongly
connected components. (c) WC graph containing a two-tree forest.
Lemma 2 Let G∗ be the condensation digraph of G . Then: i) G is SC if and
only if G∗ is composed by a single node; ii) G is QSC if and only if G∗ contains
a spanning directed tree; iii) if G is WC, then G∗ contains either a spanning
directed tree or a (weakly) connected directed forest.
The concept of condensation digraph is useful to understand the network syn-
chronization behaviors [135] and leadership problems in coordinated multi-agent
systems [114]. Some examples of graph topologies are shown in Figure 2.2, where
we report three topologies, namely: (a) an SC digraph, (b) a QSC digraph with
three SCCs, and (c) a WC (not QSC) digraph with a two-tree forest. For each
digraph, we also sketch its decomposition into SCCs and RSCCs.
2.4.2 Algebraic Graph Theory
We recall now some basic relationships between the connectivity properties of
the digraph and the spectral properties of the matrices associated to the digraph,
since they play a fundamental role in the stability analysis of the system proposed
in this work. In the following, we denote by 1M and 0M the M -length column
78
2.4 Graph Theory
vector of all ones and zeros, respectively. Given a digraph G = V, E, we
introduce the following matrices associated to G :
• the M ×M adjacency matrix A is composed of entries [A]ij = aij, i, j =
1, . . . ,M , equal to the weight associated with the edge eij , if eij ∈ E , or
equal to zero, otherwise;
• The degree matrix D is a diagonal matrix with diagonal entries [D]ii =
degin(vi);
• The (weighted) Laplacian L is defined as
∑Mk 6=i=1 aik, if j = i;
−aij , if j 6= i.(2.75)
Using the adjacency matrix A and the degree matrix D, the Laplacian L
can be rewritten in compact form as L =D −A.
By definition, the Laplacian matrix L in (2.75) has the following properties:
i) it is a diagonally dominant matrix; ii) it has zero row sums; and iii) it has
nonnegative diagonal elements. From i)-iii), invoking Gersgorin’s disk Theorem
[178], we have that zero is an eigenvalue of L corresponding to a right eigenvector
in the NullL ⊇ span1M, i.e.,
L1M = 0M (2.76)
while all the other eigenvalues have positive real part. This also means that
rank(L) = M − 1. Moreover, from (23) and (26), it turns out that balanced
digraphs can be equivalently characterized in terms of the Laplacian matrix L:
A digraph is balanced if and only if 1M is also a left eigenvector of L associated
with the zero eigenvalue, i.e.,
1TML = 0TM (2.77)
or equivalently 12(L+LT ) is positive semidefinite.
The relationship between the connectivity properties of a digraph and the
spectral properties of its Laplacian matrix are given by the following.
79
2.4 Graph Theory
Lemma 3 Let G = V, E be a digraph with Laplacian matrix L. The multiplic-
ity of the zero eigenvalue of L is equal to the minimum number of directed trees
contained in a spanning directed forest of G.
Corollary 1 The zero eigenvalue of L is simple if and only if G contains a
spanning directed tree (or, equivalently, it is QSC).
Observe that, since the strong connectivity of the digraph implies QSC, the re-
sults provided for SC digraphs, can be obtained as special case of Corollary 2.
Specifically, we have the following.
Corollary 2 Let G = V, E be a digraph with Laplacian matrix L. If G is SC,
then L has a simple zero eigenvalue and a positive left-eigenvector γ associated
to the zero eigenvalue.
According to Corollary 2, because of (2.76), the Laplacian of a QSC digraph has
an isolated eigenvalue equal to zero, corresponding to a right eigenvector in the
span1M. Observe that, for undirected graphs, Corollary 3 can be stated as
follows: rank(L) = M − 1 if and only if G is connected. For directed graphs,
instead, the only if part does not hold. We describe now the structure of the
left-eigenvector γ of the Laplacian matrix L associated to the zero eigenvalue, as
a function of the network topology. We have the following.
Lemma 4 [135] Let G = V, E be a digraph withM nodes and Laplacian matrix
L. Assume that G is QSC with K SCC’s G1 = V1, E1, . . . , GK = VK , EK,
with Vi ⊆ V, Ei ⊆ E, |Vi| = ri and∑
i ri = M , numbered w.l.o.g. so that G1
coincides with the RSCC of G. Then, the left-eigenvector γ = [γ1, . . . , γM ]T of L
associated to the zero eigenvalue has the following structure
γi =
> 0, iff vi ∈ V;
= 0, otherwise.(2.78)
If G1 is also balanced, then γr1 = [γ1, . . . , γr1 ]T ∈ span1r1, where r1 , |V1|.
80
2.4 Graph Theory
Undirected graphs
In this section, we focus on undirected graphs and the spectral properties of
the associated matrices. Let model the interaction among the network nodes as
an undirected graph G = (V, E), where V = 1, 2, ...,M denotes the set of nodes
and E ⊆ V × V is the edge set. The structure of the graph is described by the
symmetric M ×M adjacency matrix A := aij, whose entries aij are either
positive or zero, if there is or not a link between nodes i and j, respectively.
Properties : Let G = (V, E) be an undirected graph of order M with a symmetric
non-negative adjacency matrix A = AT . Then, these statements hold true:
1. L is a positive semidefinite matrix that satisfies the following sum-of-squares
(SOS) property
xTLx =1
2
M∑
i=1
M∑
j=1
aij(xj − xi)2, x ∈ R
n; (2.79)
2. The graph has c ≥ 1 connected components iff rank(L) = M − c. In
particular, G is connected iff rank(L) =M − 1;
3. Let G be a connected graph, then for any x such that x ⊥ 1M , we have 1
0 < λ2(L) ≤xTLx
‖x‖2≤ λM (L). (2.80)
The quantity λ2(L) is known as the algebraic connectivity of the graph and is a
measure of performance/speed of consensus algorithms [132]. If the symmetric
graph modeling the interaction among the nodes is connected, the multiplicity
of the null eigenvalue of the Laplacian is one. In particular, we consider M -
dimensional graph Laplacians defined by
L = L⊗ In (2.81)
1We recall that, for a connected graph, the nullspace of L has dimension 1 and it is spanned
by the vector 1. Hence, a vector x ⊥ 1M indicates a vector lying in a subspace orthogonal to
the nullspace of L.
81
2.4 Graph Theory
where ⊗ denotes Kronecker product. This multidimensional Laplacian satisfies
the following property:
xT Lx =1
2
M∑
i=1
M∑
j=1
aij‖xj − xi‖2, x ∈ R
nM . (2.82)
where x := (xT1 , . . . ,x
TM )T and xi ∈ R
n. Furthermore, the spectrum of the
multidimensional Laplacian matrix is such that
λ(L) = λ(L)⊗ 1n (2.83)
where λ(L) = [λ1(L), . . . , λM (L)] ∈ RM is the vector containing the eigenvalues
of L.
82
Chapter 3
Distributed Resource
Allocation Based on Swarming
Mechanisms
The goal of this chapter is to propose a bio-inspired radio access mechanism for
cognitive networks mimicking the behavior of a flock of birds swarming in search
for food in a cohesive fashion without colliding with each other. The equivalence
between swarming and radio resource allocation is established by modeling the
interference distribution in the resource domain, e.g. frequency and time, as the
spatial distribution of food, while the position of the single bird represents the
radio resource chosen by each radio node. The swarming mechanism is enforced
by letting every node allocate its resources (power/bits) in the time-frequency re-
gions where the interference is minimum (the food density is maximum), avoiding
collisions with other nodes (birds), yet limiting the spread in the time-frequency
domain (i.e., maintaining the swarm cohesion). The solution is given as the dis-
tributed minimization of a functional, borrowed from social foraging swarming
models, containing the average interference plus repulsion and attraction terms
that help to avoid conflicts and maintain cohesiveness, respectively.
83
3.1 Introduction on Cognitive Radio and Dynamic Radio Access
3.1 Introduction on Cognitive Radio and Dynamic
Radio Access
The radio frequency spectrum is a scarce natural resource and its efficient use
is of the utmost importance. The spectrum bands are usually licensed to certain
services, such as mobile, fixed, broadcast, and satellite, to avoid harmful interfer-
ence between different networks and users. Most spectrum bands are allocated
to certain services but worldwide spectrum occupancy measurements show that
only portions of the spectrum band are fully used. Moreover, there are large
temporal and spatial variations in the spectrum occupancy. In the development
of future wireless systems the spectrum utilization functionalities will play a key
role due to the scarcity of unallocated spectrum. Moreover, the trend in wireless
communication systems is going from fully centralized systems into the direction
of self-organizing systems where individual nodes can instantaneously establish
ad-hoc networks whose structure is changing over time. Cognitive radios [2],
with the capabilities to sense the operating environment, learn and adapt in real
time according to environment creating a form of mesh network, are seen as a
promising technology. Cognitive radio is an intelligent wireless communication
system that is aware of its surrounding environment, learns from the environ-
ment and adapts its internal states to statistical variations in the incoming RF
stimuli by making corresponding changes in certain operating parameters in real
time [3]. The primary objectives of the cognitive radio are to provide highly
reliable communications whenever and wherever needed and to utilize the radio
spectrum efficiently. The key issues in the cognitive radio are awareness, intel-
ligence, learning, adaptivity, reliability, and efficiency. The term cognitive radio
was first suggested by Mitola [2]. He defines the cognitive radio as a radio driven
by a large store of a priori knowledge, searching out by reasoning ways to deliver
the service the users want. The cognitive radio is reconfigurable and built on the
software-defined radio (SDR). The aim of the cognitive radio is to use the natural
resources efficiently including frequency, time, and transmitted energy. Spectral
84
3.1 Introduction on Cognitive Radio and Dynamic Radio Access
efficiency is playing an increasingly important role as future wireless communica-
tion systems will accommodate more and more users and high performance (e.g.
broadband) services. Cognitive radio technologies can be used in lower priority
secondary systems that improve spectral efficiency by sensing the environment
and then filling the discovered gaps of unused licensed spectrum with their own
transmissions [2,3]. The opportunistic access of the radio spectrum by unlicensed
users is a problem that is attracting a large interest in the research community
as well as in the industry sector, as a way to improve the efficiency thanks to
a dynamic radio resource allocation as opposed to conventional rigid spectrum
access [2,3]. Transmission techniques for cognitive radio systems include overlay,
underlay and interweave [4]. Underlay or interference avoidance model allows con-
current transmission of primary and secondary users in ultra wideband (UWB)
fashion where the primary users are protected by enforcing spectral masks on the
secondary signals so that the generated interference is below the noise floor for
the primary user. However, underlay allows only short-range communication due
to the power constraints. Overlay or known interference model also allows con-
current transmission of primary and secondary users. The secondary users use
part of their transmission power for relaying the data of primary users and part
of the power for their own secondary transmission. In the interweave model the
cognitive radio monitors the radio spectrum periodically and opportunistically
communicates over the spectrum holes. The three major tasks of the cognitive
radio include [3]:
1. radio-scene analysis,
2. channel identification, and
3. dynamic spectrum management and transmit-power control.
The radio-scene analysis includes the detection of spectrum holes by for example
sensing the radio frequency spectrum. The channel identification includes estima-
tion of the channel state information which is needed at the receiver for coherent
85
3.1 Introduction on Cognitive Radio and Dynamic Radio Access
detection. The transmitter power control and dynamic spectrum management
select the transmission power levels and frequency holes for transmission based
on the results of radio scene analysis and channel identification. The first two
tasks are usually carried out in the receiver (RX) while the third task is carried
out in the transmitter (TX), which requires some form of feedback from the RX.
The cognitive radio approach can be extended to cognitive networks. A cogni-
tive radio network is an intelligent multiuser wireless communication system that
perceives the radio-scene, adapts to variations in the environment, facilitates
communication between users by cooperation, and controls the communication
through proper allocation of resources [3]. The cognitive network encompasses
a cognitive process that can perceive current network conditions, and then plan,
decide, and act on those conditions [5]. The network can learn from adaptations
and use them to make future decisions taking into account end-to-end goals.
Cognitive networks require a software adaptable network to implement the ac-
tual network functionality and allow the cognitive process to adapt the network.
The basic idea in cognitive networks is to have a hierarchical structure where
unlicensed users, also known as secondary users (SU’s), are allowed to use tem-
porally unoccupied communication resources, like frequency bands, time slots or
user codes, under the constraint of not interfering (or producing a tolerable in-
terference) towards licensed, or primary, users. A key step in cognitive networks
is the ability of the opportunistic users to sense the resource domains, either
time slots or frequency subchannels, and then use the unoccupied resources un-
til possible and release them as soon as primary users access them. A broad
survey on cognitive radios is [92], whereas a more specific survey on dynamic
spectrum access methods is given in [93]. Besides cognitive radios, another in-
teresting area of application of dynamic radio access is femtocell networks, where
a potential massive deployment of femto-access points can determine an intoler-
able interference towards macrocell station users. In this case, the high number
of femto-access points demands for decentralized radio access strategies, aided
with proper channel sensing. Dynamic access based on sensing has been studied
86
3.1 Introduction on Cognitive Radio and Dynamic Radio Access
in a series of works, see e.g. [95–101, 104–111]. In all these works it has been
emphasized how the sensing and access strategies should be designed jointly to
optimize the system performance. In particular, the authors of [100] show how
to maximize the throughput of a secondary user considering the identification of
spectral opportunities, the sensing strategy and the access strategy jointly. The
primary user activity is modeled as a discrete-time Markov chain and the effect
of channel estimation errors is taken into account. Discrete-time Markov models
assume some kind of synchronization between primary and secondary users. In
situations where this synchronization cannot be taken for granted, as in WLAN
for instance, a continuous-time Markov process is assumed to model the primary
user activity, as in [99]. The proposed methods do not prevent collisions, but try
to maximize throughput under a collision constraint. In [102] it is shown that,
when the collision constraint is tight, the strategy can be implemented with a
simple memoryless policy with periodic channel sensing. The previous works con-
centrate on the decisions about accessing the available channels or not. In [104]
it was shown how to determine the decision thresholds in order to maximize
the opportunistic throughput, in a multicarrier setup, for a given set of rates
over the available subcarriers. In [106] it was then shown, in the same multi-
carrier framework, that a substantial performance improvement can be obtained
by choosing decision thresholds and power allocation jointly, rather than sepa-
rately. Most works concentrate on the access from a single secondary user into
a system partially occupied by primary users, whereas relatively fewer papers
address the uncoordinated access from multiple secondary users. The multiuser
case was specifically addressed in [103]. Game-theoretic approaches have also
been proposed, as a way to derive decentralized access strategies. In particular,
in [107] the multiuser access problem was formulated as a game whose players are
the secondary nodes, who aim at maximizing their rate under the constraint of
inducing no interference at all, or only limited interference, to the primary users.
The joint optimization of detection thresholds and power allocation in a multi-
ple secondary user scenario was addressed in [111]. In [107,111], every cognitive
87
3.1 Introduction on Cognitive Radio and Dynamic Radio Access
user was assumed to have perfect knowledge of the aggregated interference from
primary users. In practice, this is a rather idealistic assumption. A Bayesian
game-theoretic approach was considered in [108–110].
In this work, we follow a rather alternative path and, inspired by biological
models, we formulate the search for radio resources, i.e. time and frequency slots,
as the search for food by a flock of birds swarming in a cooperative manner, but
without any centralized control. The equivalence between the two problems is
the following. The interference distribution in the time-frequency plane takes
the role of the food spatial distribution: The birds (radio nodes) fly (allocate
their resources) over the regions (time-frequency domain) where there is more
food (less interference). During the flight, the birds move (choose their time-
frequency slots) in a coordinated way, even in the absence of any central control, in
order to avoid collisions (conflicts over common radio resources), yet maintaining
the swarm cohesion (i.e., avoiding unnecessary spread in the occupancy of the
time-frequency plane). Inspired by the swarm models proposed by Gazi and
Passino in [117, 118], properly modified according to our goals, we show how
the decentralized resource allocation can be formulated as the minimization of a
proper functional including the interference distribution over the radio resource
plane, plus the combination of a repulsion and an attraction term, introduced
to avoid conflicts over common resources while preventing, at the same time, an
excessive spread in the resource domain.
The basic contributions of this chapter are the following:
1) we propose the application of swarming mechanisms to radio resource alloca-
tion in cognitive radios;
2) we provide upper and lower bounds on the spread of the swarm, as a function
of the swarm connectivity;
3) we propose fast versions of the swarming algorithm, useful for our application,
and we apply such procedures to the dynamic resource allocation in the frequency
domain;
88
3.1 Introduction on Cognitive Radio and Dynamic Radio Access
4) we apply the proposed procedure to the case where the primary users in a
cognitive radio are modeled as statistically independent homogeneous continuous-
time Markov processes;
5) we extend the swarming algorithms to the case of inter-nodes communications
affected by random disturbances;
6) we derive the convergence properties of the proposed algorithms in the presence
of random disturbances such as link failures, quantization and estimation errors.
The chapter is organized as follows. Section 3.2 describes the swarm model and
formulates the search of available time/frequency slots as the distributed mini-
mization of a global potential function. In Section 3.3 and 3.4, we introduce a
continuous-time swarm model, analyzing the cohesiveness of the swarm in case
of local interactions among the nodes and providing closed form expressions for
the upper and lower bounds of the swarm size. This analysis is useful to capture
the effect of the network topology and of the swarm parameters on the spread
over the resource domain. In Section 3.5, we study in more detail the swarming
mechanism in a one-dimensional domain, e.g. the frequency domain. We provide
first a local stability analysis useful to show that the introduction of the attrac-
tion and unbounded repulsion terms in the functional to be minimized does not
affect the stability of the system. Then, we propose fast swarming methods based
on a proper selection of the descent direction of a scaled gradient optimization.
The first method is an approximation of a Newton based optimization and im-
proves the convergence speed of the algorithm; the second method adapts the
swarming speed with respect to the interference power perceived by the swarm.
Numerical examples show the main algorithm’s features. We provide also a nu-
merical validation of our theoretical findings about the spread of the swarm, as
a function of the main system parameters. In Section 3.6, we describe the appli-
cation of the proposed model to the distributed resource allocation problem on
a time-frequency plane. We consider both a static interference scenario, where
the interference activity is assumed to be known and constant along the dura-
89
3.2 Problem Statement
tion of the swarming algorithm, and a dynamic interference scenario. In this
latter case, the interference activity over each frequency subchannel is modeled
as a continuous-time Markov chain. Section 3.7 describes the discrete-time im-
plementation of the proposed swarming procedure, giving convergence results for
unprojected and projected swarming methods. Finally, in Section 3.8 we ex-
tend the swarm-based resource assignment mechanism to the more realistic case
where the packets exchanged among the cognitive nodes, while running the dis-
tributed assignment mechanism, are randomly dropped and the transmitted data
are encoded with a finite number of bits. Then, using stochastic approximation
arguments, we derive the convergence properties of the proposed algorithms in
the presence of random disturbances such as link failures, quantization noise and
estimation errors. Several simulation examples are also given in order to corrob-
orate the theoretical results and show the effect of the radio channel impairments
on the performance of the proposed algorithms.
3.2 Problem Statement
The problem we wish to solve is the assignment of time and/or frequency
slots to cognitive (or femto) users in order to minimize interference towards pri-
mary (or macrocell) users and avoid conflicts among the cognitive users, while
keeping the spread in frequency and time as reduced as possible. A centralized
controller knowing the spatial distribution of the primary users’ activity in the
time-frequency domain, as perceived by each cognitive node, could solve this
non-trivial assignment problem. However, besides the computational complexity
aspects, a centralized approach would require considerable signaling between the
secondary nodes and the controller. Furthermore, since femto-access points are
owner-operated devices, they are not necessarily under the control of a central
authority. It is then of interest to examine decentralized resource assignment
techniques. We formulate the problem as follows. Let us consider now a set of
M secondary users whose goal is to allocate resources (bits/power) dynamically
90
3.2 Problem Statement
in a domain typically occupied by primary users, with the aim of minimizing the
interference towards the primary users. A typical setting is the case where the
resource space is a time-frequency frame, as in the 4G mobile communication
standard LTE [187]. In such a case, the goal of every secondary user is to find
out a time slot and/or a frequency subchannel, temporally unoccupied by other
users. The problem arises when the number of secondary users is very high and
there is no central authority assigning the resources on demand. In such a case,
it is necessary to devise a decentralized mechanism to assign resources under the
constraint of keeping the interference towards primary and secondary users as
low as possible. To have a notation as general as possible, we denote by n the
dimension of the resource domain and the single resource selected by node i is
described by a vector xi ∈ Rn, whose entries denote, for example, a frequency
subchannel and a time slot (n = 2 in this case). In the presence of many SUs,
the problem is how to access time and/or frequency slots that are vacant, while
at the same time avoiding conflicts between SUs, without requiring a centralized
coordination node. To avoid conflicts, we propose an iterative algorithm where,
at each iteration, every node broadcasts the vector that is planning to occupy
to its nearest neighbors. The interaction among the SU nodes is modeled as an
undirected graph G = (V,E). We assume that there is a link (edge) between two
nodes if the distance between them is less than a prescribed value (the coverage
radius), dictated by the node’s transmit power and the radio channel characteris-
tics. The graph of the network topology can be described by the adjacency matrix
A := akl, composed of nonnegative entries aij ≥ 0, the degree diagonal matrix
D, whose diagonal entries are dii :=∑M
l=1 aij , and the Laplacian L, defined as
L =D −A. The set of neighbors of a node i is Nk, defined as
Ni = l ∈ V : aij > 0. (3.1)
Node k communicates (and interferes) with node j if i is a neighbor of i (or
aij > 0). We denote by Ii(xi) ∈ C1 : Rn×R→ R the interference power over the
slot having coordinate vector xi (e.g., a frequency subchannel or a time-frequency
91
3.2 Problem Statement
slot) perceived by node i. The goal of node i is to select the time-frequency slot,
having coordinates xi, where Ii(xi, t) is at a minimum. At the same time, each
node wants to prevent conflicts with the other SUs, while avoiding an excessive
spread over the resource domain.
The resource allocation problem can then be formulated mathematically as
the search of the resource vector x =[
xT1 , . . . ,x
TM
]T∈ R
nM , from the whole
population of cognitive nodes, that minimizes in a distributed fashion the global
potential function:
J(x) =M∑
i=1
Ii(xi) +1
2
M∑
i=1
M∑
j=1
aij[Ja(‖xj − xi‖)− Jr(‖xj − xi‖)], (3.2)
whose first term∑M
i=1 Ii(xi) represents the overall interference power over the
optimization domain (e.g., the time-frequency plane) perceived by the swarm,
while the second and third term are two penalty terms taking into account,
respectively the spread of resources and the collisions, in the resource domain.
Neglecting for a moment the second term on the right-hand side (RHS) of (3.2),
the minimization of (3.2) leads every node to find a position xi such that the
overall interference power, as perceived by the swarm, is minimum. This is a way
to let the SUs to fill the gaps in the time-frequency domain. However, such a
solution would not prevent two different SUs to choose the same position, thus
conflicting with each other. Actually, if the sum of interference functions Ii(xi)
had a common single minimum, every node would tend to occupy the same value,
thus inducing an overall conflict among SUs. To avoid this situation, the second
term on the RHS of (3.2) contains a repulsion function 12
∑Mi=1
∑Mj=1 aijJr(‖xj −
xi‖) that is maximum when two nodes occupy the same position (i.e., xj = xi).
Hence, the purpose of the repulsion term is to avoid collisions. However, the
repulsion term alone might lead to an excessive dispersion of the slots occupied
by the whole set of nodes, in the resource domain. This is also undesirable,
because it may imply the occupation of an unjustified large region in the spectral
domain or of distant time slots in the time domain, thus running into a non-
92
3.2 Problem Statement
stationarity problem. To avoid excessive dispersion, we introduce an attraction
function 12
∑Mi=1
∑Mj=1 aijJa(‖xj −xi‖) that is minimum when the vectors xi are
all close to each other. Hence, in summary, the attraction and repulsion terms
in (3.2) are chosen so that the overall system tends to remain cohesive, without
creating conflicts. Furthermore, there is a unique distance at which the attraction
and repulsion forces balance: the so called equilibrium distance in the biological
literature [130], [131]. This distance, in our case, is related to the bandwidth of
the frequency slot or the duration of the time interval.
Remark: Function (3.2) is reminiscent of the social foraging function, introduced
by Gazi and Passino in [118] and subsequently generalized in [119]. The objective
of [118] and [119] was to model the behavior of a swarm of birds searching for food,
while moving collectively as a swarm and yet avoiding collisions. In their case,
the three terms represented, respectively, the spatial distribution of food, the
attraction and repulsion forces among the birds. In [118], the interaction among
birds was modeled as a fully connected graph, i.e. aij = 1,∀i, j. The model was
then generalized in [119] to deal with an asymmetric graph. In our work, the
coefficients aij take into account the existence of a radio link between two nodes
and then they depend on radio channel characteristics, more specifically, on the
physical distance between the nodes i and j. In our set-up, the coefficients aij may
assume any real non-negative value, but because of the radio channel reciprocity,
they satisfy the symmetry condition aij = aji. This simplifies our analysis with
respect to [119], yet providing more general results than [118]. In particular,
both [118] and [119] concentrated on the swarm cohesiveness, or stability. The
main goal of their analysis was then to provide an upper bound for the spatial
spread of the swarm. Conversely, in our application, we are concerned with two
main issues, collisions and spread: We want to limit the spread in the resource
domain and avoid, or limit as much as possible, collisions, especially between
nearby nodes. For this reason, in the next section, the goal of our analysis is
to provide both a lower and an upper bound of the swarm, to be able to assess
93
3.3 Continuous-Time Distributed Optimization
the swarm properties in terms of overall occupancy and collision. Interestingly,
thanks to the symmetry of our graph, we will provide simple expressions for the
lower and upper bounds which yield physical insight about the role played by the
graph connectivity. Before starting our analysis, it is worth to point out that the
function (3.2), while mathematically similar to the function studied in [118], [119],
it is functionally different. In fact, in [118], [119], the vectors xi indicate a spatial
position and the coefficients aij are also related to the spatial positions of nodes
i and j. Conversely, in our case, the vectors xi indicate the resource domain,
i.e. time/frequency, while the coefficients aij are related to the spatial domain,
as they depend on the spatial distance between the nodes. We will see that this
makes a significant difference in the final result because it enables spatial reuse
of radio resources. For the moment, we consider, for simplicity, time-invariant
coefficients. Since in our application the exchange of information among nodes
occurs over wireless channels, in the following we will analyze the case where the
coefficients aij are random, in order to incorporate channel fading phenomena,
quantization and noise.
3.3 Continuous-Time Distributed Optimization
Our goal in this work is the distributed minimization of (3.2). A possible way
to achieve the solution in decentralized form is to use a simple gradient based
optimization, so that every node starts with an initial guess, let us say xi(0), and
then it updates its own resource allocation vector xi(t) in time according to the
following dynamical system:
xi(t) = −∇xiJ(x(t))
= −∇xiIi(xi(t)) +
M∑
j=1
aij g(xj(t)− xi(t)), (3.3)
i = 1, . . . ,M , with g(·) denoting a vector function defined as
g(xj − xi) = [ga(‖xj − xi‖)− gr(‖xj − xi‖)](xj − xi), (3.4)
94
3.3 Continuous-Time Distributed Optimization
where
ga(‖xj − xi‖)(xj − xi) =1
2∇xiJa(‖xj − xi‖), (3.5)
gr(‖xj − xi‖)(xj − xi) =1
2∇xiJr(‖xj − xi‖). (3.6)
There are several ways to choose the function in (3.4). In this work we consider
a constant attraction term, i.e.
ga(‖xj − xi‖) = cA cA > 0, (3.7)
and unbounded repulsion, i.e.
gr(‖xj − xi‖) =cR
‖xj − xi‖2, cR > 0, ∀ ‖xj − xi‖, (3.8)
or bounded repulsion, i.e.
gr(‖xj − xi‖)‖xj − xi‖ ≤ cR, cR > 0, ∀ ‖xj − xi‖. (3.9)
An example of repulsion function satisfying the boundness assumption in (3.9) is
given by:
gr(‖xj − xi‖) = cR exp
(
−‖xj(t)− xi(t)‖
2
cG
)
, (3.10)
cR, cG > 0, ∀ ‖xj − xi‖.
Examples of the resulting attraction and repulsion behaviors are shown in Fig.
3.1 and 3.2. These choices are instrumental to endow the system with the desired
behavior and they are simple enough to allow for mathematical tractability. The
constant attraction term in (3.7) determines an intensity of the attraction force in
(3.4) that is directly proportional to the distance between resources. Unbounded
repulsion is appealing in our intended application as it prevents collisions among
nodes and ensures the existence of a lower bound on the swarm size, as we will
see in the following section. At the same time, the bounded repulsion behavior
in (3.10) makes the coupling function in (3.4) continuously differentiable.
95
3.3 Continuous-Time Distributed Optimization
−5 0 5−10
−8
−6
−4
−2
0
2
4
6
8
10
Distance between the individuals
Ma
gn
itu
de
Figure 3.1: Magnitude of the coupling function g(·) in (3.4) with linear attraction
(3.7) and unbounded repulsion (3.8), using the values cA = 1 and cR = 2. The
distance between the red points and zero is the equilibrium distance between the
swarm agents.
The parameters of the functions ga(·) and gr(·) are chosen so that, at large
distances (in the resource domain), the attraction term dominates, while at short
distances is the repulsion term to dominate, and there is a unique distance where
attraction and repulsion balance. The red dots in Fig. 3.1 and 3.2 show examples
of equilibrium distance between swarm resources considering the unbounded and
bounded repulsion functions. In our setting, this equilibrium distance is chosen
proportional to the bandwidth of the frequency slot, in the frequency domain, or
to the duration of the elementary time slot and can be adjusted acting on the
swarm parameters cA and cR. It is important to remark, about the updating rule
(3.3), that each individual in the swarm has to estimate only local parameters:
the gradient of the interference level, evaluated only on its intended running
position xi, and the balance of attraction and repulsion forces with its neighbors.
96
3.4 Stability and Cohesion Analysis
−5 0 5−8
−6
−4
−2
0
2
4
6
8
Distance between the agents
Ma
gn
itu
de
Figure 3.2: Magnitude of the coupling function g(·) in (3.4) with linear attraction
(3.7) and bounded repulsion (3.10), using the values cA = 1, cR = 10 and cG = 2.
The distance between the red points and zero is the equilibrium distance between
the swarm agents.
3.4 Stability and Cohesion Analysis
Before studying the stability of the swarm (3.3), it is useful to analyze the
motion of the swarm center: x = 1/M∑M
i=1 xi. The trajectory of the center is
given by:
˙x = −1
M
M∑
i=1
∇xiIi(xi) +1
M
M∑
i=1
M∑
j=1
aij g(xj − xi)
= −1
M
M∑
i=1
∇xiIi(xi), (3.11)
where the equality of the second term of the RHS of (3.11) to zero follows from
the symmetry condition aij = aji and from the fact that g(·) is an odd function.
The above equation means that the center of the swarm moves until the agents
97
3.4 Stability and Cohesion Analysis
reach a position where the average gradient is zero. The fact that is the average
gradient to determine the motion of the swarm center, rather than the individual
gradients, is appealing in our context because the averaging operation reduces the
effect of undesired zero-mean fluctuations due to observation noise or to errors in
the estimate of the gradient.
3.4.1 Profiles with Bounded Gradient
We will now analyze the cohesiveness of the swarm under some assumptions
on the attraction/repulsion functions and on the interference profile. To this
end, we define the displacement vector between the position xi of node i and the
center of the swarm as ∆i = xi − x. Deriving bounds on the magnitude of the
vector ∆ = [∆1T , . . . ,∆M
T ]T is useful to quantify the size of the swarm and
then, ultimately, the spread in the resource allocation domain. The assumptions
needed for our derivations are the following.
Assumption A.1 : The interference profile functions Ii(y) ∈ C1 and there exists
a constant σ > 0 such that
‖∇yIi(y)‖ ≤ σ, ∀ i,y. (3.12)
This assumption is quite general and it only requires the gradient of the profile to
be bounded. This hypothesis is indeed very reasonable in the context of interest.
Under assumption A.1,
∥
∥
∥
∥
∇xiIi(xi)−1
M
M∑
j=1
∇xjIj(xj)
∥
∥
∥
∥
≤2σ(M − 1)
M= σ. (3.13)
Assumption A.2 : Given the initialization vector x0, the set Ω0 ≡ x : J(x) <
J(x0) is compact.
In our application, the resource allocation domain, either a frequency band or
a time interval (or both), is always a compact set. The incorporation of the
98
3.4 Stability and Cohesion Analysis
frequency and/or time interval limits in our problem can be done either imposing
box constraints on our optimization or by adding a barrier to the interference
profiles Ii(xi), for all i, i.e., a positive continuous term that starts from the
boundary of the resource domain and goes to infinity linearly, with constant
derivative σ, in order to keep satisfying Assumption A.2 1. Under this choice,
Assumption A.1 holds true. In the following derivations, we follow this second
approach. We are now able to state the main theorems on swarm behavior.
Unbounded Repulsion
Theorem 7 : Let us consider the swarming algorithm described in (3.3), with
attraction/ repulsion functions given by (3.7) and (3.8). Under assumptions A.1
and A.2, the state x(t) converges to the largest invariant subset of the set Ωe =
x ∈ Ω0 : x = 0 and the modulus of the displacement vector is upper and lower
bounded as follows:
β2 ≤ ‖∆‖ ≤ γ2, (3.14)
where
β2 = −σ
2cAλM (L)+
1
2
√
(
σ
cAλM (L)
)2
+ 2cRcA
tr(D)
λM (L)(3.15)
and
γ2 =σ
2cAλ2(L)+
1
2
√
(
σ
cAλ2(L)
)2
+ 2cRcA
tr(D)
λ2(L)(3.16)
with tr(D) denoting the trace of D.
Proof. In the following, we drop the dependency on time t, to avoid an ex-
cessive overcrowding of the formulas, and we introduce the notation ∇Tx :=
1To preserve C1 continuity it is also necessary to ensure a smooth transition from the original
interference profile to the modified profile incorporating the barrier.
99
3.4 Stability and Cohesion Analysis
(∇Tx1, . . . ,∇T
xM). The evolution of the time derivative of the global potential
function (3.2) along the trajectory described by (3.3) is
J(x) = [∇xJ(x)]T x =
M∑
i=1
[∇xiJ(x)]T xi
=
M∑
i=1
[−xi]T xi = −
M∑
i=1
‖xi‖2 ≤ 0 ∀t. (3.17)
This means that, while moving along the trajectory given by (3.3), the potential
function J(x) is always nonincreasing and it stops decreasing (i.e., J(x) = 0)
only if xi = 0, ∀i = 1, . . . ,M . If the set defined as Ω0 ≡ x : J(x) < J(x0) is
compact, then using LaSalle’s Invariance Principle [179], we can conclude that, as
t→∞, the state x(t) converges to the largest invariant subset of the set defined
as
Ω1 ≡ x ∈ Ω0 : J(x) = 0 ≡ x ∈ Ω0 : x = 0. (3.18)
Let us consider now the displacement vector ∆i. The time derivative of the
distance ∆i is given by
∆i = xi − ˙x = −∇xiIi(xi) +1
M
M∑
j=1
∇xjIj(xj) +
M∑
j=1
aij g(xj − xi). (3.19)
Defining a cumulative Lyapunov function as V =∑M
i=1 Vi, with Vi =12‖∆i‖
2,
and taking its time derivative along the system trajectory (3.19), we can write
V =
M∑
i=1
Vi =
M∑
i=1
∆Ti ∆i = −
M∑
i=1
[
∇xiIi(xi)−1
M
M∑
j=1
∇xjIj(xj)
]T
∆i +
+
M∑
i=1
M∑
j=1
aij [ga(‖xj − xi‖)− gr(‖xj − xi‖)] (xj − xi)T∆i (3.20)
From (3.20), the time derivative of the Lyapunov function along the system tra-
jectory can be rewritten as
V = −M∑
i=1
[
∇xiIi(xi)−1
M
M∑
j=1
∇xjIj(xj)
]T
∆i+
100
3.4 Stability and Cohesion Analysis
+M−1∑
i=1
M∑
j=i+1
aijg(‖xj − xi‖)](xj − xi)T∆i +
+
M−1∑
i=1
M∑
j=i+1
aijg(‖xi − xj‖)(xi − xj)T∆j
= −M∑
i=1
[
∇xiIi(xi)−1
M
M∑
j=1
∇xjIj(xj)
]T
∆i +
−1
2
M∑
i=1
M∑
j=1
aij ga(‖xj − xi‖)‖xj − xi‖2 +
+1
2
M∑
i=1
M∑
j=1
aij gr(‖xj − xi‖)‖xj − xi‖2. (3.21)
Exploiting the features of linear attraction and unbounded repulsion of the cou-
pling function g(·), as expressed in (3.7) and (3.8), we obtain
V = −M∑
i=1
[
∇xiIi(xi)−1
M
M∑
j=1
∇xjIj(xj)
]T
∆i +
−cA2
M∑
i=1
M∑
j=1
aij‖xj − xi‖2 +
cR2tr(D). (3.22)
The state vector x can always be decomposed into the motion of the center of
the swarm plus the displacement vector ∆, as follows:
x = 1M ⊗ x+∆ (3.23)
where the disagreement vector ∆ = (∆T1 , . . . ,∆
TM )T ∈ R
nM satisfies ∆ ⊥ 1M ⊗
el, l = 1, . . . , n, where el is the vector of the canonical basis, with all entries
equal to zero, except the l-th component, equal to one. The vector ∆ belongs
to an (nM − n)-dimensional subspace (the disagreement eigenspace of L) that
is orthogonal to the nullspace of L. As a consequence of (2.82) and (3.23),
expression (3.22) can be recast as
V = −M∑
i=1
[
∇xiIi(xi)−1
M
M∑
j=1
∇xjIj(xj)
]T
∆i − cA∆T L∆+
cR2tr(D) (3.24)
101
3.4 Stability and Cohesion Analysis
Applying (2.80) and considering the expression of λ(L) in (2.83), we obtain an
upper bound
V ≤ −M∑
i=1
[
∇xiIi(xi)−1
M
M∑
j=1
∇xjIj(xj)
]T
∆i − cAλ2(L)‖∆‖2 +
cR2tr(D)
(3.25)
and a lower bound
V ≥ −M∑
i=1
[
∇xiIi(xi)−1
M
M∑
j=1
∇xjIj(xj)
]T
∆i − cAλM (L)‖∆‖2 +cR2tr(D)
(3.26)
of the time derivative of the Lyapunov function. Exploiting the Cauchy-Schwarz
inequality and the assumption A.1, expressions (3.25) and (3.26) can be further
bounded as
V ≤ −cAλ2(L)‖∆‖2 + σ
M∑
i=1
‖∆i‖+cR2tr(D), (3.27)
V ≥ −cAλM (L)‖∆‖2 − σM∑
i=1
‖∆i‖+cR2tr(D) (3.28)
Now, considering the inequality∑M
i=1 ‖∆i‖ ≤√
∑Mi=1 ‖∆i‖2 = ‖∆‖, the final
expressions for the lower and upper bounds of the time derivative of the global
Lyapunov function take the form
V ≤ −cAλ2(L)‖∆‖2 + σ‖∆‖+
cR2tr(D), (3.29)
V ≥ −cAλM (L)‖∆‖2 − σ‖∆‖ +cR2tr(D). (3.30)
Let us consider now the two bounds in more detail.
Upper Bound
For the upper bound case, expression (3.29) assumes the form
V ≤ −c1(
‖∆‖2 − c2‖∆‖ − c3)
, (3.31)
102
3.4 Stability and Cohesion Analysis
with c1, c2, c3 > 0. The RHS of (3.31) is a polynomial in ‖∆‖, having two
real roots, with opposite sign, which we denote, respectively, as γ1 and γ2, with
γ2 > γ1. Clearly, only the positive root γ2 is feasible, as ‖∆‖ is certainly non-
negative. Introducing the 2D plane having axes V and ‖∆‖, the evolution of the
system occurs below the solid parabola shown in Fig. 3.3. Clearly, at convergence,
i.e., when V = 0, it must be
‖∆‖ ≤ γ2. (3.32)
where
γ2 =σ
2cAλ2(L)+
1
2
√
(
σ
cAλ2(L)
)2
+ 2cRcA
tr(D)
λ2(L). (3.33)
This result proves the swarm cohesiveness.
Lower Bound
For the lower bound case, expression (3.30) assumes the form
V ≥ −c4(
‖∆‖2 + c5‖∆‖ − c6)
, (3.34)
with c4, c5, c6 > 0. Also in this case, the RHS of (3.34) is a parabola, with two
real roots having opposite sign, which we indicate as β1 and β2, with β2 > β1.
Only β2 is feasible in this case. In this case, the evolution of the system occurs
in the space above the dashed parabola depicted in Fig. 3.3. At convergence, it
must be
‖∆‖ ≥ β2, (3.35)
where
β2 = −σ
2cAλM (L)+
1
2
√
(
σ
cAλM (L)
)2
+ 2cRcA
tr(D)
λM (L). (3.36)
This proves that the vectors ∆i cannot become all zero, i.e., the vectors xi cannot
collapse all to the same value. This is a result of the repulsion force.
103
3.4 Stability and Cohesion Analysis
||∆||
V.
β2
γ2θ
uβ
1θ
lγ1
Figure 3.3: Upper and lower bounds of the potential function time derivative.
Combining both upper and lower bounds, the evolution of the system occurs in
the dashed area sketched in Fig. 3.3 and then, at convergence, (3.14) must hold
true. We can easily check that, indeed, β2 < γ2. This concludes our proof.
Remark: The inequality (3.14) implies that the modulus of ‖∆‖ cannot be
zero, avoiding the overall collapse on the swarm center, but does not prevent
some pairs of nodes to end up with the same resource allocation vector. This
does not happen if the graph is fully connected, as in such a case the repulsion
between nodes with the same allocation vector would go to infinity. However, in
a sparse graph, two nodes i and j with no direct link between them (i.e., with
aij = 0), may end up with the same resource vector. Actually, in our intended
application, where the coefficients aij depend on the distance between the nodes,
it may happen that two nodes get the same resource vector only if they are
not neighbors. But this is indeed a positive behavior as it gives rise to what is
typically known as spatial reuse of frequency slots. In Section V we will exhibit
some numerical results showing that the proposed approach is intrinsically able
to provide a spatial reuse of frequencies.
104
3.4 Stability and Cohesion Analysis
Bounded Repulsion
Theorem 8 : Let us consider the swarming algorithm described in (3.3), with
attraction/ repulsion functions given by (3.7) and (3.9). Under assumptions A.1
and A.2, the state x(t) converges to the largest invariant subset of the set Ωe =
x ∈ Ω0 : x = 0 and the modulus of the displacement vector is upper bounded
as follows:
‖∆‖ ≤ γ, (3.37)
where
γ =σ + cRdmax
cAλ2(L). (3.38)
Proof. The proof of the system stability follows the same steps as in precedence.
In the following, we derive the upper bound γ on the displacement vector ∆ in the
case of bounded repulsion as in (3.9). From equation (3.20), the time derivative
of the Lyapunov function along the system trajectory can be rewritten as:
V = −M∑
i=1
[
∇xiIi(xi)−1
M
M∑
j=1
∇xjIj(xj)
]T
∆i +
−1
2
M∑
i=1
M∑
j=1
aij ga(‖xj − xi‖)‖xj − xi‖2 +
−M∑
i=1
M∑
j=1
aij gr(‖xj − xi‖)(xj − xi)T∆i. (3.39)
As a consequence of (2.82) and (3.23), expression (3.39) can be recast as
V = −M∑
i=1
[
∇xiIi(xi)−1
M
M∑
j=1
∇xjIj(xj)
]T
∆i − cA∆T L∆+
−M∑
i=1
M∑
j=1
aij gr(‖xj − xi‖)(xj − xi)T∆i. (3.40)
105
3.4 Stability and Cohesion Analysis
Applying (2.80) and considering the expression of λ(L) in (2.83), we obtain the
upper bound
V ≤ −M∑
i=1
[
∇xiIi(xi)−1
M
M∑
j=1
∇xjIj(xj)
]T
∆i − cAλ2(L)‖∆‖2 +
−M∑
i=1
M∑
j=1
aij gr(‖xj − xi‖)(xj − xi)T∆i. (3.41)
Now, exploiting the Cauchy-Bunyakovsky-Schwarz (CBS) inequality, assumption
A.1, and the inequality in (3.9), expression (3.41) can be further bounded as:
V ≤ −cAλ2(L)‖∆‖2 + σ
M∑
i=1
‖∆i‖+M∑
i=1
dii‖∆i‖, (3.42)
where dii =∑M
j=1 aij is the connectivity degree of node i. Then, using the
inequality∑M
i=1 ‖∆i‖ ≤ ‖∆‖ and denoting with dmax the maximum connectivity
degree of the network, expression (3.42) can be bounded as:
V ≤ −cAλ2(L)‖∆‖2 + (σ + cRdmax)‖∆‖
≤ ‖∆‖ [σ + cRdmax − cAλ2(L)‖∆‖] . (3.43)
Arguing as in Theorem 1, the evolution of the system in the plane (V , ‖∆‖) occurs
in the region below the parabola described by equation (3.43), which intersects
the ‖∆‖-axis in zero and γ. In particular, at convergence, i.e. when V = 0, we
must have
‖∆‖ ≤ γ, (3.44)
where
γ =σ + cRdmax
cAλ2(L). (3.45)
This result proves the swarm cohesiveness.
Remark: As we have shown before, unbounded repulsion determines the exis-
tence of both an upper bound and a lower bound on the swarm size. On the
106
3.4 Stability and Cohesion Analysis
contrary, in the bounded repulsion case, the swarm admits only an upper bound
on the displacement vector ‖∆‖. A lower bound is not guaranteed as the effect
of a sufficiently strong interference profile could determine the overall collapse of
the swarm into a single point. This occurrence can be avoided by selecting the
repulsion coefficient cR large enough with respect to the profile effect.
Theorems 1 and 2 have been derived under the assumption of a bounded norm
of the profile gradient. In the following, we will provide closed form expressions
for the lower and upper bounds in the case of an unbounded gradient, e.g., in the
case in which the profile is quadratic.
3.4.2 Quadratic Profile
We consider now a quadratic interference profile given by
I(y) =Aσ
2‖y − cσ‖
2 + bσ (3.46)
where Aσ ∈ R, bσ ∈ R, and cσ ∈ Rn. Its gradient at a point y ∈ R
n is
∇yI(y) = Aσ(y − cσ). (3.47)
Unbounded Repulsion
Theorem 9 : Given the swarming algorithm described in (3.3), with attraction/
repulsion functions given by (3.7) and (3.8), under the assumption of a quadratic
profile as in (3.46), we have
β ≤ ‖∆‖ ≤ γ (3.48)
where the lower and upper bounds are respectively given by
β =
√
cRtr(D)
2(cAλM (L) +Aσ), γ =
√
cRtr(D)
2(cAλ2(L) +Aσ). (3.49)
with β ≤ γ.
107
3.4 Stability and Cohesion Analysis
This proves swarm cohesiveness and prevents the individuals to collapse on the
swarm center. Moreover, if the network is fully connected, the swarm converges
to the only stable equilibrium of the system, given by
‖∆‖ =
√
cRM(M − 1)
2(cAM +Aσ). (3.50)
Proof. Under the hypothesis of quadratic profiles we have
[
∇xiI(xi)−1
M
M∑
j=1
∇xjIj(xj)
]T
∆i = Aσ‖∆i‖2. (3.51)
As a consequence, the upper bound (3.25) can be rewritten as
V ≤ −cAλ2(L)M∑
i=1
‖∆i‖2 −Aσ
M∑
i=1
‖∆i‖2 +
cR2tr(D)
= −[
cAλ2(L) +Aσ
]
M∑
i=1
‖∆i‖2 +
cR2tr(D)
= −[
cAλ2(L) +Aσ
][
‖∆‖2 − γ2]
. (3.52)
Similarly, the lower bound (3.26) becomes
V ≥ −cAλM (L)
M∑
i=1
‖∆i‖2 −Aσ
M∑
i=1
‖∆i‖2 +
cR2tr(D)
= −[
cAλM (L) +Aσ
]
M∑
i=1
‖∆i‖2 +
cR2tr(D)
= −[
cAλM (L) +Aσ
] [
‖∆‖2 − β2]
. (3.53)
Arguing as in Theorem 1, the evolution of the system in the plane (V , ‖∆‖) occurs
in the region comprised between the two parabolas described by equations (3.52)
and (3.53), both centered on the axis ‖∆‖ = 0. In particular, at convergence, i.e.
when V = 0, we must have
β ≤ ‖∆‖ ≤ γ, (3.54)
108
3.4 Stability and Cohesion Analysis
where
β =
√
cRtr(D)
2(cAλM (L) +Aσ), γ =
√
cRtr(D)
2(cAλ2(L) +Aσ), (3.55)
with β ≤ γ. If the network is fully connected, expressions (3.52) and (3.53) hold
with strict equality and the swarm converges to the unique stable equilibrium
point of the dynamical system. The final convergence value can be obtained
substituting, in one of the bounds in (3.55), tr(D) = M(M − 1) and λ2(L) =
λM (L) =M , thus achieving
V = 0 ⇒ ‖∆‖ =
√
cRM(M − 1)
2(cAM +Aσ). (3.56)
This concludes the proof of the theorem.
Bounded Repulsion
Theorem 10 : Given the swarming algorithm described in (3.3), with attrac-
tion/ repulsion functions given by (3.7) and (3.9), under the assumption of a
quadratic profile as in (3.46), we have
‖∆‖ ≤ γ (3.57)
where the lower and upper bounds are respectively given by
γ =cRdmax
cAλ2(L) +Aσ. (3.58)
Proof. Exploiting (3.51), the upper bound (3.41) can be written as:
V ≤ −cAλ2(L)M∑
i=1
‖∆i‖2 −Aσ
M∑
i=1
‖∆i‖2 + cR
M∑
i=1
dii‖∆i‖
= −[
cAλ2(L) +Aσ
]
M∑
i=1
‖∆i‖2 + cRdmax
M∑
i=1
‖∆i‖
≤ −[
cAλ2(L) +Aσ
]
‖∆‖[
‖∆‖ − γ]
. (3.59)
109
3.5 Swarming in the Frequency Domain
As in Theorem 2, the evolution of the system in the plane (V , ‖∆‖) occurs in
the region below the parabola described by equation (3.59), which intersects the
‖∆‖-axis in zero and γ. In particular, at convergence, i.e. when V = 0, we must
have
‖∆‖ ≤ γ, (3.60)
where
γ =cRdmax
cAλ2(L) +Aσ. (3.61)
This concludes the proof of the theorem.
A numerical validation of our theoretical findings will be provided in the Sections
2.6 and 2.7. In the next section, we consider in more detail the situation where the
resource domain is monodimensional (1D), like the frequency axis, for example.
In the ensuing section we will then provide some examples of application for 2D
allocation domain, e.g. time-frequency domain.
3.5 Swarming in the Frequency Domain
In this section we focus our attention on the swarming over a 1D domain, for
example the frequency axis. First, we provide a local stability analysis useful to
show that the introduction of the attraction and unbounded repulsion terms in
the functional to be minimized does not affect the stability of the system. More-
over, considering the swarm discrete-time model, we provide an upper bound for
the step size, depending on simple system parameters, that assures the local con-
vergence to an equilibrium point. Then, we propose fast swarming methods based
on a proper selection of the descent direction of a scaled gradient optimization.
The first method is an approximation of the Newton-based optimization and im-
proves the convergence speed of the algorithm; the second method adapts the
swarming speed with respect to the interference power perceived by the swarm.
Finally, we show some numerical results to assess the performance of the resource
allocation technique based on swarming.
110
3.5 Swarming in the Frequency Domain
3.5.1 Local Stability Analysis
In this section we analyze the behavior of the swarm in proximity of a solution
point x∗ where the gradient of the potential function J(x) is equal to zero. To
study the local stability of the system around a solution point, we consider the
second-order Taylor series expansion of the scalar-valued function J(x) around
x = x∗, given by
J(x) ≃ J(x∗) +∇xJ(x∗)T (x− x∗) +
1
2(x− x∗)TH(x∗)(x− x∗), (3.62)
where H(x∗) is the Hessian matrix, computed in x∗, whose entries are:
Hij(x) :=∂2J(x)
∂xj∂xi. (3.63)
If H(x∗) is positive definite, the potential function J(x) is locally approximated
by a positive definite quadratic form, in the neighborhood of x∗. This guarantees
the local stability of the system. In the one-dimensional case, xi ∈ R and x =
(x1, . . . , xM )T ∈ RM . Considering a nonlinear coupling function characterized
by linear attraction and unbounded repulsion as given in (3.7) and (3.8), the
potential function J(x) assumes the form
J(x) =M∑
i=1
Ii(xi) +1
4
M∑
i=1
M∑
j=1
aij[
cA(xj − xi)2 − cR log(xj − xi)
2]
. (3.64)
The entries Hij(x) of the Hessian matrix are then
Hii(x) =∂2J(x)
∂x2i=
∂2Ii(xi)
∂x2i+
M∑
j=1
aij
[
cA +cR
(xj − xi)2
]
,
Hij(x) =∂2J(x)
∂xj∂xi= − aij
[
cA +cR
(xj − xi)2
]
= Hji(x). (3.65)
According to Gershgorin theorem, H(x∗) is a positive definite matrix with all
the eigenvalues greater than zero if
Hii(x∗) >
∑
j
|Hij(x∗)| , ∀i = 1, . . . ,M. (3.66)
111
3.5 Swarming in the Frequency Domain
In our case, this gives
∂2Ii(x∗i )
∂x2i+
M∑
j=1
aij
[
cA +cR
(x∗j − x∗i )
2
]
>M∑
j=1
aij
∣
∣
∣
∣
cA +cR
(x∗j − x∗i )
2
∣
∣
∣
∣
⇒∂2Ii(x
∗i )
∂x2i> 0 , ∀i = 1, . . . ,M. (3.67)
As a consequence, the convexity of the interference profiles Ii(xi), evaluated in the
system equilibrium point x∗, guarantees the local stability. This is an important
result as it shows that the introduction of the attraction and unbounded repulsion
terms in the functional to be minimized does not affect the system stability.
3.5.2 Discrete-Time Implementation
The swarm evolution has been described, up to now, in continuous time. In
practice, the exchange of information between nodes of the networks requires a
discrete-time implementation. The time discretization of (3.3) yields
xi[k + 1] = xi[k] + α
[
−∂Ii(xi[k])
∂xi+
+
M∑
j=1
aij
(
cA −cR
(xj [k]− xi[k])2
)
(
xj[k]− xi[k])
]
, (3.68)
where the step size α must be sufficiently small to ensure convergence. In the
following, we will provide some upper bounds on ǫ, in order to guarantee conver-
gence, at least in the neighborhood of the solution points.
In the neighborhood of the equilibrium point x∗, the continuous-time dynam-
ical system can be approximated as
x = −∇xJ(x) ≃ −H(x∗)(x− x∗). (3.69)
The discretization of this vector differential equation leads to the following dif-
ference equation
x[k + 1] = x[k]− αH(x∗)(x[k]− x∗) = αH(x∗)x∗ + (I − αH(x∗))x[k]
= αH(x∗)x∗ +W (x∗)x[k] (3.70)
112
3.5 Swarming in the Frequency Domain
where W (x∗) = I −αH(x∗) represents the iteration matrix of the discrete time
algorithm evaluated at the equilibrium point. In order to assure the convergence
of the swarming algorithm, the discrete time mapping must be a contraction
having fixed point x∗ such that x∗ = αH(x∗)x∗+W (x∗)x∗. In this way, letting
y = x− x∗, the discrete time procedure can be rewritten as
y[k + 1] =W (x∗)y[k], (3.71)
and the error vector y converges to zero if the spectral radius (W (x∗)) of the
iteration matrix is less than one in modulus. This condition holds true if
H(x∗) ≻ 0 and 0 < α <2
λMAX(H(x∗)). (3.72)
From the analysis carried out in section 3.5.1, the positive definiteness of the
Hessian matrix H(x∗) is assured by the convexity of the interference profiles
evaluated at x∗. Under this condition, the step size ǫ must be chosen in order
to satisfy the previous bound. The upper limit may be difficult to evaluate
in a distributed manner. Nevertheless, using again Gershgorin’s theorem, the
maximum eigenvalue of H(x∗) can be upper bounded as
λMAX(H(x∗)) ≤ maxi
∂2Ii(x∗i )
∂xi2+ 2
∑
j
aij
(
cA +cR
(x∗j − x
∗i )
2
)
< σ′′
MAX + 2
(
cA +cR
r2MIN
)
dMAX = λσM (3.73)
where σ′′
MAX is the maximum convexity of the interference profile, r2MIN =
mini,j ‖x∗j − x
∗i ‖
2 is determined by the repulsion constant cR and dMAX is the
maximum degree of the network connectivity. Hence, the convergence of the
discrete time algorithm is ensured by choosing the step size ǫ in the interval
0 < α <2
λσM, (3.74)
whose upper bound depends now on global parameters that can be exchanged
through the network.
113
3.5 Swarming in the Frequency Domain
3.5.3 Fast Swarming Algorithms
One of the main drawbacks of gradient-based methods is their speed of con-
vergence, which is known to be low. Clearly, a distributed technique is amenable
for resource allocation only if it guarantees convergence in a few iterations. In
this section we modify the basic swarming algorithm (3.3) in order to increase its
convergence speed. In general, the minimization of the functional in (3.2) can be
achieved through a general scaled-gradient method given by
x = −B(x)∇xJ(x) (3.75)
where B(x) is some positive definite matrix representing a scaling along the di-
rection of steepest descent. The introduction of B(x) is useful to increase the
speed of the algorithm or to enforce particular behaviors on the swarm individ-
uals. The evolution of the time derivative of the global potential function (3.2)
along this modified system trajectory (3.75) is given by
J(x) = [∇xJ(x)]T x = −xT [B(x)]−T x ≤ 0 ∀t. (3.76)
where (·)−T means inverse and transpose of a matrix. This means that, mov-
ing along the trajectory given by (3.3), the potential function J(x) is always
nonincreasing and it stops decreasing (i.e., J(x) = 0) only if x = 0. Un-
der assumption A.2, the set Ω0 ≡ x : J(x) < J(x0) is compact, then us-
ing the LaSalle’s Invariance Principle [179] we can conclude that, as t → ∞,
the state x(t) converges to the largest invariant subset of the set defined as
x ∈ Ω0 : J(x) = 0 ≡ x ∈ Ω0 : x = 0.
Typically, B(x) is chosen equal to the inverse of the Hessian matrix H(x) in
order to implement a Newton recursion having an improved convergence speed
with respect to the normal steepest descent. The Newton’s method approximates
at each iteration J(x) by a quadratic function, as in (3.62), and then it moves
towards the minimum of that quadratic function. However, the matrix H(x) is
a full matrix, with elements given by (3.65). This implies that the computation
114
3.5 Swarming in the Frequency Domain
of B(x) requires a centralized mechanism. One suboptimal, but parallelizable,
solution consists in approximating the Hessian matrix by retaining only its diago-
nal entries. This simplifies the inversion of the Hessian matrix and allows parallel
computation. We consider then the trajectory (3.75), where the diagonal scaling
matrix B(x) has entries given by
Bii(x) =
(
∂2J(x)
∂x2i
)−1
=
(
∂2Ii(xi)
∂x2i+
M∑
j=1
aij
[
cA +cR
(xj − xi)2
])−1
. (3.77)
The i-th element is computable at the correspondent node having access to the
second derivative of the profile at the local point xi and to the positions xj of its
neighbors. For general non convex profiles, the value of Bii(x) can be negative.
To ensure the positive definiteness of the matrix B(x) and to preserve the descent
direction of the algorithm, we can add a scaled identity matrix, with the scale
adjusted at each iteration, to the Hessian’s approximation.
An alternative solution to improve the convergence speed of the swarming
algorithm uses a scaling matrix whose diagonal elements are functions of the
power Ii(xi) perceived at each node i. The goal is to accelerate the motion of
the resources perceiving a high interference and, at the same time, to slow down
the resources that are allocating on idle sub-bands. This adaptive feature can be
implemented using a variable step size depending on a monotonically increasing
function f(Ii(xi)) of the perceived interference power. The values of this function
are lower and upper bounded by positive values ensuring that the matrix B(x)
is positive definite. Examples include linear, quadratic, logarithmic functions
etc...The diagonal entries of the scaling matrix can then be expressed as
Bii(x) = f(Ii(xi)), f(·) ∈ [fmin, fmax] > 0. (3.78)
This solution improves the reaction time needed by the algorithm to perform a
resource allocation on idle bands in case of a PU’s activation. The discrete-time
recursion of node i can then be expressed as
xi[k + 1] = xi[k] + αBii(x[k])
[
−∂Ii(xi[k])
∂xi+
115
3.5 Swarming in the Frequency Domain
+M∑
j=1
aij
(
cA −cR
(xj [k]− xi[k])2
)
(
xj[k]− xi[k])
]
, (3.79)
where Bii(x[k]) is given by (3.77) or (3.78). In the following sections, we will
illustrate how these solutions outperform the convergence speed of the gradient-
based algorithm.
3.5.4 Numerical Examples
In this section we provide some numerical results to assess the performance
of the proposed algorithms.
Example 1 - Validation of the theoretical results for profiles with bounded gradi-
ent: In this example we show some numerical results supporting Theorem 1. We
consider the one-dimensional evolution of 10 agents constituting the swarm in
the presence of a bounded profile composed of the superposition of several Gaus-
sian functions. Each agent interacts with its neighbors according to a connected
0.7 0.8 0.9 1 1.1 1.21
2
3
4
5
6
7
8
9
10
Covering radius r0
||∆
||
||∆||
Upper Bound − γ2
Lower Bound − β2
Figure 3.4: Swarm size parameter versus the node covering radius.
116
3.5 Swarming in the Frequency Domain
0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.40
2
4
6
8
10
12
14
16
18
Attraction parameter cA
||∆
||||∆||
Upper Bound − γ2
Lower Bound − β2
Figure 3.5: Swarm size parameter versus the attraction parameter cA.
topology and updates its intended position according to (3.3), implemented in
discrete-time. To provide a validation of our theoretical findings, we report, on
of Fig. 3.4, the behavior of the swarm size parameter ‖∆‖ and its theoretical
bounds in (3.15) and (3.16), versus the node’s covering radius that determines
the network connectivity. The results have been averaged over 100 independent
realizations. The attraction and repulsion parameters used in this simulation are
cA = 0.2 and cR = 0.2. As we can see, the effect of an increment of connectivity
slightly reduces the swarm size and the swarm parameter ‖∆‖ remains always
inside the theoretical bound interval. The increment of the network connectivity
implies tighter bounds that converge on constant values if the network is fully
connected.
A further example is given on Fig. 3.5, where we show the behavior of the
swarm size parameter ‖∆‖ and its theoretical bounds in (3.15) and (3.16), versus
the swarm attraction constant cA. As expected, an increment of the attraction
force, while keeping constant the repulsion, decreases the swarm size. Also in
this case, we can see how the theoretical bounds are always satisfied.
117
3.5 Swarming in the Frequency Domain
0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
Figure 3.6: Network topology and allocation example.
Example 2 - Collision avoidance and spatial reuse of frequency: One of the main
features of the proposed swarming technique is the capability to prevent collisions
between nearby nodes while, at the same time, allowing for spatial reuse of the
frequency channels from secondary nodes far away from each other. The coupling
coefficients aij in (3.3) depend on the distance between the SUs and reflect the
network topology, as dictated by the coverage radius of each node. Interestingly,
as will be shown next, the proposed swarming algorithm leads naturally to spatial
reuse of frequencies, as the network topology becomes more and more sparse. To
quantify the spatial reuse of frequencies, we introduce the reuse factor ζ, defined
as the ratio between the number of resources (frequency subchannels) necessary
to guarantee one slot for each node and the number of channels really allocated
by the algorithm. As an example of channel allocation, in Fig. 3.6 we consider a
network composed of 100 nodes, where each node senses the interference spectrum
shown in Fig. 3.7. The swarm parameters are cA = 0.01 and cR = 0.1. Every
node starts from a random initial position on the spectrum, and then it updates
118
3.5 Swarming in the Frequency Domain
15 15.5 16 16.5 17 17.5 18 18.5 19 19.5 20−0.5
0
0.5
1
1.5
Frequency (MHz)
PS
D (
mW
/Hz)
Figure 3.7: Interference profile and allocation example.
its intended position according to (3.3), implemented in discrete-time. In the
application at hand, there is an intrinsic quantization of the frequency resources
given by the subchannel bandwidth. In our implementation, we let the system
evolve according to (3.68) until successive differences in allocation become smaller
than the bandwidth of a frequency subchannel. At that point, the evolution stops
and every SU is allowed to transmit over the selected channel. The final choice is
indicated by assigning a different shape to different subchannels, as shown in the
topology plot reported in Fig. 3.6. In our experiment, the number of available
channels with low interference is eight, hence much smaller than the number
of users. Interestingly, from Fig. 3.6 we can observe that the nodes that have
picked up the same channel are never neighbor of each other. This is indeed one
of the most interesting features of the proposed algorithm. This means that the
algorithm is capable of implementing a decentralized mechanism for spatial reuse
of frequencies. In this case, the network frequency reuse parameter is ζ = 12.5.
Clearly, the reuse factor depends on the sparsity of the graph describing the
network topology. To quantify the effect of the coverage radius of each node
on the reuse factor, in Fig. 3.8 we report the average behavior of ζ, averaged
over 100 independent initializations, versus the covering radius of each node,
119
3.5 Swarming in the Frequency Domain
0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2
1
1.05
1.1
1.15
1.2
1.25
1.3
1.35
Covering radius r0
Fre
quency r
euse p
ara
mete
r ζ
CR
= 0.1
CR
= 0.2
CR
= 0.3
Figure 3.8: Frequency reuse parameter versus covering radius.
considering three different values of the repulsion constant cR. In this simulation
we consider M = 15 nodes, cA = 0.2, the same interference profile as in the
bottom side Fig. 3.6 and 15 available channels on the idle band in the middle
of the spectrum. As expected, the behavior of ζ is monotonically decreasing and
it reaches the unit value when the covering radius is such that the network is
fully connected. Furthermore, Fig. 3.8 shows also that, by reducing the value of
cR, the repulsion force is weaker and this facilitates the reuse of frequency slots.
Hence, the parameter cR has to be chosen as a trade-off between reuse factor
and number of collisions. We have checked numerically that in all simulations,
choosing appropriately the swarm parameters, the final channel allocation never
determines collisions among spatial neighbors.
Example 3 - Convergence speed: One of the main issues for distributed resource
allocation algorithms is convergence speed. In this section we show some nu-
merical examples to evaluate the convergence time of the proposed allocation
algorithm and its modified version given in (3.79). The first example assumes the
same settings as in Fig. 3.6. To assess convergence time, in Fig. 3.9, we report
120
3.5 Swarming in the Frequency Domain
0 5 10 15 20 25 30 35 40 45 50
0
0.2
0.4
0.6
0.8
1
Iteration Index
Norm
aliz
ed S
yste
m P
ote
ntial F
unction
Fully connected
r0 = 0.8
r0 = 0.5
Figure 3.9: Normalized system potential function vs. time index, for different
coverage radii.
the average behavior of the evolution of the system potential function (3.9), nor-
malized with respect to the maximum and the minimum value, averaged over 500
independent realizations, vs. the iteration index. Three different coverage radii
are considered to evaluate the impact of network topology on the convergence
speed. The attraction and repulsion parameters of the swarm for this simulation
are cA = 0.2 and cR = 0.2, and the step size is equal to α = 0.05. As expected,
the convergence rate increases as the connectivity increases.
The second example compares the convergence speed of the gradient based
swarming algorithm in (3.3) and of the approximated Newton method with scal-
ing coefficients given by (3.77). In Fig. 3.10, we report the average behavior of
the evolution of the system potential function, e.g., (3.2), normalized with respect
to the maximum and the minimum value, averaged over 500 independent realiza-
tions, vs. the iteration index. The evolution of the gradient-based algorithm is
given by the dashed curve while the approximated Newton version is depicted by
the continuous curve. In this simulation we consider the same interference profile
and network topology depicted in Fig. 3.6. The parameters are the same of the
121
3.5 Swarming in the Frequency Domain
0 5 10 15 20 25 30 35 40 45 50
0
0.2
0.4
0.6
0.8
1
iteration index
Norm
aliz
ed S
yste
m P
ote
ntial F
unction
Gradient Descent
Newton Approx
Figure 3.10: Normalized system potential function vs. time index, for differ-
ent descent directions of the algorithm : gradient descent (dashed) and Newton
approximation (solid).
previous simulation and the step size for the approximated Newton version is
equal to α = 0.8. The step sizes were empirically decided because slightly greater
values determine instability of the algorithm in both cases. From Fig. 3.10, we
can notice how the approximated Newton-scaled version greatly outperforms the
gradient based algorithm. This means that the convergence time of the swarm-
ing algorithm can be considerably improved if every node is able to evaluate the
second order derivative of the system potential function given by (3.79).
Example 4 - Dynamic response of the swarm to a predator (interferer): Natural
swarms are adaptive systems whose individuals cooperate in order to improve
their food search capabilities and to increase their robustness against preda-
tors’ attacks. We show next that the proposed resource allocation increases,
as a by-product, the network robustness against the intrusion of a primary user
(predator). We consider again the network topology depicted in Fig. 3.6, plus
the inclusion of two PU’s that start emitting, at different times, thus causing a
dynamic change of the occupied spectrum. Our goal is to test the dynamic
122
3.5 Swarming in the Frequency Domain
0 50 100 150 2000.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
Iteration index
Ave
rag
e in
terf
ere
nce
leve
l
Figure 3.11: Dynamic resource allocation by swarming: Reaction time to PU’s
activations, for basic swarming.
response of the network to this changing environment. Playing again with the
swarm analogy, PU’s take now the role of predators whose positions must be
avoided by the swarm individuals.
To give an example of the reaction time needed by the algorithm to react
to the PU’s intrusion and adjust the resource allocation consequently, in Fig.
3.11 we show the behavior of the average interference perceived by the swarm
versus the time index. The two peaks at the iterations 67 and 123 correspond to
the two PU’s activation times. The low power value represents the noise level.
In particular, we compare the results of the gradient-based algorithm in (3.3),
in Fig. 3.11, and its scaled version in (3.78), in Fig. 3.12, that adapts the
convergence speed with respect to the perceived interference. The attraction and
repulsion parameters used in this simulation are cA = 0.2 and cR = 0.2; the step
size is equal to α = 0.05 for both algorithms. We can notice how the adaptive
scaled version needs only a small number of iterations to leave the PU’s regions,
thus outperforming the gradient-based version. This positive behavior is given
by the adaptation of the algorithm with respect to the perceived interference,
123
3.5 Swarming in the Frequency Domain
0 50 100 150 2000.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4A
vera
ge in
terf
eren
ce le
vel
Iteration index
Figure 3.12: Dynamic resource allocation by swarming: Reaction time to PU’s
activations, for adaptive scaling.
determining that resources allocating on high interference regions move faster
due to the increment of the profile gradient and the cohesion force.
These examples show that the cohesion force represents an intrinsic robust-
ness factor of the algorithm. In fact, resources allocating over high interference
bands might measure a flat spectrum, thus resulting in limited capabilities to
move out of (flat) occupied bands, if the only cause of change is spectrum gra-
dient. However, increasing the cohesion force, the agents allocating over the low
interference band tend to form cohesive blocks that exert an attraction towards
the agents trapped by mistake over the flat regions of the spectrum occupied by
the primary users. This is an example of cooperation gain.
Example 5 - Comparison with deterministic graph coloring methods: In the previ-
ous examples, we have illustrated the capability of the algorithm of implementing
a decentralized mechanism for spatial reuse of frequencies. The swarming algo-
rithm in (3.3) is indeed based on local exchange of data among SU’s, whose
resources select available channels avoiding conflicts only with spatial neighbors.
The aim of this example is to compare the number of iterations needed by the
124
3.5 Swarming in the Frequency Domain
10 20 30 40 50 60 70 80 90 1000
20
40
60
80
100
120
Number of nodes
Ave
rag
e n
um
be
r o
f ite
ratio
ns
Graph Coloring α = 1
Graph Coloring α = 1.25
Graph Coloring α = 1.5
Swarming α = 1
Swarming α = 1.25
Swarming α = 1.5
Figure 3.13: Average number of iterations to obtain convergence versus number
of nodes, for different degrees of network connectivity.
swarming algorithm to convergence with respect to a graph coloring algorithm
[87]– [90]. As a comparison, we consider a deterministic distributed graph color-
ing method from [87]– [88], having the same complexity of the swarming algorithm
in terms of number of exchanged messages per iteration. Several other methods
having a better convergence rate, which is paid by a greater complexity of the
algorithm, are also present in the literature.
In this example, we generate the interference graph among SU’s as a random
geometric graph over a unitary area. The covering radius of each node is chosen
as
r0 = α
√
2 logM
πM(3.80)
which guarantees network connectivity with probability one as the number of
nodes goes to infinity [91]. The selection of the parameter α allows a tuning
of the node’s covering radius, thus affecting the network connectivity. For each
network configuration, we consider the presence of at least dmax+1 available chan-
125
3.6 Swarming in the Time-Frequency Domain
nels, where dmax is maximum number of neighbors of a node (maximum degree)
in the interference graph. In Fig. 3.13, it is illustrated the average behavior of the
number of iterations needed by two different methods to converge versus number
of nodes composing the network, for different degrees of network connectivity.
The behavior of the swarming algorithm is depicted with dashed lines, whereas
the graph coloring method is plotted using solid lines. As we can notice from
Fig. 3.13, the average number of iterations needed by the graph coloring method
to converge grows increasing the number M of SU’s and the connectivity of the
interference graph (tuned by the parameter α). Interestingly, we notice how, fix-
ing the number of network nodes, the swarming algorithm generally outperforms
the graph coloring method, especially for large and highly connected networks.
Indeed, increasing the connectivity of the interference graph, the graph coloring
performance gets worse whereas the swarming algorithm improves its conver-
gence speed. This positive behavior is due to the attraction force of the swarm,
which actually improves the convergence speed of the algorithm, thus helping the
resource allocation performance.
3.6 Swarming in the Time-Frequency Domain
The proposed swarming procedure can be extended to the two-dimensional
domain, representing, for instance, the time-frequency plane. In this case, the vec-
tor xi referred to node i has two entries representing the position of the frequency
subchannel and the time slot that node i intends to occupy. Before occupying
the slot, each secondary node interacts with its neighbors exchanging information
on the intended slot. Every node then updates its intended position according
to (3.3), implemented in discrete-time. In the application at hand, there is an
intrinsic quantization of the time-frequency resources given by the subchannel
bandwidth and by the duration of the time slot. In our implementation, we let
the system evolve according to (3.3) until successive differences in allocation be-
come smaller than the discretization step in the time-frequency domain. At that
126
3.6 Swarming in the Time-Frequency Domain
0 5 10 15 20 25 30 350
5
10
15
20
25
30
Initial positions
Evolution
Final positions
center of the swarm
Figure 3.14: Example of 2D allocation, considering a quadratic profile.
point, the evolution stops and every SU is allowed to transmit over the selected
resource, i.e. a pair of frequency subchannel/time slot. A major difference with
respect to the frequency domain is that the allocation over successive time slots
requires the knowledge of the primary users’ activities through time. Of course,
this noncausal knowledge of the future is not available. However, if we have a
statistical model of the interference activity, we may derive a resource alloca-
tion mechanism, based on our statistical model. In this section we provide some
examples of swarming in the time-frequency plane. We consider first the ideal
case of known interference profile, which can represent a limit case of a static
environment. Then, we will consider the more realistic case in which the PU’s
activity is modeled as a Markov chain.
3.6.1 Swarming in a Static Interference Environment
In the first example we provide numerical support to Theorem 3. In Fig.
3.14 we show the two-dimensional evolution of 15 agents constituting the swarm,
considering the presence of a quadratic profile having a global minimum (Aσ > 0)
127
3.6 Swarming in the Time-Frequency Domain
0 2 4 6 8 10
4
4.5
5
5.5
6
6.5
7
7.5
8
8.5
9
9.5
Aσ
||∆
||||∆||
Upper Bound − γ
Lower Bound − β
Figure 3.15: Swarm size versus the magnitude of the quadratic profile Aσ.
at the center of the plane. Each agent interacts with its neighbors according
to the depicted topology and updates its intended position according to (3.3),
implemented in discrete-time. The initial guesses of the agents are represented
as blue dots, scattered randomly across the time-frequency plane. The evolution
is depicted by the red curves and the final allocations are given by the violet dots.
The profile magnitude Aσ is equal to 1 and the attraction/repulsion constants are
cA = 1 and cR = 5. It is evident how the swarm moves toward the minimum of
the profile avoiding collisions among nearby agents. At convergence, the swarm
exhibits an equilibrium configuration characterized by a norm of the distance
vector ‖∆‖ = 7.67, which falls inside the interval constituted by the theoretical
lower and upper bounds, respectively β = 4.45 and γ = 9.22. To verify our
theoretical findings, on Fig. 3.15, we report the behavior of the swarm size
parameter ‖∆‖ and its theoretical bounds in (3.55), versus the magnitude Aσ
of the quadratic profile. The results have been averaged over 100 independent
realizations. The swarm parameters are cA = 1 and cR = 5. As we can see, the
effect of an increment of the profile magnitude reduces the swarm size and the
swarm parameter ‖∆‖ remains always inside the theoretical bound interval. The
128
3.6 Swarming in the Time-Frequency Domain
0 2 4 6 8 10
4
4.5
5
5.5
6
6.5
7
7.5
8
8.5
9
9.5
Aσ
||∆
||
||∆||
Upper Bound − γ
Lower Bound − β
Figure 3.16: Swarm size versus the magnitude of the quadratic profile Aσ.
increment of the network connectivity implies tighter bounds on the final swarm
size and the presence of a unique equilibrium point in case of a fully connected
network. This behavior is shown in Fig. 3.16, where we repeat the previous
simulation increasing the coverage radius of each node and, as a consequence, the
connectivity of the network graph. A second example is reported in Fig. 3.17.
The boxes in Fig. 3.17 represent the power allocation of primary users, in the
time-frequency domain, supposed to be static and known, within the convergence
time. These are the regions that do not have to be occupied by the SUs. The
initial guesses of the SUs are represented as squares, scattered randomly across
the time-frequency plane. The evolution of the resource allocation is depicted
by the dotted curves and the final allocations are given by dots. It is evident
how the SUs avoid the positions occupied by the primary users, tend to keep the
spread in the time-frequency plane as small as possible and, at the same time,
they avoid collisions with each other. The previous examples refer to the idealistic
case where the interference is assumed to be static and known, at least within
the convergence time of the swarming algorithm. Next section generalizes the
approach to the case where the interference is known only in a statistical sense.
129
3.6 Swarming in the Time-Frequency Domain
−5 0 5 10 15 20 25 30 35
0
5
10
15
20
25
30
Frequency Axis
Tim
e A
xis
Figure 3.17: Example of time-frequency allocation.
3.6.2 Swarming in the Presence of Markovian Interference
In this section, we model the interference activity, over each frequency sub-
channel, as a two-state continuous-time homogeneous Markovian chain. The
two (on/off) states refer to the cases where the interferer is transmitting or is
idle. This model gives a map of the expected interference power over the time-
frequency plane, conditioned to the measurement performed in the initial time
slot. This expected profile is then used by the swarm algorithm to allocate re-
sources according to the assumed statistical model.
Let us denote by
Qk =
(
−λk µk
λk −µk
)
(3.81)
the transition matrix of the interferer over the k-th frequency subchannel. At each
time t, each subchannel can be either idle or active. Let us denote by Pk0(t) and
Pk1(t) the probabilities that subchannel k, at time t, is idle or active, respectively.
Let us also introduce the probability vector Pk(t) := [Pk0(t),Pk1(t)]T . Given the
130
3.6 Swarming in the Time-Frequency Domain
vector Pk(0) at time 0, the vector probability Pk(t) at time t is
Pk(t) = eQk tPk(0), (3.82)
where
eQk t =
(
1− λkλk+µk
(
1− e−(λk+µk)t) µk
λk+µk
(
1− e−(λk+µk)t)
λkλk+µk
(
1− e−(λk+µk)t)
1− µkλk+µk
(
1− e−(λk+µk)t)
)
. (3.83)
Let us suppose that, on each channel, it is known, through preliminary estimation,
the average power pk and the transition rates from idle to idle λk and from active
to active µk. Let us denote by pk(t; 0) the expected power on channel k, at time
t, conditioned to the knowledge of the channel status at time 0.
Suppose now that at time 0, the channel k is sensed as idle. The probability
vector that, at time t > 0, the channel will be either idle or busy is then
Pk(t) =
(
1− λkλk+µk
(
1− e−(λk+µk)t)
λkλk+µk
(
1− e−(λk+µk)t)
)
. (3.84)
Conversely, if, at time 0, the channel is sensed as active, the probability vector
at time t is
Pk(t) =
(
µkλk+µk
(
1− e−(λk+µk)t)
1− µkλk+µk
(
1− e−(λk+µk)t)
)
. (3.85)
Hence, the expected power, at time t on channel k can be expressed as
pk(t; idle at 0) =λk pkλk + µk
(
1− e−(λk+µk)t)
, (3.86)
pk(t; active at 0) = pk −µkpkλk + µk
(
1− e−(λk+µk)t)
. (3.87)
The expressions (3.86) and (3.87) model the time evolution of the expected in-
terference power over each subchannel. This model allows the construction of
a map of the expected interference distribution, conditioned to the estimation
performed in the first slot. This profile is then used by the swarm to dynamically
allocate power over the time-frequency plane. As an example, Fig. 3.18 shows
the evolution of the swarm in the time-frequency plane. The grey level represents
131
3.7 Discrete-Time Distributed Optimization
Frequency Axis
Tim
e A
xis
0 5 10 15 20
0
2
4
6
8
10
12
14
16
18
20
1
2
3
4
5
6
7
8
9
Figure 3.18: Example of time-frequency allocation with Markovian interference.
the expected interference profile. At time 0, there are two disjoint sub-bands with
a high level of interference and three interference-free regions. The evolution of
the (expected) interference in time is given by (3.86), in case the channel at time
0 is sensed as idle, or by (3.87), in case the channel is sensed as busy. The initial
guesses of the SU’s are represented as squares, scattered randomly across the
time-frequency plane. The evolution of the resource allocation is depicted by the
dotted curves and the final allocations are given by dots. This figure shows that
the swarm tends to keep the spread in the time-frequency plane as limited as
possible while avoiding, at the same time, collisions. Interestingly, the swarm
tends to stay as close as possible, to time 0, where the prediction is better, under
the constraint of avoiding collisions.
3.7 Discrete-Time Distributed Optimization
The swarm evolution (3.3) has been described, so far, mainly in continuous
time. In practice, the implementation is generally performed in discrete-time.
Our goal in this work is the distributed minimization of (3.2). In general, the
132
3.7 Discrete-Time Distributed Optimization
objective function in (3.2) is not convex in the resource allocation vector x and,
as a consequence, the problem may have multiple local optima. A local solution
can be found in a centralized manner using standard optimization algorithms.
However, we focus on distributed solutions where it is allowed a local coordination
among SUs through a limited exchange of data. A possible way to achieve the
solution in decentralized form is to use a simple gradient based optimization, so
that every node starts with an initial guess, let us say xi[0], and then it updates
its own resource allocation vector xi[k] in time according to the following discrete-
time recursion:
xi[k + 1] = xi[k]− αi[k]∇xiJ(x) (3.88)
= xi[k]− αi[k]
∇xiIi(xi[k])−M∑
j=1
aij g(xj[k]− xi[k])
,
i = 1, . . . ,M , where k is the time index, and αi[k] is a positive iteration-dependent
step-size. In this section, we consider a coupling function g(·) with linear attrac-
tion (3.7) and bounded repulsion (3.10). Under these conditions on the coupling
function g(·), the potential function (3.2) can be written as:
J(x) =
M∑
i=1
Ii(xi) +cA2
M∑
i=1
M∑
j=1
aij‖xj − xi‖2 +
+ cGcR2
M∑
i=1
M∑
j=1
aij exp
(
−‖xj − xi‖
2
cG
)
=M∑
i=1
Ii(xi) +cA2
M∑
i=1
M∑
j=1
aij‖xj − xi‖2
+ cGcR2
M∑
i=1
M∑
j=1
aijexp
(
−‖xj−xi‖2
cG
)
‖xj − xi‖2‖xj − xi‖
2
= tr(Σ(x)) + cAxT (L⊗ In)x+ cGcRx
T (Lr,x ⊗ In)x (3.89)
133
3.7 Discrete-Time Distributed Optimization
where tr(·) denotes the trace of a matrix. The matrices Σ(x) and Lr,x write as
Σ(x) = diag(I1(x1), . . . , IM (xM )) (3.90)
Lr,x = Dr,x −Ar,x (3.91)
where Ar,x denotes a symmetric state dependent adjacency matrix whose entries
are given by
[
Ar,x
]
ij=
aij‖xj−xi‖2
exp
(
−‖xj−xi‖2
cG
)
,
[
Ar,x
]
ii= 0
(3.92)
In our setup, the foraging profiles Ii(xi) represent the interference power dis-
tribution on the resource domain, hence, the term tr(Σ(x)) is always positive.
Moreover, the matrix Lr,x is positive semi-definite for all x implying that the
corresponding quadratic form in (3.89) is always grater than or equal to zero,
thus leading to J(x) ≥ 0 for all x. Furthermore, considering the choice of a
coupling function g(·) with linear attraction (3.7) and bounded repulsion (3.10)
and resorting to the assumptions A.1-A.2, the potential function J(x) is also
continuously differentiable, with continuous second-order partial derivatives. We
are now able to state the convergence result on the discrete time procedure (3.88).
Theorem 11 Consider the discrete-time swarming algorithm in (3.88) with ar-
bitrary initial state x[0]. Under the assumptions A.1-A.2 and considering a
coupling function g(·) with linear attraction (3.7) and bounded repulsion (3.10),
the algorithm converges to a stationary point of the function J(x) in (3.89), as
k →∞, if the step size α is chosen such that
0 < α <2
L, (3.93)
where L is the Lipschitz constant of the potential function J(x) in (3.89).
Proof. As shown before, the potential function J(x) in (3.89) is continuously
differentiable (Lipschitz-continuous) and bounded from below. Since we are min-
imizing this function using a descent algorithm, Theorem 1 applies and the con-
vergence result follows.
134
3.7 Discrete-Time Distributed Optimization
3.7.1 Projected Swarming Algorithms
Up to now, we have considered an unconstrained distributed optimization
of the global potential function (3.2). In our derivations, we assumed the opti-
mization set to be compact. In our application, the resource allocation domain,
either a frequency band or a time interval (or both), is always a compact set.
The incorporation of the frequency and/or time interval limits in our problem
can be done either imposing box constraints on our optimization or by adding
a barrier to the interference profiles I(xi), for all i, i.e., a positive continuous
term that starts from the boundary of the resource domain and goes to infinity
linearly, with constant derivative σ, in order to keep satisfying Assumptions A.1
and A.2. In this section, we consider the first approach, imposing box constraints
on the values that the resource allocation vector can assume.
Let us consider the following constrained optimization problem:
minx
J(x)
s.t. xil 4 xi 4 x
iu i = 1, . . . ,M
(3.94)
where x = (xT1 , . . . ,x
TM )T , and 4 denotes component-wise inequality, e.g., a 4 b
if ai ≤ bi, ∀i = 1, . . . , n. The box constraints in (3.94) denote a frequency band
or a time interval (or both) where the i-th cognitive SU can allocate its resources.
As before, the objective function in (3.2) is not concave in the resource allocation
vector x and, as a consequence, the problem may have multiple local optima.
A local optimum x∗ = [x∗1T , . . . ,x∗
MT ]T of problem (3.2) is a regular point 2
and, as a consequence, it satisfies the Karush-Kuhn-Tucker (KKT) conditions
[188]. In particular, we focus on distributed solutions where it is allowed a local
coordination among SU’s through a limited exchange of data. The problem is
amenable for distributed solutions because the optimization set X =∏M
i=1Xi is
given by Cartesian product of sets Xi, allowing the parallel computation of the
algorithm. Then, a possible way to achieve the solution in decentralized form
2A feasible point is said to be regular if the equality constraints gradients and the active
inequality gradients are linearly independent [188].
135
3.7 Discrete-Time Distributed Optimization
is to use a gradient projection optimization, so that every node starts with an
initial guess, let us say xi(0), and then it updates its own resource allocation
vector xi(k) in time according to the following discrete-time dynamical system:
xi[k + 1] = [xi[k] + α∇xiJ(x[k])]Xi= T i(x[k]), (3.95)
k ≥ 0, i = 1, . . . ,M , where [·]Xi denotes the projection over the feasible set Xi, α
is the step size, and ∇xiJ(x[k]) ∈ Rn is the discrete-time version of (3.3). We are
now able to state the convergence result on the discrete time procedure (3.95).
Theorem 12 Consider the discrete-time swarming algorithm in (3.95) with ar-
bitrary initial state x[0] and let x[k] be the sequence generated by it. Further-
more, consider the assumptions A.1-A.2 and a coupling function g(·) with linear
attraction (3.7) and bounded repulsion (3.10). Then, by selecting the step-size as
0 < α <2
L, (3.96)
where L is the Lipschitz constant of the potential function J(x) in (3.89), if x∗
is an accumulation point of the sequence x[k], the optimal local solution x∗ is
a fixed point of the mapping T (x) = colT i(x)Mi=1, such that x∗ = T (x∗).
Proof. As shown before, the potential function J(x) in (3.89) is continuously
differentiable (Lipschitz-continuous) and bounded from below. Since we are min-
imizing this function using a projection-based descent algorithm, Theorem 2 ap-
plies and the convergence result follows.
In this section, we showed the convergence properties of discrete-time swarming
algorithms, minimizing the global cost function (3.2) in a distributed fashion
using descent approaches. In the next section, we will consider the effect that
realistic channels have on the proposed resource allocation technique based on
swarming, showing how to handle the randomness introduced by fading and noise
in order to ensure the convergence of the proposed techniques.
136
3.8 The Effect of Noise and Realistic Channels
3.8 The Effect of Noise and Realistic Channels
The swarming mechanism studied up to now assumed ideal communications
among the cognitive nodes. However, in a realistic scenario, the wireless channel
is affected by random fading and additive noise, which induce errors in the re-
ceived packets. In such a case, the receiving node could request the retransmission
of the erroneous packets, but this would imply random delays in the communica-
tion among the cognitive nodes and it would be complicated to implement over
a totally decentralized system. It is then of interest to analyze networks where
the erroneous packets are simply dropped. Moreover, the data exchanged among
the nodes is usually quantized using a finite number of bits, and then the effect
of quantization noise on the swarm mechanism should be properly taken into
account. The goal of this section is to extend the swarm-based resource assign-
ment mechanism to the more realistic case where the packets exchanged among
the cognitive nodes, while running the swarm mechanism, are randomly dropped
and the transmitted data are encoded with a finite number of bits.
Several swarm models have been analyzed in the control literature, see, e.g.
[120]- [125], in the case where the graph describing the interaction among the
swarm individuals varies with time, thus inducing a switching topology. In [122]
the authors showed that, if the network graph is always connected, a stable flock-
ing motion can be achieved by using a set of switching control laws given by a
combination of attractive/repulsive and alignment forces. Reference [123] consid-
ered a swarm model affected by a switching topology and proved the convergence
of the swarm to a common velocity vector and the stabilization of inter-agent
distances, regardless of switching, as long as the network remains connected all
the time. The cohesiveness of a hybrid swarm model, suitable to describe swarm
aggregation with limited sensing ability, was also analyzed in [124] taking into
account switching topologies. The flocking behavior of multi-agent systems with
switching topology in a noisy environments was considered in [125], where it was
shown that, although the information is contaminated by noise, all agents can
137
3.8 The Effect of Noise and Realistic Channels
form and maintain the flocking behavior if the gradient of the environment is
bounded and the interaction graph is jointly connected.
The effect of random graphs on consensus algorithms has been thoroughly
studied in a series of works, such as [137]- [141], which focused on the conver-
gence of consensus protocols in the presence of random disturbances. In [137],
the authors use a decreasing sequence of weights to prove the convergence of
consensus protocols to an agreement space in the presence of additive noise un-
der a fixed network topology. A distributed consensus algorithm in which the
nodes utilize probabilistically quantized information to communicate with each
other was proposed in [138]. As a result, the expected value of the consensus
is equal to the average of the original sensor data. A stochastic approximation
approach was followed in [139], which considered a stochastic consensus problem
in a strongly connected directed graph where each agent has noisy measurements
of its neighboring states. The study of a consensus protocol that is affected by
both additive channel noise and a random topology was considered in [141]. The
resulting algorithm relates to controlled Markov processes and the convergence
analysis relies on stochastic approximation techniques. In this work, inspired by
these recent results, we propose a decentralized algorithm to solve the resource as-
signment problem in the presence or random disturbances (such as fading, noise,
and quantization) and we prove its convergence in the presence of random link
failures and quantization noise.
The section is organized as follows. First, we briefly recall some basic concepts
from random link failures model and dithered quantization that will be used
throughout the section. Then, we will demonstrate the stochastic convergence
of the swarming mechanisms previously introduced. We also provide simulation
examples corroborating the theoretical results and showing the effect of the radio
channel impairments on the performance.
138
3.8 The Effect of Noise and Realistic Channels
3.8.1 Random Link Failures
In a realistic communication scenario, some packets may be lost at random
times. To account for this fact, we allow the links among the network nodes to
fail at some probability, inducing a time-varying, or switched, network topology,
depending on the link failures. In this case, we model the network at time k as
an undirected graph, G[k] = V,E[k] where the graph Laplacians is taken as a
sequence of i.i.d. matrices L[k] of the form:
L[k] = L+ L[k] (3.97)
where L denote the mean matrix and L[k] are i.i.d. perturbations around the
mean. We do not make any assumptions on the link failure model. Although
the link failures and the Laplacians are independent over time, during the same
iteration, the link failures can still be spatially correlated (even during the same
iteration). Moreover, connectedness of the graph is an important issue. We do
not require the random instantiations G[k] of the graph be connected for all k; We
only require that the graph is connected on average. This condition is captured by
having the second eigenvalue of the expected Laplacian matrix strictly positive,
i.e., λ2(L) > 0.
3.8.2 Dithered Quantization
We assume that each inter-node communication channel uses a uniform quan-
tizer, which is defined by the following vector mapping, q(·) : Rn → Qn,
q(y) = [b1∆, . . . , bn∆]T = y + e(y), (3.98)
where the entries of the vector y, the quantization step ∆ > 0, and the error e
satisfy
(bm − 1/2)∆ ≤ ym ≤ (bm + 1/2)∆, 1 ≤ m ≤ n,
−∆/2 1n ≤ e(y) ≤ ∆/2 1n, for all y.(3.99)
139
3.8 The Effect of Noise and Realistic Channels
The quantization alphabet is
Qn = [b1∆, . . . , bn∆]T |bm ∈ Z,∀m. (3.100)
Conditioned on the input, the quantization error e(y) is deterministic. This
strong correlation of the error with the input influences the statistical properties
of the error and can influence the convergence of the algorithm. To avoid these
effects, we consider dithered quantization [175]- [176], which endows the quanti-
zation error with some useful statistical properties. The dither added to random-
ize the quantization effects satisfies a special condition, namely the Schuchman
conditions, as in subtractively dithered systems, see [177]. Then, at every time
instant k, adding to each component ym[k] a dither sequence νm[k]k≥0 of i.i.d.
uniformly distributed random variables on [−∆/2,∆/2) independent of the input
sequence, the resultant error sequence ǫm[k]k≥0 becomes
ǫm[k] = q(ym[k] + νm[k])− (ym[k] + νm[k]). (3.101)
The sequence ǫm[k]k≥0 is now an i.i.d. sequence of uniformly distributed ran-
dom variables on [−∆/2,∆/2), which is independent of the input sequence.
Thus, by randomizing the input to the uniform quantizer, we can render the
quantization error to be independent of the input and uniformly distributed on
[−∆/2,∆/2).
3.8.3 Stochastic Convergence
In this section, using stochastic approximation arguments, we demonstrate
the convergence properties of the proposed swarming algorithms impaired by
random link failures, quantization and noise.
Basic Swarming Algorithm
In this section we reformulate the swarming problem as the search for the zeros
of a deterministic function, whose value is corrupted by random disturbance and
140
3.8 The Effect of Noise and Realistic Channels
can be observed at each time instant. We will provide conditions for the almost
sure convergence of the search procedure. For our subsequent derivations, we
consider the assumptions A.1-A.2, and the following condition.
Assumption A.3 : To ensure the swarm cohesion and, hence, a finite swarm
size, we assume that the graph describing the network topology is connected. As
we will see, due to the random nature of the network graph, we only require a
topology connected on average. This condition is captured by having the second
eigenvalue of the expected Laplacian strictly positive, i.e. λ2(L) > 0.
In an ideal case communication case, since the function (3.2) is Lipschitz continu-
ous and bounded from below, it is possible to prove the convergence of a discrete-
time gradient algorithm by using a classical descent approach [189]. Then, by
selecting the step-size of the algorithm sufficiently small (smaller than the inverse
of the Lipshitz constant), the discrete-time version of the iterative procedure (3.3)
will asymptotically converge to a local minimum of the potential function (3.2).
However, in an imperfect communication scenario, where the network links may
fail randomly and communication is corrupted by quantization noise, the nodes
will have access to a random subset of their neighbors and the received data will
likely be corrupted. Furthermore, each node needs to estimate the interference
profile Ii(xi) or at least its gradient for use in (3.3). Estimating this gradient vec-
tor will be subject to errors and we therefore denote the estimate of the gradient
by:
∇xi Ii(xi) = ∇xiIi(xi) + ηi (3.102)
where ηi is a zero mean i.i.d. vector noise sequence of bounded variance. Under
these non-ideal conditions, the convergence of (3.3) to a local minimum is not
assured and the swarming algorithm needs to be adjusted in order to handle the
data imperfections.
The swarm evolution (3.3) has been mainly described, so far, in continuous
time. In practice, the implementation is generally performed in discrete-time.
141
3.8 The Effect of Noise and Realistic Channels
A discrete time version of (3.3) that accounts for random link failures, dithered
quantization noise and estimation errors, can be written as:
xi[k + 1] = xi[k] + α[k]
[
−∇xi[k]Ii(xi[k]) − ηi[k] + (3.103)
+
M∑
j=1
aij [k] g(q(xj [k] + νij [k])− xi[k])
]
, i = 1, . . . ,M,
where α[k] is a positive iteration dependent step-size. Now, exploiting the feature
of subtractively dithered systems in (3.101), the previous expression is given by:
xi[k + 1] = xi[k] + α[k]
[
−∇xi[k]Ii(xi[k])− ηi[k] + (3.104)
+
M∑
j=1
aij[k] g(xj[k]− xi[k] + νij[k] + ǫij [k])
]
, i = 1, . . . ,M.
Starting from some initial position in the resource domain, xi[0] ∈ Rn, each node
generates via (3.104) a sequence of resource allocations, xi[k]k≥0. The position
xi[k + 1] at the i-th node at time k + 1 is a function of: its previous position;
the communicated quantized postions at time k of its neighboring sensors; and
the new estimate of the profile gradient ∇xi[k]I(xi[k]). As described in Section
II-B, the data is subtractively dithered quantized, such that the quantized data
received by the i-th sensor from the j-th sensor at time k is q(xj[k] + νij [k]).
It then follows from the discussion in Section II-B that the quantization error
ǫij [k] is a random vector, whose components are i.i.d., uniformly distributed on
[−∆/2,∆/2), and independent of xj[k]. In the presence of small quantization
noise, we can appeal to a first-order Taylor approximation of the vector function
g(·), and approximate the updating rule (3.104) as:
xi[k + 1] ≃ xi[k] + α[k]
[
−∇xi[k]Ii(xi[k]) +M∑
j=1
aij [k] g(xj [k]− xi[k]) +
− ηi[k] +
M∑
j=1
aij[k] Jg(xj [k]− xi[k])(ν ij[k] + ǫij [k])
]
,
i = 1, . . . ,M, (3.105)
142
3.8 The Effect of Noise and Realistic Channels
where Jg(xj [k]−xi[k]) is the Jacobian matrix of g(·) evaluated at (xj[k]−xi[k]).
Now, exploiting the structure of the function g(·) in (3.4) and the features of linear
attraction in (3.7) and bounded repulsion in (3.8), the recursion (3.105) can be
expressed as:
xi[k + 1] ≃ xi[k] + α[k]
[
−∇xi[k]Ii(xi[k]) +
+
M∑
j=1
aij [k]
[
cA − cR exp
(
−‖xj [k]− xi[k]‖
2
cG
)]
(xj [k]− xi[k]) +
− ηi[k] +
M∑
j=1
aij [k] Jg(xj [k]− xi[k])(ν ij[k] + ǫij [k]),
i = 1, . . . ,M. (3.106)
To rewrite (3.106) in compact form, we introduce the random vectors Υx[k] and
Ψx[k] ∈ RnM with vector components
[
Υx[k]]
i=
M∑
j=1
aij [k]Jg(xj [k]− xi[k])ν ij[k], (3.107)
[
Ψx[k]]
i=
M∑
j=1
aij [k]Jg(xj [k]− xi[k])ǫij[k]. (3.108)
The vectors Υx[k] and Ψx[k] are the state dependent aggregated contribution of
quantization and dithering. It follows from the conditions on the dither, that
E[Υx[k]] = E[Ψx[k]] = 0, ∀k,
supx,k
[
‖Υx[k]‖2]
= supx,k
[
‖Ψx[k]‖2]
≤M(M − 1)n∆2λmax(Jg)
12,
from which we get
supx,k
[
‖Υx[k] +Ψx[k]‖2]
≤ 2 supx,k
[
‖Υx[k]‖2]
+ 2 supx,k
[
‖Ψx[k]‖2]
≤M(M − 1)n∆2λmax(Jg)
3= ζq. (3.109)
143
3.8 The Effect of Noise and Realistic Channels
with λmax(Jg) = maxx λmax(JTg (x)Jg(x)) > 0. To prove the validity of the
bound in (3.109), we have still to show that λmax(Jg) is finite or upper bounded
by a finite value. The choice of the coupling function g(·) in (3.7)-(3.9), with
linear attraction and bounded repulsion, ensures the boundedness of the partial
derivatives in Jg(x). In the scalar case, this is straightforward to see. Indeed, in
this case, we have
g(y) = [cA − cR exp(−y2/cG)]y (3.110)
⇒ g′(y) = cA − cR exp(−y2/cG)(1− 2y2/cG), (3.111)
and it is easy to see that g′(y) can assume only bounded values. In the vector case,
at the same way, we can guarantee that the elements of Jg(x) take values from
a finite set. Then, as a consequence of the Gershgorin circles theorem [178], the
eigenvalues of the matrix JTg (x)Jg(x) assume bounded values. Let Jmax denote
the maximum value (in modulus) assumed by the elements of JTg (x)Jg(x), ∀
x. Then, by the Gershgorin theorem, an upper bound for λmax(Jg) is given by
n2M2J2max.
The overall evolution dynamics can then be expressed in compact form as:
x[k + 1] = x[k] + α[k][
−Σ′(x[k])−(
Lx[k]⊗ In)
x[k] +Υx[k] +
+ Ψx[k]−Ξ[k]]
, (3.112)
where Υx[k] and Ψx[k] are the state dependent aggregated contribution of quan-
tization noise in (3.107) and (3.108), Σ′(x[k]) = col[∇xi[k]Ii(xi[k])]i=1,...,M , and
Ξ[k] = col[ηi[k]]i=1,...,M is the overall estimation noise vector. The state-dependent
Laplacian matrix is given by
Lx[k] =Dx[k]−Ax[k] (3.113)
whereAx[k] denotes a symmetric state-dependent adjacency matrix whose entries
are given by
[
Ax[k]]
ij= aij [k]
[
cA − cR exp(
−‖xj [k]−xi[k]‖2
cG
)]
,[
Ax[k]]
ii= 0.
(3.114)
144
3.8 The Effect of Noise and Realistic Channels
We consider four assumptions on the stochastic procedure (3.112):
Assumption B.1 : (Estimation noise) We assume that the observation noise
process Ξ[k] = colηi[k]i=1,...,M in (3.102) is an i.i.d. zero mean process, with
finite second order moment, i.e.,
E[Ξ[k]TΞ[k]] ≤ ϕe, for all k. (3.115)
Assumption B.2 : (Independence) The sequences Lx[k]k≥0, Υx[k]k≥0,
Ψx[k]k≥0 and Ξ[k]k≥0 are mutually independent.
Assumption B.3 : (Markov) Consider the filtration Fxk k≥0, given by
Fxk = σA
(
x(0), Lx[n],Υx[n],Ψx[n],Ξ[n]0≤n<k
)
(3.116)
where σA(·) denotes sigma algebra. It then follows that the random quantities
Lx[k], Υx[k], Ψx[k] and Ξ[k] are independent of Fxk , implying that x[k],Fx
k k≥0
is a Markov process.
Assumption B.4 : (Persistence) To obtain convergence, we assume that the
step size α[k] satisfies the following conditions:
α[k] > 0,
∞∑
k=0
α[k] =∞,∞∑
k=0
α2[k] <∞. (3.117)
Condition (3.117) ensures that the step-size decays to zero, but not too fast. An
example of step-size sequence that satisfies (3.117) is
α[k] =α0
(k + 1)β, α0 > 0, 0.5 < β ≤ 1. (3.118)
The following theorem presents a classical result from stochastic approxima-
tion theory from [183] regarding the convergence properties of generic stochastic
recursive procedures; the result will be used to establish the convergence of the
swarming algorithm.
145
3.8 The Effect of Noise and Realistic Channels
Theorem 13 Let x[k]k≥0 be a random vector defined by the difference equation
x[k + 1] = x[k] + α[k][
R(x[k]) + Γ(k,x[k], ω)]
(3.119)
with initial condition x[0] = x0, where R(·) : Rn → R
n is Borel-measurable,
Γ(k,x[k], ω) is a family of zero-mean random vectors in Rn, defined on some
probability space (Ω,F ,P), and ω ∈ Ω is a canonical element of Ω. Consider the
following set of conditions:
Condition C.1 : The function Γ(k, ·, ·) : Rn ×Ω→ Rn is Bn ⊗F measurable 3
for all k.
Condition C.2 : There exists a filtration Fkk≥0 of F , such that, for every
k, the family of random vectors Γ(k,x[k], ω)x∈Rn is Fk measurable and indepen-
dent of Fk−1. (Under the conditions C.1 and C.2, the random vector sequence
x[k]k≥0 is a Markov process.)
Condition C.3 : There exists a nonnegative function V (x) ∈ C2 with bounded
second-order partial derivatives satisfying the conditions
lim‖x‖→∞
V (x) = ∞ (3.120)
supx∈Uǫ,1/ǫ(S)
< R(x),∇xV (x) > < 0 for ǫ > 0 (3.121)
where < ·, · > denotes the inner product operator, S = x : R(x) = 0 is the
solution set and Uǫ,1/ǫ(S) = x ∈ Rn : ǫ < ‖x− xs‖ < 1/ǫ,xs ∈ S, ǫ > 0.
Condition C.4 : There exist a constant K > 0, such that,
‖R(x)‖2 + E‖Γ(k,x, ω)‖2 ≤ K(1 + V (x)). (3.122)
Condition C.5 : The step size sequence α[k]k≥0 satisfies (3.117).
Let the conditions C.1-C.5 hold for the process x[k]k≥0. Then, x[k]k≥0 is a
Markov process and, starting from an arbitrary initial condition x0, it converges
3B
n denotes the Borel algebra of Rn.
146
3.8 The Effect of Noise and Realistic Channels
almost surely (a.s.) as k → ∞, either to a point of the solution set S = x :
R(x) = 0, or to the boundary of one of its connected components.
Proof. The proof can be found in [183] (Theorem 5.2.3).
In the following, we will use Theorem 1 to establish the a.s. convergence of
the iterative swarming procedure (3.112). By decomposing the state dependent
Laplacian matrix Lx[k] into the sum of a mean part plus a random part as in
(3.97), expression in (3.112) can be written as:
x[k + 1] = x[k] + α[k][
−Σ′(x[k])−(
Lx ⊗ In)
x[k]−(
Lx[k]⊗ In)
x[k] +
+ Υx[k] +Ψx[k]−Ξ[k]]
. (3.123)
In the notation of Theorem 1, equation (3.123) can be recast as in (3.119), where
R(x[k]) = −Σ′(x[k])−(
Lx[k]⊗ In)
x[k], (3.124)
Γ(k,x[k], ω) = −(
Lx[k]⊗ In)
x[k] +Υx[k] +Ψx[k]−Ξ[k]. (3.125)
In this way, the original swarming problem has been converted into the search
for the zeros of the deterministic function R(x[k]), whose value is measurable
at each time instant k and corrupted by an additive zero-mean random distur-
bance Γ(k,x[k], ω). We are now able to state the main theorem of the swarming
behavior in the presence of random disturbances.
Theorem 14 Consider the discrete swarming algorithm in (3.104) with arbitrary
initial state x0. Under the hypothesis of a small additive quantization noise and
the assumptions A.1-A.3 and B.1-B.4, the algorithm converges almost surely
(a.s.) as k →∞ to one of the zeros of the function R(x) in (3.124) or, equiva-
lently, to a local minimum of the function J(x) in (3.2) evaluated for the expected
graph. Then,
P
[
limk→∞
ρ(x[k], S) = 0
]
= 1 (3.126)
where ρ(·) is the standard Euclidean metric norm and S = x : R(x) = 0 is the
solution set.
147
3.8 The Effect of Noise and Realistic Channels
Proof. The proof follows by showing that the process x[k]k≥0, generated by
the swarming algorithm, satisfies the conditions C.1-C.5 of Theorem 1. Recall
the filtration given in equation (3.116). Under the Assumptions B.1-B.4, the
random family Γ(k + 1,x[k], ω) is Fxk+1 measurable, zero mean and independent
of Fxk . As a consequence, the conditions C.1, C.2 of Theorem 2 are satisfied
and the random vector sequence x[k]k≥0 is a Markov process. We will show
now the existence of a stochastic potential function V (x) such that the swarming
algorithm in (3.112) satisfies the conditions C.3, C.4. To this end, we define
V (x) = tr(Σ(x)) + cAxT (L⊗ In)x+ cGcR x
T (Lr,x ⊗ In)x (3.127)
which coincides with the system potential function in (3.89) evaluated for the
expected graph Laplacian L. As shown for the expression in (3.89), V (x) ∈ C2 is
a nonnegative function and, under the profile’s smoothness assumption A.1 and
the choice of attraction and repulsion functions in (3.7) and (??), it has bounded
second order partial derivatives. Under Assumption A.3, the vector x lies on a
subspace orthogonal to N(
L⊗In)
, where N (·) denotes the nullspace of a matrix.
Hence, applying (2.80) and considering the expression of λ(L⊗ In) in (2.83), we
find that the expression (3.127) admits the lower bound:
V (x) ≥ tr(Σ(x)) + cAλ2(L)‖x‖2 + cGcR x
T (Lr,x ⊗ In)x. (3.128)
It is then straightforward to see how the potential function V (x) satisfies condi-
tion (3.120). Since the gradient of V (x) is given by
∇xV (x) = Σ′(x) +(
Lx ⊗ In)
x, (3.129)
it then follows that
< R(x),∇xV (x) > = −[
Σ′(x) +(
Lx ⊗ In)
x]T [
Σ′(x) +(
Lx ⊗ In)
x]
= −‖R(x)‖2 < 0 for all x 6= xs ∈ S. (3.130)
Thus, condition C.3 of Theorem 2 is satisfied. From equation (3.124), applying
148
3.8 The Effect of Noise and Realistic Channels
the Cauchy-Schwartz inequality, we get the following upper bound
‖R(x)‖2 =∥
∥−Σ′(x)−(
Lx ⊗ In)
x∥
∥
2
= ‖Σ′(x)‖2 + 2Σ′(x)T(
Lx ⊗ In)
x+ ‖(
Lx ⊗ In)
x‖2
≤ ‖Σ′(x)‖2 + 2‖Σ′(x)‖‖(
Lx ⊗ In)
x‖+ ‖(
Lx ⊗ In)
x‖2. (3.131)
Under assumption A.1, we have ‖Σ′(x)‖ ≤ σM , and the previous bound can be
recast as
‖R(x)‖2 ≤ σ2M + 2σM |λM (Lx)|‖x‖+ λ2M (Lx)‖x‖2 (3.132)
where λM (Lx) is the maximum (in modulus) eigenvalue of Lx. The state depen-
dent mean Laplacian Lx = Dx−Ax in (3.113) depends on x through a bounded
function. Hence, for any values of the swarm constants cA, cR and cG, it always
exists a constant c1 > 0 such that |λM (Lx)| ≤ c1 and
‖R(x)‖2 ≤ σ2M + 2σMc1‖x‖+ c21‖x‖2
= c2 + c3‖x‖+ c4‖x‖2 (3.133)
where c2 = σ2M > 0, c3 = σMc1 > 0 and c4 = c21 > 0. Now, adding the function
c3/2(1 − ‖x‖)2 ≥ 0, the previous bound gives
‖R(x)‖2 ≤ c2 + c3/2 + (c4 + c3/2)‖x‖2 = c5 + c6‖x‖
2 (3.134)
where c5 = c2 + c3/2 > 0 and c6 = c4 + c3/2 > 0.
From equation (3.125) and the independence assumption B.2,
E‖Γ(k,x, ω)‖2 = E∥
∥−(
Lx[k]⊗ In)
x+Υx[k] +Ψx[k]−Ξ[k]∥
∥
2
= xTE[(
Lx[k]⊗ In)T
(Lx[k]⊗ In)]
x+
+ E‖Υx[k] +Ψx[k]‖2 + E‖Ξ[k]‖2. (3.135)
The eigenvalues of the Laplacian error Lx are bounded because this matrix takes
values from a finite set and it depends on the state x through a bounded function.
149
3.8 The Effect of Noise and Realistic Channels
Then, considering the bounds in (3.109) and (3.115), we get
E‖Γ(k,x, ω)‖2 ≤ E[
λM (Lx[k]T Lx[k])
]
‖x‖2 + ζq + ϕe
= c7 + c8‖x‖2 (3.136)
where c7 = ζq + ϕe > 0 and c8 = maxx E[
λM (LTxLx)
]
> 0. We then have from
(3.134) and (3.136)
‖R(x)‖2 + E‖Γ(k,x, ω)‖2 ≤ c9 + c10‖x‖2 (3.137)
where c9 = max(c5, c7) > 0 and c10 = max(c6, c8) > 0. Exploiting now Assump-
tion A.3, the vector x lies on a subspace orthogonal to N(
L⊗In)
and, under the
assumption that the repulsion force is strong enough to avoid the overall collapse
of the swarm onto the center, the overall consensus over x is never reached. This
means that the inequality 0 < λ2(L)‖x‖2 ≤ xT
(
L ⊗ In)
x holds for all x, and,
substituting it in (3.137), we get
‖R(x)‖2 + E‖Γ(k,x, ω)‖2 ≤ c9 +c10λ2(L)
xT(
L⊗ In)
x
= c9 + c11xT(
L⊗ In)
x, (3.138)
where c11 = c10/λ2(L) > 0. Summing now the potential function V (x) ≥ 0 in
(3.127) to the last expression, we can write the bound
‖R(x)‖2 + E‖Γ(k,x, ω)‖2 ≤ c9 + c11xT(
L⊗ In)
x+ V (x)
≤ c9 + tr(Σ(x)) + (cA + c11)xT(
L⊗ In)
x+ cGcRxT(
Lr,x ⊗ In)
x
= c9 + tr(Σ(x)) + c12cAxT(
L⊗ In)
x+ cGcRxT (Lr,x ⊗ In
)
x
≤ c9 + c12V (x) ≤ K(1 + V (x)) (3.139)
where c12 = (1 + c11/cA) > 1 and K = max(c9, c12) > 0. This verifies also condi-
tion C.4 of Theorem 1 and condition C.5 is satisfied by the choice of α[k]k≥0
made in the B.4. This concludes our proof.
150
3.8 The Effect of Noise and Realistic Channels
Fast Swarming Algorithms
As in the preceding section, we can consider a general scaled gradient opti-
mization in order to improve the performance of the algorithm. Considering the
presence of random disturbances, the discrete time version of a general scaled
gradient optimization can be recast in compact form as:
x[k + 1] = x[k] + α[k]B(x[k])[
−Σ′(x[k])−(
Lx[k]⊗ In)
x[k] +Υx[k]
+ Ψx[k]−Ξ[k]]
. (3.140)
where B(x[k]) = diag(fi(Ii(xi(k)))In)i=1,...,M . In the notation of Theorem 1,
the equation (3.140) can be written as in (3.119) where
R(x[k]) = −B(x[k])
[
Σ′(x[k]) +(
Lx[k]⊗ In)
x[k]
]
(3.141)
Γ(k,x[k], ω) = −B(x[k])
[
(
Lx[k]⊗ In)
x[k]−Υx[k]−Ψx[k] +Ξ[k]
]
(3.142)
The following convergence result holds for the adaptive swarming in (3.140).
Theorem 15 Consider the discrete swarming algorithm in (3.140) with arbitrary
initial state x0. Under the hypothesis of a small additive quantization noise and
the assumptions A.1-A.2 and B.1-B.4, the algorithm converges a.s. as k →∞
to one of the zeros of the function R(x) in (3.142) or, equivalently, to a local
minimum of the function J(x) in (3.2) evaluated for the mean graph. That is
P
[
limk→∞
ρ(x[k], S) = 0
]
= 1 (3.143)
where ρ(·) is the standard Euclidean metric norm and S = x : R(x) = 0 is the
solution set.
Proof. The proof follows the same steps as in Theorem 2. Under assumption
B.3, the sequence generated by the swarming algorithm in (3.140) is a Markov
process. We consider again the nonnegative function V (x) in (3.127). SinceR(x)
in (3.142) is a scaled gradient descent direction for the optimization of V (x), it
151
3.8 The Effect of Noise and Realistic Channels
is easy to show that the Lyapunov condition in (3.121) is always verified for all
x outside the solution set S. Applying now the assumptions A.1, A.3, B.1 and
B.2 and considering the boundedness of the eigenvalues of the diagonal matrix
B(x), some algebra shows that the inequality in (3.122) holds, thus concluding
the proof.
Remark: The matrix B(x) in (3.140) is a full rank matrix. Hence, the zeros of
the function R(x) in (3.142) coincide with those of the function in (3.124).
In the following sections, we will present numerical results illustrating how the
proposed algorithm (3.140) outperforms the basic swarming algorithm (3.123) in
terms of convergence speed and resilience against channel imperfections.
Projected Methods
In the previous section, we showed how the gradient of the sum-rate is affected
by the randomness introduced by the link failures and by the quantization error
present on the data exchanged between SUs. In this section, we consider the dis-
tributed solution of the constrained optimization problem (3.94), which leads to
a projected swarm method. To find a solution of the problem (3.94) affected by
random disturbances, stochastic approximation algorithms assume relevance. In
the remainder of this section we introduce a stochastic approximation scheme for
solving the problem in (3.94) in a distributed manner. In particular, we consider
a projection-based Robbins-Monro (RM) stochastic approximation procedure.
The problem is amenable for distributed solutions because the optimization set
X =∏M
i=1Xi is given by Cartesian product of sets Xi, allowing the parallel com-
putation of the algorithm. In particular, we consider a projection-based swarming
algorithm where at each time k, every SU simultaneously updates its resource
allocation vector according to
x[k + 1] = [x[k] + α[k]∇xJ(x[k])]X = T (x[k]), (3.144)
k ≥ 0, q = 1, . . . , Q,
152
3.8 The Effect of Noise and Realistic Channels
where α[k] is an iteration-dependent step size, and ∇xJ(x[k]) ∈ RnM is the
discrete-time version of (3.3). As shown in the previous section, due to the effect
of the random link failures and quantization, the gradient of the potential function
in (3.89) can be written as:
∇xJ(x[k]) = Σ′(x[k])−(
Lx[k]⊗ In)
x[k] +Υx[k] +Ψx[k]−Ξ[k]
= R(x[k]) + Γ(k,x[k], ω), (3.145)
with R(x[k]) and Γ(k,x[k], ω) given by (3.124) and (3.125), respectively. To
prove the convergence of the iterative procedures in (3.144), we use a known
result from supermartingale theory, which we provide for convenience.
Lemma 5 Let Y [k], W [k], Z[k] be three sequences such that Wk is non-negative
for all k. If almost surely∑∞
k=1 Z[k] <∞, and
Y [k + 1] ≤ Y [k]−W [k] + Z[k], k ≥ 0, (3.146)
then, almost surely, either Y [k] → −∞ or else Y [k] converges to a finite
value and∑∞
k=1W [k] <∞.
Proof. The proof can be found in [174], Lemma 1.
It follows the convergence result on the projection-based swarming algorithm in
the presence of random disturbances.
Theorem 16 Let x[k] be the sequence generated by the distributed stochastic
swarming algorithm in (3.144), with step-size satisfying the conditions in (3.117).
Then, the potential sequence J(x[k])k≥0 converges almost surely to a finite value
J∗, i.e.,
Prob
[
limk→∞
J(x[k]) = J∗
]
= 1, (3.147)
where Prob[E ] denotes the probability of the event E. Furthermore, let x∗ be an
accumulation point of the sequence x[k], almost surely the optimal local solution
x∗ is a fixed point of the mapping T (x) = colT i(x)Mi=1, such that x∗ = T (x∗).
153
3.8 The Effect of Noise and Realistic Channels
Proof. To prove the convergence of the sum-rate J(x), we use the following
Lemma.
Lemma 6 (Descent Lemma [189]): If F : RN → R is continuously differentiable
and its gradient is Lipschitz continuous, i.e.,
‖∇F (x)−∇F (y)‖2 ≤ L‖x− y‖2 ∀x,y ∈ RN (3.148)
then,
F (x+ y) ≤ F (x) + yT∇F (x) +L
2‖y‖22 ∀x,y ∈ R
N . (3.149)
Proof. The proof can be found in [188].
In the k-th iteration of the algorithm, each SU simultaneously updates its resource
allocation. One sufficient condition for Lipschitz continuity is that the L2-norm
of the Hessian matrix of J(x) is bounded, in which case this bound can be used
for the Lipschitz constant L. Considering the choice of a coupling function g(·)
with linear attraction (3.7) and bounded repulsion (3.10) and resorting to the
assumption A.1, it can be shown this is true for J(x) and, specifically, there
exists a positive constant L which upper bounds the L2-norm of the Hessian
matrix of J(x). Applying the descent Lemma to J(x), we get
J(x[k+1]) ≤ J(x[k])+[x[k+1]−x[k]]T∇xJ(x[k])+L
2‖x[k+1]−x[k]‖22 (3.150)
Now, we recall some basic properties of projection mappings from [189]:
1. For every x ∈ RN , there exists a unique z ∈ X that minimizes ‖z − x‖2
over all z ∈ X, and will be denoted as [x]X ;
2. Given some x ∈ RN , a vector z ∈ X is equal to [x]X if and only if (y −
z)T (x− z) ≤ 0 for all y ∈ X;
3. The mapping f : RN → X defined by f(x) = [x]X is continuous and
nonexpansive, that is, ‖[x]X − [y]X‖ ≤ ‖x− y‖2 for all x,y ∈ RN .
154
3.8 The Effect of Noise and Realistic Channels
Applying property 2 of the projection mapping and using the expression in
(3.144), we have
[x[k]− x[k + 1]]T (x[k]− α[k]∇xJ(x[k])− x[k + 1]) ≤ 0 (3.151)
which yields α[k][x[k+1]−x[k]]T∇xJ(x[k]) ≤ −‖x[k+1]−x[k]‖22. Substituting
the last expression in (3.150), we get
J(x[k + 1]) ≤ J(x[k])−1
α[k]‖x[k + 1]− x[k]‖22 +
L
2‖x[k + 1]− x[k]‖22. (3.152)
In the notation of Lemma 1, expression (3.152) can be recast as in (3.146), where
Y [k] = J(p[k]), W [k] = (1/α[k])‖x[k + 1]− x[k]‖22,
Z[k] = (L/2)‖x[k + 1]− x[k]‖22. (3.153)
By choosing a positive step size α[k], the sequence W [k] is nonnegative for all
k. Moreover, exploiting the expression in (3.144) and the fact that x[k] ∈ X , we
have
Z[k] =L
2‖x[k + 1]− x[k]‖22 =
L
2‖[x[k] + α[k]∇xJ(x[k])]X − [x[k]]X ‖
22 .
Now, using the non-expansivity of the projection operator and substituting the
expression in (3.145), we get
Z[k] ≤L
2‖α[k]∇xJ(x[k])‖
22 =
L
2α2[k] ‖R(x[k]) + Γ(k,x[k], ω)‖22
≤ Lα2[k][
‖R(x[k])‖22 + ‖Γ(k,x[k], ω)‖22
]
. (3.154)
SinceR(x[k]) is the gradient of J(x) with respect to x, evaluated for the expected
graph L, the resource allocation vector lies into a bounded set, and considering
Assumption A.1, the function ‖R(x[k])‖22 can be upper bounded by a positive
constant C2, for all k. To proceed with the proof, we will use the following lemma.
Lemma 7 Let b[k] be a sequence of random variables with each b[k] being Fk+1
measurable, and suppose that E[b[k]|Fk] = 0 and E[‖b[k]‖2|Fk] ≤ B, where B is
155
3.8 The Effect of Noise and Realistic Channels
some deterministic constant. Then, the sequences
∞∑
k=0
α[k]b[k] and
∞∑
k=0
α2[k]‖b[k]‖2 (3.155)
converge to finite limits (with probability 1), if the step size α[k] satisfies condi-
tions (3.117).
Proof. Since∑∞
k=0 α[k]b[k] is a martingale whose variance is bounded by the
series B∑∞
k=0 α[k], it must have a finite limit by the martingale convergence
theorem. Furthermore,
E
[
∞∑
k=0
α[k]2‖b[k]‖2
]
≤ B∞∑
k=0
α2[k], (3.156)
showing that∑∞
k=0 α2[k]‖b[k]‖2 converges to a finite limit with probability 1.
Now, considering the expression of Γ(k,x[k], ω) in (3.125), we have from
(3.136) that
E[‖Γ(k,x[k], ω)‖Fk ]2 ≤ C3 +C4‖x‖
2, (3.157)
and, since the resource allocation vector lies into a bounded set, we have ‖x‖2 ≤
C5. As a consequence of the previous bound, we get
E[‖Γ(k,x[k], ω)‖2Fk] ≤ C3 + C4C5 = C6 > 0. (3.158)
Now, resorting to the bound on the sequence Z[k] and on ‖R(x[k])‖22, we have
∞∑
k=0
Z[k] ≤∞∑
k=0
Lα2[k][
‖R(x[k])‖22 + ‖Γ(k,x[k], ω)‖22
]
≤ LC2
∞∑
k=0
α2[k] + L
∞∑
k=0
α2[k]‖Γ(k,x[k], ω)‖22. (3.159)
Exploiting the conditions in (3.117), the first term of the right hand side of (3.159)
has a finite limit. Moreover, due to the fact that E[Γ(k,x[k], ω)|Fk ] = 0 and
E[‖Γ(k,x[k], ω)‖2|Fk] ≤ C6, for all k, lemma 3 applies and also the second term
156
3.8 The Effect of Noise and Realistic Channels
of the right hand side of (3.159) has a finite limit. Then, we conclude that the
series∑∞
k=0 Z[k] <∞, having a finite limit with probability 1. All the conditions
of Lemma 1 are then satisfied and the result applies. Hence, almost surely, either
Y [k] → −∞ or else Y [k] converges to a finite value and∑∞
k=0W [k] < ∞.
Since the function J(x) is bounded from below, the sequence Y [k] = J(x[k])
cannot diverge to −∞ and it has to converge almost surely to a finite value.
Furthermore, almost surely it must be that
∞∑
k=1
W [k] =
∞∑
k=0
1
α[k]‖x[k + 1]− x[k]‖22 =
∞∑
k=0
1
α[k]‖T (x[k])− x[k]‖22 <∞,
(3.160)
where T (x[k]) = col [T i(x[k])]Mi=1. However, if the step size α[k] satisfies (3.117),
the sequence 1/α[k] is divergent. Then, since∑∞
k=1W [k] <∞, it must follow
limk→∞
‖T (x[k])− x[k]‖22 = 0 a.s. (3.161)
Then, if x∗ is an accumulation point of the sequence, we can a.s. guarantee
subsequence convergence such that T (x∗) = x∗. This concludes our proof.
Numerical Examples
In this section, we provide numerical examples to illustrate the performance
and main features of the proposed swarming techniques impaired by channel
randomness.
Numerical Example 1 - Swarming in the Presence of Link Failures, Quantization
and Estimation Errors: The purpose of this first example is to show the per-
formance of the proposed allocation algorithm in the presence of random packet
drops, errors in the estimation of the profile gradient and quantization noise. We
consider a connected network composed of 15 SUs, plus two PUs. The topology of
the network corresponding to the case in which all packets are correctly delivered
is shown in Fig. 3.19, where the SUs are represented by dots, while the PUs are
indicated by squares. We consider two examples of interference profiles (supposed
157
3.8 The Effect of Noise and Realistic Channels
0 0.2 0.4 0.6 0.8 1
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
PU
PU
Figure 3.19: Secondary network. The square nodes denote primary users and the
circle nodes denote secondary users.
to be the same for all the nodes), as shown in Fig. 3.20, where the the black curve
represents the true spectrum, whereas the red lines report the noisy observation.
The number of resources (frequency subchannels) to be allocated is assumed to be
15, equal to the number of cognitive users. The resources are initially scattered
randomly across the frequency spectrum. At the k-th iteration of the updating
rule (3.104), each node communicates to its neighbors the position it intends to
occupy, i.e., the scalar xi[k] representing a frequency subchannel. Because of fad-
ing and additive noise, a communication link among two neighbors has a certain
probability p to be established correctly. The values to be exchanged are also af-
fected by quantization noise, supposed to be small with respect to the bandwidth
of the frequency subchannel. The error in the estimation of the profile gradient
is assumed to be Gaussian distributed with zero mean and variance σ2e = 1. Two
examples of resource allocation are shown in Fig. 3.20, where the dots represent
the final frequency channels chosen at convergence by the network nodes. The
parameters of the swarm are cA = 0.02, cR = 0.5, cG = 1 and we considered
158
3.8 The Effect of Noise and Realistic Channels
20 20.5 21 21.5 22 22.5 23 23.5 24 24.5 25−0.5
0
0.5
1
1.5
Frequency (Mhz)
PS
D (
mW
/Hz)
30 35 40 45−0.5
0
0.5
1
1.5
2
2.5
3
3.5
Frequency (Mhz)
PS
D (
mW
/Hz)
Figure 3.20: Examples of resource allocation by swarming.
p = 0.7. In both cases, it is evident how the resources avoid the position occu-
pied by primary users, tend to keep the spread as small as possible while avoiding
collisions among the allocations of different users. Observe that the number of
allocated channels is less than the number of requested resources. This means
that a certain number of nodes have picked up the same channels. We have
checked numerically that, by choosing appropriately the swarm parameters, the
final channel allocation does not lead to collisions among spatial neighbors. This
means that the algorithm is capable of implementing a decentralized mechanism
for spatial reuse of frequencies.
To show the effect of link failures on resource allocation, in Fig. 3.21 we
159
3.8 The Effect of Noise and Realistic Channels
0 50 100 150 2000.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
0.55
Iteration index
Avera
ge Inte
rfere
nce
Ideal Case
p = 0.7
p = 0.5
p = 0.3
Figure 3.21: Average interference perceived by the swarm vs. time index, for
different probabilities of correct packet reception.
report the average interference level perceived by the swarm versus the iteration
index, averaged over 100 independent realizations of random drops. We consider
different probabilities p of correct packet reception; the ideal case, which corre-
sponds to p = 1, is shown as a benchmark. The interference profile is the one
shown in the left side of Fig. 3.20; the network topology is the one depicted in
Fig. 5.1, the swarm parameters are cA = 0.02, cR = 0.5, cG = 1. The iteration
dependent step size is given by α[k] = α0/k, with α0 = 0.5, in order to satisfy
(3.117). From Fig. 5.1 we notice that, after sufficient time, the network always
reaches an equilibrium state that coincides with a swarm cohesion in the low
interference region of the spectrum. Interestingly, we can observe that the final
interference level is always the same, independently of the link failure probability
1−p. From Fig. 3.21, we see that the only effect of the random link failures is to
slow down the convergence, but without affecting the final average interference
level perceived by the swarm. This observation illustrates the robustness of the
proposed algorithm.
160
3.8 The Effect of Noise and Realistic Channels
0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10.1
0.15
0.2
0.25
0.3
0.35
Probability to establish a link
Avera
ge Inte
rfere
nce
cA = 0.01
cA = 0.02
cA = 0.03
Figure 3.22: Average interference perceived by the swarm at convergence, versus
the probability to establish a communication link, for different values of the swarm
attraction parameter cA.
In some applications, the number of iterations must be limited to avoid ex-
cessive delays. It is then of interest to assess the performance of the distributed
resource allocation strategy, fixing a maximum number of iterations Nmax. To
this end, in Fig. 3.22 we report the average interference level perceived by the
swarm, versus the probability p to establish a communication link, averaged over
the frequency slots occupied by the SUs, after Nmax = 50 iterations. The results
are averaged over 100 independent realizations, considering three different values
of the swarm attraction parameter cA. The other parameters of the swarm are
cR = 0.5, cG = 1; the iteration dependent step size is chosen as before, with
α0 = 0.1. From Fig. 3.22, we notice that, for that given number of iterations, at
low values of the probability p, the interference perceived by the swarm is high
because some nodes are trapped in regions occupied by the primary users and
characterized by a low gradient profile. This happens because the duration of the
161
3.8 The Effect of Noise and Realistic Channels
20 30 40 50 60 70 80 90 10020
40
60
80
100
120
140
160
180
Number of nodes
Avera
ge c
onverg
ence t
ime (
itera
tions)
nb = 3
nb = 4
nb = 5
Ideal case
Figure 3.23: Average convergence time versus number of nodes, for different
number of bits used for quantization.
iterative algorithms has not been sufficient to move out the resources trapped
in wrong locations. However, as p increases, the perceived interference decreases
because the attraction exerted by the swarm is finally able to move the resources
toward the interference-free region.
What is also interesting to observe from Fig. 3.22 is that, increasing the at-
traction parameter, for any given p, the performance of the resource allocation
improves. This example shows that the cohesion force represents an intrinsic
robustness factor of the algorithm. In fact, resources allocating over high in-
terference bands might measure a flat spectrum, thus resulting in limited capa-
bilities to move out of (flat) occupied bands, if the only cause of change is the
spectrum gradient. However, increasing the cohesion force, the agents allocating
over the low interference band tend to form cohesive blocks that exert an attrac-
tion towards the agents trapped by mistake over the flat regions of the spectrum
occupied by the primary users. This is an example of cooperation gain.
Finally, it is of interest to check the effect of network size and quantization
162
3.8 The Effect of Noise and Realistic Channels
20 30 40 50 60 70 80 90 10020
40
60
80
100
120
140
160
Number of nodes
Avera
ge c
onverg
ence t
ime (
itera
tions)
connectivity parameter β = 1.1
connectivity parameter β = 1.3
connectivity parameter β = 1.5
Figure 3.24: Average convergence time versus number of nodes, for different
degrees of network connectivity.
noise on the convergence rate. In Fig. 3.23, we report the average number
of iterations needed by the algorithm to converge versus the number of nodes
composing the network, averaged over 200 different network configurations, con-
sidering different numbers of bits to quantize the exchanged messages. The ideal
case, which corresponds to absence of quantization, is shown as a benchmark.
The network graph is a random geometric graph with node’s covering radius
r0 = β√
2 log(M)/(πM), where β = 1.3 tunes the average network connectivity.
The swarm parameters are cA = 0.2, cR = 1, cG = 1 and the initial step size is
α0 = 0.5. As expected, from Fig. 3.23, we can notice how, increasing the number
of nodes and reducing the number of bits used for quantization, the algorithm
needs more time to converge. In a noise-free case, a higher network connectivity
implies faster convergence, whereas, in the noisy case, the algorithm needs more
time to converge due to the larger variance of the disturbance that affects the
system. To show this behavior, in Fig. 3.24, we repeat the previous simulation,
fixing the number of quantization bits to nb = 4 and considering different val-
163
3.8 The Effect of Noise and Realistic Channels
0 20 40 60 80 100 1200.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
0.55
Iteration index
Avera
ge Inte
rfere
nce
Basic Swarming − Ideal Case
Basic Swarming − p = 0.5
Adaptive Scaling − Ideal Case
Adaptive Scaling − p = 0.5
Figure 3.25: Average interference perceived by the swarm vs. time index, for
different algorithms and probabilities of correct packet reception.
ues of the network connectivity parameter β. As we can notice from Fig. 3.24,
increasing the network connectivity, the algorithm needs more time to converge.
Numerical Example 2 - Effect of Adaptation on Performance: In this example we
aim to show the benefits achievable by introducing adaptation and learning as
in (3.140). We consider linear scaling functions fi(Ii(xi(k))) = ai + biIi(xi(k)),
where ai = 0.1 (for all i) and the slope parameter bi is chosen in order to increase
the convergence speed of the nodes perceiving a high interference. We assume
the presence of the interference profile shown in the left side of Fig. 3.20 and
the network topology depicted in Fig. 5.1. In the first example we compare the
convergence speed of the gradient based swarming algorithm and the adaptive
method in (3.140) with scaling coefficients weighted by the perceived interference
power. In Fig. 3.25, we report the average interference level perceived by the
swarm, averaged over 100 independent realizations, versus the iteration index,
considering two different values of probability p to establish a link. In the simu-
164
3.8 The Effect of Noise and Realistic Channels
2 4 6 8 10 12 14 16 18 200.1
0.12
0.14
0.16
0.18
0.2
0.22
0.24
0.26
0.28
Slope parameter b
Avera
ge Inte
rfere
nce
cA = 0.02 p = 0.9
cA = 0.02 p = 0.5
cA = 0.03 p = 0.9
cA = 0.03 p = 0.5
Figure 3.26: Average interference perceived by the swarm at convergence, versus
the slope parameter of the linear scaling functions, for different probabilities of
correct packet reception and different values of the attraction parameter cA.
lation, we consider a linear scaling function with parameters ai = 0.1 and bi = 2
for all i. The parameter of the swarm are cA = 0.02, cR = 0.5, cG = 1 and the
step size is chosen such that α0 = 0.5 for both the algorithms. The simulation
shows how the swarming algorithm with adaptive scaling is robust with respect
to link failures, achieving the same performance level of the basic algorithm and
greatly outperforming it in terms of convergence speed. This means that the
convergence time of the swarming algorithm can be considerably reduced if every
node adapts its convergence speed according to the perceived interference.
To measure the effectiveness of the distributed resource allocation strategy
in the presence of a limited maximum number of iterations, in Fig. 3.26 we
report the interference level, versus the slope parameter bi of the linear scaling
functions, averaged over the frequency slots occupied by the SUs, after Nmax =
20 iterations. The result is averaged over 100 independent realizations. We
165
3.8 The Effect of Noise and Realistic Channels
considered two different values of the probability p and of the swarm attraction
parameter cA. The other swarm parameters are equal to cR = 0.5 and cG = 1.
From Fig. 3.26, we notice that, at low values of the parameter bi, the movement
of the resources is very limited and, inside the maximum number of iterations,
some resources cannot move out form the regions occupied by the primary users
because of the random impairments affecting the algorithm. As bi increases, the
resources perceiving a high power move faster toward the interference-free region
due to the increment of the average profile gradient and the cohesion force, thus
making the overall swarm experience a smaller total interference. This means
that the performance of the swarming algorithm can be considerably improved
if every node adapts its scaling function according to the perceived interference.
Furthermore, from Fig. 3.26, we notice how an increment of the cohesion force
induces a better performance thanks to the cooperation among network nodes,
similarly to what we had observed from Fig. 3.22. From Fig. 3.26, we also
notice, as expected, how a lower probability to establish a communication link
determines worst performance.
Numerical Example 3 - Distributed Graph Coloring: One of the most interesting
features of the proposed swarming technique is its capability to induce a spatial
reuse of frequency channels from secondary nodes far away from each other, using
a decentralized approach under random packet dropping. As an example of chan-
nel allocation, in Fig. 3.27 we consider a network composed of 50 nodes, where
each node senses the interference profile shown in the left side of Fig. 3.20. The
swarm parameters are cA = 0.01, cR = 0.5 and cG = 1. Every node starts from a
random initial position on the spectrum, and then it updates its intended position
according to (3.104), where the probability to establish correctly a communica-
tion link between secondary users is p = 0.7. In the application at hand, there
is an intrinsic quantization of the frequency resources given by the subchannel
bandwidth. In our implementation, we let the system evolve according to (3.104)
until successive differences in allocation become smaller than the bandwidth of a
166
3.8 The Effect of Noise and Realistic Channels
0 0.2 0.4 0.6 0.8 1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Figure 3.27: Example of resource allocation by swarming.
frequency subchannel. At that point, the evolution stops and every SU is allowed
to transmit over the selected channel. The channel chosen by each cognitive node
at the end of the iterations is indicated by the symbol assigned to each vertex
in Fig. 3.27. Each symbol identifies a frequency subchannel uniquely. In our
simulation, the low interference band on the spectrum is divided in 8 different
channels. Hence, the number of available resources is much smaller than the
number of users. From Fig. 3.27 we can observe that the nodes that have picked
up the same channel are never neighbor of each other. This shows the capability
of the proposed method to implement a decentralized mechanism for spatial reuse
of frequencies.
167
3.9 Conclusion
3.9 Conclusion
In this chapter we have proposed a distributed resource allocation strategy for
the access of opportunistic users in cognitive networks based on a swarm model
mimicking the foraging behavior of a swarm, where the interference distribution
over the radio resource domain plays the role of spatial distribution of food. The
swarm mechanism includes an attraction force, useful to minimize the spread over
the resource domain, and a repulsion force, useful to avoid collisions among radio
nodes. In the proposed model, each agent is supposed to listen only to nearby
nodes, in a narrow band spectral interval, over consecutive time slots. This is
useful to limit the complexity of the secondary users’ equipments and, interest-
ingly enough, it produces, as a by-product, an intrinsic capability of the system
to provide spatial reuse of frequency, through a purely decentralized mechanism.
We have derived closed form expressions, validated by numerical results, for the
upper and lower bounds of the final spread as a function of the main system
parameters and of the network topology.
Specific attention has been devoted to the analysis of the proposed swarm-
ing mechanism in the frequency domain, where a local stability analysis has
shown that the introduction of the attraction and unbounded repulsion terms
in the functional to be minimized does not affect the stability of the system.
Furthermore, we have introduced fast swarming methods, useful to improve the
convergence rate and the algorithm’s reaction time in dynamic environments.
Simulation results confirm that these techniques outperform the basic swarming
algorithm. The application of the proposed model to the distributed resource al-
location problem on a time-frequency plane has been also described. We consider
both a static interference scenario, where the interference activity is assumed to
be known and constant along the duration of the swarming algorithm, and a
dynamic interference scenario. In this latter case, the interference activity over
each frequency subchannel is modeled as a continuous-time Markov chain. Inter-
estingly, simulation results show how the swarm tends to stay as close as possible
168
3.9 Conclusion
to time 0, where the prediction of the expected interference is better, under the
constraint of avoiding collisions.
In the last part of the chapter, we have extended the swarming algorithm
for allocating resources in cognitive radio networks considering the presence of
channel imperfections, such as link failures, estimation errors, and quantization
noise. We showed that the swarm converges almost surely to an equilibrium
configuration dependent on the mean graph of the network. We also established
that the resource allocation algorithm is robust against channel imperfections
such as random link failures, quantization noise and estimation errors, whose
effect is only to slow down the convergence process. In particular, reducing the
probability to establish a communication link, the network requires more time
to reach the final equilibrium state. Simulation results show that the cohesion
force introduces cooperation gain among the SU’s and represents an intrinsic
robustness factor of the algorithm. Finally, an adaptive swarming approach was
applied to allocate resources in the frequency domain. The proposed algorithm
adapts the speed of the swarm individuals according to the perceived interference
distribution resulting in an improved allocation performance, convergence speed,
adaptation and learning capability.
169
170
Chapter 4
Distributed Cooperative
Spectrum Sensing Based on
Diffusion Adaptation
In this chapter, a distributed technique for cooperative spectrum estimation
in cognitive radio systems is introduced, based on a basis expansion model of the
power spectral density (PSD) map in frequency. Joint estimation of the model
parameters enables identification of the (un)used frequency bands, thus facilitat-
ing spatial frequency reuse. The proposed method, based on diffusion adaptation
algorithms [143, 145], estimates and learns the interference profile through local
cooperation and without the need for a central processor. Compared to well stud-
ied incremental methods, diffusion methods do not require the use of a cyclic path
over the nodes and are robust to node and link failures. The diffusion mechanism
endows the networks with powerful learning and tracking abilities that enable the
individual nodes to continue learning even when the cost function changes with
time, exploiting both the time- and spatial-diversity of the data. The diffusion of
information across the network results in various forms of self-organizing behavior
and collective intelligence, which is well-suited to model, e.g., animal swarming.
171
4.1 Introduction
4.1 Introduction
Spectrum sensing is a critical prerequisite in envisioned applications of wire-
less cognitive radio (CR) networks which promise to resolve the perceived band-
width scarcity versus under-utilization dilemma. Creating an interference map
of the operational region plays an instrumental role in enabling spatial frequency
reuse and allowing for dynamic spectrum allocation in a hierarchical access model
comprising primary and secondary users [93], [94]. The non-coherent energy de-
tector has been widely used to this end because it is simple and obviates the need
for synchronization with unknown transmitted signals; see e.g., [157], [158], [159],
and [104]. Power information (or other statistics [155], [156]) collected locally per
CR is fused centrally by an access point in order to decide absence or presence of
a primary user per frequency band. At the expense of commensurate communi-
cation overhead [158], these cooperative sensing and detection schemes have been
shown to increase reliability, reduce the average detection time, cope with fading
propagation effects, and improve throughput [156], [157], [159], [104]. Recently,
the possibility of spatial reuse has received growing attention. It was noticed
that even if a frequency band is occupied, there could be locations where the
transmitted power is low enough so that these frequencies can be reused with-
out suffering from or causing harmful interference to the primary system. These
opportunities are discussed in [160], and a statistical model for the transmitters’
spatial distribution is advocated in [161].
Communications over radio networks is concerned with efficient techniques
for dynamic access to spectral resources [2, 3] and for self-organization (SO) ca-
pabilities. SO is important in femtocell networks, where the deployment of a
potentially large number of user-operated femto-access points makes centralized
schemes hard to implement and prone to heavy signaling traffic. While decentral-
ized resource allocation strategies are certainly more appealing, relying on pure
decentralization with adaptation and learning abilities can lead to inefficient im-
plementations. A more viable approach consists in endowing the radio nodes
172
4.1 Introduction
with the capability to learn from the environment and to exchange information
only with immediate neighbors in order to identify the most appropriate radio
resources. It is only natural that the deployment of cognitive terminals can bene-
fit from highly decentralized radio access and sensing strategies. The deployment
of distributed sensing strategies was proposed, for example, in [172, 173], where
cooperative spectrum sensing techniques exploited the intrinsic sparsity of the
radio resource allocation.
The decentralized approach that is pursued in this work is inspired by behavior
encountered in nature and is meant to endow the cognitive nodes with adaptation
and learning abilities. Biological networks tend to exhibit robust behavior and
are are capable of solving difficult organizational tasks through local cooperation
among the individual agents without central control. In this chapter, we propose
a distributed technique for cooperative spectrum estimation in cognitive radio
systems based on diffusion adaptation algorithms. In comparison with other
distributed approaches that rely on, for example, consensus-based techniques
[132], [133], [134], [136], adaptive networks avoid the need to iterate over data
and do not require all nodes to converge to the same equilibrium (or consensus).
The basic contributions of this chapter are:
1) the exploitation of sparsity in the distributed estimation problem over adaptive
networks;
2) the derivation of mean-square analysis for the sparse diffusion adaptive filter;
3) a real-time distributed spectrum estimation technique based on diffusion adap-
tation techniques;
4) the derivation of the mean-square properties of the diffusion adaptive filter
applied to the spectrum estimation problem.
The chapter is organized as follows. In section 4.2 we describe the basic diffu-
sion algorithm from [145]. Section 4.3 introduces novel regularized diffusion LMS
173
4.2 Diffusion Adaptation
techniques for distributed estimation over adaptive networks, which are able to
exploit sparsity in the underlying system model. Convergence and mean square
analysis of the sparse adaptive diffusion filter show under what conditions we
have dominance of the proposed method with respect to its unregularized coun-
terpart in terms of steady-state performance. Simulation results also confirm the
potential benefits of the proposed filter under the sparsity assumption on the
true coefficient vector. From section 4.4 we illustrate the proposed distributed
spectrum estimation technique based on diffusion adaptation. We first intro-
duce a basis expansion model, which is useful to model the PU’s transmission,
allowing distributed cooperative sensing. Then, we propose a normalized ver-
sion of the Adapt then Combine (ATC) diffusion algorithm [145], which enables
the network to learn and track the time-varying interference profile. Convergence
and mean-square performance analysis of the proposed normalized ATC diffusion
filter, applied to the spectrum estimation problem, is also derived.
4.2 Diffusion Adaptation
In this section we briefly recall some results on adaptive estimation over net-
works from [145]. We consider the problem of distributed estimation, where a
set of nodes is required to collectively estimate some parameter of interest from
noisy measurements by relying solely on in-network processing. Thus, consider a
set of M nodes distributed over some geographic region. At every time instant
k, every node i takes a scalar measurement di(k) of some random process δi(k)
and a 1×N regression vector, ui,k, of some random process υi,k. The objective
is for every node in the network to use the collected data di(k),ui,k to estimate
some parameter vector wo in a distributed manner. In the centralized solution
to the problem, every node in the network transmits its data di(k),ui,k to a
central fusion center for processing. This approach has the disadvantage of being
non-robust to failure by the fusion center. Moreover, in the context of wireless
sensor networks, centralizing all measurements in a single node lacks scalability,
174
4.2 Diffusion Adaptation
and may require large amounts of energy and communication resources [1]. On
the other hand, in distributed implementations, every node in the network com-
municates with a subset of the nodes, and processing is distributed among all
nodes in the network. For the following derivations, we assume the presence of a
linear measurement model where, at every time instant k, every node i takes a
measurement according to the model:
di(k) = ui,kwo + vi(k) (4.1)
where vi(k) is a zero mean random variable with variance σ2v,k, independent of
ui,k for all k and i, and independent of vj(l) for l 6= k and i 6= j. Linear models
as in (4.1) are customary in adaptive filtering since they are able to capture
many cases of interest. The objective of the network is to estimate wo in a fully
distributed manner and in real-time, where each node is allowed to interact only
with its neighbors. That is, the nodes would like to estimate the global parameter
wo that minimizes the following cost function:
Jw(w) =M∑
k=1
E|di(k)− ui,kw|2 (4.2)
where E(·) denotes the expectation operator. Useful diffusive adaptation schemes
were developed for such purpose in [143,145] and their mean-square performance
was studied there in great detail. One example is the so-called Adapt-then-
Combine (ATC) diffusion algorithm in [145], which operates as follows:
ψi,k = wi,k−1 + µi∑
j∈Nicj,iu
∗j,k[dj(k) − uj,kwi,k−1]
(adaptation step)
wi,k =∑
j∈Niaj,iψj,k (diffusion step)
(4.3)
i = 1, . . . ,M , where µi is a positive step-size chosen by node i, and the operator ∗
denotes complex conjugate transposition. The first step in (4.3) is an adaptation
step, where the coefficients cj,i determine which nodes j ∈ Ni should share their
measurements dj(k),uj,k with node i. The second step is a diffusion step where
175
4.3 Sparse Diffusion Adaptation
the intermediate estimates ψj,k, from the neighborhood j ∈ Ni, are combined
through the coefficients aj,i. The non-negative combination matrices C =
cj,i ∈ RM×M and A = aj,i ∈ R
M×M satisfy
cj,i > 0, aj,i > 0 if j ∈ Ni, (4.4)
1TC = 1T , C1 = 1, and 1TA = 1T . (4.5)
At the same way, reversing the order of the processing steps in (4.3), we also
consider the Combine-then-Adapt strategy for the distributed minimization of
(4.44):
ψi,k−1 =∑
j∈Niaj,iwj,k−1 (diffusion step)
(adaptation step)
wi,k = ψi,k−1 + µi∑
j∈Nicj,iu
∗j,k[dj(k)− uj,kψi,k−1]
(4.6)
A detailed convergence and mean-square analysis of these adaptive diffusion
schemes can be found in [145]. In the next section we introduce regularized dif-
fusion LMS techniques for distributed estimation over adaptive networks, which
are able to exploit sparsity in the underlying system model. Convergence and
mean square analysis of the sparse adaptive diffusion filter are also derived and
simulation results confirm the potential benefits of the proposed method under
the sparsity assumption on the true coefficient vector.
4.3 Sparse Diffusion Adaptation
For adaptive distributed estimation purposes, several diffusion adaptation
techniques were proposed and studied in [143, 145], where the nodes exchange
information locally and cooperate in order to estimate wo without the need for
a central processor. In many scenarios, the vector parameter wo can be sparse,
containing only a few large coefficients among many negligible ones. Exploiting
the prior information about the sparsity of wo can help improve the estima-
tion performance and this fact has already been investigated in the literature
176
4.3 Sparse Diffusion Adaptation
for several years. Specifically, motivated by LASSO [162] and works on com-
pressive sensing [163,164], several algorithms have been proposed in the context
of sparse adaptive filtering frameworks such as LMS [166], RLS [167, 168], and
projection-based methods [169]. A distributed algorithm implementing LASSO
over an ad-hoc network of nodes has been also proposed in the context of spec-
trum sensing for cognitive radio [172] and for sparse linear regression [171]. The
basic idea of these techniques is to introduce a convex penalty, i.e., an ℓ1-norm
term, into the cost function to favor sparsity. However, none of the earlier works
considered the design of adaptive distributed solutions that are able to process
data online and exploit sparsity at the same time. Doing so would endow net-
works with learning abilities and would allow them, for example, to learn the
sparse structure from incoming data recursively and also to track variations in
the sparsity of the underlying vector. In this work, we consider adaptive networks
running diffusion techniques under general constraints enforcing sparsity. In par-
ticular, we consider two convex regularization functions. First, we consider the
ℓ1-norm, which acts as a uniform zero-attractor. Then, to improve the estima-
tion performance, we employ a reweighted regularization to selectively promote
sparsity on the zero elements of w0, rather than uniformly on all the elements.
We provide convergence analysis of the proposed methods, giving a closed form
expression for the bias on the estimate due to the regularization. We also provide
a mean-square analysis, showing the conditions under which the sparse diffusion
filter outperforms its unregularized version in terms of steady-state performance.
Interestingly, it turns out that, if the system model is sufficiently sparse, it is pos-
sible to tune a single parameter to achieve better performance than the standard
diffusion algorithm.
4.3.1 Sparse ATC Diffusion
Assuming the presence of a linear observation model as in (4.1), the cooper-
ative sparse estimation problem can be cast as the distributed minimization of
177
4.3 Sparse Diffusion Adaptation
the following cost function:
Jw(w) =
M∑
k=1
E|di(k)− ui,kw|2 + ρf(w) (4.7)
where E(·) denotes the expectation operator, and f(w) is a convex regularization
term weighted by the parameter ρ > 0, which is used to enforce sparsity. Pro-
ceeding as in [145], it is possible to develop several diffusion adaptation schemes
for such purpose. In this paper, we consider the Adapt-then-Combine (ATC)
strategy and refer to the following algorithm as the ATC-sparse diffusion (or
ATC-SD) version:
ψi,k = wi,k−1 + µi∑
j∈Nicj,iu
∗j,k[dj(k) − uj,kwi,k−1]
−µiρ∂f(wi,k−1) (adaptation step)
wi,k =∑
j∈Niaj,iψj,k (diffusion step)
(4.8)
i = 1, . . . ,M , where µi is a positive step-size chosen by node i, the operator ∗
denotes complex conjugate transposition, and ∂f(w) is the sub-gradient of the
convex function f(w). The first step in (4.8) is an adaptation step, where the
coefficients cj,i determine which nodes j ∈ Ni should share their measurements
dj(k),uj,k with node i. The second step is a diffusion step where the interme-
diate estimates ψj,k, from the neighborhood j ∈ Ni, are combined through the
coefficients aj,i. The non-negative combination matrices C = cj,i ∈ RM×M
and A = aj,i ∈ RM×M satisfy (4.4).
In this paper we consider two different convex regularization terms. Motivated
by LASSO [162] and work on compressive sensing [163], we first propose the ℓ1-
norm as penalty function, i.e.,
f1(w) = ‖w‖1, (4.9)
in the global cost function (4.44). This choice leads to an algorithm update in
(4.3) where the subgradient vector is given by ∂f1(w) = sign(w), where sign(x)
178
4.3 Sparse Diffusion Adaptation
is a component-wise function defined as
sign(x) =
x/|x|, x 6= 0
0, x = 0. (4.10)
This update leads to what we shall refer to as the zero-attracting (ZA) ATC
diffusion algorithm. The ZA update uniformly shrinks all components of the
vector, and does not distinguish between zero and non-zero elements. Since all
the elements are forced toward zero uniformly, the performance would deteriorate
for systems that are not sufficiently sparse. Motivated by the idea of reweighting
in compressive sampling [164], we also consider the following cost function:
f2(w) =N∑
n=1
log(1 + ε|wn|), (4.11)
which behaves more similarly to the l0-norm than the l1-norm [164], thus enhanc-
ing the sparsity recovery of the algorithm. The algorithm in (4.3) is then updated
by using
∂f2(w) = εsign(w)
1 + ε|w|, (4.12)
leading to what we shall refer to as the reweighted zero-attracting (RZA) ATC
diffusion algorithm. The update in (4.12) selectively shrinks only the components
whose magnitudes are comparable to 1/ε, and there is little effect on components
satisfying |wn| ≫ 1/ε.
4.3.2 Performance Analysis
In this section we analyze the performance of the ATC-SD algorithm. In what
follows we view the estimates wi,k as realizations of a random process ωi,k and
analyze the performance of the algorithm in terms of its mean square behavior.
To proceed with the analysis, we assume a linear measurement model as in
(4.1). Using (4.8), we define the error quantities wk,i = wo − wi,k, ψi,k =
179
4.3 Sparse Diffusion Adaptation
wo −ψi,k, and the global vectors:
wk =
w1,k
...
wM,k
, wk =
w1,k
...
wM,k
, ψi =
ψ1,k...
ψM,k
. (4.13)
We also introduce a diagonal matrix
M = diagµ1IN , . . . , µMIN, (4.14)
and the extended weighting matrices
C = C ⊗ IN , A = A⊗ IN , (4.15)
where ⊗ denotes the Kronecker product operation. We further introduce the
following random quantities:
Dk = diag
M∑
j=1
cj,1u∗j,kuj,k, . . . ,
M∑
j=1
cj,Mu∗j,kuj,k
(4.16)
gk = CTcolu∗
1,kv1(k), . . . ,u∗M,kvM (k). (4.17)
Then, we can write (4.3) in compact form as:
ψk = wk−1 − M [Dkwk−1 + gk] + ρM∂f(wk−1)
wk = ATψk (4.18)
where ∂f(wk−1) = col[∂f(w1,k−1), . . . ,∂f(wM,k−1)], or, equivalently,
wk = AT[I − MDk]wk−1 − A
TMgk + ρA
TM∂f(wk−1). (4.19)
Mean Stability
Assuming all regressors ui,k are spatially and temporally independent and
taking the expectation of (4.19), we get
Ewk = AT [I − MEDk]Ewk−1 + ρATME∂f(wk−1). (4.20)
180
4.3 Sparse Diffusion Adaptation
Since the subgradient vector ∂f(wk−1) has bounded entries, the algorithm (4.56)
converges in the mean if the matrix AT[I−MEDk] is a stable matrix. Since the
entries on the columns of AT
add up to one, and since M is diagonal, we can
show that the previous condition holds if the matrix I −MD, where D = EDk,
is stable. Using (4.51) we conclude that the algorithm converges in the mean for
any step-size satisfying:
0 < µi <2
λmax
(
∑Mj=1 cj,iRu,j
) , i = 1, . . . ,M, (4.21)
where λmax(X) denotes the maximum eigenvalue of a Hermitian matrix X, and
Ru,i = Eu∗i,kui,k is the covariance matrix of the regression data at the i-th node.
Furthermore, taking the limit as k →∞ of equation (4.20), we have
Ew∞ = wo − ρ[
I − AT [I − MD
]
]−1A
TME∂f(w∞), (4.22)
thus concluding that the estimate wk is asymptotically biased; moreover, the
smaller the value of ρ, the smaller the bias.
Mean-Square Performance
In this section we examine the mean-square performance of the diffusion filter
(4.56). Now, following the energy conservation framework of [143, 145], we can
evaluate the weighted norm of wk, obtaining:
E‖wk‖2Σ= E‖wk−1‖
2Σ
′ + E[g∗kMAΣATMgk] + φΣ,k(ρ) (4.23)
where Σ is an Hermitian positive-definite matrix that we are free to choose, and
Σ′ = E(I −DkM )T AΣAT(I − MDk), (4.24)
φΣ,k(ρ) = ρβΣ,k
(
ρ−αΣ,k
βΣ,k
)
, (4.25)
where
βΣ,k = E‖∂f(wk−1)‖2MAΣAT M
> 0, (4.26)
αΣ,k = −2E∂f(wk−1)TMAΣA
T [I − MD
]
wk−1. (4.27)
181
4.3 Sparse Diffusion Adaptation
Moreover, setting
G = E[gkg∗k] = C
Tdiagσ2v,1Ru,1, . . . , σ
2v,MRu,MC, (4.28)
we can rewrite (4.64) in the form
E‖wk‖2Σ = E‖wk−1‖
2Σ
′ +Tr[ΣATMGMA] + φΣ,k(ρ) (4.29)
where Tr(·) denotes the trace operator. Let σ = vec(Σ) denote the vector that is
obtained by stacking the columns of Σ on top of each other. Using the Kronecker
product property vec(UΣV ) = (V T ⊗ U)vec(Σ), we can vectorize Σ′ in (4.67)
as σ′ = vec(Σ′) = Fσ, where the matrix F is given by:
F = (I ⊗ I)I − I ⊗ (DM)− (DM )⊗ I
+ E(DkM)⊗ (DkM)(A⊗ A). (4.30)
Then, using the property Tr(ΣX) = vec(XT )Tσ and taking the limit as k →∞
(assuming the step-sizes are small enough to ensure convergence to steady-state),
we deduce from (4.29) that:
E‖w∞‖2Σ−Σ
′ = [vec(ATMGMA)]Tσ + ρβΣ,∞
(
ρ−αΣ,∞
βΣ,∞
)
.
The steady-state mean-square deviation (MSD) of the network is defined as:
MSDnet = limk→∞
1
M
M∑
i=1
E‖wi,k‖2. (4.31)
Then, if the step sizes µi are small enough so that the matrix (I − F ) is
invertible and choosing σ = (I − F )−1vec(I ⊗ I), the network MSD tends to
MSDnet =1
M[vec(A
TMGTMA)]T (I − F )−1vec(I ⊗ I)
+1
MρβΣ,∞
(
ρ−αΣ,∞
βΣ,∞
)
. (4.32)
182
4.3 Sparse Diffusion Adaptation
The first term on the right-hand side of (4.75) is the network MSD of the standard
diffusion algorithm (compare with (48) in [145]), whereas the second term is due
to the regularization. Then, if
αΣ,∞ > 0, and 0 < ρ <αΣ,∞
βΣ,∞, (4.33)
the ATC-SD algorithm would perform better than the standard diffusion [145].
Let us examine the interpretation of the condition ασ,∞ > 0, where ασ,i is
given by (4.27), relating this condition to the sparsity of the vector wo. In-
deed, since f(·) is a convex regularization function, it holds that f(x + y) −
f(x) ≥ ∂f(x)Ty. Then, choosing x = w∞ and y = BΣ(wo − w∞), where
BΣ = 2MAΣAT [I − MD
]
, the first condition in (4.33) can be recast as
αΣ,∞ ≥ E[f(w∞)− f(w∞ +Bσ(wo −w∞))] > 0. (4.34)
If the step-sizes are sufficiently small, we can approximate BΣ ⋍ 2MAΣAT,
neglecting the second term that depends on µ2. Then, we have wB∞ = w∞ +
BΣ(wo−w∞) ⋍ w∞−2MAΣA
T(w∞−w
o). This expression can be interpreted
as a gradient descent update minimizing the function ‖w∞−wo‖2
AΣAT , yielding
wB∞ closer to wo than w∞. As a consequence, if wo is sparse, wB
∞ will be more
sparse than w∞ and the condition in (4.34) will likely be true. Then, by selecting
properly the sparsity coefficient ρ, the ATC-SD algorithm will have better MSD
than the standard ATC diffusion algorithm. On the other hand, if wo is not
sparse, condition (4.34) in general would not be true, thus leading the ATC-SD
algorithm to perform worse than standard ATC diffusion.
4.3.3 Numerical Results
In this section, we provide some numerical examples to illustrate the perfor-
mance of the ATC-SD algorithm. We consider a connected network composed
of 20 nodes. The regressors have size N = 50 and are zero-mean white Gaus-
sian distributed with covariance matrices Ru,i = σ2uI, with σ2u = 0.1, for all
183
4.3 Sparse Diffusion Adaptation
0 200 400 600 800 1000 1200 1400 1600 1800
−30
−25
−20
−15
−10
−5
0
5
10
15
Iteration index
MS
D (
dB
)LMS
ZA LMS
RZA LMS
ATC Diffusion
ZA ATC Diffusion
RZA ATC Diffusion
Figure 4.1: Transient network MSD for the non-cooperative approaches LMS
[190], ZA-LMS [166], RZA-LMS [166], and the diffusion techniques ATC [145],
ZA-ATC (eq.(4.3)-(4.9)), RZA-ATC (eq.(4.3)-(4.11)).
i. The background white noise power is set to σ2v = 0.01. The first example
aims to show the tracking and steady-state performance for the ATC-SD algo-
rithm. In Fig. 4.1, we report the learning curves in terms of network MSD of
6 different adaptive filters: ATC diffusion LMS [145], ZA-ATC (eq.(4.3)-(4.9))
and RZA-ATC diffusion (eq.(4.3)-(4.11)), and the corresponding non cooperative
approaches from [166]. The simulations use a value of µ = 0.2 and the results
are averaged over 100 independent experiments. The sparsity parameters are set
equal to ρLMS = 5 × 10−3 for the non cooperative approaches, ρZA = 10−3 for
ZA-ATC, ρRZA = 0.25 × 10−3 for RZA-ATC, and ǫ = 10. In this simulation, we
consider diffusion algorithms without measurement exchange, i.e., C = I, and a
combination matrixA that simply averages the estimates from the neighborhood,
hence, such that aj,i = 1/Ni for all j. Initially, only one of the 50 elements of
wo is set equal to one while the others are equal to zero, making the system very
sparse. After 600 iterations, 25 elements are randomly selected and set equal to
184
4.3 Sparse Diffusion Adaptation
0 0.001 0.002 0.003 0.004 0.005 0.006 0.007−4
−3
−2
−1
0
1
2
Sparsity parameter ρ
Diffe
rential M
SD
(d
B)
Sparsity ratio = 1/50
Sparsity ratio = 5/50
Sparsity ratio = 10/50
Sparsity ratio = 20/50
Sparsity ratio = 40/50
Figure 4.2: Differential MSD versus sparsity parameter ρ for ZA-ATC Diffusion
LMS, for different degrees of system sparsity.
1, making the system have a sparsity ratio of 25/50. After 1200 iterations, all the
elements are set equal to 1, leaving a completely non-sparse system. As we see
from Fig. 5.4, when the system is very sparse both ZA-ATC and RZA-ATC yield
better steady-state performance than standard diffusion. The RZA-ATC outper-
forms ZA-ATC thanks to the reweighted regularization. When the vector wo is
only half sparse, the performance of ZA-ATC deteriorates, performing worse than
standard diffusion, while RZA-ATC has the best performance among the three
diffusion filters. When the system is completely non-sparse, the RZA-ATC still
performs comparably to the standard diffusion filter. Finally, we can also notice
the gain of the diffusion schemes with respect to the non-cooperative approaches
from [166,190]. To quantify the effect of the sparsity parameter ρ on the perfor-
mance of the ATC-SD filters, we consider two additional examples. Considering
the same settings of the previous simulation, in Fig. 4.2, we show the behavior
of the difference (in dB) between the network MSD of ATC-ZA and standard
diffusion, versus ρ, for different sparsity degrees of wo. The results are averaged
185
4.3 Sparse Diffusion Adaptation
0 0.001 0.002 0.003 0.004 0.005 0.006−8
−7
−6
−5
−4
−3
−2
−1
0
1
2
Sparsity parameter ρ
Diffe
rential M
SD
(d
B)
Sparsity ratio = 1/50
Sparsity ratio = 5/50
Sparsity ratio = 10/50
Sparsity ratio = 20/50
Sparsity ratio = 40/50
Sparsity ratio = 50/50
Figure 4.3: Differential MSD versus sparsity parameter ρ for RZA-ATC Diffusion
LMS, for different degrees of system sparsity.
over 100 independent experiments and over 100 samples after convergence. As
we can see from Fig. 4.2, reducing the sparsity of wo, the interval of ρ values
that yield a gain for ATC-ZA with respect to standard diffusion becomes smaller,
until it reduces to zero when the system is not sparse enough. In this case, in
fact, the ATC-ZA diffusion algorithm only introduces a bias in the final estimates
without improving the system performance. In Fig. 4.3, we repeat the same ex-
periment considering the ATC-RZA algorithm. As wee can see, ATC-RZA gives
better performance than ZA-ATC and yields a performance loss with respect to
standard diffusion, for any ρ, only when the vector w0 is completely non-sparse.
This is an effect of the selective shrinking of the algorithm, which acts only on
the components whose magnitudes are comparable to 1/ε, whereas there is little
effect on components satisfying |wn| ≫ 1/ε.
In the next section, we introduce a basis expansion model useful to describe
PU transmissions in a cognitive radio system. This model will be instrumental
to enable the network with adaptive spectrum estimation capabilities.
186
4.4 Basis Expansion Model of the Spectrum
4.4 Basis Expansion Model of the Spectrum
Consider Ntx active PU’s transmitters and let yi(k) denote the signal received
at time k by the SU i, given by the superposition of the transmitted signals
uq(k), q = 1, . . . , Ntx convolved with a linear, and possibly time-varying, fading
tapped-delay-line channel with impulse response hiq(k, l)Liq
l=1, and observed in
the presence of additive white noise n(k):
yk(i) =
Ntx∑
q=1
Liq∑
l=1
hiq(k, l)uq(k − l) + n(k). (4.35)
We consider the two following assumptions on the transmitted signals and on the
communication channels.
Assumption 1 : The source signals uq(k) are stationary, mutually uncorrelated
and independent of the channels hiq(k, l).
Assumption 2 : The channels hiq(k, l) are zero mean and uncorrelated across
the lag variable l and the spatial variables q and i.
Spatial uncorrelatedness of the channels is well justified since PU’s are physically
sufficiently far apart relative to the high carrier frequencies, which determine
short wavelengths. Since the channels are uncorrelated across lags l, the channel
gain piq(k) is frequency invariant, but possibly time-variant, and given by
piq(k) =
Lqi∑
l=1
σ2iq(k, l), (4.36)
where σ2iq(k, l) = E|hiq(k, l)|2. This model is quite general and takes into account
the possibility of node mobility and possible variations in the channel conditions
over time. In this paper, we assume the gain piq(k) can be acquired via train-
ing. Then, cooperation between primary and secondary users is allowed. The
case where such cooperation is not allowed was analyzed in [172], where the au-
thors used a grid of candidate PU’s locations and a known model to describe
187
4.4 Basis Expansion Model of the Spectrum
15 20 25 300
0.5
1
1.5
Frequency (MHz)
PS
D (m
W/H
z)
Figure 4.4: Example of basis expansion using Gaussian pulses. The dotted curves
represent the Gaussian basis functions, whereas the continuous curve denotes the
behavior of a generic interference profile described by 6 Gaussian pulses.
the path loss between transmitter and receiver. In the following, we introduce a
basis expansion model, which is useful to model the PU’s transmission, allowing
distributed cooperative sensing.
Let Φq(f) denote the power spectral density (PSD) of the stationary random
signal uq(k) transmitted by the q-th primary user. The PSD can be represented
as a linear combination of some preset J basis functions, say, as:
Φq(f) =J∑
j=1
bj(f)wqj = bT0 (f)wq (4.37)
where b0(f) = [b1(f), ..., bJ (f)]T is the vector of basis functions evaluated at
frequency f , wq = [wq1, ..., wqJ ] is a vector of weighting coefficients representing
the power transmitted by the q-th PU over each basis, and J is the number
of basis functions. For J sufficiently large, the basis expansion in (4.37) can
approximate well the transmitted spectrum. Several choices for the set of basis
bj(f)Jj=1 are possible. In particular, we consider continuously differentiable
188
4.4 Basis Expansion Model of the Spectrum
basis functions, such as raised cosines, Gaussian pulses, etc. An example of basis
expansion using Gaussian pulses is shown in Fig. 4.4. Assuming Ntx active users
are transmitting, the overall transmitted spectrum can be expressed as:
IT (f) =
Ntx∑
q=1
J∑
j=1
bj(f)wqj = bT1 (f)w
o (4.38)
where wo = [w1, . . . ,wNtx] ∈ RJNtx and b1(f) = 1⊗b0(f), with ⊗ and 1 ∈ R
Ntx
denoting, respectively, the Kronecker product operation and the vector of all ones.
The propagation medium introduces path loss attenuation between primary and
secondary users. Let piq(k) be the path loss coefficient between the q-th primary
user transmitter and the i-th secondary user. Then, considering a single source q,
due to the assumptions 1 and 2, we can express the autocorrelation of the signal
received by SU i in absence of noise, as
φi(m) = φq(m)
Liq∑
l=1
E|hiq(k, l)|2, (4.39)
and hence the received PSD, at time k, as Ii(f, k) = piq(k)Φq(f). Considering
now the presence of Ntx active PUs and receiver noise, under the assumption of
spatial uncorrelatedness of the channels and sources, the signal received by the
secondary node i can be expressed as:
Ii(f, k) =
Ntx∑
q=1
piq(k)
J∑
j=1
bj(f)wqj + σ2n,i = bTi,k(f)w
o + σ2n,i, (4.40)
where pk,i = [pk1(i), ..., pkNtx(i)] ∈ RNtx is the vector of path-loss coefficients
between every transmitter and the i-th receiver, σ2n,i is the noise power at the
i-th receiver node, and
bi,k(f) = pi,k ⊗ b0(f). (4.41)
Expression (4.40) models the power received by node i in terms of an unknown
vector wo; this vector represents the expansion of the received power in the basis
defined by the bi,k(f).
189
4.5 ATC Diffusion for Adaptive Spectrum Estimation
4.5 ATC Diffusion for Adaptive Spectrum Estimation
At every time instant k, every node i observes noisy measurements of the
PSD Ii(f, k) described by (4.40) over Nc frequency samples fm = fmin : (fmax −
fmin)/Nc : fmax, for m = 1, . . . , Nc, according to the model:
dmi (k) = bTi,k(fm)wo + σ2n,i + vmi (k) (4.42)
where wo is the true vector parameter, and vmi (k) is a zero mean random variable
with variance σ2v,m. The temporal index k in the regressor expression bTi,k(fm)
takes into account the possibility of node mobility and possible variations in the
channel conditions over time. The receiver noise power σ2n,i can be pre-estimated
with high accuracy using an energy detection over an idle band. It can then
be removed from expression (4.1). Collecting measurements over Nc contiguous
channels, we obtain a vector linear model:
di,k = Bi,kwo + vi,k (4.43)
where Bi,k = [bTi,k(fm)]Ncm=1 ∈ R
Nc×JNtx and vi,k is a zero mean random vector
with covariance matrix Rv,i. Given the interference measurements di,k across
all M secondary users, these users can now cooperate to estimate the modeling
vector wo in a distributed and adaptive manner. They can do so by seeking to
minimize the the following cost function:
JE(w) =M∑
i=1
E‖di,k −Bi,kw‖2 (4.44)
where E(·) denotes the expectation operator. The minimization of (4.44) can be
computed using a centralized algorithm, which can be run by a fusion center once
all nodes transmit their data di,k,Bi,k, for all k, to it. However, our emphasis
is on a distributed solution, where the nodes estimate the interference profile
by relying solely on in-network processing within their neighborhoods. Several
diffusion adaptation schemes have been developed for such purpose in [143,145].
190
4.5 ATC Diffusion for Adaptive Spectrum Estimation
In this paper, we employ a vector version of the Adapt-then-Combine (ATC) al-
gorithm without measurement exchange from [145]. For the vector minimization
problem in (4.44), the ATC algorithm reads as follows:
ψi,k = wi,k−1 + µiH i,kBTi,k[di,k −Bi,kwi,k−1] (adaptation step)
wi,k =∑
j∈Nicijψj,k (combination step)
(4.45)
where µi is a positive step-size chosen by node i, and Hi,k ∈ RJNtx×JNtx can be
properly chosen to normalize the algorithm in (5.6). The first step in (5.6) involves
local adaptation where node k updates using the new observations di,k,Bi,k.
The choiceH i,k = I, for all i, leads to the classical ATC diffusion scheme, whereas
by selecting Hi,k = (BTi,kBi,k)
†, where (·)† denotes the pseudo-inverse operation,
leads to an approximate Newton version with improved convergence speed. The
second step in (5.6) is a combination step where the intermediate estimates ψj,k,
from the neighborhood j ∈ Ni, are combined through the coefficients cij. The
combination matrix C = cij ∈ RM×M satisfies cij ≥ 0 if j ∈ Ni and 1TC = 1T .
The resulting estimate of node i at time k is denoted by wi,k. In the case in which
the unknown parameter wo varies slowly with time, the ATC diffusion algorithm
allows online tracking of the interference profile variations.
4.5.1 Performance Analysis
In this section we analyze the performance of the normalized diffusion algo-
rithm (5.6) following the approach of [145] by extending it to the case of a vector
linear model as in (4.43). In what follows we view the estimates wi,k as realiza-
tions of a random process ωk,i and analyze the performance of the algorithms in
terms of their mean square behavior.
We consider a general algorithmic form that includes various normalized diffu-
sion algorithms as special cases. Thus, we consider a general normalized diffusion
191
4.5 ATC Diffusion for Adaptive Spectrum Estimation
filter of the form:
φi,i =
M∑
j=1
c1,j,iwj,k−1
ψi,k = φi,k + µi
M∑
j=1
sj,iH i,kBTi,k[di(k)−Bi,kφi,k] (4.46)
wi,k =
M∑
j=1
c2,j,iφi,k
i = 1, . . . ,M , where the coefficients c1,j,i,sj,i and c2,j,i are generic non-
negative real coefficients corresponding to the (j, i) entries of the matrices C1,
S, and C2, respectively, satisfying
1TC1 = 1T ,1TS = 1T ,1TC2 = 1T . (4.47)
Different types of algorithms can be obtained as special cases of (4.46) by choosing
different matrices C1,S,C2. The ATC diffusion algorithm without measure-
ment exchange in (5.6) is obtained by choosing C1 = S = I and C2 = C. To
proceed with the analysis, we assume a linear measurement as in (4.43). Us-
ing (4.46), we define the error quantities wi,k = wo − wi,k, ψi,k = wo − ψi,k,
φi,k−1 = wo − φi,k−1 and the global vectors:
wk =
w1,k
...
wM,k
, ψk =
ψ1,k...
ψM,k
, φk−1 =
φ1,k−1...
φM,k−1
. (4.48)
We also introduce the diagonal matrix
M = diagµ1IJNtx , . . . , µMIJNtx, (4.49)
and the extended weighting matrices
C1 = C1 ⊗ IJNtx , C2 = C2 ⊗ IJNtx, S = S ⊗ IJNtx . (4.50)
192
4.5 ATC Diffusion for Adaptive Spectrum Estimation
We further introduce the matrices
Dk = diag
M∑
j=1
sj,iHj,kBTj,kBj,k
M
i=1
, (4.51)
gk = ST· col
H i,kBTi,kvi,k
M
i=1. (4.52)
The matrices H i,k and Bi,k depend on the path loss vector pi,k ∈ RNtx of node
i at time k. Let γk =[
pT1,k, . . . ,pTM,k
]T∈ R
MNtx denote the overall path loss
vector of the network at time k. Then, the noise vector gk is a function of γk and
we have
φk−1 = CT1 wk−1 (4.53)
ψk = φk−1 − M [Dk(γk)φk−1 + gk(γk)] (4.54)
wk = CT2 ψk (4.55)
or, equivalently,
wk = CT2 [I − MDk(γk)]C
T1 wk−1 − C
T2 Mgk(γk). (4.56)
Mean Stability
Assuming the regression data are spatially and temporally white and taking
the expectation of (4.56), we get
Ewk = CT2 [I − MEDk(γk)]C
T1 Ewk−1 (4.57)
The algorithm (4.56) converges in the mean if the matrix CT2 [I−MDk(γk)]C
T1 ,
where Dk(γk) = EDk(γk), is a stable matrix for all k. To proceed, we call upon
results from [145], [147], and [148]. Following [147], let z = colzT1 , . . . ,zTM
T
denote a vector that is obtained by stacking M subvectors on top of each other
(as is the case with wk). The block maximum norm of z is defined as
‖z‖b,∞ = max1≤k≤M ‖zk‖, (4.58)
193
4.5 ATC Diffusion for Adaptive Spectrum Estimation
where ‖ · ‖ denotes the Euclidean norm of its vector argument. Likewise, the
induced block maximum norm of a block matrix B is defined as
‖B‖b,∞ = maxz 6=0‖Bz‖b,∞‖z‖b,∞
. (4.59)
Following [148] and applying the triangle inequality of norms to (4.57), we find
that
‖Ewk‖b,∞ ≤ ‖CT2 ‖b,∞ · ‖I − MDk(γk)‖b,∞ · ‖C
T2 ‖b,∞ · ‖Ewk−1‖b,∞. (4.60)
It was argued in [147] that ‖CT1 ‖b,∞ and ‖C
T2 ‖b,∞ are bounded by one. Therefore,
for the vector Ewk to converge to zero, it suffices to require
supk‖I − MDk(γk)‖b,∞ < 1. (4.61)
Then, exploiting the inequality in (4.61) and the structure of the matrices in
(4.49) and (4.51), we conclude that wk is asymptotically unbiased if the step-size
satisfy
0 < µi <2
λmax(Dk(γk)), for all i, k, (4.62)
where λmax(X) denotes the maximum eigenvalue of a Hermitian matrix X. If
H i,k in (5.6) is chosen such that H i,k = (BTi,kBi,k)
†, exploiting the expression
in (4.51), we have that the algorithm converges in the mean for any step-size
satisfying
0 < µi < 2, for all i. (4.63)
Mean-Square Performance
In this section we examine the mean-square performance of the adaptive diffu-
sion filter (4.46). Following the energy conservation arguments of [143,145,191],
we evaluate the weighted norm of wk:
E‖wk‖2Σ
= E‖wk−1‖2C1(I−Dk(γk)M)T C2ΣC
T2 (I−MDk(γk))C
T1
+
+ E[gTk (γk)MC2ΣCT2 Mgk(γk)] (4.64)
194
4.5 ATC Diffusion for Adaptive Spectrum Estimation
where Σ is an arbitrary Hermitian positive-definite matrix that we are free to
choose. If we let
G(γk) = E[gk(γk)gTk (γk)] = S
TEdiagH i,kB
Ti,kRv,iBi,kH
Ti,k
Mi=1S, (4.65)
then we can rewrite (4.64) as a variance relation of the form
E‖wk‖2Σ= E‖wk−1‖
2Σ
′ +Tr[ΣCT2 MG(γk)MC2] (4.66)
where Tr(·) is the trace operator, and
Σ′ = C1C2ΣCT2 C
T1 − C1MC2ΣC
T2 C
T1 − C1C1ΣC
T2 MC
T1
+ C1MC2ΣCT2 MC
T1 . (4.67)
We further introduce the notation
σ = vec(Σ), and Σ = vec−1(σ), (4.68)
where the vec(·) notation stacks the columns of the matrix Σ on top of each
other, and vec−1(·) is the inverse operation. We will also use the notation ‖z‖2σ
or ‖z‖2Σ
to interchangeably denote the same squared weighted norm of a vector
z. Using the Kronecker product property [191]:
vec(UΣV ) = (V T ⊗U)vec(Σ) (4.69)
and the fact that the expectation and vectorization operators commute, we can
rewrite Σ′ in (4.67) as σ′ = vec(Σ′) = Fσ, where the matrix F is given by
F = (C1 ⊗ C1)I − I ⊗ (Dk(γk)M)− (Dk(γk)M )⊗ I +
+ E(Dk(γk)M)⊗ (Dk(γk)M )(C2 ⊗ C2). (4.70)
Assumption : The path loss vector γk → γ0, where γ0 is a fixed constant
vector, as k →∞.
Then, using the property Tr(ΣX) = vec(XT )Tσ and taking the limit of
(4.66) as k →∞, we can recast (4.66) as follows:
E‖w∞‖2σ = E‖w∞‖
2Fσ
+ [vec(CT2 MG(γ0)
TMC2)]Tσ. (4.71)
195
4.6 Conclusion
The steady-state mean-square deviation (MSD) at node k is defined as:
MSDi = limk→∞
E‖wi,k‖2. (4.72)
Then, if the step sizes µi are small enough so that the matrix (I − F ) is
invertible and choosing σ = (I − F )−1mi, the MSD of node i tends to
MSDi = [vec(CT2 MG(γ0)
TMC2)]T (I − F )−1mi, (4.73)
where mi = vec(diag(ei)⊗ IJNtx), with ei denoting the column vectors with a
unity entry at position i and zeros elsewhere. The network MSD is defined as the
average MSD across all nodes in the network and is given by
MSDnet =1
M
N∑
i=1
MSDi. (4.74)
Then, from (4.71), the network MSD can be obtained as:
MSDnet =1
M[vec(C
T2 MG(γ0)
TMC2)]T (I − F )−1m, (4.75)
where m = vec(IJNtx ⊗ IJNtx).
In the following chapter, we will illustrate how these theoretical expressions
match well with simulation results.
4.6 Conclusion
The key challenge in developing cognitive wireless transceivers is enabling
them to sense the ambient power spectral density at arbitrary locations in space.
In this chapter, we addressed this challenging task through a parsimonious basis
expansion model of the PSD in frequency. This model reduces the sensing task
to estimating a common vector of unknown parameters. The resulting estimator
relies on diffusion adaptation algorithms, where the cognitive radios exchange
information locally only with their one-hop neighbors, eliminating the need for a
fusion center.
196
4.6 Conclusion
First, we have proposed a novel class of diffusion LMS strategies, regularized
by convex sparsifying penalties, for distributed estimation over adaptive net-
works. Convergence and mean square analysis of the sparse adaptive diffusion
filter show under what conditions we have dominance of the proposed method
with respect to its unregularized counterpart in terms of steady-state perfor-
mance. Two different penalty functions have been employed, the ℓ1-norm, which
uniformly attracts to zero all the vector elements, and a reweighted function,
which selectively shrinks only the elements with small magnitude. Numerical re-
sults show the potential benefits of using such strategies. Other penalty functions
can also be useful. Adaptive diffusion strategies for the distributed optimization
of convex cost functions are further considered in [154].
Then, we have illustrated the proposed distributed spectrum estimation tech-
nique based on diffusion adaptation. We introduce a basis expansion model,
which is useful to model the PU’s transmission, allowing distributed cooperative
sensing. Then, we have proposed a normalized version of the Adapt then Com-
bine (ATC) diffusion algorithm, which enables the network to learn and track
the time-varying interference profile. Convergence and mean-square performance
analysis of the proposed normalized ATC diffusion filter, applied to the spectrum
estimation problem, has been derived.
197
198
Chapter 5
Swarming for Dynamic Radio
Access Based on Diffusion
Adaptation
The goal of this chapter is to study the learning abilities of adaptive networks
in the context of cognitive radio networks and to investigate how well they assist
in allocating power and communications resources in the frequency domain. The
allocation mechanism is based on a social foraging swarm model that lets every
node allocate its resources (power/bits) in the frequency regions where the inter-
ference is at a minimum while avoiding collisions with other nodes. We employ
adaptive diffusion techniques to estimate the interference profile in a cooperative
manner and to guide the motion of the swarm individuals in the resource domain.
The resulting bio-inspired network cooperatively estimates the interference pro-
file in the resource domain of a cognitive network and allocates resources through
purely decentralized mechanisms. The proposed approach endows the cognitive
network with powerful learning and adaptation capabilities, allowing fast reaction
to dynamic changes in the spectrum.
199
5.1 Swarm Model
The basic contributions of this chapter are:
1) the extension of the social foraging model proposed in chapter 3 to incor-
porate a real-time distributed spectrum estimation technique based on diffusion
adaptation;
2) the application of the proposed procedure to the dynamic resource allocation
problem in the frequency domain.
The chapter is organized as follows. In Section 5.1 we describe the swarm model,
formulating the search of available time/frequency slots as the distributed mini-
mization of a time-varying global potential function. Section 5.2 illustrates the
proposed distributed spectrum estimation technique based on diffusion adapta-
tion, which enables the network to learn and track the time varying interference
profile. In section 5.3, combining the diffusion step from section 5.2 and the
swarming behavior from section 5.1, we illustrate how the proposed adaptive
swarming algorithm performs dynamic resource allocation. Section 5.4 provides
some numerical examples aimed to illustrate the theoretical findings and assess
the performance and adaptation capabilities of the proposed technique.
5.1 Swarm Model
In this section, we describe an improved version of the swarm-based resource
allocation strategy proposed in chapter 2, extending the swarming algorithm to
dynamic environments. For this purpose, we denote by Ii(xi, t) ∈ C1 : Rn ×
R → R the interference power over the slot having coordinate vector xi (e.g.,
a frequency subchannel or a time-frequency slot) perceived by node i at time t.
The resource allocation problem can then be formulated mathematically as the
search of the resource vector x, from the whole population of M cognitive nodes,
that minimizes in a distributed fashion the global function:
J(x, t) =
M∑
i=1
Ii(xi, t) +1
2
M∑
i=1
M∑
j=1
aijJar(‖xj − xi‖), (5.1)
200
5.1 Swarm Model
where x := (xT1 , . . . ,x
TM )T , and the second term on the right-hand side of (3.2)
incorporates an attraction/repulsion potential function given by
Jar(‖xj − xi‖) = Ja(‖xj − xi‖)− Jr(‖xj − xi‖). (5.2)
This potential incorporates a short range repulsion term Jr(‖xj − xi‖), whose
effect is to avoid collisions among the cognitive nodes, and a long range attraction
term Ja(‖xj − xi‖), whose goal is to induce a swarm cohesion behavior, e.g. to
avoid an excessive spread of the selected radio resources in the time-frequency
domain. In summary, minimizing (3.2) leads each node to dynamically allocate
its own resources in time-frequency regions where there is less interference, helps
to avoid conflicts among users, and limits the spread of the occupied domain.
Remark : Differently from the swarm model proposed in chapter 3, the function
(5.1) takes into account the time variability of the interference profile Ii(xi, t)
sensed by each node. This is an important difference because it enables the
swarm to dynamically allocate in the resource domain, tracking the changes in
the interfering environment and reacting to them. A similar scenario arises in the
modeling of bird flight formations through adaptive networks [149, 150], where
the total upwash (cost) function evolves dynamically as the birds move in search
of the optimal (peak) location.
Considering again the swarm analogy, the occupied zones in the resource domain
take the role of dangerous regions that must be avoided by the swarm individuals
as fast as possible, while idle bands represent regions rich of food that the agents
have to occupy reducing their speed. Mimicking this natural learning capability,
we consider a distributed minimization of (3.2) based on a scaled gradient descent
optimization, so that every node starts with an initial guess, let us say xi[0], and
then updates its resource allocation vector xi in time according to the following
discrete-time implementation:
xi[k + 1] = xi[k]− αi[k]∇xiJ(x, kT ) (5.3)
201
5.2 Diffusion Adaptation for Cooperative Spectrum Sensing
= xi[k]− αi[k]
∇xiIi(xi, kT )−M∑
j=1
aij g(xj[k]− xi[k])
,
i = 1, . . . ,M , where T is the sampling time, k is the time index, αi[k] > 0 is a
positive iteration-dependent step-size and g(·) is a vector function defined as in
(3.4). The step size is given by
αi[k] = f(Ii(xi, kT )) ∈ [αmin, αmax] > 0, (5.4)
where f(·) is a monotonically increasing function of the interference power per-
ceived at time i by every node at its current position on the resource domain.
Examples include linear, quadratic, logarithmic functions, etc. The goal is to
accelerate the motion of the resources perceiving a high interference and, at the
same time, to slow down the resources that are allocating on idle sub-bands. This
adaptive behavior considerably improves the reaction capability of the algorithm
to changes in the environment. In this chapter, we consider dynamic swarm-
ing in the frequency domain, where xi ∈ R is a scalar denoting the position of
the i-th resource on the frequency axis. To update the swarming behavior in
(5.3), the SUs need to estimate the interference profile Ii(xi, kT ) and its gradi-
ent ∇xiIi(xi, kT ) on the current position xi in the frequency domain, at time k.
Then, in the next section, we show how to adaptively estimate these quantities
in a distributed manner and through local cooperation.
5.2 Diffusion Adaptation for Cooperative Spectrum
Sensing
As shown in the previous chapter, At every time instant k, every node i
observes noisy measurements of the PSD Ii(f, k) described by (4.40) over Nc
frequency samples fm = fmin : (fmax − fmin)/Nc : fmax, for m = 1, . . . , Nc,
according according to the vector linear model:
di,k = Bi,kwo + vi,k (5.5)
202
5.2 Diffusion Adaptation for Cooperative Spectrum Sensing
where Bi,k = [bTi,k(fm)]Ncm=1 ∈ R
Nc×JNtx and vi,k is a zero mean random vector
with covariance matrix Rv,i. The temporal index k in the regressor expression
Bi,k takes into account the possibility of node mobility and possible varia-
tions in the channel conditions over time. Given the interference and regression
measurements di,k,Bi,k across all M secondary users, these users can now co-
operate to estimate the modeling vector wo in a distributed and adaptive manner.
For this purpose, we employ a vector version of the Adapt-then-Combine (ATC)
algorithm without measurement exchange from [145]. For the vector minimiza-
tion problem in (4.44), the ATC algorithm reads as follows:
ψi,k = wi,k−1 + µiH i,kBTi,k[di,k −Bi,kwi,k−1] (adaptation step)
wi,k =∑
j∈Nicijψj,k (combination step)
(5.6)
where µi is a positive step-size chosen by node i, and Hi,k ∈ RJNtx×JNtx can be
properly chosen to normalize the algorithm in (5.6). In the case in which the un-
known parameter wo varies slowly with time, the ATC diffusion algorithm allows
online tracking of the interference profile variations, thus enabling the swarming
agents to dynamically allocate resources on the frequency domain. Using its local
estimates wi,k, and using (4.40), every node i can determine an estimate for the
transmitted interference profile and of its derivative at the frequency location
fm = xi at time i as:
Ii(xi, k) =
Ntx∑
q=1
piq(k)
J∑
j=1
bj(xi)wq,ji,k , (5.7)
dIi(xi, k)
dxi=
Ntx∑
q=1
piq(k)
J∑
j=1
b′j(xi)wq,ji,k . (5.8)
where b′j(f) is the known derivative of the j-th basis function. Expressions (5.7-
5.8) can be used in (3.3) to update the swarming behavior. In wide area ad-hoc
networks, the PU’s communications reach only a subset of SU’s with a high inter-
fering power. In such scenario, estimating the received spectrum through (5.7-5.8)
would enable remote cognitive users to dynamically reuse the PU’s resources.
203
5.3 Adaptive Swarming
5.3 Adaptive Swarming
Combining the adaptive diffusion step (4.3), for estimating and tracking the
interference profile, with the swarming update (3.3), for dynamic resource allo-
cation, we arrive to the following adaptive swarming algorithm to coordinate the
learning and adaptation capabilities of the bio-inspired cognitive network.
Algorithm 1 : Adaptive swarming algorithm
For each node i, start with wi,−1 = 0,ψi,−1 = 0, xi[0] ∈ [fmin, fmax]. Every
node i then performs the following steps for k ≥ 0:
1. The node knows the position xi[k] of its resource on the frequency domain
and has access to the local data di,k,Bi,k.
2. Perform an adaptation step to adaptively estimate the weight vector wo as:
ψi,k = wi,k−1 + µiH i,kBTi,k[di,k −Bi,kwi,k−1], (5.9)
wi,k =∑
j∈Ni
cijψj,k. (5.10)
3. Compute estimates of the interference spectrum and of its derivative in the
current position xk at time i as:
Ii(xi, k) =
Ntx∑
q=1
piq(k)
J∑
j=1
bj(xi)wq,ji,k , (5.11)
dIi(xi, k)
dxi=
Ntx∑
q=1
piq(k)J∑
j=1
b′j(xi)wq,ji,k . (5.12)
4. Update the position of resource i on the frequency axis as:
xi[k + 1] = xi[k]− αi[k]
dIi(xi, k)
dxi−
M∑
j=1
aij g(xj [k]− xi[k])
. (5.13)
204
5.4 Simulation Results
5.4 Simulation Results
In this section, we provide numerical examples to illustrate the main fea-
tures of the proposed technique combining the swarm-based resource allocation
method, illustrated in Section 4.1, and the distributed cooperative sensing algo-
rithm using ATC diffusion adaptation, shown in Section 4.2.
Numerical Example 1 - Performance : In the following examples, we aim to show
the performance of the cognitive network based on the adaptive swarming algo-
rithm. We consider a connected network composed of 15 SUs, plus the inclusion
of two PUs. The topology of the network is shown in Fig. 5.1, where the SUs
are depicted by dots, whereas the PUs by squares. In particular, a PU moves
from the initial position given by the white square to the final position depicted
by the black square, so that the interference perceived by the secondary network
is time-varying. We consider a polynomial path loss model piq(diq) = (diq/d0)−α
(α = 2), where diq is the distance between the q-th PU and the k-th SU, and
d0 = 1 is a reference distance. The cognitive SU’s scan Nc = 80 channels be-
tween 30 and 45 MHz and use J = 15 Gaussian basis functions to model the
basis expansion of the transmitted spectrum. The single Gaussian basis function
is expressed as:
bj(f) =A
√
2πσ2b
exp
(
−(f − fj)
2
2σ2b
)
. (5.14)
with amplitude normalized to one, and σ2b = 0.5. An example of basis expansion
using Gaussian pulses is shown in Fig. 4.4. To estimate the spectrum profile, we
employ the normalized ATC diffusion algorithm in (4.3) where the step sizes are
set equal to µk = 1, for all k. Furthermore, we consider a combination matrix
C that simply averages the intermediate estimate from the neighborhood, hence,
cj,i = 1/Ni for all l. The white Gaussian noise variance in (4.1) is set equal
to σ2n,i = 0.5 × 10−3, for all i. We assume the presence of 15 resources (to be
allocated) that are initially scattered randomly across the frequency spectrum.
At the k-th iteration of the updating rule (3.3), each node communicates to its
205
5.4 Simulation Results
0 0.2 0.4 0.6 0.8 1
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
PU
PU
Figure 5.1: Secondary network. The square nodes denote primary users and the
dot nodes denote secondary users.
neighbors the position it intends to occupy, i.e., the scalar xi[k] representing a fre-
quency subchannel. In the application at hand, there is an intrinsic quantization
of the frequency resources given by the subchannel bandwidth. In our implemen-
tation, we let the system evolve according to (3.3) until successive differences in
allocation become smaller than the bandwidth of a frequency subchannel. At that
point, the evolution stops and every SU is allowed to transmit over the selected
channel. We consider an interference profile as in Fig. 5.2, where the dashed curve
depicts the true transmitted spectrum, whereas the solid and dot dashed curves
represent, respectively, the estimation at convergence through ATC diffusion and
without cooperation among nodes. We notice how diffusion adaptation fits well
the spectrum profile while the non-cooperative approach leads to poor estimation.
To evaluate the performance of the distributed estimation technique, in Fig. 5.3
we show the steady-state MSD of the ATC diffusion algorithm compared with the
theoretical result in (4.73). The steady-state values are obtained by averaging
206
5.4 Simulation Results
30 35 40 45−0.5
0
0.5
1
1.5
2
2.5
3
3.5
Frequency (Mhz)
PS
D (
mW
/Hz)
True Spectrum
ATC Diffusion
No Cooperation
Figure 5.2: Comparison of the result of spectrum estimation through cooperative
diffusion adaptation and without cooperation among the users.
over 200 independent experiments and over 100 time samples after convergence.
It can be observed that the simulation results match well the theoretical values.
An example of resource allocation is shown in Fig. 5.2, where the dots on the
frequency axis represent the final frequency sub-channels chosen at convergence
by the network nodes. The parameters of the swarm are cA = 0.025, cR = 0.25.
We also considered a fixed step size ε0 = 0.05, thus leading to a simple gradient
descent version of the algorithm in (3.3). It is evident how the resources avoid the
positions occupied by primary users, tend to keep the spread as small as possible
while avoiding collisions among the allocations of different users. Observe that
the number of allocated channels is less than the number of requested resources.
This means that a certain number of nodes have picked up the same channels.
We have checked numerically via simulations that, by choosing appropriately the
swarm parameters, the final channel allocation does not lead to collisions among
spatial neighbors. This means that the algorithm is capable of implementing a
decentralized mechanism for spatial reuse of frequencies.
To measure the effectiveness of the distributed resource allocation strategy, in
207
5.4 Simulation Results
2 4 6 8 10 12 14−40
−38
−36
−34
−32
−30
Node index
MS
D (
dB
)
Simulation results
Theoretical values
Figure 5.3: Steady-state MSD versus node index.
Fig. 5.4 we report the interference level, versus the number of nodes composing
the secondary network, perceived over the frequency slots occupied by the SUs,
after convergence. The result is averaged over 200 independent realizations. We
considered two different values of the receiver noise power σ2n, which determines
the variance of the estimation noise vmi (k) in (4.1). The parameters of the swarm
are cA = 0.025, cR = 0.25 and the interference profile is the same considered in
Fig. 5.2. From Fig. 5.4, we notice that, using a non cooperative approach, the
estimation of the interference profile gradient is quite poor and some resources
end up being allocated by mistake in the regions occupied by the primary users,
trapped because of the estimation errors affecting the algorithm. This explains
the high level of interference perceived in this case. The performance of the allo-
cation can be remarkably improved adopting the cooperative diffusion adaptation
approach. Indeed, as the estimation accuracy improves, each resource tends to
move towards the interference-free regions, thus making the overall swarm ex-
perience a smaller total interference. As the number of nodes N increases, the
allocation performance improves because the swarming algorithm exploits a co-
operative capability given by the cohesion force. This intrinsic robustness deter-
mines that the agents, allocated over the low interference bands, tend to form
cohesive blocks that exert an attraction towards the agents trapped by mistake
over the regions of the spectrum occupied by the primary users. Moreover, in the
208
5.4 Simulation Results
10 12 14 16 18 20 22 24 26 28 30
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
Number of nodes
Avera
ge inte
rfere
nce (m
W)
No Cooperation σ2 = 0.1mW
ATC Diffusion σ2 = 0.1mW
No Cooperation σ2 = 0.5mW
ATC Diffusion σ2 = 0.5mW
Figure 5.4: Average interference perceived by the swarm at convergence, for the
non cooperative estimation case and for adaptive diffusion.
cooperative case, an increase in the number of nodes also improves the estimation
performance, thus simplifying the resource allocation task. From Fig. 5.4, we
also note, as expected, how a stronger noise leads to worst allocation performance
in both cases. Nevertheless, the performance of the cooperative approach is less
sensitive. This means that the performance of the resource allocation based on
the swarming algorithm can be considerably improved if every node cooperates
with its own neighbors to adaptively estimate the interference profile.
Numerical Example 2 - Learning and adaptation : In this example, we aim to
show the learning and adaptation capability of the cognitive network based on
the adaptive swarming algorithm. Natural swarms are adaptive systems whose
individuals cooperate in order to improve their food search capabilities and to
increase their robustness against predators’ attacks. We show next that the
proposed resource allocation increases, as a by-product, the network robustness
against the intrusion of a primary user (predator). We consider again the network
topology depicted in Fig. 5.1, where the two PU’s start emitting at different
209
5.4 Simulation Results
30 35 40 45−0.5
0
0.5
1
1.5
2
2.5
3
3.5
Frequency (MHz)
PS
D (
mW
/Hz)
True spectrum
ATC Diffusion
30 35 40 45−0.5
0
0.5
1
1.5
2
2.5
3
3.5
Frequency (MHz)
PS
D (
mW
/Hz)
True spectrum
ATC Diffusion
30 35 40 45
0
1
2
3
Frequency (MHz)
PS
D (
mW
/Hz)
True spectrum
ATC Diffusion
30 35 40 45
0
1
2
3
Frequency (MHz)
PS
D (
mW
/Hz)
True spectrum
ATC Diffusion
i = 125 i = 375
i = 625 i = 875
Figure 5.5: Different resource assignments in dynamic environment.
times, thus causing a dynamic change of the occupied spectrum. Our goal is to
test the dynamic response of the network to this changing environment. The
parameters are the same considered in the previous simulation. In Fig. 5.5 we
show an example of spectrum estimation and swarm-based resource assignment
in the case the PU’s interference is dynamic. As before, the dots on the frequency
axis represent the final channels chosen at convergence by the network nodes. At
the beginning of time, the two PU’s are silent, and the first PU starts to transmit
only at the iteration i = 250. The first PU becomes silent at iteration i = 500
while, at the same time, the second PU starts to transmit. After iteration i = 750,
the two PU’s are both transmitting at the same time. In particular, in Fig. 5.5
it is shown the evolution of the spectrum estimation and resource allocation
210
5.4 Simulation Results
0 200 400 600 800 1000−40
−35
−30
−25
−20
−15
−10
−5
0
5
10
15
iteration index
MS
D (
dB
)
Figure 5.6: ATC diffusion learning curve, in terms of MSD.
at 4 different time iterations, i = 125, 375, 625, 875. We notice from Fig. 5.5
how diffusion adaptation fits always well the spectrum profile, thus proving good
tracking performance. To give an example of the tracking capability of the ATC
diffusion filter, in Fig. 5.6, we show the learning curve of the algorithm in terms
of MSD. As we can see, ATC diffusion reacts to the changes in the environment,
learning the spectrum profile through local cooperation. At the same way, from
Fig.5.5, we also notice how the swarm reacts to the PU’s activations, avoiding to
select radio channels occupied by primary transmissions. Resorting again with
the swarm analogy, PU’s take now the role of predators whose positions must be
avoided by the swarm individuals. In this context it is reasonable that the swarm
agents closer to the predator’s positions move faster to avoid the dangerous zones.
To allow this adaptive swarming behavior, in this example, we have considered
a time-varying step size that depends on the perceived interference through a
linear scaling function, e.g. εi[k] = ai + biIi(xi, k), for all i, where ai = 0.05 and
bi = 1. As a consequence, the swarm model in (3.3) accelerates the motion of
the resources perceiving a high interference, improving the reaction time needed
211
5.5 Conclusion
0 200 400 600 800 1000−0.1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
iteration index
Avera
ge inte
rfere
nce (
mW
)
Figure 5.7: Average perceived interference versus iteration index.
by the algorithm to perform a resource allocation on idle bands in case of a PU’s
activation. To give an example of the time needed by the algorithm to react to
the PU’s intrusions and adjust the resource allocation consequently, in Fig. 5.7
we show the behavior of the instantaneous interference perceived by the swarm
versus the iteration index. The three peaks correspond to the PU’s activation
times. From Fig. 5.7, we notice how the adaptive swarming algorithm needs
only a small number of iterations to leave the regions occupied by the PU’s.
This positive behavior is a consequence of the adaptation of the algorithm to the
perceived interference, determining that resources allocating on high interference
regions move faster due to the increment of the profile gradient and the cohesion
force.
5.5 Conclusion
In this chapter we have proposed a dynamic resource allocation technique
combining a distributed diffusion algorithm, for implementing cooperative sens-
212
5.5 Conclusion
ing, with a swarming technique, for allocating resources in a parsimonious way
(i.e., avoiding unnecessary spread in the frequency domain), yet avoiding col-
lisions. In the swarm analogy, the dynamic interference distribution over the
frequency domain takes the role of the food spatial distribution. Furthermore,
the occupied zones in the resource domain take the role of dangerous regions that
must be avoided by the swarm individuals as fast as possible, while idle bands
represent regions rich of food that the agents have to occupy reducing their speed.
The swarm mechanism includes an attraction force, useful to minimize the spread
over the resource domain, and a repulsion force, useful to avoid collisions among
swarm members. We employ a diffusion adaptation scheme, which estimates and
learn the interference profile through local cooperation, to guide the movement
of the swarm individuals. A mean-square performance analysis for the diffusion
adaptation filter has been derived and simulation results match well with the
theoretical results. Finally, the procedure has been applied to the dynamic re-
source allocation problem in the frequency domain. Numerical results show the
improvement that results in the resource allocation performance due to the coop-
erative estimation of the spectrum. Furthermore, it is shown how the proposed
technique endows the resulting bio-inspired network with powerful learning and
adaptation capabilities.
213
214
Chapter 6
Concluding Remarks
This dissertation has considered bio-inspired techniques for dynamic radio
access in cognitive radio systems. Both resource allocation and spectrum sensing
tasks have been considered. For the former, it has been proposed a radio access
mechanism that mimics the behavior of a flock of birds swarming in search for food
in a cohesive fashion without colliding with each other; specifically, the method
has been employed for dynamic radio access in the frequency and time-frequency
domains and detailed stability and convergence analysis has been provided, even
in the presence of random disturbances introduced by realistic radio channels.
For the latter, a distributed method based on diffusion adaptation algorithms
has been developed. First, we have introduced a basis expansion model of the
PU’s PSD, then we have proposed a normalized version of the Adapt then Com-
bine (ATC) diffusion algorithm, which enables the network to learn and track
the time-varying interference profile. Convergence and mean-square performance
analysis of the proposed normalized ATC diffusion filter, applied to the spectrum
estimation problem, has also been derived. Finally, we have extended the swarm
based resource allocation to incorporate a real-time distributed technique for
spectrum estimation based on diffusion adaptation. The proposed procedure has
been applied to the dynamic resource allocation problem in cognitive radio, en-
215
6.1 Conclusions
dowing the resulting bio-inspired network with powerful learning and adaptation
capabilities.
6.1 Conclusions
After giving the motivation of the dissertation in Chapter 1, an overview of the
recent advances in the field of bio-inspired signal processing and networking has
been presented.
Chapter 2 has recalled several important theories on which many results of this
dissertation are based. In particular, basic results have been reported from dy-
namical systems theory, distributed nonlinear optimization, stochastic approxi-
mation theory, and graph theory, which have been be largely used in this disser-
tation.
Chapter 3 has proposed a bio-inspired radio access mechanism for cognitive net-
works mimicking the behavior of a flock of birds swarming in search for food in
a cohesive fashion without colliding with each other. The equivalence between
swarming and radio resource allocation is established by modeling the interference
distribution in the resource domain, e.g. frequency and time, as the spatial distri-
bution of food, while the position of the single bird represents the radio resource
chosen by each radio node. The solution is given as the distributed minimization
of a functional, borrowed from social foraging swarming models, containing the
average interference plus repulsion and attraction terms that help to avoid con-
flicts and maintain cohesiveness, respectively. A stability and cohesion analysis is
derived under different assumptions on the attraction/repulsion terms, showing
the effect played by the swarm parameters and connectivity on the final swarm
size. Several examples have illustrated how the proposed method can be applied
to dynamic resource allocation on the frequency domain and the time-frequency
domain, providing an intrinsic capability of the system to provide spatial reuse
216
6.1 Conclusions
of frequency, through a purely decentralized mechanism. In the last part of the
chapter, we have also considered the swarming algorithm in the presence of chan-
nel imperfections, such as link failures, estimation errors, and quantization noise.
Thus, we have derived the almost sure convergence of the swarming procedure to
an equilibrium configuration dependent on the mean graph of the network, even
in the presence of such random disturbances.
Chapter 4 has developed adaptive methods for spectrum estimation in cognitive
radio networks based on diffusion adaptation algorithms. In particular, it has
been addressed this task through a parsimonious basis expansion model of the
PSD in frequency. This model reduces the sensing task to estimating a common
vector of unknown parameters. The resulting estimator relies on diffusion adapta-
tion algorithms, where the cognitive radios exchange information locally only with
their one-hop neighbors, eliminating the need for a fusion center. First, we have
described the basic diffusion algorithm, then we have introduced novel regular-
ized diffusion LMS techniques for distributed estimation over adaptive networks,
which are able to exploit sparsity in the underlying system model. Convergence
and mean square analysis of the sparse adaptive diffusion filter have shown un-
der what conditions we have dominance of the proposed method with respect to
its unregularized counterpart in terms of steady-state performance. Simulation
results have also confirmed the potential benefits of the proposed filter under the
sparsity assumption on the true coefficient vector. Exploiting these estimation
schemes, we have illustrated the proposed distributed spectrum estimation tech-
nique based on diffusion adaptation. We have first introduced a basis expansion
model, which is useful to model the PU’s transmission, allowing distributed co-
operative sensing. Then, we have proposed a normalized version of the Adapt
then Combine (ATC) diffusion algorithm, which enables the network to learn
and track the time-varying interference profile. Convergence and mean-square
performance analysis of the proposed normalized ATC diffusion filter, applied to
the spectrum estimation problem, has also been derived.
217
6.1 Conclusions
Chapter 5 has studied the learning abilities of adaptive networks in the context of
cognitive radio networks and investigated how well they assist in allocating power
and communications resources in the frequency domain. The allocation mech-
anism is based on a social foraging swarm model that lets every node allocate
its resources (power/bits) in the frequency regions where the interference is at a
minimum while avoiding collisions with other nodes. We have employed adaptive
diffusion techniques to estimate the interference profile in a cooperative manner
and to guide the motion of the swarm individuals in the resource domain. The
resulting bio-inspired network cooperatively estimates the interference profile in
the resource domain of a cognitive network and allocates resources through purely
decentralized mechanisms. Finally, the resulting procedure has been applied to
the dynamic resource allocation problem in the frequency domain. Numerical
results have shown the improvement that results in the resource allocation per-
formance due to the cooperative estimation of the spectrum. Furthermore, it
has been shown how the proposed technique endows the resulting bio-inspired
network with powerful learning and adaptation capabilities.
Summarizing, the main results obtained in this dissertation are:
• the application of swarming mechanisms to radio resource allocation in
cognitive radios;
• the stability and cohesion analysis of social foraging swarms, applied to
the resource allocation problem in cognitive radios, which provides upper
and lower bounds on the spread of the swarm, as a function of the swarm
connectivity;
• fast versions of the swarming algorithm, useful for our application, and the
application of such procedures to the dynamic resource allocation in the
frequency domain;
• the application of the proposed procedure to resource allocation in the time-
frequency domain, where the primary users in a cognitive radio system are
218
6.1 Conclusions
modeled as statistically independent homogeneous continuous-time Markov
processes;
• the derivation of the convergence properties of the proposed algorithms in
the presence of random disturbances such as link failures, quantization noise
and estimation errors;
• the application of compressive sensing techniques in the distributed estima-
tion problem over adaptive networks;
• the derivation of mean-square analysis for the sparse diffusion adaptive
filter;
• a real-time distributed spectrum estimation technique based on diffusion
adaptation;
• the derivation of the mean-square properties of the diffusion adaptive filter
applied to the spectrum estimation problem;
• the extension of the basic social foraging swarming model to incorporate
a real-time distributed spectrum estimation technique based on diffusion
adaptation.
219
220
Bibliography
[1] D. Estrin, L. Girod, G. Pottie, and M. Srivastava, “Instrumenting the world
with wireless sensor networks,” in Proc. International Conference on Acoustic,
Speech, and Signal Processing (ICASSP), Salt Lake City, UT, May 2001, vol.4,
pp. 2033–2036.
[2] J. Mitola, “Cognitive radio for flexible mobile multimedia communications,”
Mobile Networks and Applications, vol. 6, no. 5, pp. 435–441, 2001.
[3] S. Haykin, “Cognitive radio: Brain-empowered wireless communications,”
IEEE Jornal on Selected Areas in Communications, vol. 23, no. 2, pp. 201–
220, February 2005.
[4] S. Srinivasa and S. A. Jafar, “The throughput potential of cognitive radio: A
theoretical perspective,” IEEE Communications Magazine, vol. 45, no. 5, pp.
73–79, May 2007.
[5] R. W. Thomas, D. H. Friend, L. A. DaSilva, and A. B. MacKenzie, “Cognitive
networks: Adaptation and learning to achieve end-to-end performance objec-
tives,” IEEE Communications Magazine, vol. 44, no. 12, pp. 51–57, December
2006.
[6] R. L. Ashok, D. P. Agrawal, “Next-Generation Wearable Networks,” IEEE
Computer, vol. 36, no. 11, 2003, pp. 31–39.
221
BIBLIOGRAPHY
[7] B. Atakan, O. B. Akan, “An Information Theoretical Approach for
Molecular Communication,” in 2nd IEEE/ACM International Conference
on Bio-Inspired Models of Network, Information and Computing Systems
(IEEE/ACM BIONETICS 2007), Budapest, Hungary, 2007, pp. 33–40.
[8] I. F. Akyildiz, D. Pompili, T. Melodia, “Underwater acoustic sensor networks:
research challenges,” Elsevier Ad Hoc Networks, vol. 3, no. 3, 2005, pp. 257–
279.
[9] I. F. Akyildiz, I. H. Kasimoglu, “Wireless Sensor and Actor Networks: Re-
search Challenges,“ Elsevier Ad Hoc Networks, vol. 2, 2004, pp. 351–367.
[10] F. Dressler, “A Study of Self-Organization Mechanisms in Ad Hoc and Sensor
Networks,” Elsevier Computer Communications, vol. 31, no. 13, 2008, pp.
3018–3029.
[11] F. Dressler, Self-Organization in Sensor and Actor Networks, John Wiley
and Sons, 2007.
[12] I. F. Akyildiz, W. Su, Y. Sankarasubramaniam, E. Cayirci, “Wireless sensor
networks: a survey,” Elsevier Computer Networks, vol. 38, 2002, pp. 393–422.
[13] I. F. Akyildiz, X. Wang, W. Wang, “Wireless mesh networks: a survey,”
Elsevier Computer Networks, vol. 47, no. 4, 2005, pp. 445–487.
[14] O. B. Akan, I. F. Akyildiz, “Event-to-Sink Reliable Transport in Wireless
Sensor Networks,” IEEE/ACM Transactions on Networking (TON), vol. 13,
no. 5, 2005, 1003–1016.
[15] B. Atakan, O. B. Akan, “Immune System Based Distributed Node and Rate
Selection in Wireless Sensor Networks,” in 1st IEEE/ACM International Con-
ference on Bio-Inspired Models of Network, Information and Computing Sys-
tems (IEEE/ACM BIONETICS 2006), Cavalese, Italy, 2006, pp.1-8.
222
BIBLIOGRAPHY
[16] I. Chlamtac, M. Conti, J. J. Liu, “Mobile ad hoc networking: imperatives
and challenges,” Elsevier Ad Hoc Networks, vol. 1, no. 1, 2003, pp. 13–64.
[17] B. Metcalfe, “The next-generation Internet,” IEEE Internet Computing, vol.
4, no. 1, 2000, pp. 58–59.
[18] F. Dressler and O. B. Akan, “A survey on bio-inspired networking,” Elsevier
Computer Networks, vol. 54, no. 6, pp. 881–900, April 2010.
[19] F. Dressler and O. B. Akan, “Bio-inspired networking: from theory to prac-
tice,” IEEE Communications Magazine, vol. 48, no. 11, pp. 176–183, Novem-
ber 2010.
[20] J. Timmis, M. Neal, J. Hunt, “An Artificial Immune System for Data Anal-
ysis”, Biosystems, vol. 55, 2000, pp. 143–150.
[21] S. Camazine, J.-L. Deneubourg, N. R. Franks, J. Sneyd, G. Theraula,
E. Bonabeau, Self-Organization in Biological Systems, Princeton University
Press, 2003.
[22] E. Bonabeau, M. Dorigo, G. Theraulaz, Swarm Intelligence: From Natural
to Artificial Systems, Oxford University Press, 1999.
[23] M. Dorigo, V. Maniezzo, A. Colorni, “The Ant System: Optimization by
a colony of cooperating agents,” IEEE Transactions on Systems, Man, and
Cybernetics, vol. 26, no. 1, 1996, pp. 1–13.
[24] W. Vogels, R. van Renesse, K. Briman, “The Power of Epidemics: Robust
Communication for Large-Scale Distributed Systems, ACM SIGCOMM Com-
puter Communication Review, vol. 33, no. 1, 2003, pp. 131–135.
[25] S. A. Hofmeyr, S. Forrest, “Architecture for an Artificial Immune System,”
Evolutionary Computation, vol. 8, no. 4, 2000, pp. 443–473.
223
BIBLIOGRAPHY
[26] T. C. Henderson, R. Venkataraman, G. Choikim, “Reaction-Diffusion Pat-
terns in Smart Sensor Networks, in IEEE International Conference on
Robotics and Automation (ICRA 2004), New Orleans, LA, 2004, pp. 654–
658.
[27] C. A. Richmond, “Fireflies Flashing in Unison,” Science, vol. 71, no. 1847,
1930, pp. 537–538.
[28] F. Dressler, “Self-Organized Event Detection in Sensor Networks using Bio-
inspired Promoters and Inhibitors,” 3rd ACM/ICST International Confer-
ence on Bio-Inspired Models of Network, Information and Computing Systems
(Bionetics 2008), ACM, Hyogo, Japan, 2008.
[29] A. Boukerche, H. Oliveira, E. Nakamura, A. Loureiro, “Vehicular Ad Hoc
Networks: A New Challenge for Localization-Based Systems,” Elsevier Com-
puter Communications, vol. 31, no. 12, 2008, pp. 2838–2849.
[30] K. Leibnitz, N. Wakamiya, M. Murata, “Biologically-Inspired Self-Adaptive
Multi-Path Routing in Overlay Networks,” Communications of the ACM,
Special Issue on Self-Managed Systems and Services, vol. 49, no. 3, 2006, pp.
63–67.
[31] M. Eigen, P. Schuster, The Hypercycle: A Principle of Natural Self Organi-
zation, Springer, 1979.
[32] W. R. Ashby, Principles of the Self-Organizing System, in Principles of Self-
Organization, Pergamon Press, 1962, pp. 255–278.
[33] M. Wang, T. Suda, “The Bio-Networking Architecture: A Biologically
Inspired Approach to the Design of Scalable, Adaptive, and Surviv-
able/Available Network Applications,” in IEEE Symposium on Applications
and the Internet (SAINT), San Diego, CA, 2001, pp. 43-53.
224
BIBLIOGRAPHY
[34] J. Suzuki, T. Suda, “Adaptive Behavior Selection of Autonomous Objects
in the Bio-Networking Architecture,” in Annual Symposium on Autonomous
Intelligent Networks and Systems, Los Angeles, CA, 2002.
[35] C. Lee, H. Wada, J. Suzuki, Towards a Biologically-inspired Architecture for
Self-Regulatory and Evolvable Network Applications, in Advances in Biolog-
ically Inspired Information Systems - Models, Methods, and Tools, vol. 69
of Studies in Computational Intelligence (SCI), Springer, Berlin, Heidelberg,
New York, 2007, pp. 21–46.
[36] B. Webb, “What does robotics offer animal behaviour?,” Animal Behavior,
vol. 60, no. 5, 2000, pp. 545–558.
[37] E. Bonabeau, M. Dorigo, G. Theraulaz, Swarm Intelligence: From Natural
to Artificial Systems, Oxford University Press, 1999.
[38] Z. S. Ma, A. W. Krings, “Insect Sensory Systems Inspired Computing and
Communications,” Elseview Ad Hoc Networks, vol. 7, no. 4, 2009, pp. 742–
755.
[39] M. Farooq, Bee-Inspired Protocol Engineering: From Nature to Networks, in
Natural Computing, Springer, 2009.
[40] G. Di Caro, F. Ducatelle, L. M. Gambardella, “AntHocNet: An adaptive
nature-inspired algorithm for routing in mobile ad hoc networks,”, European
Transactions on Telecommunications, Special Issue on Self-organization in
Mobile Networking, vol. 16, 2005, pp. 443–455.
[41] G. Di Caro, M. Dorigo, “AntNet: Distributed Stigmergetic Control for
Communication Networks,” Journal of Artificial Intelligence Research, vol.
9, 1998, pp. 317–365.
[42] A. Forestiero, C. Mastroianni, G. Spezzano, “Antares: an Ant-Inspired P2P
Information System for a Self-Structured Grid,” in 2nd IEEE/ACM Interna-
225
BIBLIOGRAPHY
tional Conference on Bio-Inspired Models of Network, Information and Com-
puting Systems (IEEE/ACM BIONETICS 2007), Budapest, Hungary, 2007,
pp.151-158.
[43] R. E. Mirollo, S. H. Strogatz, “Synchronization of Pulse-Coupled Biological
Oscillators,” SIAM Journal on Applied Mathematics, vol. 50. no. 6, 1990,
1645–1662.
[44] A. Tyrrell, G. Auer, “Imposing a Reference Timing onto Firefly Synchroniza-
tion in Wireless Networks,”, in 65th IEEE Vehicular Technology Conference
(VTC2007-Spring), Dublin, Ireland, 2007, pp. 222–226.
[45] A. M. Turing, “The Chemical Basis for Morphogenesis,” Philosophical Trans-
actions of the Royal Society of London. Series B, Biological Sciences, vol. 237,
no.641, 1952, pp. 37–72.
[46] K. Hyodo, N. Wakamiya, E. Nakaguchi, M. Murata, Y. Kubo, K. Yanagi-
hara, “Experiments and Considerations on Reaction-Diffusion based Pattern
Generation in a Wireless Sensor Network,” in IEEE International Symposium
on a World of Wireless, Mobile and Multimedia Networks (IEEE WoWMoM
2007), Helsinki, Finland, 2007, pp. 1–6.
[47] G. Neglia, G. Reina, “Evaluating Activator-Inhibitor Mechanisms for Sensors
Coordination,” in 2nd IEEE/ACM International Conference on Bio-Inspired
Models of Network, Information and Computing Systems (IEEE/ACM BIO-
NETICS 2007), Budapest, Hungary, 2007.
[48] I. Dietrich, F. Dressler, “On the Lifetime of Wireless Sensor Networks,” ACM
Transactions on Sensor Networks (TOSN), vol. 5, no. 1, 2009, pp. 1–39.
[49] A. Yoshida, K. Aoki, S. Araki, “Cooperative control based on reaction-
diffusion equation for surveillance system,” in 9th International Conference
on Knowledge-Based Intelligent Information and Engineering Systems (KES
2005), Vol. LNCS 3684, Melbourne, Australia, 2005.
226
BIBLIOGRAPHY
[50] L. N. de Castro, J. Timmis, Artificial Immune Systems: A New Computa-
tional Intelligence Approach, Springer, 2002.
[51] J. O. Kephart, “A Biologically Inspired Immune System for Computers, in:
4th International Workshop on Synthesis and Simulation of Living Systems,”
MIT Press, Cambridge, MA, 1994, pp. 130–139.
[52] S. Stepney, R. E. Smith, J. Timmis, A. M. Tyrrell, M. J. Neal, A. N. W.
Hone, “Conceptual Frameworks for Artificial Immune Systems,” International
Journal of Unconventional Computing, vol. 1, no. 3, 2005, pp. 315–338.
[53] C. C. Zou, W. Gong, D. Towsley, L. Gao, “The Monitoring and Early De-
tection of Internet Worms,” IEEE/ACM Transactions on Networking (TON),
vol. 13, no. 5, 2005, pp. 961–974.
[54] M. Vojnovic, A. J. Ganesh, “On the Race of Worms, Alerts, and Patches,”
IEEE/ACM Transactions on Networking (TON), vol. 16, no. 5, 2008, pp.
1066–1079.
[55] J. Kleinberg, “Computing: The wireless epidemic,” Nature, vol. 449, 2007,
pp. 287–288.
[56] A. Khelil, C. Becker, J. Tian, K. Rothermel, “An Epidemic Model for In-
formation Diffusion in MANETs, in 5th ACM International Symposium on
Modeling, Analysis and Simulation of Wireless and Mobile Systems (ACM
MSWiM 2002), ACM, Atlanta, GA, 2002, pp. 54–60.
[57] I. Carreras, D. Miorandi, G. S. Canright, K. Engo-Monsen, “Understanding
the Spread of Epidemics in Highly Mobile Networks,” in 1st IEEE/ACM In-
ternational Conference on Bio-Inspired Models of Network, Information and
Computing Systems (IEEE/ACM BIONETICS 2006), Cavalese, Italy, 2006,
pp. 1-8.
227
BIBLIOGRAPHY
[58] H. Hayashi, T. Hara, S. Nishio, “On Updated Data Dissemination Exploiting
an Epidemic Model in Ad Hoc Networks,” in 2nd International Workshop on
Biologically Inspired Approaches to Advanced Information Technology (Bio-
ADIT 2006), Vol. LNCS 3853, Springer, Osaka, Japan, 2006, pp. 306–321.
[59] E. Ahi, M. Caglar, O. Ozkasap, “Stepwise Probabilistic Buffering for Epi-
demic Information Dissemination,” in 1st IEEE/ACM International Confer-
ence on Bio-Inspired Models of Network, Information and Computing Systems
(IEEE/ACM BIONETICS 2006), IEEE, Cavalese, Italy, 2006, pp. 1-8.
[60] X. Zhang, G. Neglia, J. Kurose, D. Towsley, “Performance Modeling of Epi-
demic Routing,” Elsevier Computer Networks, vol. 51, no. 10, 2007, pp. 2867–
2891.
[61] V. Pappas, D. Verma, B.J. Ko, A. Swami, “A Circulatory System Approach
for Wireless Sensor Networks,” Elsevier Ad Hoc Networks Available online:
10.1016/j.adhoc.2008.04.009.
[62] C. Bustamante, Y. Chelma, N. Forde, D. Izhaky, “Mechanical processes in
biochemistry,” Annual Review of Biochemistry, vol. 73, 2004, pp. 705–748.
[63] T. Nakano, T. Suda, M. Moore, R. Egashira, A. Enomoto, K. Arima,
“Molecular Communication for Nanomachines Using Intercellular Calcium
Signaling,” in 5th IEEE Conference on Nanotechnology (IEEE NANO 2005),
Nagoya, Japan, 2005, pp. 478–481.
[64] T. Suda, M. Moore, T. Nakano, R. Egashira, A. Enomoto, “Exploratory Re-
search on Molecular Communication between Nanomachines,” in Conference
on Genetic and Evolutionary Computation (GECCO 2005), ACM, 2005.
[65] S. Hiyama, Y. Moritani, T. Suda, R. Egashira, A. Enomoto, M. Moore, T.
Nakano, “Molecular Communication,” in NSTI Nanotech 2005, NSTI, 2005.
228
BIBLIOGRAPHY
[66] M. Moore, A. Enomoto, T. Nakano, R. Egashira, T. Suda, A. Kayasuga, H.
Kojima, H. Sakakibara, K. Oiwa, “A Design of a Molecular Communication
System for Nanomachines Using Molecular Motors,” in 4th IEEE Interna-
tional Conference on Pervasive Computing and Communications Workshops
(PERCOMW’06), Washington, DC, 2006, p. 554.
[67] B. Atakan, O. B. Akan, “On Channel Capacity and Error Compensation
in Molecular Communication,” Springer Transactions on Computational Sys-
tems Biology (TCSB), LNBI 5410, 2008, pp. 59–80.
[68] B. Atakan, O. B. Akan, “On Molecular Multiple-Access, Broadcast, and
Relay Channel in Nanonetworks,” in 3rd ACM/ICST International Confer-
ence on Bio-Inspired Models of Network, Information and Computing Systems
(Bionetics 2008), ACM, Hyogo, Japan, 2008.
[69] Y. Moritani, S. Hiyama, T. Suda, R. Egashira, A. Enomoto, M. Moore, T.
Nakano, “Molecular Communications between Nanomachines,” in 24th IEEE
Conference on Computer Communications (IEEE INFOCOM 2005), Miami,
FL, 2005.
[70] Y. Moritani, S. Hiyama, S. Nomura, K. Akiyoshi, T. Suda, “A Commu-
nication interface using vesicles embedded with channel forming proteins
in molecular communication,” in 2nd IEEE/ACM International Conference
on Bio-Inspired Models of Network, Information and Computing Systems
(IEEE/ACM BIONETICS 2007), Budapest, Hungary, 2007, pp. 147–149.
[71] Y. Moritani, S. Hiyama, T. Suda, “Molecular Communication for Health
Care Applications,” in 4th IEEE International Conference on Pervasive Com-
puting and Communications Workshops (PERCOMW’06), Washington, DC,
2006, p. 549.
229
BIBLIOGRAPHY
[72] S. Barbarossa and G. Scutari, “Bio-inspired sensor network design: dis-
tributed decision through self-synchronization” IEEE Signal Processing Mag-
azine, Volume 24, Issue 3, pp. 26–35, May 2007.
[73] R. Pagliari, Y.-W. Hong and A. Scaglione, “Bio-inspired algorithms for de-
centralized round-robin and proportional fair scheduling,” in IEEE Journal
on Selected Areas in Communications, vol. 28, no. 4, pp. 564–575, May 2010.
[74] X. Liang and Y Xiao, “Studying bio-inspired coalition formation of robots
for detecting intrusions using game theory,” IEEE Transactions on Systems,
Man, and Cybernetics, Part B, Special Issue on Game Theory, Vol. 40, No.
3, June 2010, pp. 683–693.
[75] T. Renk, C. Kloeck, D. Burgkhardt, F. K. Jondral, D. Grandblaise, S. Gault
and J.C. Dunat, “Bio-inspired algorithms for dynamic resource allocation in
cognitive wireless networks,” in Proc. International Conference on Cognitive
Radio Oriented Wireless Networks and Communications (CrownCom), Aug.
2007, Orlando, pp. 351–356.
[76] B. Atakan and O. B. Akan, “BIOlogically-inspired spectrum sharing in cog-
nitive radio networks,” in Proc. IEEE Wireless Communications and Net-
working Conference, March 2007, Hong Kong, pp. 43-48.
[77] X. Mao and H. Ji, “Biologically-inspired distributed spectrum access for cog-
nitive radio network,” in Proc. International Conference on Wireless Com-
munications Networking and Mobile Computing (WiCOM), pp. 1–4, Wuhan,
Sept. 2010
[78] M. Neal, J. Timmis, “Once More Unto the Breach: Towards Artificial Home-
ostasis”, in Recent Developments in Biologically Inspired Computing, 2005,
pp. 340–365.
[79] I. F. Akyildiz, F. Brunetti, C. Blazquez, “Nanonetworks: A New Communi-
cation Paradigm,” Elsevier Computer Networks, vol. 52, 2008, pp. 2260–2279.
230
BIBLIOGRAPHY
[80] B. Alberts, D. Bray, J. Lewis, M. Raff, K. Roberts, J. D. Watson, Molecular
Biology of the Cell, 3rd Edition, Garland Publishing, Inc., 1994.
[81] T. H. Labella, F. Dressler, A Bio-Inspired Architecture for Division of Labour
in SANETs, in Advances in Biologically Inspired Information Systems - Mod-
els, Methods, and Tools, Vol. 69 of Studies in Computational Intelligence
(SCI), Springer, Berlin, Heidelberg, New York, 2007, pp. 209–228.
[82] M. Fiedler, “Algebraic connectivity of graphs,” in Czechoslovak Math. J.,
vol. 23, no. 98, pp. 298–305, 1973.
[83] B. Mohar, “The Laplacian spectrum of graphs,” in Graph Theory, Combi-
natorics, and Applications, New York: Wiley, 1991, pp. 871–898.
[84] R. Merris, “Laplacian matrices of a graph: A survey,” in Linear Algebra its
Appl., vol. 197, pp. 143–176, 1994.
[85] C. Godsil and G. Royle, “Algebraic Graph Theory,” ser. Graduate Texts in
Mathematics, Springer-Verlag, 2001, vol. 207.
[86] R.A. Brualdi and H.J. Ryser, “Combinatorial Matrix Theory,” Cambridge,
UK: Cambridge Univ. Press, 1991.
[87] N. Linial “Locality in distributed graph algorithms. SIAM Journal on Com-
puting, vol. 21, no. 1, pp. 193–201, February 1992.
[88] F. Kuhn and R. Wattenhofer, “On the Complexity of Distributed Graph
Coloring,” in Proc. of the twenty-fifth ACM symposium on Principles of dis-
tributed computing (PODC), 2006, New York, USA.
[89] K. Duffy, N. O’Connell, A. Sapozhnikov, “Complexity analysis of a decen-
tralised graph colouring algorithm”, Information Processing Letters, vol. 107,
no. 2, pp. 60–63, 2008.
231
BIBLIOGRAPHY
[90] P. R. Walenga Junior, M. Fonseca, A. Munaretto, A. C. Viana, “A Dis-
tributed Channel Assignment Algorithm for Cognitive Radio Networks”, in
EURASIP Journal on Wireless Communications and Networking, accepted
for pubblication, 2011.
[91] P. Gupta and P. Kumar, “Critical power for asymptotic connectivity in
wireless networks,” In Stochastic Analysis, Control, Optim. and Appl.: A
Volume in Honor of W.H. Fleming, Birkhauser, Boston, pp. 547–566, 1998.
[92] I. Akyildiz, W. Lee, M. Vuran, and S. Mohanty, “NeXt generation/dynamic
spectrum access/cognitive radio wireless networks: A survey,” Computer Net-
works, vol. 50, no. 13, pp. 2127–2159, Sep. 2006.
[93] Q. Zhao and B. M. Sadler, “A survey of dynamic spectrum access,” IEEE
Signal Processing Magazine, vol. 24, no. 3, pp. 79–89, May 2007.
[94] Q. Zhao and A. Swami, “A survey of dynamic spectrum access: Signal pro-
cessing and networking perspectives,” in Proc. 55th Int. Conf. Acoust., Speech,
Signal Process., Honolulu, HI, April 15-20, 2007, pp. 1349–1352.
[95] Q. Zhao, L. Tong, and A. Swami, “Decentralized cognitive MAC for dynamic
spectrum access,” in Proc. 1st IEEE International Symposyum New Frontiers
on Dynamic Spectrum Access Networks, Baltimore, MD, Nov. 2005, pp. 224–
232.
[96] Q. Zhao, L. Tong, A. Swami, and Y. Chen, “Decentralized cognitive MAC for
opportunistic spectrum access in ad hoc networks: A POMDP framework,”
IEEE Journal on Selected Areas in Communications, vol. 25, no. 3, pp. 589–
600, Apr. 2007.
[97] Q. Zhao and A. Swami, “A decision-theoretic framework for opportunistic
spectrum access,” IEEE Wireless Communication Magazine (Special Issue
Cognitive Wireless Networks), vol. 14, no. 4, pp. 14–20, Aug. 2007.
232
BIBLIOGRAPHY
[98] S. Geirhofer, L. Tong, and B. Sadler, “Cognitive medium access: constrain-
ing interference based on experimental models,” IEEE Journal on Selected
Areas in Communications, Special Issues on Cognitive Radio: Theory and
Applications, vol. 26, no. 1, pp. 95–105, Jan. 2008.
[99] S. Geirhofer, L. Tong, and B. Sadler, “Opportunistic spectrum access via
periodic channel sensing,” IEEE Transactions on Signal Processing, vol. 56,
no. 2, pp. 785–796, Feb. 2008.
[100] Y. Chen, Q. Zhao, A. Swami “Joint design and separation principle for op-
portunistic spectrum access in the presence of sensing errors,” IEEE Trans-
actions on Information Theory, vol. 54, no. 5, pp. 2053–2071, May 2008.
[101] Q. Zhao, B. Krishnamachari, and K. Liu, “On myopic sensing for multi-
channel opportunistic access: structure, optimality, and performance,” IEEE
Transactions on Wireless Communications, vol. 7, no. 12, pp. 5431–5440, Dec.
2008.
[102] X. Li, Q. Zhao, X. Guan, L. Tong, “Optimal cognitive access of markovian
channels under tight collision constraints”, 2010 IEEE International Confer-
ence on Communications (ICC), Cape Town, South Africa, July 2010, pp.
1–5.
[103] S. Chen, L. Tong, “Multiuser cognitive access of continuous time Markov
channels: Maximum throughput and effective bandwidth regions”, Informa-
tion Theory and Applications Workshop (ITA), 2010, San Diego, CA, Jan.-
Feb. 2010, pp. 1–10.
[104] Z. Quan, S. Cui, H. V. Poor and A. H. Sayed, “Optimal multiband joint de-
tection for spectrum sensing in cognitive radio networks,” IEEE Transactions
on Signal Processing, vol. 57, pp. 1128–1140, March 2009.
233
BIBLIOGRAPHY
[105] J. Unnikrishnan, V.V. Veeravalli, “Algorithms for dynamic spectrum access
with learning for cognitive radio,” IEEE Transactions on Signal Processing,
vol.58, no.2, pp. 750-760, Feb. 2010.
[106] S. Barbarossa, S. Sardellitti, G. Scutari, “Joint optimization of detection
thresholds and power allocation for opportunistic access in multicarrier cog-
nitive radio networks,” Proc. of International Workshop on Computational
Advances in Multi-Sensor Adaptive Processing (CAMSAP) 2009, Aruba, Dec
12-14, 2009, pp. 404-407.
[107] G. Scutari, D. P. Palomar and S. Barbarossa, “Cognitive MIMO radio,”
IEEE Signal Processing Magazine, vol. 25, pp. 46–59, Nov. 2008.
[108] V. Krishnamurthy, “Decentralized spectrum access amongst cognitive ra-
dios - an interacting multivariate global game-theoretic approach,” IEEE
Transactions on Signal Processing, Vol. 57, no. 10, pp. 3999–4013, Oct. 2009.
[109] J. Huang, V. Krishnamurthy, “Transmission control in cognitive radio as
a Markovian dynamic game: Structural result on randomized threshold poli-
cies,” IEEE Transactions on Communications, Vol. 58 , no. 1, pp. 301–310,
Jan. 2010.
[110] M. Maskery, V. Krishnamurthy, and Q. Zhao “Cecentralized dynamic spec-
trum access for cognitive radios: cooperative design of a non-cooperative
game,” IEEE Transactions on Communications, vol. 57, no. 2, pp. 459–469,
Feb. 2009.
[111] S. Barbarossa, S. Sardellitti, G. Scutari, “Joint optimization of detection
thresholds and power allocation in multiuser wideband cognitive radios”,
Proc. of Cognitive systems with Interactive Sensors (COGIS) 2009, Paris,
Nov. 16-18, 2009.
234
BIBLIOGRAPHY
[112] R. Olfati-Saber and R. M. Murray, “Consensus protocols for networks of
dynamic agents,” in Proc. of the 2003 American Control Conference, Denver,
June 4-6, 2003, pp. 951-956.
[113] W. Ren, R. W. Beard, and T. W. McLain, “Coordination Variables and
Consensus Building in Multiple Vehicle Systems,” in Proc. of the Block Island
Workshop on Cooperative Control, Springer-Verlag Series, vol. 309, pp. 171–
188, 2005.
[114] C. W. Wu, “Agreement and Consensus Problems in Groups of Autonomous
Agents with Linear Dynamics,” in Proc. of IEEE International Symposium
on Circuits and Systems (ISCAS 2005), pp. 292–295, 23-26 May 2005.
[115] R. Olfati-Saber, “Flocking for multi-agent dynamic systems: algorithms
and theory”, IEEE Transactions on Automatic Control, vol. 51, no. 3, pp.
401–420, Mar. 2006.
[116] V. Gazi and K.M. Passino, “Stability analysis of swarms,” IEEE Transac-
tions on Automatic Control, Vol. 48, No. 4, pp. 692–697, April 2003.
[117] V. Gazi, K. M. Passino, “A class of attractions/repulsion functions for
stable swarm aggregations,” International Journal on Control, 2004, vol. 77,
no. 18, pp. 1567–1579, 2004
[118] V. Gazi, K. M. Passino, “Stability analysis of social foraging swarms,” IEEE
Transactions On Systems, Man, And Cybernetics - Part B: Cybernetics, vol.
34, No. 1, pp. 539–557, February 2004.
[119] W. Li, “Stability analysis of swarms with general topology,” IEEE Trans-
actions On Systems, Man, And Cybernetics - Part B: Cybernetics, vol. 38,
No. 4, pp. 1084–1097, August 2008.
235
BIBLIOGRAPHY
[120] R. Olfati-Saber, and R.M. Murray, “Consensus problems in networks of
agents with switching topology and time-delays,” in IEEE Transactions on
Automatic Control, vol. 49, no.9, pp. 1520-1533, Sep. 2004.
[121] A. Jadbabaie, J. Lin, and A. S. Morse “Coordination of groups of mo-
bile autonomous agents using nearest neighbor rules,” IEEE Transactions on
Automatic Control, Vol. 48 , No. 6, June 2003, pp. 988–1001.
[122] H. Shi, L. Wang, T. Chu, G. Xie, M. Xu, “Flocking coordination of multi-
ple interactive dynamical agents with switching topology,” in Proc. of IEEE
International Conference on Systems, Man and Cybernetics (SMC), Taipei,
pp. 2684–2689, Oct. 2006
[123] H. G. Tanner, A. Jadbabaie, G. J. Pappas, “Flocking in fixed and switching
networks,” in IEEE Transactions on Automatic Control, vol. 52, no. 5, pp.
863–868, May 2007.
[124] J. Hu, J. Yao, L. Wang, “Cohesiveness analysis of hybrid swarm systems
based on artificial potential functions,” in Proc. International Conference on
Modelling, Identification and Control (ICMIC), Okayama, pp. 639–644, 17-19
July 2010.
[125] Z. Li, Y. Jia, J. Du, S. Yuan, “Flocking for multi-agent systems with switch-
ing topology in a noisy environment,” in Proc. American Conctrol Conference,
Seattle, pp. 111–116, June 2008
[126] P. Di Lorenzo and S. Barbarossa, “Distributed resource allocation in cog-
nitive radio systems based on social foraging swarms,” in The 11th IEEE
International Workshop on Signal Processing Advances in Wireless Commu-
nications (SPAWC), Marrakech, pp. 1–5, June 2010
[127] P. Di Lorenzo and S. Barbarossa, “A bio-inspired swarming algorithm for
decentralized access in cognitive radio,” IEEE Trans. on Signal Processing,
vol. 59, no. 12, December 2011, pp. 6160-6174.
236
BIBLIOGRAPHY
[128] P. Di Lorenzo and S. Barbarossa, “Bio-inspired swarming models for de-
centralized radio access incorporating random links and quantized communi-
cations,” Proc. of International Conference on Acoustics, Speech and Signal
Processing (ICASSP) 2011, pp. 5780–5783, Prague, May 2011.
[129] P. Di Lorenzo, S. Barbarossa, and Ali H. Sayed “Bio-inspired swarming for
dynamic radio access based on diffusion adaptation,” in Proc. of the 19-th
European Signal Processing Conference (EUSIPCO 2011), Barcelona, Spain,
August-September 2011, pp. 402-406.
[130] A. Mogilner, L. Edelstein-Keshet, L. Bent and A. Spiros, “Mutual interac-
tions, potentials, and individual distance in a social aggregation,” Journal of
Mathematical Biology, vol. 47, pp. 353–389, 2003.
[131] D. Grunbaum and A. Okubo, “Modeling social animal aggregations”, in
Frontiers in Theoretical Biology, New York: Springer-Verlag, 1994,
[132] R. Olfati-Saber, J. A. Fax, R. M. Murray, “Consensus and cooperation in
networked multi-agent systems,” in Proceedings of the IEEE, vol. 95, no. 1,
pp. 215–233, Jan. 2007.
[133] L. Xiao, S. Boyd, and S. Lall,“A scheme for robust distributed sensor fu-
sion based on average consensus” in Proc. Information Processing in Sensor
Networks, Los Angeles, CA, pp. 63–70, Apr. 2005.
[134] S. Barbarossa and G. Scutari, “Bio-inspired sensor network design: dis-
tributed decision through self-synchronization” IEEE Signal Processing Mag-
azine, Volume 24, Issue 3, pp. 26–35, May 2007.
[135] G. Scutari, S. Barbarossa, and Loreto Pescosolido, “Distributed Decision
Through Self-Synchronizing Sensor Networks in the Presence of Propagation
Delays and Asymmetric Channels ” IEEE Transactions on Signal Processing
, Volume 56, Issue 4, pp. 1667–1684, April 2008.
237
BIBLIOGRAPHY
[136] I. D. Schizas, G. B. Giannakis, S. D. Roumeliotis and A. Ribeiro, “Consen-
sus in Ad Hoc WSNs with noisy links - part II: distributed estimation and
smoothing of random signals,” IEEE Transactions on Signal Processing, vol.
56, no. 4, pp. 1650–1666, April 2008.
[137] Y. Hatano, A. K. Das, and M. Mesbahi, “Agreement in presence of noise:
pseudogradients on random geometric networks,” in Proc. IEEE Conference
on Decision and Control (CDC), Seville, Spain, pp. 6382–6387, December
2005.
[138] T. C. Aysal, M. Coates, and M. Rabbat, “Distributed average consensus
using probabilistic quantization,” in Proc. IEEE/SP Workshop on Statistical
Signal Processing Workshop (SSP), Maddison, Wisconsin, USA, August 2007,
pp. 640–644.
[139] M. Huang and J. Manton, “Stochastic approximation for consensus seeking:
mean square and almost sure convergence,” in Proc. IEEE Conference on
Decision and Control (CDC), New Orleans, LA, USA, pp. 306–311, Dec.
2007.
[140] S. Kar and J. M. F. Moura, “Sensor networks with random links: Topology
design for distributed consensus,” IEEE Transactions on Signal Processing,
vol. 56, no. 7, pp. 3315–3326, July 2008.
[141] S. Kar and J.M.F. Moura, “Distributed consensus algorithms in sensor
networks with imperfect communication: link failures and channel noise,”
IEEE Transactions on Signal Processing vol. 57, no. 5, pp. 355–369, January
2009.
[142] K. Srivastava, Angelia Nedic, and D. M. Stipanovic “Distributed con-
strained optimization over noisy networks,” in Proc. the 49th IEEE Confer-
ence on Decision and Control, Atlanta, Georgia, December 2010, pp. 1945–
1950
238
BIBLIOGRAPHY
[143] C. G. Lopes and A. H. Sayed, “Diffusion least-mean squares over adaptive
networks: Formulation and performance analysis,” IEEE Transactions on
Signal Processing, vol. 56, no. 7, pp. 3122–3136, July 2008.
[144] F. S. Cattivelli, C. G. Lopes and A. H. Sayed, “Diffusion recursive least-
squares for distributed estimation over adaptive networks ,” IEEE Transac-
tions on Signal Processing, vol. 56, no.5, pp. 1865–1877, March 2008.
[145] F. S. Cattivelli and A. H. Sayed, “Diffusion LMS strategies for distributed
estimation,” IEEE Transactions on Signal Processing, vol. 58, no.3, pp. 1035–
1048, March 2010.
[146] F. S. Cattivelli and A. H. Sayed, “Diffusion detection over adaptive net-
works using diffusion adaptation,” IEEE Transactions on Signal Processing,
vol. 59, no.5, pp. 1035–1048, March 2010.
[147] N. Takahashi, I. Yamada, and A. H. Sayed, “Diffusion least-mean squares
with adaptive combiners: Formulation and performance analysis,” IEEE
Transactions on Signal Processing, vol. 58, no. 9, pp. 4795–4810, Sep. 2010.
[148] J. Chen, S-Y. Tu and A. H. Sayed, “Distributed optimization via diffusion
adaptation,” in Proc. IEEE International Workshop on Computational Ad-
vances in Multi-Sensor Adaptive Processing (CAMSAP), San Juan, Puerto
Rico, December 2011.
[149] F. Cattivelli and A. H. Sayed, “Self-organization in bird flight formations
using diffusion adaptation,” in Proc. 3rd International Workshop on Com-
putational Advances in Multi-Sensor Adaptive Processing (CAMSAP), pp.
49–52, Aruba, Dutch Antilles, December 2009.
[150] F. Cattivelli and A. H. Sayed, “Modeling bird flight formations using dif-
fusion adaptation,” IEEE Transactions on Signal Processing, vol. 59, no. 5,
pp. 2038-2051, May 2011.
239
BIBLIOGRAPHY
[151] S-Y. Tu and A. H. Sayed, “Foraging behavior of fish schools via diffusion
adaptation,” in Proc. International Workshop on Cognitive Information Pro-
cessing (CIP), pp. 63-68, Elba Island, Italy, June 2010.
[152] S-Y. Tu and A. H. Sayed, “Mobile adaptive networks,” IEEE Journal on
Selected Topics on Signal Processing, vol. 5, no. 4, pp. 649–664, August 2011.
[153] J. Chen, X. Zhao, and A. H. Sayed, “Bacterial motility via diffusion adap-
tation,” Proc. 44th Asilomar Conference on Signals, Systems and Computers,
pp. 1930–1934, Pacific Grove, CA, Nov. 2010.
[154] J. Chen and A. H. Sayed, “Bio-inspired cooperative optimization with appli-
cation to bacteria motility,” in Proc. of International Conference on Acous-
tics, Speech and Signal Processing (ICASSP), Prague, Czech Republic, pp.
5788–5791, May 2011.
[155] R. Chen, J. M. Park, and K. Bian, “Robust distributed spectrum sensing
in cognitive radio networks,” in Proc. 27th Conference Computer Communi-
cations, Phoenix, AZ, Apr. 13-18, 2008, pp. 1876–1884.
[156] C. R. C. da Silva, B. Choi, and K. Kim, “Distributed spectrum sensing for
cognitive radio systems, in Proc. Workshop Information Theory and Applica-
tions, San Diego, CA, Feb. 2, 2007, pp. 120–123.
[157] G. Ganesan, Y. Li, B. Bing, and S. Li, “Spatiotemporal sensing in cognitive
radio networks,” in IEEE Jornal on Selected Areas in Communications, vol.
26, pp. 5–12, Jan. 2006.
[158] A. Ghasemi and E. S. Sousa, “Spectrum sensing in cognitive radio net-
works: The cooperation-processing tradeoff,” in Wireless Communication
Mobile Computing, vol. 7, no. 9, pp. 1049–1060, 2007.
240
BIBLIOGRAPHY
[159] S. M. Mishra, A. Sahai, and R. W. Brodersen, “Cooperative sensing among
cognitive radios,” in Proc. 42nd International Conference on Communica-
tions, Istanbul, Turkey, Jun. 11-15, 2006, pp. 1658–1663.
[160] K. Nishimori, R. D. Taranto, H. Yomo, P. Popovski, Y. Takatori, R. Prasad,
and S. Kubota, “Spatial opportunity for cognitive radio systems with hetero-
geneous path loss conditions,” in Proc. 65th Vehicular Technology Conference,
Dublin, Ireland, Apr. 22-25, 2007, pp. 2631–2635.
[161] J. Riihijarvi and P. Mahonen, “Exploiting spatial statistics of primary and
secondary users towards improved cognitive radio networks,” in Proc. 3rd In-
ternational Conference on Cognitive Radio Oriented Wireless Network Com-
munications, Singapore, May 15-17, 2008, pp. 1–7.
[162] R. Tibshirani, “Regression shrinkage and selection via the LASSO,” Journal
Royal Statistics Soc B., vol. 58, pp. 267–288, 1996.
[163] R. Baraniuk, “Compressive sensing,”, IEEE Signal Processing Magazine,
vol. 25, pp. 21–30, March 2007.
[164] E.J. Candes, M. Wakin, and S. Boyd, “Enhancing sparsity by reweighted ℓ1
minimization,” Journal of Fourier Analysis and Applications, vol. 14, pp.877–
905, 2007.
[165] Y. Tsaig and D. L. Donoho, “Extensions of compressed sensing,” in Elsevier
Signal Processing - Sparse approximations in signal and image processing, vol.
86, no. 3, March 2006, pp. 549-571.
[166] Y. Chen, Y. Gu, and A.O. Hero, “Sparse LMS for system identification,”
in Proc. International Conference on Acoustics, Speech and Signal Processing
(ICASSP), pp. 3125–3128, Taipei, May 2009.
241
BIBLIOGRAPHY
[167] D. Angelosante, J.A. Bazerque, and G.B. Giannakis, “Online adaptive esti-
mation of sparse signals: where RLS meets the ℓ1-norm,” IEEE Transactions
on Signal Processing, vol. 58, no. 7, pp. 3436–3447, 2010.
[168] B. Babadi, N. Kalouptsidis, and V. Tarokh, “SPARLS: The sparse RLS
algorithm,” IEEE Transactions on Signal Processing, vol. 58, no. 8, pp. 4013–
4025, 2010.
[169] Y. Kopsinis, K. Slavakis, and S. Theodoridis, “Online sparse system iden-
tification and signal reconstruction using projections onto weighted ℓ1 balls,”
IEEE Transactions on Signal Processing, vol. 59, no. 3, pp. 936–952, 2010.
[170] J. A. Bazerque, and G. B. Giannakis, “Distributed spectrum sensing for
cognitive radio networks by exploiting sparsity,” IEEE Transactions on Signal
Processing, vol 58, No. 3, pp. 1847–1862, March 2010.
[171] G. Mateos, J. A. Bazerque, and G. B. Giannakis, “Distributed sparse linear
regression,” IEEE Transactions on Signal Processing, vol 58, No. 10, pp.
5262–5276, Oct. 2010.
[172] J. A. Bazerque, and G. B. Giannakis, “Distributed spectrum sensing for
cognitive radio networks by exploiting sparsity,” IEEE Transactions on Signal
Processing, vol 58, No. 3, pp. 1847-1862, March 2010.
[173] S. J. Kim, E. Dall’Anese and G. B. Giannakis, “Cooperative spectrum
sensing for cognitive radios using kriged Kalman filtering,” IEEE Journal on
selected topics in signal processing, vol 5, No. 1, pp. 24–36, Feb. 2011.
[174] D.P. Bertsekas and J. Tsitsiklis,“Gradient convergence in gradient methods
with errors,” SIAM Journal on Optimization, 2000, vol.10, no.3, pp. 627-642.
[175] S. P. Lipshitz, R. A. Wannamaker, and J. Vanderkooy, “Quantization and
dither: A theoretical survey,” Journal of Audio Engeneering Society, vol. 40,
pp. 355–375, May 1992.
242
BIBLIOGRAPHY
[176] R. Wannamaker, S. Lipshitz, J. Vanderkooy, and J. Wright, “A theory of
nonsubtractive dither,” IEEE Transactions on Signal Processing, vol. 48, no.
2, pp. 499–516, February 2000.
[177] L. Schuchman, “Dither signals and their effect on quantization noise,” IEEE
Transactions Communication Technologies, vol. COMM-12, pp. 162–165, De-
cember 1964.
[178] S. Gerschgorin, “Uber die abgrenzung der eigenwerte einer matrix,” Izv.
Akad. Nauk. USSR Otd. Fiz.-Mat., Nauk 7, pp. 749-754, 1931.
[179] J.P. LaSalle, “Some extensions of Liapunov’s second method,” IRE Trans-
actions on Circuit Theory, CT-7, pp. 520-527, 1960.
[180] H. Robbins and S. Monro, “A stochastic approximation method,” Annals
of Mathematical Statistics, Vol. 22, No. 3, 400–407, 1951.
[181] J. Kiefer and J. Wolfowitz, “Stochastic estimation of the maximum of a
regression function,” Annals of Mathematical Statistics, Vol. 23, 462–466,
1952.
[182] P. Bianchi, G. Fort, W. Hachem, and J. Jakubovics, “Performance analysis
of a distributed Robbins-Monro algorithm for sensor networks” in Proc. of the
19-th European Signal Processing Conference (EUSIPCO 2011), Barcelona,
Spain, August-September 2011, pp. 1030–1034.
[183] M. Nevelson and R. Hasminskii, Stochastic Approximation and Recursive
Estimation, Providence, RI: American Mathematical Society, 1973.
[184] H. J. Kushner and G. G. Yin, Stochastic Approximation Algorithms and
Applications, New York: Springer-Verlag, 1997.
[185] A. E. Albert and L. A. Gardner, Stochastic Approximation and Nonlinear
Regression, The MIT press, 2003.
243
BIBLIOGRAPHY
[186] R. A. Horn and C. R. Johnson, Matrix Analysis, Cambridge University
Press, 1985.
[187] F. Khan, LTE for 4G Mobile Broadband – Air Interface Technologies and
Performance, Cambridge University Press, 2009
[188] D.P. Bertsekas and J.N. Tsitsiklis, “Nonlinear programming,” 2nd ed.,
Athena Scientific, Belmont, 1999.
[189] D.P. Bertsekas and J.N. Tsitsiklis, “Parallel and distributed computation:
numerical methods,” Belmont, MA, Athena Scientific, 1997.
[190] B.Widrow and S.D. Stearns, Adaptive Signal Processing, New Jersey: Pren-
tice Hall, 1985.
[191] A. H. Sayed, Adaptive Filters, Wiley, NJ, 2008.
244