Entropia muestral

8/9/2019 Entropia muestral

1/15

c om pu te r m et ho ds a nd p ro gr am s i n b io me di ci ne 1 0 4 ( 2 0 1 1 ) 382396

j o u r n a l h om e pa ge : www. i n t l . e l s e v i e r he a l t h . c om / j ou r na l s / c m pb

Fast computation of sample entropy and approximateentropy in biomedicine

Yu-Hsiang Pan a , , Yung-Hung Wang b, Sheng-Fu Liang c, Kuo-Tien Lee aa Department of Environmental Biology and Fisheries Science, National Taiwan Ocean University, 2 Pei-Ning Road, Keelung 20224,Taiwanb Society of Streams, R.O.C., 1F., No. 15, Alley 70, Lane 12, Sec. 3, Bade Rd., Taipei City 10558, Taiwanc Department of Computer Science and Information Engineering, National Cheng Kung University, No. 1, Ta-Hsueh Road, Tainan,Taiwan, ROC

a r t i c l e i n f o

Article history:Received 2 April 2010Received in revised form28 November 2010Accepted 6 December 2010

Keywords:Approximate entropy ( AE)Computational geometryMultiscale entropy (MSE)Sample entropy ( SE)Sliding kd tree (SKD)

a b s t r a c t

Both sample entropy and approximate entropy are measurements of complexity. The twomethods have received a great deal of attention in the last few years, and have been suc-cessfully veried and applied to biomedical applications and many others. However, thealgorithms proposed in the literature require O(N2) execution time, which is not fast enoughfor online applications and for applications with long data sets. To accelerate computation,the authors of the present paper have developed a new algorithm that reduces the com-putational time to O(N3/2)) using O(N) storage. As biomedical data are often measured withinteger-type data, the computation time can be further reduced to O(N) using O(N) storage.The execution times of the experimental results with ECG, EEG, RR, and DNA signals showa signicant improvement of more than 100 times when compared with the conventionalO(N2) method for N =80,000 (N = length of the signal). Furthermore, an adaptive version of the new algorithm has been developed to speed up the computation for short data length.Experimental results show an improvement of more than 10 times when compared withthe conventional method for N >4000.

2010 Elsevier Ireland Ltd. All rights reserved.

1. Introduction

Both approximate entropy ( AE) [1] and sample entropy ( SE) [2]

are measurementsof a systems complexity, which are impor-tant for the analysis of biomedical signals [3,4]and signals inother elds [5,6]. Pincus introduced approximate entropy, a setof measures of system complexity closely related to entropy,which is easilyapplied to biomedical signals andothers. How-ever, AE has two deciencies. First, AE strongly depends onthe record length and is uniformly lower than expected forshort records. Second, AE lacks relative consistency. The sam-ple entropy was introduced by Richman et al., which requires

Corresponding author . Tel.: +886 987 153 779; fax: +886 2 82366969.E-mail addresses: [email protected] , [email protected] (Y.-H. Pan).

much shorter data sets than the approximate entropy. Both AE and SE quantify theregularity (orderliness) of a time series.However, both the measurements indicate a higher complex-ity for certain pathologic processes associated with random

outputs than for healthy dynamics exhibiting long-range cor-relations. For this reason, Costa et al. [3,7,8] introduced themultiscale entropy (MSE), which measures complexity by tak-ing into account multiple time scales. For each scale, thesample entropy is computed. All these quantities are denedin Section 2.

The algorithms for computing complexity proposed in theliterature require O(N2), which is not realistic for applicationswith long data sets or for online diagnosis of a time-varying

0169-2607/$ see front matter 2010 Elsevier Ireland Ltd. All rights reserved.doi:10.1016/j.cmpb.2010.12.003
http://localhost/var/www/apps/conversion/tmp/scratch_1/dx.doi.org/10.1016/j.cmpb.2010.12.003http://www.intl.elsevierhealth.com/journals/cmpbmailto:[email protected]:[email protected]://localhost/var/www/apps/conversion/tmp/scratch_1/dx.doi.org/10.1016/j.cmpb.2010.12.003http://localhost/var/www/apps/conversion/tmp/scratch_1/dx.doi.org/10.1016/j.cmpb.2010.12.003mailto:[email protected]:[email protected]://www.intl.elsevierhealth.com/journals/cmpbhttp://localhost/var/www/apps/conversion/tmp/scratch_1/dx.doi.org/10.1016/j.cmpb.2010.12.003


2/15

c om p ut er m et ho ds a nd p ro gr am s i n b io me di ci ne 1 0 4 ( 2 0 1 1 ) 382396 383

system,especially fordatacollected withhigh sample rates. Inaddition, to nd the statistical meaning of the signals, a largeamount of data and many parameters are typically collected,from which many parameters are determined.

The method developed by Manis [9] has reduced the exe-cution time for computing approximate entropy. He used abucket-assisted technique, similar to a bucket sort, to excludeimpossible matches of similarities early in the process. Thisimproved the execution time, but it was still an O(N2) algo-rithm. To monitor the change in SE with time for the onlinemonitoring of a systems health, Sugisaki and Ohmori [10]derived a recursive sample entropy algorithm for overlappedrolling windows. However, this algorithm has two drawbacks:rst, the computed sample entropy is only approximate; sec-ond, the computational efciency decays with a decreasing overlap length, with the improvement reachingzerowhen theoverlap length is zero.

Bucket-assisted and recursive sample entropy algorithmsare O(N2) algorithms. Does a nonquadratic algorithm exist?If the time series consists only of binary numbers 0 and 1[11], then the distance r in the approximate (sample) entropyterm disappears. In this situation, a linear time algorithmexists. In [12], Pincus stated: The discreteness of the statespace affords the possibility of such linear time calculations,in contrast with inherently quadratic time algorithms forcontinuous state space. However, Pincus did not give a the-oretical proof. Pan et al. [13] proved that the nonquadraticexecution time can be achieved by transforming the nmi (n

mi )

term in the sample (approximate) entropy into an orthogonalrange-counting problem by transforming the template seriesinto m-dimensional space point sets. All these quantities aredened in Section 2. The kd tree, or the range tree combinedwith the fractional cascading technique, is then applied tocompute the sample and the approximate entropies. Theseimprove the time complexity to O(N5/3) using linear storage(memory) or O(N log 2 N) using O(N log 2 N) storage for m = 2.Although the range tree affords less time complexity, the stor-age is not linear; it requires a lot of overhead, and hence,performs worse than the kd tree algorithms for signals witha practical length ( N must be greater than a certain thresh-old).

This paper has developed for the rst time a sliding kdtree (SKD) algorithm that reduces the time complexity of akd tree algorithm by one dimension while using linear stor-age. In other words, the time complexity for SKD is O(N3/2) form =2 for the real-type data, and is O(Bm 1N) for the integer-type data as in digitalized biomedical signals, where B is theresolution of the data as described in Section 3.3. Second,for N under a certain threshold, the performance of the SKDalgorithm is inferior to the conventional brute force method.For this reason, the authors have developed an adaptive SKDalgorithm to accelerate the computational time for short datalength.

Theremainder of thepaperis organizedas follows:Section2 provides a review of the sample entropy, the approximateentropy, and the multiscale entropy, and a review of the kdtree algorithm. In Section 3, the SKD algorithm and its adap-tive version are developed to compute the entropy. Section 4presents experimental results to illustrate the effectiveness of the SKD algorithm, and in Section 5 conclusions are given.

2. Review of the computation of sampleentropy and approximate entropy

2.1. Review of sample entropy, approximate entropy,and multiscale entropy

Consider a time series of length N: X={X1 . . . Xi . . . XN}. A pat-ternlength m (length of sequences to be compared) is selected,and, for each i, a vector (template) of size m is dened:

(Xi)m = { Xi, Xi+ 1, . . . , X i+ m 1} (1)

Two vectors ( Xi)m and ( X j)m are dened as similar if theirdistance is less than r, where r is dened as the tolerance foraccepting matches.

|(Xi+ k) (X j+ k)| < r, k, 0 k m 1 (2)

A variable (i, j,m,r) is dened; its value is 1 if Eq. (2) holds;otherwise, its value is zero.

Furthermore , n mi =N m+ 1

j= 1

(i , j ,m,r ) is dened (3)

which is the number of vectors ( X j)m within r of (Xi)m .Then, the probability that any vector ( X j)m is within r of

(Xi)m is

Cmi (r) =n mi

N m + 1

Then, m(r) = 1/N m + 1 N m+ 1i= 1 log C

mi (r), which is the aver-

age of the natural logarithms of Cmi (r).Subsequently, the approximate entropy [1] is calculated as

follows:

AE(m,r,N ) = m(r) m+ 1(r) (4)

The sample entropy SE [2] is computed as:

SE(scale,m,r,N ) = ln

N mi= 1 n

mi

N mi= 1 n

m+ 1i

= ln nnnd

(5)

where

nmi =N m

j= i+ 1

(i , j ,m,r ) (6)

nmi in Eq. (6)differs from nmi in Eq. (3)to the extent that, for

SE, self matches are not counted ( i /= j) and i ranges from 1 to(N m).

Description of the MSE analysis [3]: rst, the original timeseries is divided intononoverlapping windowsof length ,andsecond, the data points inside each window are averaged. Ingeneral, each element of a coarse-grained time series is cal-
http://localhost/var/www/apps/conversion/tmp/scratch_1/dx.doi.org/10.1016/j.cmpb.2010.12.003http://localhost/var/www/apps/conversion/tmp/scratch_1/dx.doi.org/10.1016/j.cmpb.2010.12.003


3/15

384 c om p ut er m e th od s a nd p ro gr am s i n b io me di ci ne 1 0 4 ( 2 0 1 1 ) 382396

culated according to the equation

X j =1

j

i= ( j 1) + 1

Xi, 1 j N , N > (7)

Then, for each time scale , SE is computed using Eq. (5).

This article has focused on the computation of SE and AE,which is similar as seen in Eqs. (3) and (6).

2.2. kd tree algorithm

In [13], it was shown that the computation of the nmi term canbe transformed into an orthogonal range-counting problem,andthe procedure is brieyreviewed here. For each i, the tem-plate ( Xi)m has been transformedinto an m-dimensional pointset Pi by setting

xi = Xi, y i = Xi+ 1, z i = Xi+ 2 (8)

Then, nmi is equivalent to the number of points inside thebounding box W i:

W i = [(xLB)i : (xUB)i] [( yLB)i : ( yUB)i] [(zLB)i : (zUB)i] (9a)

where the subscript LBand UBstand for the lower bound andupper bound of the box, and

(xLB)i = xi r, (xUB)i = xi + r( yLB)i = yi r, ( yUB)i = yi + r(zLB)i = zi r, (zUB)i = zi + r . . .

(9b)

Given an orthogonal range (bounding box) in the d-dimensional space, the number of points in each box isqueried, which is called an orthogonal range-counting prob-lem in the eld of computational geometry. Hence, for eachpoint Pi and its associated bounding box W i, the computationof nmi (n

m+ 1i ) isequivalentto an m (m + 1) dimensionalorthogo-

nal range-counting problem. Once nmi and nm+ 1i are computed,

nn , nd, and SE can be calculated directly from Eq. (5). Com-paring the dimension of nm+ 1i with n

mi , the time complexity

is dominated by the rst term. Hence, the time complexityof computing SE is determined by nm+ 1i . Fig. 1 shows the rst10,000 points of an ECG (electrocardiographic) signal. For agiven point Pi and distance r, Fig. 2 demonstrates the compu-tation of n1i from a geometric point of view. In the geometricview, the algorithm proposed in [3] can be interpreted as fol-lows. Foreach point Pi, calculateits boundingbox W i. Traveling each point p j with index j> i (Eq. (6)), nmi is equivalent to thenumber of points in W i. This algorithm requires double loopsi and j. Hence, it is an O(N2) algorithm and can be interpretedas a brute force algorithm, because it does not preclude anyimpossible queries (matches).

The kd tree [14,15]can be applied to the orthogonal range-counting problem [13] and it showed that it is an effectivealgorithm in computing SE. The fundamental concept is tostore the point sets {P} in a specially designed data structure;subsequently for a given box, the query would be faster. Thekd tree, proposed by Bentley in 1975, is a binary tree, whoseeach node v is associated with a rectangle Bv. If Bv contains

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000-150

-100

-50

0

50

100

150

200

250

points

a m p

l i t u d e

ECG TIme series

Fig. 1 Time series for the rst 10,000 points of an ECGsignal.

-150 -100 -50 0 50 100 150 200 250-150

-100

-50

0

50

100

150

200

250

x

y

x-y plot for ECG signal

2d search

point i

1d search

Fig. 2 Demonstration of the geometric view for an ECGsignal for m = 1. Computing n1i is equivalent to atwo-dimensional search and computing n0i is equivalent toa one-dimensional search.

only point in itsinterior, v is a leaf. Otherwise, Bv is partitionedinto two rectangles by drawing a horizontal or a vertical line,such that each rectangle contains, at most, half of the points;the splitting lines are alternately horizontal and vertical. Akd tree can be extended to higher dimensions in an obviousmanner.

Table 1 summarizes the performance of the kd tree algo-rithm.

Table 1 Time and storage complexity of the kd treealgorithms.Construct tree time O(N log N)Search time O(log N) for d = 1

O(N1 (1/d)) for d > 1Storage O(N)


4/15


3. The sliding kd tree algorithm

3.1. Developing the SKD algorithm

An algorithm called the sliding kd tree (SKD) [16] wasoriginally developed to speed up the range search in very

large-scale integration systems (VLSI) in electrical engineer-ing. However, its restriction is that the length of the query boxcan be xed in any direction. It can be observed that in Eq. (9),the distance r is identicalfor alldirections, andone cansubse-quently apply the SKD to further reduce the time complexityfor computing MSE. We focus on computing nmi , n

m+ 1i can be

computed in the same way.Forconvenience, rewriteEq. (9)as W i = (W x)i (W hd )iwhere

(W x)i = [(xLB)i : (xUB)i] (10a)

and

(W hd )i = [( yLB)i : ( yUB)i] [(zLB)i : (zUB)i] . . . (10b)

have been dened. The subscript x and hd in Eq. (10) standfor the bounding box in the x and higher ( d 1) dimensionrespectively, and subscript LB and UB denote the lower andupperbound of thebox.Also,index i denotes thepoints index.

The idea: The d (d = m) dimensional box W i is a one dimen-sional box ( W x)i intersected with a ( d 1) dimensional box(W hd )i. Thus nmi can be interpreted as the number of pointsrst in ( W x)i and then in ( W hd)i as demonstrated in Fig. 2. TheSKD algorithm is composed of a one dimensional and a ( d 1)dimensional range counting,and therimecomplexity is deter-mined by the latter one. The SKD algorithm of computing nmiis presented as follows.

Step1: Sort the point sets { p} by the x (rst) component inascending order, and then index each point in the sortedarray. Unless otherwise specied, { p} represents the sortedarray hereafter.Step 2: For i = 1:N m, report the points inside ( W x)i, which isequivalent to nd the points with index j satisfy

iLB(i) j iUB(i) (11)

where iLB(i) and iUB(i) represent the lower and upper indexof points inside ( W x)i.Step 3: Build the ( d 1) dimensional kd tree. The tree initiallycontains points in ( W x)i=1 and only the higher dimensioncoordinates ( y, z, . . .) are stored into the trees node.Step 4: Begin with i = 1, nmi is equivalent to the number of points inside ( W hd )i which is already in ( W x)i as illustrated inFig. 2. The d 1 dimensional kd tree search (search y, z, . . .)is applied to obtain nmi .Step 5: Slide (move) from point i to i +1, query the numberof points in ( W hd )i+1 intersecting ( W x)i+1. Firstly, report thepoints in ( W x)i+1 using the points in ( W x)i as a clue. Becausethe points within ( W x)i and ( W x)i+1 may be different, oldpoints (points in ( W x)i, but not in ( W x)i+1; that is, points withindexes iLB(i 1) j iLB(i) 1) have to be removed from thetree,andnewpoints(pointsin( W x)i+1,butnotin( W x)i;thatis,

Fig. 3 Demonstration of moving the box from point pi to pi+ 1 ; insertion of the new points and removal of the oldpoints.

points withindexes iUB(i) j iUB(i + 1))mustbe inserted intothe tree, as illustrated in Fig. 3. After the deleting and insert-ing operations, one obtains the points in ( W x)i+1. Secondly,the d 1 dimensional kd tree search is applied to obtain nmi+ 1.Step 6: Repeat Step 5 for i =1to N m.

Discussion and modications :

Step 1: Quick sort can be used to sort the point sets { p}.

Step 2: Find iLB(i) and iUB(i) for i =1to N m:

Given point pi, iLB(i) istheindexof the rstpointin { p} sat-isfying ( xLB)i x (xUB)i. That is, j= iLB(i) is the index satises

x j 1 < (xLB)i and x j (xLB)i (12a)

since the boxs length (2 r) in the x direction is a constantand { p} is a sorted array, then iLB(i) must be in ascending (strictly speaking, non-descending) order. Thus, we can visit j (the index to record iLB(i)) from 1 to N m to nd all iLB(i)for i = 1:N m. Starting from i = 1 and j=1, if the conditionx j 1 < (xLB)i and x j (xLB)i holds, then record iLB(i) = j and

increase i by one; Otherwise increase j by one. Similarly, iUB(i)is the index j which satises

x j (xUB)i and x j+ 1 > (xUB)i (12b)

The same procedure isapplied tond iUB(i)for i =1to N m.

Step 5: The structure of the kd tree is described in Section2.2. The kd tree is, in nature, static and a balanced binarytree. Thedeletionandinsertionoperationswould destroythealready balanced structure of the kd tree, hence increase thetree depth and search time. This difculty can be circum-vented as follows. Initially, build the d 1 dimensional treeusing { p} (all thepoints in thedomain).Once thetree is built,


5/15


6/15


Algorithm 1: .1: r = r *SD(X); % Normalize r by the standard deviation (SD) of

the time series {X}2: for scale = 1: max do3: X = coarseGrainSeries(scale, X) % nd the coarse-grain

series s using Eq. (7)4: pointArray=transform to space( m, X)

5: for (k =m 1:m) do6: d = k;7: pointArray =sort points by x coord(pointArray); % step

18: [iLB,iUB]= nd lower upper index(r, pointArray) % Eq.

(12)9: pointArray = compress data(pointArray);10: tree=buildKDTree(d, pointArray); % build kd tree

structure11: for ( j= 1: iUB[1] 1)12: insert new points(pointArray[j],tree); % insert point

pj into the tree13: end do14: for (i = 1: N m) do % The Sliding Loop15: delete old points([pointArray[i]] ,tree); % deletepoint pi

from the tree16: (W hd )i = nd bounding box( m,r,PointArray[i]) % using

Eq. (10b)17: nmi = kd search(( W hz )i, tree); %compute n

mi (k =

m 1)or nm+ 1i (k = m);18: if (k = =m 1)19: nn(i) = nn(i) + nmi % numerator term in Eq. (5)20: else21: nn(i) = nn(i) + nmi % denominator term in Eq. (5)22: endif 23: for ( j= iUB[i]+ 1: iUB[i +1]) do24: insert new points(pointArray[j],tree); % insert new

points to tree25: end do % j26: end do % i27: end do % k28: SE(scale,m,r ) = ln(nn /n d) % Eq.(5)29: release tree memory(tree);29: end do % scale

data as discussed in Section 3.2, the data compression tech-nique is applied in this algorithm.

Input: max : the maximum scale, m: the pattern length, r:the distance of accepting similarity, and the time series {X}dened in Section 2.1.Ouput: SE for scale = 1: max .

We explain the details of the implementation of Algorithm1 in computing the MSE in natural language.

Line 1: Normalize r by the standard deviation of the originaltime series in Eq. (1).Line3: Calculate thecoarse-grainedtime series fromthe orig-inal time series by Eq. (7).Line4: For each( X )i, transform thetemplate vector ( Xi)m intoa d dimensional space point sets using Eq. (8).Line 7: Input point array in space, output the indexed sortedpoint array { p}. Points are stored in a structure containing their ( x, y,z. . .) coordinate. Quick sort algorithm is applied tosort the point sets by the their x coordinate.

Line 8: Input { p} and r, Output the iLUand iUBarray denedEq. (11). The detailed is discussed in step 2 in Section 3.1.Line 9: compress data(): Input thesortedpoint array { p}. Out-put the new compressed point array. Compress the pointarray. If points have the same coordinates, only the rst oneis stored in the database. The counter in the trees nodedescribedin Step 5 inSection 3.1 is used to record thenumberof points in a compressed point.

To operate the kd tree, the following tree functions areneeded:

(1) build kd tree(): Input ( d 1) dimensional points;Ouput thekd tree structure;

(2) delete old points(): this function is to delete points fromthe tree. Input points to be deleted from the tree. Thetree nodes counters corresponding to these points areimplicitly subtracted from the tree and these points aredisjointed from the leaf node as discussed in Step 5 inSection 3.1.

(3) insert new points(): similar to the delete old points().(4) kd search(): Input the bounding box ( W hd )i and the tree;

output: nmi(5) release tree memory(): thisfunction is to release the trees

memory once the tree operations are nished. Input thetree, andthe memory is implicitly cleaned in thisfunction.

The kd tree isa popular algorithm. Freecomputerprogramscan be found in the public domain.

Line 10: build the kd tree structure.Line 1113: for i = 1, insert the point sets into the tree.Line 15: is the deletion operation.Line 16: input the point array, m, and r. Output the bounding box (W hd )i dened in Eq. (10b).Line 17: compute nmi and n

m+ 1i for each i by the ( d-1) dimen-

sional kd tree algorithm as described in Step 4 in Section3.1.Line1822: compute nn(nd) by taking the contribution fromnmi (n

m+ 1i ) for each i.

Line 2325: the insertion operation.Line 28: compute SE from nn and nd using Eq. (5)Line 29: release the memory after the computation of SE isnished.

From Eq. (6), the index j starts from i +1 in computing SE.However, from Eq. (3), the index j starts from 1 in computing AE. Then, line 15 must be modied as the following 3 lines:

for ( j= iLB[i-1]:iLB[i] 1)deleteOldPoints(pointArray[i]);

end do

3.5. The adaptive SKD algorithm

3.5.1. Developing the adaptive SKDalgorithmWhen the data length N is short (under certain threshold),the brute force algorithm performs better than the SKD algo-rithm because both building and operating the tree involvea lot of overheads. The threshold is around N =1000 with


7/15


m = 2 from numerical experiments. For short data length, thecomputational time can be further improved by the following algorithm.

Keep thesortingalgorithmfortherst dimensionin step (1)given in Section 3.1. Replace the kd tree search in steps (2)(3)given in Section 3.1 f or thehigher dimensionby thebrute forcealgorithm. The computer program, which adaptively switchesfrom SKD to this algorithm when N is smaller than a certainthreshold, can be implemented. Therefore, this algorithm iscalled the adaptive SKD algorithm. Now, the details of theimplementation of the adaptive SKD algorithm in computing the SE in general language are explained.

3.5.2. Algorithm 2: the adaptive SKDalgorithmIn this section, we present the pseudo code of computing thenn dened in Eq. (5) by the adaptive SKD algorithm. It can beeasily modied to compute SE as described in Algorithm 1.Replace m by m +1, we obtain nd. Then SE can be computed byEq. (5).

Input: m: the pattern length, r: thedistance of accepting sim-ilarity, and the time series {X} dened in Section 2.1.Ouput: nn

1: pointArray= sort points by x coord( X);2: [iLB,iUB] = nd lower upper index ( r, pointArray)3: for (i = 1: N m) do4: i idx= pointArray[ i].id5: for ( j= i + 1:iUB[i]) do6: j idx = pointArray[ j].id7: for (k = 1: m)8: if (abs(y[i idx + p] y[j idx+ p])

9: nn(i)=nn(i)+110: else11: break;12: end do13: end do % k14: end do % j15: end do % iLine 1 is exactly the same as line 7 in Algorithm 1 except

that the ( y, z, . . .) coordinate is not needed in the pointArraystructure. Line 2 is exactly the same as line 8 in Algorithm1. The pointArray structure in line 1 also contains the origi-nal index (before sorting). Thus we can remember its originalindex after sorting. Line 3 to line 15 is to compute the number

ofthepointsinside the boundingbox inEq. (9).Line5 to14 istocompute nmi , whichis quitesimilar to thebrute forcealgorithmprovided in [3]except that we must transform from thesortedcoordinate system back to its original coordinate system (line4 and 6) to perform the similarity testing for k = 1,2,3 (line 7 to13). To compute the AE, the index j startsfrom1 instead of i + 1,then we must modify line 5 to for ( j= iLB[i]:iUB[i]) do.

3.5.3. Time complexity analysisFrom the pseudo code of Algorithm 2 , it can be observed thatthree loops are required, namely, 1 i N m, i+ 1 j iUB[i],and 1 k m. Hence, the time complexity is

T (N) = O(m Nnbr N) = O(m Nnbr N) (16)

where Nnbr is the average number of [ iUB(i) iLB(i)] in Eq. (12),which is approximately equal to rN where r


8/15


RR

Brute

kd

SKD

10000 20000 30000 40000 50000 60000 70000 80000 90000 1e+05 1.1e+05N

0

20

40

60

80

100

120

S a m p

E n

t i m e

( s e o n

d . )

RR

SKD

kd

10000 20000 30000 40000 50000 60000 70000 80000 90000 1e+05 1.1e+05

N

0

0.5

1

1.5

2

2.5

3

3.5

S a m p

E n

t i m e

( s e c o n

d )

a

b

Fig. 4 (a) Execution times versus N for RR intervals and SE (scale= 1, m = 2, r =0.15 SD ). The black squares and line represent the execution times of the brute force algorithm, the blue crossed lines represent the corresponding values for the kd treealgorithm, and the red crossed line represents the corresponding values for the SKD algorithm. (b) Execution times versus Nfor RR intervals and SE (scale=1, m = 2, r =0.15 SD ). The blue circles and line represent the corresponding values for the kdtree algorithm, and the red crossed line represents the corresponding values for the SKD algorithm.

SKD =122 and kd =26, which shows that the SKD algorithmis signicantly faster than the kd tree algorithm and the bruteforce algorithm. Furthermore, Fig. 4b shows that the SKD isapproximately an O(N) algorithm, as can be seen that the SKD

curve is approximately a straight line.Figs. 57 show the T N plot for the ECG, EEG, and the 8-bit

random signal. Similar results areobtainedas in theRR series.In thesethree cases, theSKD isfoundto outperformthe kdtreealgorithmby 2.55 timesandthebruteforce algorithmby morethan 100 times for N = 80,000. Note that for N = 80,000, T SKD forthe8-bit random signalis about3 timesslower than the others.For the random signal, x and y are not correlated; thus, thepoints are evenly distributed in the Poincare plot (see Fig. 2).Therefore, Ndiff for the random signal is much larger than thatof a biological signal, thereby taking longer execution times.

Fig. 8(a) shows the T N plot fordifferent r.Itshowsthat T SKDincreases slightly with r. As nmi increases with r, the depth of

the kd tree also increases. Therefore, ( T )SKD increases with r.

Fig. 8(b) shows the T N plot for different m. It shows thatthe ( T )SKD increases with m initially and converges (stabilizes)to one curve nally ( m 8).

Fig. 8(c) shows the T N plot for different b (bit number) for

the random signal. The time complexity of the SKD for m = 2,as discussed in Section 3.3, is bounded by O(B N) if N >B2. Thefollowing is observed: rst, ( T )SKD increases with b for xed N.Second, if N B2, then the T N plot is close to a straight line.Third, if the condition N B2 fails, then the SKD algorithmwould switch to an O(N3/2) algorithm, which is then indepen-dent from b. For this reason, the T N plots converge (stabilize)to one curve for N > 80,000 and b >10.

4.2. Experiment 2

This experiment tests the performance of the SKD in com-puting A E (scale=1, m = 2, r =0.2 SD). Fig. 9a and b shows theT N plot for the RR series and the EEG signal, respectively. To


9/15


ECG Signala

b

Brutekd

SKD

10000 20000 30000 40000 50000 60000 70000 80000 90000 1e+05 1.1e+05 1.2e+05number of points(N)

0

20

40

60

80

100

120

S a m p

E n

t i m e

( s e c o n

d )

ECG Singal

SKD

kd

10000 20000 30000 40000 50000 60000 70000 80000 90000 1e+05 1.1e+05 1.2e+05N

0

1

2

3

4

5

S a m p

E n

t i m e

( s e c o n

d )

Fig. 5 (a) Execution times versus N for the ECG signal and SE (scale=1, m = 2, r =0.15 SD ). The black circles and linerepresent the execution times of the brute force algorithm, the blue crossed lines represent the corresponding values for thekd tree algorithm, and the red crossed line represents the corresponding values for the SKD algorithm. (b) Execution times versus N for the ECG signal and SE (scale = 1, m = 2, r =0.15 SD ). The blue circles and line represent the execution times forthe kd tree algorithm and the red crossed line represents the corresponding values for the SKD algorithm.

make a fair comparison of the performance of the SKD withthat of the bucket-assisted algorithm developed by Manis [9],( T )SKD and the estimated( T )bucket assist were measured. As canbe seen from Fig. 4 of [9], ( T )bucket assist for both RR and EEG sig-nals are less than 4 for N =80,000. Fig. 9(a) and (b) shows that( T )SKD =106s/0.61s =173 for RR, and 105s/0.38s =276 for EEG,which indicates a signicant improvement over the bucket-assistedalgorithm. On comparing Fig. 4with Fig. 9(a)and Fig. 6with Fig. 9b, one can observe that computation of AE is abouttwo times more than that of SE. The reason is that when using Eq. (6)in computing AE, the index j starts from 1 instead of j+ 1as in computing SE, andmore logarithms areinvolved in Eq. (4).

To mimic the computations of the complexity of the DNAsequence [3,11], the approximate entropy AE (scale=1, m = 2)of a 1-bit random signal is computed. Fig. 9(c) shows the T Nplot using the SKD algorithm. Table 2 shows the executiontimes versus m for N = 106 using the SKD algorithm. It showsthat ( T )SKD


10/15


EEG signal

Brute

kd

SKD10000 20000 30000 40000 50000 60000 70000 80000 90000 1e+05 1.1e+05 1.2e+05

N

0

20

40

60

80

100

120

S a m p

E n

t i m e ( s e c o n

d )

EEG Signal

kd

SKD

10000 20000 30000 40000 50000 60000 70000 80000 90000 1e+05 1.1e+05 1.2e+05N

0

0.2

0.4

0.6

0.8

1

1.2

S a m p

E n

t i m e

( s e c o n

d )

a

b

Fig. 6 (a) Execution times versus N for the EEG signal and SE (scale = 1, m = 2, r =0.15 SD ). The black squares and linerepresent the execution times of the brute force algorithm, the blue crossed lines represent the corresponding values for thekd tree algorithm, and the red crossed line represents the corresponding values for the SKD algorithm. (b) Execution times versus N for the EEG signal and SE (scale=1, m = 2, r =0.15 SD ). The blue circles and line represent the execution times forthe kd tree algorithm and the red crossed line represents the corresponding values for the SKD algorithm.

4.3. Experiment 3

This experiment is to test the performance of the SKD and

the adaptive SKD algorithms. The rst test is the EEG signalin computing SE (scale= 1:1, m = 2, r =0.15 SD). The results forN 20,000are listedin Table3 . Table3 shows that theadaptiveSKDalgorithmis thefastest one in the rangeof approximately3008000. ( T )adp skd is around5 for N = 500, and isaround10 forN around 4000. Furthermore, ( T )adp skd is higher than ( T )SKDfor N approximately less than 8000 and is higher than ( T )kdfor N approximately less than 10,000.

The second test is the 1 / f (pink) noise in computing MSE(scale= 1:20, m = 2, r =0.15 SD). The results for the data lengthranging from 300 to 0.8 million are listed in Table 4 . Table 4shows that ( T )SKD is around 40 for N =100,000 and 160 forN = 800,000. For N 1000, the benet of SKD is not clear, as dis-

cussed in Section 3.5. It canbe observed that the adaptive SKD

algorithm is faster than the SKD algorithm for N < 20,000, andis faster than the brute force algorithm for N approximatelygreater than 400. Furthermore, the adaptive SKD algorithm

is faster than the kd tree algorithm for N < 8 105. The resultis surprising, because the adaptive SKD algorithm has beendesigned for only handling short data length. On comparing T bruteforce and T adp skd , the adaptive SKD is consistently 1015times faster than the brute force algorithm for N 4000. TheO(N2) property of the adaptive SKD can be veried as follows.The execution time increases approximately by 4 times whenN is doubled as can be seen from Tables 3 and 4 .

4.4. Experiment 4

To test the SKD algorithm and its adaptive version in the han-dling of time-varyingMSE,16-bit EOG datacollectedovernightare partitioned into nonoverlapping windows to analyze the


11/15


Random

kdSKD

Brute

10000 20000 30000 40000 50000 60000 70000 80000 90000 1e+05 1 .1e+05 1 .2e+05N

0

20

40

60

80

100

120

140

S a m p

E n e x e c u

t i o n

t i m e

( s e c . )

Random: 8 bit

kd

SKD

10000 20000 30000 40000 50000 60000 70000 80000 90000 1e+05 1.1e+05 1.2e+05

N

0

2

4

6

8

S a m p

E n

t i m e

( s e c o n

d )

a

b

Fig. 7 (a) Execution times versus N for the 8-bit uniform distributed random signal and SE (scale= 1, m = 2, r =0.15 SD ). The black squares and line represent execution times of the brute force algorithm, the blue circle lines represent thecorresponding values for the kd tree algorithm, and the red crossed line represents the corresponding values for the SKDalgorithm. (b) Execution times versus N for the 8-bit uniform distributed signal and SE (scale=1, m = 2, r =0.15 SD ). The bluecircles and line represent the execution times for the kd tree algorithm and the red crossed line represents thecorresponding values for the SKD algorithm.

relation between the MSE and the sleep stages. Each win-

dow contains 7680 (30s 256Hz) data points and the MSEprocedure is repeated for each window. The MSE analysis of the rst 30,000s of one subjects EOG is shown in Fig. 10.

Fig. 10(a) shows the MSE values of different scales ( x-axis:

time, y-axis: scale, color: MSE value), while Fig. 10(b) showsthe results of averaging the 20 MSE values in each window.Fig. 10(c) shows the manual sleep scoring by the expert, and

Table 3 Execution times versus N for EEG for m = 2, r =0.15 SD, scale= 1 for the brute force, kd tree, SKD, and adaptiveSKD algorithms.

Number of points Brute force (s) kd tree (s) SKD (s) Adaptive SKD (s)300 8.5 10 4 2.3 10 3 1.8 10 3 2.4 10 4

500 2.5 10 3 4.2 10 3 1.8 10 3 5.0 10 4

1000 9.1 10 3 6.5 10 3 4.0 10 3 1.3 10 3

2000 3.6 10 2 1.4 10 2 7.3 10 3 3.5 10 3

4000 0.14 0.031 0.018 0.014

8000 0.57 0.066 0.035 0.04120,000 3.63 0.18 0.077 0.23


12/15


EEG

r=0.05

r=0.3

r=0.1r=0.15

r=0.2

20000 30000 40000 50000 60000 70000 80000 90000 1e+05 1.1e+05 1.2e+05N

0.1

0.2

0.3

0.4

0.5

0.6

0.7

S a m p

E n

t i m e

( s e c o n

d )

RR

m=1m=2

m=3

m=4

m=8,10

20000 30000 40000 50000 60000 70000 80000 90000 1e+05 1.1e+05 1.2e+05N

0

1

2

3

4

5

S a m p

E n

t i m e

( s e c o n

d )

Random Signal

b=1

b=6

b=10

b=14,15

b=4

b=8

b=2

b=12

20000 30000 40000 50000 60000 70000 80000 90000 1e+05 1.1e+05 1.2e+05N

0

0.5

1

1.5

2

S a m p

E n

t i m e

( s e c o n

d )

a

b

c

Fig. 8 (a) Execution times versus N for the EEG signal for different r using the SKD algorithm in computing SE (scale= 1,m = 2). (b) Execution times versus N for the RR series for different m using the SKD algorithm in computing SE (scale=1,r = 0.15). (c) Execution times versus N for the random signal for different number of bits using the SKD algorithm incomputing SE (scale = 1, m = 2, r =0.15).


13/15


14/15


Table 4 Execution times versus N for 1/ f noise for m = 2, r =0.15 SD, scale = 1:20 for the brute force, kd tree, SKD, andadaptive SKD algorithms.

Number of points Brute force (s) kd tree (s) SKD (s) Adaptive SKD (s)300 0.0019 0.014 0.012 0.0021500 0.0075 0.017 0.010 0.005211000 0.015 0.032 0.02 0.00662000 0.059 0.059 0.039 0.00934000 0.29 0.13 0.08 0.0268000 0.921 0.29 0.16 0.0720,000 5.39 1.01 0.46 0.4250,000 32.4 4.18 1.34 2.14105 128.4 15.16 3.15 9.012 105 501.0 51.3 7.35 34.24 105 2063 176.7 18.5 1468 105 8716 655.0 52.5 54016 105 153 32 105 411

the scores dening wake, rapid eye movement (REM), stages 1

(S1) and 2 (S2), and slow wave sleep (SWS) are 0, 1, 2, 3,and 4, respectively. The correlation coefcient between theaverage MSE values and the manual scoring of sleep stagescan reach 0.7628. This result demonstrates the feasibility of applying MSE to sleep staging. The execution times of the

brute force method, the SKD, and the adaptive SKD algo-

rithms to analyze 8-h EOG data (7,365,376 samples) were 320,40.0, 24.5s, respectively. With such a level of performance,this experiment demonstrates that the SKD algorithm can beintegrated into a chip in a consumer product for online diag-nosis.

Fig. 10 The relation between the MSE and the sleep stages. (a) The MSE values using EOG of different scales of one subject;(b) results after averaging the 20 MSE values in each epoch; (c) manual sleep staging reviewed by the expert.


15/15


4.5. Summary

The experiment veries that the SKD algorithm and its adap-tive version are robust for all parameters including differentbiomedical signals,datalength ( N), data resolution ( B), patternlength ( m), distance ( r), and scales.

The SKD algorithm is O(N3/2) for m = 2 and for the real-typedata, and is O(N) for the integer-type data as in the digital-ized biological signal. From all the experiments, it is observedthat the SKD algorithm is signicantly better than the kd treealgorithm, the bucket-assisted algorithm, and the brute forcealgorithm in the literature bya factorof 26, more than 20,andmore than100, respectively, for N = 80,000. For small N,theSKDis observed to be inferior to the brute force algorithm. For thisreason, an adaptive SKD algorithm has been developed and itis more than 10 times faster than thebrute force algorithm forr = 0.15 and N 2000.

5. Conclusions

Sample entropy and approximate entropy are measurementsof complexity used to analyze biomedical and other sig-nals. The time complexity of the algorithms proposed inthe literature requires O(N2), which is not fast enough formany applications. This paper has developed a sliding kdtree algorithm to reduce the time complexity. For a typicalvalue of pattern length m = 2, the time complexity is reducedto O(N3/2) for the real-type data, and is reduced to O(BN)for the integer-type data as in the digitalized biomedicaldata. Experimental results show a signicant improvement more than 100 times when compared with the con-ventional brute force method, and 26 times than that of

the kd tree algorithm for N = 80,000. The adaptive SKD algo-rithm was also developed to handle short data length. Theadaptive SKD algorithm is more than 10 times faster thanthe conventional brute force method for N 4000. As theexecution time is substantially reduced, the SKD (adaptiveSKD) algorithm can be integrated into a chip in a consumerproduct for online diagnosis and can efciently computethe complexity of biological signals over long periods of time.

Conict of interest

None.

Acknowledgment

This work was funded in part by the Industrial DevelopmentBureau Ministry of Economic Affairs, Taiwan (ROC).

r e f e r e n c e s

[1] S.M. Pincus, Approximate entropy as a measure of systemcomplexity, Proceedings of the National Academy of Sciences 88 (1991) 22972301.

[2] J.S. Richman, J.R. Moorman, Physiological time series

analysis using approximate entropy and sample entropy,American Journal of Physiology 278 (6) (2000) H2039H2049.[3] M. Costa, A.L. Goldberger, C.K. Peng, Multiscale entropy

analysis of biological signals, Physical Review E 71 (2005)021906.

[4] L.Angelini, R. Maestri, D. Marinazzo, L. Nitti, M. Pellicoro, G.Pinna, S. Stramaglia, S.A. Tupputi, Multiscale analysis of short term heart beat interval, arterial blood pressure, andinstantaneous lung volume time series, ArticialIntelligence in Medicine 41 (2007) 237250.

[5] R. Hernndez-Prez, L. Guzmn-Vargas, A. Ramrez -Rojas, F.Angulo-Brown, Multiscale entropy analysis of electroseismic time series, Natural Hazards and Earth SystemSciences 8 (2008) 855860.

[6] R. Yan, R.X. Gao, Approximate entropy as a diagnostic tool

for machine health monitoring, Mechanical Systems andSignal Processing 21 (2007) 824839.[7] M. Costa, A.L. Goldberger, C.K. Peng, Multiscale entropy

analysis of complex physiologic time series, Physical ReviewLetters 89 (2002) 068102.

[8] M. Costa, C.K. Peng, A.L. Goldberger, J.M. Hausdorff,Multiscale entropy analysis of human gait dynamics,Physica A 330 (2003) 5360.

[9] G. Manis, Fast computation of approximate entropy,Computer Methods and Programs in Biomedicine 91 (2008)4854.

[10] K. Sugisaki, H. Ohmori, Online estimation of complexityusing variable forgetting factor, in: Proceedings of the SICEAnnual Conference, 2007, pp. 16.

[11] S. Pincus, B.H. Singer, Randomness and degrees of

irregularity, Proceedings of the National Academy of Science93 (March) (1996) 20832088.[12] S. Pincus, R.E. Kalman, Not all (possibly) random

sequences are created equal, Proceedings of the NationalAcademy of Science 94 (1997) 35133518.

[13] Y.H. Pan, W.Y. Lin, Y.H. Wang, K.T. Lee, Computing multiscaleentropy with orthogonal range search, Journal of MarineScience and Technology 19 (2011).

[14] J.L. Bentley, Multidimensional binary search trees used forassociative searching, Communications of the ACM 18 (9)(1975) 509517.

[15] J.L. Bentley, Decomposable searching problems, InformationProcessing Letters 8 (1979) 244251.

[16] Y.H. Pan, Y.H. Wang, K.T. Lee, An optimal two-dimensionalorthogonal range search algorithm in VLSI design

automation, in: Proceedings of the InternationalSymposium on Computer, Communication, Control andAutomation (3CA IEEE), May, 2010, 2010, pp. 5356.

[17] F. Marcaino, M.L. Migaux, D. Acanfora, G. Furgi, F. Rango,Quantication of Poincare maps for the evaluation of heartrate variability, IEEE Computer in Cardiology (1994) 577580.

Documents

Entropia muestral