87531018 summary-of-efica

Homework Help

https://www.homeworkping.com/

Research Paper help


Online Tutoring


Summary of EFICA

A basic assumption in ICA is that the elements of Original Source Signals(S),denoted by s ij, are mutually independent identically distributed random variables with probability density functions (pdf’s) being defined as pi(si)j;i=1….d . The row variables sijfor all j=1…N , having the same density, are thus an independent identically distributed (i.i.d.) sample of one of the independent sources denoted by . The key assumptions for the identifiability of the model (1), or solving both the Mixing Matrix (A) and Original Source (S) up to some simple ambiguities, are that all but at most one of the densities are non-Gaussian, and the unknown matrix has full rank, i.e., it has column/row rank, whichever is the maximum(Rank corresponds to the Linearly Independent seta(i.e. Either a row or a Column).Column Rank therefore would be the maximum number of Lineraly Independent Columns. A full rank would be possible if all Columns are independent, given No. of Columns> Number of rows.) < -Part of ICA Intro

The basic ICA problem and its extensions and applications have been studied widely and many algorithms have been developed. One of the main differences is how the unknown probability density functions pi() of the original signals are estimated or replaced by suitable nonlinearities in the ICA contrast functions. Non-Gaussianity is the key property. For instance, JADE [SELF] is based on the estimation of kurtosis via cumulants, NPICA [SELF] uses a nonparametric model of the density functions, and RADICAL [SELF] uses an approximation of the entropy of the densities based on order statistics. The FastICA algorithm uses either kurtosis [FastICA’s Paper] or other measures of non-Gaussianity in entropy approximations in the form of suitable nonlinear functions G()<- Part of Intro.Please include the other non linearities as well here

Though ICA has been very successful in large scale practical problems, it still suffers from some issues, like the Theoretical Accuracy of the Algorithm when considering the various inputs. To Prove the general


http://www.homeworkping.com/





Validity that the algorithm is correct and efficient, it should reach Cramer-Rao’ s Lower Bound.(Need to add info about CRB here)Also write in the CRB that practically the demixing matrix is not exactly the inverse of Original Sources A and the Estimated Sources is approximation of original signals with the variance being calculated as WA-unit matrix(WA is the multiplication of Demixing and Mixing matrix, and assuming a dxd Unit matrix(same dimenstions as that of A and W)

An asymptotic performance analysis of the FastICA algorithm in is compared with the CRBfor ICA [SELF] and showed that the accuracy of FastICA is very close, but not equal to, the CRB. The condition for this is that the nonlinearity G() in the FastICA contrast function is the integral of the score function of the original signals, or the negative log density

[SELF]When the asymptotic performance achieves the CRB, the absolute accuracy is reached, which cannot be improved further.

Use the SectionII.A in FastICA, as a chapter.Use the last Paragraph for EFICA, along with the figure.(If possible, try to understand the maths of section B)EFFICIENT FASTICA: EFICAThe proposed efficient version of FastICA is based on the followingobservations: i) The symmetric FastICA algorithm canbe run with different nonlinearity for different sources; ii) inthe symmetrization step of each iteration, it is possible to introduceauxiliary constants, that can be tuned to minimize meansquare estimation error in one (say th) row of the estimateddemixing matrix. These estimations can be performed in parallelfor all rows—to obtain an estimate of the whole demixingmatrix, that achieves the corresponding CRB, if the nonlinearitiescorrespond to score functions of the sources; and iii) thealgorithm remains to be asymptotically efficient (attaining theCRB) if the theoretically optimum auxiliary constants in the algorithmare replaced by their consistent estimates.The proposed algorithm EFICA models all independent signalsas they have a generalized Gaussian (GG) distribution withappropriate parameters ’s. The algorithm is summarized inFig. 1. Note that the output is not constrained, unlike symmetricFastICA, in the sense that the separated components need nothave exactly zero sample correlations.In order to explain the proposed algorithm in more details, thenotion of “generalized symmetric FastICA” is introduced, andits efficiency is studied in Section III-A. The algorithm EFICAwill be presented in detail in Section III-B.A. Generalizing the Symmetric FastICA to Attain the CRBConsider now a version of the symmetric version of FastICAwhere two changes have been made.First, as it is not possible to attain the CRB if only oneNonlinearity g()is used, different nonlinear functions, gk(),k=1,2…..d will be used for estimation of each row of W+

Second, the first step of the iteration will be followed by multiplying each row of W+ with a suitable positive number ci , i=1,2….d before the symmetric orthogonalization Thiswill change the length (norm) of each row, which will affect theorientations of the rows after orthonormalization.

The true score functions are rarely known in advance,and the generalized symmetric FastICA has only a theoreticalmeaning. It can be proved, however, that the asymptotic efficiencyof the algorithm is maintained if the score functions andthe optimum coefficients are replaced by their consistentestimates.For the consistent estimation, it is necessary to have a consistentinitial estimate of the mixing or demixing matrix. Theordinary symmetric FastICA is one possible choice. Second,one needs a consistent estimate of the score functions computedfrom the sample distribution functions of the components. Thisis a widely studied task, and numerous approaches have beendeveloped either parametric [21] or nonparametric [11], [12],[22]. Note, however, that not every score function can serve asuitable nonlinearity for use in FastICA iteration. Suitable nonlinearitymust be continuous and differentiable.

B. Proposed AlgorithmIn this section, an algorithm, called for brevity EFICA is proposed,which combines the idea of the generalized symmetricFastICA with an adaptive choice of the function , which isbased on modelling of the distribution of the independent componentby GG distribution [15].The algorithm consists of three stepsStep 1) Running the Symmetric FastICA Until Convergence: The purpose of Step 1) is to quickly and reliably get preliminaryestimates of the original signals. In this step, therefore, theoptional nonlinearity in the original symmetric FastICA g(s)=tanh(s)is used due to its universality, but other possibilitiesseem to give promising results as well, e.g., .g(s)=s/(1+s2)Also, the test for saddle points as introduced in [8] is performedto get reliable source estimates.

2) adaptive choice of different nonlinearities gk to estimatethe score functions of the found sources, based on theoutcome of step 1);Assume that ukis the kth estimated independent signal obtained in Step 1).In many real situations, the distributions of the signals areunimodal and symmetric. In this paper,we focus on a parametricchoice of gk that works well for the class of GG distributions with parameter (Symbol:alpha)denoted as GG((Symbol:Aplha)).The score function of this function is

A problem with the score function of the distribution GG((Symbol:alpha))is that it is not continuous for (Symbol:alpha) < 1 and thus it is not a valid nonlinearity for FastICA. For these (Symbol:alpha) ’s the statistical efficiency cannot be achieved by the algorithm using this score function.They therefore take the Super Gaussian (Symbol:alpha) >2) and Sub Gaussian ((Symbol:alpha) <2) separately.In summary, the nonlinearity of our choice is

where m4k is the estimated fourth-order moment of the kth source signal, and

((Symbol:alpha))k is , where v1 is approx 0.2096 and v2 is approx 0.1851

Step 3): The Refinement: a refinement or fine-tuning for each of the found source components by one-unit FastICA, using the nonlinearities found in step 2), and another fine-tuning using the optimal Ck parameters (where ck parameters are

)

The refinement of the initial estimateproceeds in two steps.The first step, denoted R1, is a more sophisticated implementationof the relation (15). Theoretically, it would suffice to perform(15) once, starting from the initial estimate of W. However,better results are obtained if it is performed separately foreach k as series of one unit FastICA iterations, until a convergenceis achieved. In the last iteration, however, the normalizationstep is skipped.This method works well, if the preliminary estimates of theoriginal signals uk from the first step (symmetric FastICA) ofthe proposed method lie in the right domain of attraction. Itmight happen, however, that some of the components are difficultto separate from some other components, and the one-unititerations converge to a wrong component. This pathologicalcase can be excluded by checking the condition whether theangle between the component separated by the initial solutionand the one unit solution is not too big. If it happens, then theone unit solution should be replaced by the initial estimate.

Homework Help




Math homework help


Research Paper help


Algebra Help


Calculus Help


Accounting help


Paper Help


Writing Help


Online Tutor


Online Tutoring




















Education

87531018 summary-of-efica