Unmixing signals recorded on several sensors.
Principal component analysis (PCA) and its cousin independent component analysis (ICA) both write data as a few directions times amounts; PCA asks for the directions of most spread, ICA for directions whose amounts are statistically independent, and non-negative matrix factorization (NMF) for parts that are never negative.
ICA fits once, in batch, on a whole recording; after that, unmixing a new sample is subtracting the training mean, then one matrix multiply.
Each method writes the data as a few blocks times amounts; they differ in what they ask of the blocks.
| PCA | ICA | NMF | |
|---|---|---|---|
| What it asks for | most spread | independent amounts, like two voices in one room | parts that only add, like notes in a spectrogram |
| Input | one data matrix | several sensors, one column each | one non-negative data matrix |
| Sign of blocks | any | any | never negative |
| Blocks at right angles? | yes | not in general | not in general |
What PCA, ICA and NMF ask of the same data.
PCA is where ICA begins, since ICA starts by whitening, a step taught in Principal Component Analysis. NMF gets its own page, Non-negative Matrix Factorization, next in Classical ML.
Two people talk in one room, and two phones or meeting-room mics record them. Each mic hears its own blend of the two voices.
Mic 1 hears 2 parts voice 1 and 3 parts voice 2, mic 2 hears 2 parts voice 1 and 1 part voice 2, and these four weights make the mixing matrix A. The weights come from Hyvärinen and Oja (2000), the ICA tutorial this page follows.
An illustrative toy with 4 samples keeps every step checkable by hand. Each voice is on (+1) or off (−1); off is −1, not 0, because each voice is centred on its own average, so its mean is zero. Every on and off pair appears once, so knowing one voice says nothing about the other. The same steps run on the speech-like and music-like stand-in signals of the tool in Section 2.7.
| Sample 1 | Sample 2 | Sample 3 | Sample 4 | |
|---|---|---|---|---|
| Voice 1, s1 | 1 | 1 | −1 | −1 |
| Voice 2, s2 | 1 | −1 | 1 | −1 |
| Mic 1 | 5 | −1 | 1 | −5 |
| Mic 2 | 3 | 1 | −1 | −3 |
The two voices and the two mic readings at each of the 4 samples.
ICA first whitens the mic readings, as taught in PCA variants. Whitening removes their correlation and sets every spread to 1, so only a rotation is left to find.
| Sample | Mics x | Whitened z̃ |
|---|---|---|
| 1 | 5, 3 | 1.411, −0.088 |
| 2 | −1, 1 | −0.088, −1.411 |
| 3 | 1, −1 | 0.088, 1.411 |
| 4 | −5, −3 | −1.411, 0.088 |
| Spread along PCA axes | 17.062, 0.938 | 1, 1 |
The four mic samples before and after whitening.
Neither whitened coordinate equals a voice: whitening removed correlation, not all of the mixing.
Round is not separated; Section 2.3 says what is left to undo.
Independent means knowing one source tells you nothing about the other. The whitened samples are uncorrelated, yet z̃1 = 1.411 tells you z̃2 = −0.088 in the Section 2.2 table. Uncorrelated only rules out a straight-line relation; independent rules out every relation.
| Value | What it shows | |
|---|---|---|
| Blend left after whitening, M | 0.662, 0.750−0.750, 0.662 | a turn, no flip |
| M times its transpose | 1, 00, 1 | rows of length 1, at right angles |
| A−1 | −0.25, 0.750.5, −0.5 | undoes A |
| A−1 times A | 1, 00, 1 | undoing after mixing changes nothing |
| A−1 times sample 1, (5, 3) | 1, 1 | the voices of sample 1 |
What whitening leaves of the blend, and the inverse of A.
Whitening made the cloud round. ICA fixes each source to variance 1, and the on/off voices already have it, so the sources were round too; what is left can only turn or flip the cloud, as row 2 shows.
Every rotation keeps the cloud round; Section 2.4 says which one is right.
Gaussian is another name for the bell curve, and non-Gaussian means shaped unlike a bell.
Adding independent signals usually makes a more bell-shaped signal, the central limit theorem, so a blend looks more like a bell than either source. Turn the whitened cloud until the output looks least like a bell, and that output is one source. It works only if at most one source is a bell (the tutorial, section 3.3).
For a signal y scaled to spread 1, kurtosis is the average of y4 minus 3. The FastICA steps of Section 2.5 use it, and the purple ring of Figure 7 is the size of the kurtosis in each direction.
Both signals are 2 s synthetic stand-ins made in NumPy and embedded in the page, syllables with pauses between them and plucked notes, so every number from them is illustrative. Recorded speech is spiky in the same way: its samples fit a Laplace curve, the spiky shape, better than a bell (Aroudi et al. 2015).
| Signal | Average of y4 | Kurtosis |
|---|---|---|
| Bell | 3 | 0 |
| On/off voice s1 | 1 | −2 |
| Toy mic 1 | 1.852 | −1.148 |
| Laplace curve, the fit to recorded speech | 6 | 3 |
| Speech-like, illustrative | 11.03 | 8.03 |
| Music-like, illustrative | 8.51 | 5.51 |
| Mic 1 blend, illustrative | 5.31 | 2.31 |
Average of y4 and kurtosis of seven signals, each scaled to spread 1.
The toy is flat, the stand-ins are spiky, and blends sit closer to 0.
Each direction in the whitened plane is an angle θ from the first PCA axis, and reading the cloud along it gives one output signal.
The on/off voices are flat, so here least bell-like means most negative; the spiky voices of Figure 7 peak instead.
Kurtosis is “very sensitive to outliers” (the tutorial, section 4.2.1), so FastICA's default uses log cosh, the logarithm of the hyperbolic cosine, a U-shaped curve; it is a smooth stand-in for y4 that grows slowly for large values, so one loud outlier cannot decide the answer.
| Signal | Average of log cosh | Gap from the bell |
|---|---|---|
| Bell | 0.375 | 0 |
| Toy along the PCA axis, 0° | 0.390 | 0.015 |
| Toy along a source, 41.4° | 0.434 | 0.059 |
Average of log cosh for a bell and for the toy at two angles, spread 1.
Scanning every angle works with two mics; Section 2.5 finds the best angle in a few steps with any number of mics.
FastICA climbs J of Section 2.4 using g, the slope (derivative) of G. The slope of log cosh is tanh, the hyperbolic tangent, an S-curve from −1 to 1, and the slope of a quarter of y4 is y3, the kurtosis form of the first-step table and Figure 6. As w turns, the average of G(y) changes at the rate given by the average of z̃ g(y), the first term of the update; at the peak that average points straight along w. FastICA moves w to that average, minus the part a bell would give, and reaches a source in a few steps.
| Step | Result |
|---|---|
| Read along w = (1, 0), y = z̃1 | 1.411, −0.088, 0.088, −1.411 |
| Cube each value | 2.812, −0.001, 0.001, −2.812 |
| Average of z̃ y3 | 1.985, −0.123 |
| Subtract 3w | −1.015, −0.123 |
| Rescale to length 1 (length 1.023) | a line at 6.9 degrees |
The first FastICA step in the kurtosis form, from w = (1, 0).
Only the line matters, because w and −w give the same source up to sign (the tutorial, section 6.1).
Figure 6. Each step turns w toward a dashed source line; bars show y.
ICA undoes an instant blend across several sensors; Section 2.6 says what that assumes.
Basic ICA needs at least as many mics as sources (the tutorial, section 2.1). Extra mics are fine, since whitening drops the extra dimensions (the tutorial, sections 4.5 and 5.2); with fewer mics than sources the outputs stay blends.
ICA cannot recover the order, sign or scale of the sources (the tutorial, section 2.2). As in Section 2.3, each output is fixed to variance 1, and the order and sign can change between runs.
The model also assumes an instant blend with no time delays (the tutorial, section 1). A real room breaks this: a voice reaches each mic first by the direct path, then as quieter echoes off the walls a little later, so each mic hears a blurred sum of the voice over recent time, the voice convolved with the room’s impulse response. One number per arrow of Figure 2 cannot hold that, so x = As breaks, and since the echoes boost some frequencies and cut others, the blend differs at each frequency.
The standard fix is frequency-domain ICA (Smaragdis 1998): take the STFT of each mic and run ICA separately in each frequency bin, where the blend is again one matrix, of complex numbers. Each bin returns its voices in its own order, so the last step puts every bin in the same order, the permutation problem. For one recording in a room, NMF can model the reverberation itself. Anushiravani (2016, section 5.11) uses convolutive NMF, whose parts each span several time frames, on spectrograms to split a reverberant recording into the dry sound, the voice with no room echo, and a reverberation kernel, the room’s echo pattern over time.
Whitening leaves one rotation, and the angle that makes the outputs least like a bell separates the voices.
Figure 7. Illustrative stand-ins mixed by A; purple dots mark sources.
Take any two mono WAV recordings of your own at one sample rate, for example a voice and a song, and mix them by the A of Section 2.1; X has one row per sample and one column per mic, the shape scikit-learn expects.
import numpy as np
import scipy.io.wavfile as wf
import sklearn.decomposition as sd
_, s1 = wf.read("voice.wav")
_, s2 = wf.read("song.wav")
n = min(len(s1), len(s2))
S = np.c_[s1[:n], s2[:n]]
A = np.array([[2, 3], [2, 1]])
X = S @ A.T
ica = sd.FastICA(2)
S_hat = ica.fit_transform(X)
A_hat = ica.mixing_n_components is the one positional parameter.A_hat estimates A up to column order, sign and scale (Section 2.6). With no seed the order and sign can change from run to run.FastICA centers and whitens for you. A good fit gives outputs that look far from a bell, and a convergence warning means max_iter ran out.
| Setting | Raise it | Lower it | Extremes | Default |
|---|---|---|---|---|
n_components | more outputs, up to the number of mics in the basic model | keeps only the strongest whitened directions, which drops noise as PCA does | 1 keeps only the strongest whitened direction, one output | None, all features |
fun | A choice, not a scale. "logcosh" is the general choice, "exp" suits spiky sources, and "cube" is kurtosis. | "cube" is outlier sensitive | "logcosh" | |
max_iter | more iterations before it stops | may stop before w settles, with a convergence warning | max_iter = 1 takes one update from its start, like step 1 of Figure 6; scikit-learn starts at random, the figure at w = (1, 0) | 200 |
tol | stops sooner with a looser w | tighter w, but more iterations, and it may hit max_iter | tol = 0 always runs to max_iter | 1e-4 |
algorithm | A choice, not a scale. "parallel" finds every w at once, and "deflation" finds them one after another. | a choice; no extreme | "parallel" | |
FastICA settings in scikit-learn 1.9.1, what each one does, and its default.
| Cause | Symptom | Fix |
|---|---|---|
| Runs disagree | outputs come back in a different order, sign or scale | the model cannot know them; match outputs to sources by correlation, fix signs by a convention, or set random_state |
| Gaussian sources | no direction is least bell-like, and outputs stay blends | ICA cannot help, since at most one source may be Gaussian |
| More sources than mics | outputs carry blends | the basic model needs at least as many mics as sources; add mics or use a method built for that case |
| Reverberant room | the recording has delays and echoes | the blend is not one matrix; run frequency-domain ICA, one ICA per STFT bin, then put the bins in one order (Section 2.6) |
Four ways ICA goes wrong on a recording, what you see, and the fix.
When the data is one recording and everything is non-negative, NMF, on its own page next in Classical ML, asks a different question.