![Page 1: Fast and incoherent dictionary learning algorithms with application to fMRI Authors: Vahid Abolghasemi Saideh Ferdowsi Saeid Sanei. Journal of Signal Processing](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649eb75503460f94bc130f/html5/thumbnails/1.jpg)
Fast and incoherent dictionary learning algorithms with application
to fMRI
Authors: Vahid Abolghasemi Saideh Ferdowsi Saeid Sanei.Journal of Signal Processing Systems
----------2015/04/11
![Page 2: Fast and incoherent dictionary learning algorithms with application to fMRI Authors: Vahid Abolghasemi Saideh Ferdowsi Saeid Sanei. Journal of Signal Processing](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649eb75503460f94bc130f/html5/thumbnails/2.jpg)
Content
1. K-SVD
2. Incoherent K-SVD (IK-SVD)
3. Fast incoherent dictionary learning (FIDL)
4. Results
![Page 3: Fast and incoherent dictionary learning algorithms with application to fMRI Authors: Vahid Abolghasemi Saideh Ferdowsi Saeid Sanei. Journal of Signal Processing](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649eb75503460f94bc130f/html5/thumbnails/3.jpg)
K-SVD
Assume that the signal can be represented as a linear combination of a few atoms in dictionary such that .
The DL problem can be expressed as:
ny R 1
K
i id
n KD R y Dx
![Page 4: Fast and incoherent dictionary learning algorithms with application to fMRI Authors: Vahid Abolghasemi Saideh Ferdowsi Saeid Sanei. Journal of Signal Processing](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649eb75503460f94bc130f/html5/thumbnails/4.jpg)
K-SVD
a) Sparse coding (Keep D fixed, update X)• assume that τ is known and apply OMP to solve:
b) Updating D
![Page 5: Fast and incoherent dictionary learning algorithms with application to fMRI Authors: Vahid Abolghasemi Saideh Ferdowsi Saeid Sanei. Journal of Signal Processing](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649eb75503460f94bc130f/html5/thumbnails/5.jpg)
K-SVD
• In order to minimize (3), apply SVD to and simultaneously update and using the strongest eigenvector and eigenvalue of
TkE U V
![Page 6: Fast and incoherent dictionary learning algorithms with application to fMRI Authors: Vahid Abolghasemi Saideh Ferdowsi Saeid Sanei. Journal of Signal Processing](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649eb75503460f94bc130f/html5/thumbnails/6.jpg)
IK-SVD
High incoherence between the dictionary atoms is desired in almost all dictionary learning methods. This guarantees that the atoms are as discriminative as possible.
A remedy—a suitable tool for evaluating the coherence between the atoms is Gram matrix:
![Page 7: Fast and incoherent dictionary learning algorithms with application to fMRI Authors: Vahid Abolghasemi Saideh Ferdowsi Saeid Sanei. Journal of Signal Processing](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649eb75503460f94bc130f/html5/thumbnails/7.jpg)
IK-SVD
• Matrix G is K×K and symmetric with unit diagonal elements (note that D is column-normalized).
• The absolute values of off-diagonal elements of G represent the degree of coherence between any pair of atoms in D and therefore are desired to be very small.
![Page 8: Fast and incoherent dictionary learning algorithms with application to fMRI Authors: Vahid Abolghasemi Saideh Ferdowsi Saeid Sanei. Journal of Signal Processing](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649eb75503460f94bc130f/html5/thumbnails/8.jpg)
IK-SVD
• In order to minimize the above problem, we first take the gradient of which is computed:
![Page 9: Fast and incoherent dictionary learning algorithms with application to fMRI Authors: Vahid Abolghasemi Saideh Ferdowsi Saeid Sanei. Journal of Signal Processing](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649eb75503460f94bc130f/html5/thumbnails/9.jpg)
IK-SVD
• where γ = 4ξ > 0 is the step size controlling the convergence behavior of the algorithm, and k is the iteration counter of the incoherence constraint stage.
![Page 10: Fast and incoherent dictionary learning algorithms with application to fMRI Authors: Vahid Abolghasemi Saideh Ferdowsi Saeid Sanei. Journal of Signal Processing](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649eb75503460f94bc130f/html5/thumbnails/10.jpg)
IK-SVD
![Page 11: Fast and incoherent dictionary learning algorithms with application to fMRI Authors: Vahid Abolghasemi Saideh Ferdowsi Saeid Sanei. Journal of Signal Processing](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649eb75503460f94bc130f/html5/thumbnails/11.jpg)
FIDL
Because of computationally expensive for learning large dictionaries.
In order to design to be fast and at the same time exploits the incoherence of atoms.
![Page 12: Fast and incoherent dictionary learning algorithms with application to fMRI Authors: Vahid Abolghasemi Saideh Ferdowsi Saeid Sanei. Journal of Signal Processing](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649eb75503460f94bc130f/html5/thumbnails/12.jpg)
FIDL
Use - norm of the entire matrix X defined as
instead of forcing individual vectors to be sparse. This allows us to update the coefficients, simultaneously rather than column-by-column.
The incoherence constraint on D is also added to the cost function.
![Page 13: Fast and incoherent dictionary learning algorithms with application to fMRI Authors: Vahid Abolghasemi Saideh Ferdowsi Saeid Sanei. Journal of Signal Processing](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649eb75503460f94bc130f/html5/thumbnails/13.jpg)
FIDL
a) Coefficient update (sparse coding)• split into the sum of a smooth and a nonsmooth
sub-cost function, represented by P and Q.Gradient descent step:
Proximal step:
![Page 14: Fast and incoherent dictionary learning algorithms with application to fMRI Authors: Vahid Abolghasemi Saideh Ferdowsi Saeid Sanei. Journal of Signal Processing](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649eb75503460f94bc130f/html5/thumbnails/14.jpg)
FIDL
The proximal function here is defined using soft-thresholding (Shrink{.}), which ultimately leads to:
b) Dictionary update
![Page 15: Fast and incoherent dictionary learning algorithms with application to fMRI Authors: Vahid Abolghasemi Saideh Ferdowsi Saeid Sanei. Journal of Signal Processing](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649eb75503460f94bc130f/html5/thumbnails/15.jpg)
FIDL
![Page 16: Fast and incoherent dictionary learning algorithms with application to fMRI Authors: Vahid Abolghasemi Saideh Ferdowsi Saeid Sanei. Journal of Signal Processing](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649eb75503460f94bc130f/html5/thumbnails/16.jpg)
Results-Experiment 1
• The nonzero entries of the 20×1,000 sparse matrix X were generated randomly (from Gaussian distribution). D was selected as a random overcomplete full-rank matrix of size 15×20 with all columns normalized to one.
![Page 17: Fast and incoherent dictionary learning algorithms with application to fMRI Authors: Vahid Abolghasemi Saideh Ferdowsi Saeid Sanei. Journal of Signal Processing](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649eb75503460f94bc130f/html5/thumbnails/17.jpg)
Results-Experiment 1
![Page 18: Fast and incoherent dictionary learning algorithms with application to fMRI Authors: Vahid Abolghasemi Saideh Ferdowsi Saeid Sanei. Journal of Signal Processing](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649eb75503460f94bc130f/html5/thumbnails/18.jpg)
Results-Experiment 2
• To learn an over-complete dictionary of size 64 × 256 over 14,000 noisy image patches of size 8 × 8 extracted from Barbara image.
![Page 19: Fast and incoherent dictionary learning algorithms with application to fMRI Authors: Vahid Abolghasemi Saideh Ferdowsi Saeid Sanei. Journal of Signal Processing](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649eb75503460f94bc130f/html5/thumbnails/19.jpg)
Results-Experiment 3
• The aim was to investigate the robustness of the proposed methods against variations in the input noise.
• We considered noisy model Y = DX + V, where V was Gaussian noise with zero mean. All matrices were drawn randomly (from Gaussian distribution) with n = 15, K = 20, and N = 1,000. The number of nonzeros at each column of X was set to five.
![Page 20: Fast and incoherent dictionary learning algorithms with application to fMRI Authors: Vahid Abolghasemi Saideh Ferdowsi Saeid Sanei. Journal of Signal Processing](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649eb75503460f94bc130f/html5/thumbnails/20.jpg)
Results-Experiment 3
![Page 21: Fast and incoherent dictionary learning algorithms with application to fMRI Authors: Vahid Abolghasemi Saideh Ferdowsi Saeid Sanei. Journal of Signal Processing](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649eb75503460f94bc130f/html5/thumbnails/21.jpg)
Results-Experiment 4
• To evaluate the computational cost of the proposed methods and comparing these methods with other well-known algorithms.
• The parameters for the algorithms were similar to the first experiment. However, we increased the dictionary size from 5×10 to 500×1,000 for a fixed level of sparsity τ = 2.
![Page 22: Fast and incoherent dictionary learning algorithms with application to fMRI Authors: Vahid Abolghasemi Saideh Ferdowsi Saeid Sanei. Journal of Signal Processing](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649eb75503460f94bc130f/html5/thumbnails/22.jpg)
Results-Experiment 4
![Page 23: Fast and incoherent dictionary learning algorithms with application to fMRI Authors: Vahid Abolghasemi Saideh Ferdowsi Saeid Sanei. Journal of Signal Processing](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649eb75503460f94bc130f/html5/thumbnails/23.jpg)
Results-Synthetic fMRI data
• The simulations started by forming X of size 5×3,600 using five vectorized source images of size 60 × 60 .
• The mixtures were generated by multiplying column-normalized D of size 100 × 5 by X.
![Page 24: Fast and incoherent dictionary learning algorithms with application to fMRI Authors: Vahid Abolghasemi Saideh Ferdowsi Saeid Sanei. Journal of Signal Processing](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649eb75503460f94bc130f/html5/thumbnails/24.jpg)
Results-Synthetic fMRI data
![Page 25: Fast and incoherent dictionary learning algorithms with application to fMRI Authors: Vahid Abolghasemi Saideh Ferdowsi Saeid Sanei. Journal of Signal Processing](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649eb75503460f94bc130f/html5/thumbnails/25.jpg)
Results-Synthetic fMRI data
![Page 26: Fast and incoherent dictionary learning algorithms with application to fMRI Authors: Vahid Abolghasemi Saideh Ferdowsi Saeid Sanei. Journal of Signal Processing](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649eb75503460f94bc130f/html5/thumbnails/26.jpg)
Results-Real fMRI data
• A real auditory fMRI dataset was considered for this experiment.
• Chose K = 35 sources
![Page 27: Fast and incoherent dictionary learning algorithms with application to fMRI Authors: Vahid Abolghasemi Saideh Ferdowsi Saeid Sanei. Journal of Signal Processing](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649eb75503460f94bc130f/html5/thumbnails/27.jpg)
Results-Real fMRI data
FIDL IK-SVD
![Page 28: Fast and incoherent dictionary learning algorithms with application to fMRI Authors: Vahid Abolghasemi Saideh Ferdowsi Saeid Sanei. Journal of Signal Processing](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649eb75503460f94bc130f/html5/thumbnails/28.jpg)
Results-Real fMRI data
K-SVD FastICA
![Page 29: Fast and incoherent dictionary learning algorithms with application to fMRI Authors: Vahid Abolghasemi Saideh Ferdowsi Saeid Sanei. Journal of Signal Processing](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649eb75503460f94bc130f/html5/thumbnails/29.jpg)
Results-Real fMRI data
Lp-norm-based method SPM
![Page 30: Fast and incoherent dictionary learning algorithms with application to fMRI Authors: Vahid Abolghasemi Saideh Ferdowsi Saeid Sanei. Journal of Signal Processing](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649eb75503460f94bc130f/html5/thumbnails/30.jpg)
Thank you!