processing big datausers.isr.tecnico.ulisboa.pt/~csoares/bigdatadocs/... · processing big data...

20
Processing Big Data Compressed sensing

Upload: others

Post on 14-Jul-2020

12 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Processing Big DataCompressed sensing

Page 2: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Dimensionality reduction• Linear

• Principal Component Analysis: SVD-based

• Compressed sensing

• Matrix sketching

• Non-linear

• Kernel PCA

• Isometric mapping

Page 3: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Compressed sensing

Consider a sparse vector

i.e., a d-dimensional vector with only m non-zero entries

d = 19, m = 4

f = (0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0)<latexit sha1_base64="CMPizmkPd2D6NHAe9rDfZHIzG5w=">AAACHHicbVBNSwMxEJ2tX7V+rXr0EixCBSm7UlAPQtGLxwquLbRLyabZNjT7QZIVytI/4sW/4sWDihcPgv/GbLsFbZ3JhMd7MyTzvJgzqSzr2ygsLa+srhXXSxubW9s75u7evYwSQahDIh6Jlocl5SykjmKK01YsKA48Tpve8DrTmw9USBaFd2oUUzfA/ZD5jGClqa5Z89ElqlgnaHrs7J6lrXOm/Oqwjrtm2apak0CLwM5BGfJodM3PTi8iSUBDRTiWsm1bsXJTLBQjnI5LnUTSGJMh7tO2hiEOqHTTyXZjdKSZHvIjoStUaML+nkhxIOUo8HRngNVAzmsZ+Z/WTpR/7qYsjBNFQzJ9yE84UhHKrEI9JihRfKQBJoLpvyIywAITpQ0taRPs+ZUXgXNavajat7Vy/Sp3owgHcAgVsOEM6nADDXCAwCM8wyu8GU/Gi/FufExbC0Y+sw9/wvj6AQV6l4c=</latexit><latexit sha1_base64="CMPizmkPd2D6NHAe9rDfZHIzG5w=">AAACHHicbVBNSwMxEJ2tX7V+rXr0EixCBSm7UlAPQtGLxwquLbRLyabZNjT7QZIVytI/4sW/4sWDihcPgv/GbLsFbZ3JhMd7MyTzvJgzqSzr2ygsLa+srhXXSxubW9s75u7evYwSQahDIh6Jlocl5SykjmKK01YsKA48Tpve8DrTmw9USBaFd2oUUzfA/ZD5jGClqa5Z89ElqlgnaHrs7J6lrXOm/Oqwjrtm2apak0CLwM5BGfJodM3PTi8iSUBDRTiWsm1bsXJTLBQjnI5LnUTSGJMh7tO2hiEOqHTTyXZjdKSZHvIjoStUaML+nkhxIOUo8HRngNVAzmsZ+Z/WTpR/7qYsjBNFQzJ9yE84UhHKrEI9JihRfKQBJoLpvyIywAITpQ0taRPs+ZUXgXNavajat7Vy/Sp3owgHcAgVsOEM6nADDXCAwCM8wyu8GU/Gi/FufExbC0Y+sw9/wvj6AQV6l4c=</latexit><latexit sha1_base64="CMPizmkPd2D6NHAe9rDfZHIzG5w=">AAACHHicbVBNSwMxEJ2tX7V+rXr0EixCBSm7UlAPQtGLxwquLbRLyabZNjT7QZIVytI/4sW/4sWDihcPgv/GbLsFbZ3JhMd7MyTzvJgzqSzr2ygsLa+srhXXSxubW9s75u7evYwSQahDIh6Jlocl5SykjmKK01YsKA48Tpve8DrTmw9USBaFd2oUUzfA/ZD5jGClqa5Z89ElqlgnaHrs7J6lrXOm/Oqwjrtm2apak0CLwM5BGfJodM3PTi8iSUBDRTiWsm1bsXJTLBQjnI5LnUTSGJMh7tO2hiEOqHTTyXZjdKSZHvIjoStUaML+nkhxIOUo8HRngNVAzmsZ+Z/WTpR/7qYsjBNFQzJ9yE84UhHKrEI9JihRfKQBJoLpvyIywAITpQ0taRPs+ZUXgXNavajat7Vy/Sp3owgHcAgVsOEM6nADDXCAwCM8wyu8GU/Gi/FufExbC0Y+sw9/wvj6AQV6l4c=</latexit><latexit sha1_base64="CMPizmkPd2D6NHAe9rDfZHIzG5w=">AAACHHicbVBNSwMxEJ2tX7V+rXr0EixCBSm7UlAPQtGLxwquLbRLyabZNjT7QZIVytI/4sW/4sWDihcPgv/GbLsFbZ3JhMd7MyTzvJgzqSzr2ygsLa+srhXXSxubW9s75u7evYwSQahDIh6Jlocl5SykjmKK01YsKA48Tpve8DrTmw9USBaFd2oUUzfA/ZD5jGClqa5Z89ElqlgnaHrs7J6lrXOm/Oqwjrtm2apak0CLwM5BGfJodM3PTi8iSUBDRTiWsm1bsXJTLBQjnI5LnUTSGJMh7tO2hiEOqHTTyXZjdKSZHvIjoStUaML+nkhxIOUo8HRngNVAzmsZ+Z/WTpR/7qYsjBNFQzJ9yE84UhHKrEI9JihRfKQBJoLpvyIywAITpQ0taRPs+ZUXgXNavajat7Vy/Sp3owgHcAgVsOEM6nADDXCAwCM8wyu8GU/Gi/FufExbC0Y+sw9/wvj6AQV6l4c=</latexit>

Page 4: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Compressed sensing

Let a measurement be a random vector xi 2 {�1, 0, 1}d<latexit sha1_base64="7jApKt3CNHGN4HereefsAGduPAA=">AAAB/HicbVBNS8NAEJ34WetX/Lh5WSyCh1oSEdRb0YvHCsYWmlg22227dLMJuxuxhuJf8eJBxas/xJv/xm2bg7Y+GHi8N8PMvDDhTGnH+bbm5hcWl5YLK8XVtfWNTXtr+1bFqSTUIzGPZSPEinImqKeZ5rSRSIqjkNN62L8c+fV7KhWLxY0eJDSIcFewDiNYG6ll7z60GPKZQH525JadsusP79otu+RUnDHQLHFzUoIctZb95bdjkkZUaMKxUk3XSXSQYakZ4XRY9FNFE0z6uEubhgocURVk4+uH6MAobdSJpSmh0Vj9PZHhSKlBFJrOCOuemvZG4n9eM9WdsyBjIkk1FWSyqJNypGM0igK1maRE84EhmEhmbkWkhyUm2gRWNCG40y/PEu+4cl5xr09K1Ys8jQLswT4cggunUIUrqIEHBB7hGV7hzXqyXqx362PSOmflMzvwB9bnD4AFk4M=</latexit><latexit sha1_base64="7jApKt3CNHGN4HereefsAGduPAA=">AAAB/HicbVBNS8NAEJ34WetX/Lh5WSyCh1oSEdRb0YvHCsYWmlg22227dLMJuxuxhuJf8eJBxas/xJv/xm2bg7Y+GHi8N8PMvDDhTGnH+bbm5hcWl5YLK8XVtfWNTXtr+1bFqSTUIzGPZSPEinImqKeZ5rSRSIqjkNN62L8c+fV7KhWLxY0eJDSIcFewDiNYG6ll7z60GPKZQH525JadsusP79otu+RUnDHQLHFzUoIctZb95bdjkkZUaMKxUk3XSXSQYakZ4XRY9FNFE0z6uEubhgocURVk4+uH6MAobdSJpSmh0Vj9PZHhSKlBFJrOCOuemvZG4n9eM9WdsyBjIkk1FWSyqJNypGM0igK1maRE84EhmEhmbkWkhyUm2gRWNCG40y/PEu+4cl5xr09K1Ys8jQLswT4cggunUIUrqIEHBB7hGV7hzXqyXqx362PSOmflMzvwB9bnD4AFk4M=</latexit><latexit sha1_base64="7jApKt3CNHGN4HereefsAGduPAA=">AAAB/HicbVBNS8NAEJ34WetX/Lh5WSyCh1oSEdRb0YvHCsYWmlg22227dLMJuxuxhuJf8eJBxas/xJv/xm2bg7Y+GHi8N8PMvDDhTGnH+bbm5hcWl5YLK8XVtfWNTXtr+1bFqSTUIzGPZSPEinImqKeZ5rSRSIqjkNN62L8c+fV7KhWLxY0eJDSIcFewDiNYG6ll7z60GPKZQH525JadsusP79otu+RUnDHQLHFzUoIctZb95bdjkkZUaMKxUk3XSXSQYakZ4XRY9FNFE0z6uEubhgocURVk4+uH6MAobdSJpSmh0Vj9PZHhSKlBFJrOCOuemvZG4n9eM9WdsyBjIkk1FWSyqJNypGM0igK1maRE84EhmEhmbkWkhyUm2gRWNCG40y/PEu+4cl5xr09K1Ys8jQLswT4cggunUIUrqIEHBB7hGV7hzXqyXqx362PSOmflMzvwB9bnD4AFk4M=</latexit><latexit sha1_base64="7jApKt3CNHGN4HereefsAGduPAA=">AAAB/HicbVBNS8NAEJ34WetX/Lh5WSyCh1oSEdRb0YvHCsYWmlg22227dLMJuxuxhuJf8eJBxas/xJv/xm2bg7Y+GHi8N8PMvDDhTGnH+bbm5hcWl5YLK8XVtfWNTXtr+1bFqSTUIzGPZSPEinImqKeZ5rSRSIqjkNN62L8c+fV7KhWLxY0eJDSIcFewDiNYG6ll7z60GPKZQH525JadsusP79otu+RUnDHQLHFzUoIctZb95bdjkkZUaMKxUk3XSXSQYakZ4XRY9FNFE0z6uEubhgocURVk4+uH6MAobdSJpSmh0Vj9PZHhSKlBFJrOCOuemvZG4n9eM9WdsyBjIkk1FWSyqJNypGM0igK1maRE84EhmEhmbkWkhyUm2gRWNCG40y/PEu+4cl5xr09K1Ys8jQLswT4cggunUIUrqIEHBB7hGV7hzXqyXqx362PSOmflMzvwB9bnD4AFk4M=</latexit>

xi = (0,�1, 0, 1, 0, 0, 1, 0, 0,�1, 1, 0, 0, 0, 1, 0, 0, 1, 0)<latexit sha1_base64="Q8XureH8/qzqRBy/lYGDSTAV2Xo=">AAACIHicbVDLSgMxFM34rPU16tJNsAgVxpIRwboQim5cVnBsoR2GTJq2oZkHSUYsQ3/Fjb/ixoWK7vRrzExnoa2X5N7DOfeS3OPHnEmF0JexsLi0vLJaWiuvb2xubZs7u3cySgShDol4JNo+lpSzkDqKKU7bsaA48Dlt+aOrTG/dUyFZFN6qcUzdAA9C1mcEK015Zv3BY/ACVpEFj20L6pJnZNlWljWXAZgfu6i6BR15ZgXVUB5wHtgFqIAimp752e1FJAloqAjHUnZsFCs3xUIxwumk3E0kjTEZ4QHtaBjigEo3zTecwEPN9GA/EvqGCubs74kUB1KOA193BlgN5ayWkf9pnUT1627KwjhRNCTTh/oJhyqCmV2wxwQlio81wEQw/VdIhlhgorSpZW2CPbvyPHBOauc1++a00rgs3CiBfXAAqsAGZ6ABrkETOICAR/AMXsGb8WS8GO/Gx7R1wShm9sCfML5/ALLOmOY=</latexit><latexit sha1_base64="Q8XureH8/qzqRBy/lYGDSTAV2Xo=">AAACIHicbVDLSgMxFM34rPU16tJNsAgVxpIRwboQim5cVnBsoR2GTJq2oZkHSUYsQ3/Fjb/ixoWK7vRrzExnoa2X5N7DOfeS3OPHnEmF0JexsLi0vLJaWiuvb2xubZs7u3cySgShDol4JNo+lpSzkDqKKU7bsaA48Dlt+aOrTG/dUyFZFN6qcUzdAA9C1mcEK015Zv3BY/ACVpEFj20L6pJnZNlWljWXAZgfu6i6BR15ZgXVUB5wHtgFqIAimp752e1FJAloqAjHUnZsFCs3xUIxwumk3E0kjTEZ4QHtaBjigEo3zTecwEPN9GA/EvqGCubs74kUB1KOA193BlgN5ayWkf9pnUT1627KwjhRNCTTh/oJhyqCmV2wxwQlio81wEQw/VdIhlhgorSpZW2CPbvyPHBOauc1++a00rgs3CiBfXAAqsAGZ6ABrkETOICAR/AMXsGb8WS8GO/Gx7R1wShm9sCfML5/ALLOmOY=</latexit><latexit sha1_base64="Q8XureH8/qzqRBy/lYGDSTAV2Xo=">AAACIHicbVDLSgMxFM34rPU16tJNsAgVxpIRwboQim5cVnBsoR2GTJq2oZkHSUYsQ3/Fjb/ixoWK7vRrzExnoa2X5N7DOfeS3OPHnEmF0JexsLi0vLJaWiuvb2xubZs7u3cySgShDol4JNo+lpSzkDqKKU7bsaA48Dlt+aOrTG/dUyFZFN6qcUzdAA9C1mcEK015Zv3BY/ACVpEFj20L6pJnZNlWljWXAZgfu6i6BR15ZgXVUB5wHtgFqIAimp752e1FJAloqAjHUnZsFCs3xUIxwumk3E0kjTEZ4QHtaBjigEo3zTecwEPN9GA/EvqGCubs74kUB1KOA193BlgN5ayWkf9pnUT1627KwjhRNCTTh/oJhyqCmV2wxwQlio81wEQw/VdIhlhgorSpZW2CPbvyPHBOauc1++a00rgs3CiBfXAAqsAGZ6ABrkETOICAR/AMXsGb8WS8GO/Gx7R1wShm9sCfML5/ALLOmOY=</latexit><latexit sha1_base64="Q8XureH8/qzqRBy/lYGDSTAV2Xo=">AAACIHicbVDLSgMxFM34rPU16tJNsAgVxpIRwboQim5cVnBsoR2GTJq2oZkHSUYsQ3/Fjb/ixoWK7vRrzExnoa2X5N7DOfeS3OPHnEmF0JexsLi0vLJaWiuvb2xubZs7u3cySgShDol4JNo+lpSzkDqKKU7bsaA48Dlt+aOrTG/dUyFZFN6qcUzdAA9C1mcEK015Zv3BY/ACVpEFj20L6pJnZNlWljWXAZgfu6i6BR15ZgXVUB5wHtgFqIAimp752e1FJAloqAjHUnZsFCs3xUIxwumk3E0kjTEZ4QHtaBjigEo3zTecwEPN9GA/EvqGCubs74kUB1KOA193BlgN5ayWkf9pnUT1627KwjhRNCTTh/oJhyqCmV2wxwQlio81wEQw/VdIhlhgorSpZW2CPbvyPHBOauc1++a00rgs3CiBfXAAqsAGZ6ABrkETOICAR/AMXsGb8WS8GO/Gx7R1wShm9sCfML5/ALLOmOY=</latexit>

Taking a measurement is

f = (0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0)<latexit sha1_base64="CMPizmkPd2D6NHAe9rDfZHIzG5w=">AAACHHicbVBNSwMxEJ2tX7V+rXr0EixCBSm7UlAPQtGLxwquLbRLyabZNjT7QZIVytI/4sW/4sWDihcPgv/GbLsFbZ3JhMd7MyTzvJgzqSzr2ygsLa+srhXXSxubW9s75u7evYwSQahDIh6Jlocl5SykjmKK01YsKA48Tpve8DrTmw9USBaFd2oUUzfA/ZD5jGClqa5Z89ElqlgnaHrs7J6lrXOm/Oqwjrtm2apak0CLwM5BGfJodM3PTi8iSUBDRTiWsm1bsXJTLBQjnI5LnUTSGJMh7tO2hiEOqHTTyXZjdKSZHvIjoStUaML+nkhxIOUo8HRngNVAzmsZ+Z/WTpR/7qYsjBNFQzJ9yE84UhHKrEI9JihRfKQBJoLpvyIywAITpQ0taRPs+ZUXgXNavajat7Vy/Sp3owgHcAgVsOEM6nADDXCAwCM8wyu8GU/Gi/FufExbC0Y+sw9/wvj6AQV6l4c=</latexit><latexit sha1_base64="CMPizmkPd2D6NHAe9rDfZHIzG5w=">AAACHHicbVBNSwMxEJ2tX7V+rXr0EixCBSm7UlAPQtGLxwquLbRLyabZNjT7QZIVytI/4sW/4sWDihcPgv/GbLsFbZ3JhMd7MyTzvJgzqSzr2ygsLa+srhXXSxubW9s75u7evYwSQahDIh6Jlocl5SykjmKK01YsKA48Tpve8DrTmw9USBaFd2oUUzfA/ZD5jGClqa5Z89ElqlgnaHrs7J6lrXOm/Oqwjrtm2apak0CLwM5BGfJodM3PTi8iSUBDRTiWsm1bsXJTLBQjnI5LnUTSGJMh7tO2hiEOqHTTyXZjdKSZHvIjoStUaML+nkhxIOUo8HRngNVAzmsZ+Z/WTpR/7qYsjBNFQzJ9yE84UhHKrEI9JihRfKQBJoLpvyIywAITpQ0taRPs+ZUXgXNavajat7Vy/Sp3owgHcAgVsOEM6nADDXCAwCM8wyu8GU/Gi/FufExbC0Y+sw9/wvj6AQV6l4c=</latexit><latexit sha1_base64="CMPizmkPd2D6NHAe9rDfZHIzG5w=">AAACHHicbVBNSwMxEJ2tX7V+rXr0EixCBSm7UlAPQtGLxwquLbRLyabZNjT7QZIVytI/4sW/4sWDihcPgv/GbLsFbZ3JhMd7MyTzvJgzqSzr2ygsLa+srhXXSxubW9s75u7evYwSQahDIh6Jlocl5SykjmKK01YsKA48Tpve8DrTmw9USBaFd2oUUzfA/ZD5jGClqa5Z89ElqlgnaHrs7J6lrXOm/Oqwjrtm2apak0CLwM5BGfJodM3PTi8iSUBDRTiWsm1bsXJTLBQjnI5LnUTSGJMh7tO2hiEOqHTTyXZjdKSZHvIjoStUaML+nkhxIOUo8HRngNVAzmsZ+Z/WTpR/7qYsjBNFQzJ9yE84UhHKrEI9JihRfKQBJoLpvyIywAITpQ0taRPs+ZUXgXNavajat7Vy/Sp3owgHcAgVsOEM6nADDXCAwCM8wyu8GU/Gi/FufExbC0Y+sw9/wvj6AQV6l4c=</latexit><latexit sha1_base64="CMPizmkPd2D6NHAe9rDfZHIzG5w=">AAACHHicbVBNSwMxEJ2tX7V+rXr0EixCBSm7UlAPQtGLxwquLbRLyabZNjT7QZIVytI/4sW/4sWDihcPgv/GbLsFbZ3JhMd7MyTzvJgzqSzr2ygsLa+srhXXSxubW9s75u7evYwSQahDIh6Jlocl5SykjmKK01YsKA48Tpve8DrTmw9USBaFd2oUUzfA/ZD5jGClqa5Z89ElqlgnaHrs7J6lrXOm/Oqwjrtm2apak0CLwM5BGfJodM3PTi8iSUBDRTiWsm1bsXJTLBQjnI5LnUTSGJMh7tO2hiEOqHTTyXZjdKSZHvIjoStUaML+nkhxIOUo8HRngNVAzmsZ+Z/WTpR/7qYsjBNFQzJ9yE84UhHKrEI9JihRfKQBJoLpvyIywAITpQ0taRPs+ZUXgXNavajat7Vy/Sp3owgHcAgVsOEM6nADDXCAwCM8wyu8GU/Gi/FufExbC0Y+sw9/wvj6AQV6l4c=</latexit>

hf, xii = 2<latexit sha1_base64="TRsC3i0iYgFrQJQAhSKDgpd3DFo=">AAACAnicbVBNS8NAEJ3Ur1q/ot70slgED1KSIqgHoejFYwVjC00pm+2mXbrZhN2NWELBi3/FiwcVr/4Kb/4bt20O2vpg4O17M+zMCxLOlHacb6uwsLi0vFJcLa2tb2xu2ds7dypOJaEeiXksmwFWlDNBPc00p81EUhwFnDaCwdXYb9xTqVgsbvUwoe0I9wQLGcHaSB17z+dY9DhF4TF66DDky+nzAlU7dtmpOBOgeeLmpAw56h37y+/GJI2o0IRjpVquk+h2hqVmhNNRyU8VTTAZ4B5tGSpwRFU7m9wwQodG6aIwlqaERhP190SGI6WGUWA6I6z7atYbi/95rVSHZ+2MiSTVVJDpR2HKkY7ROBDUZZISzYeGYCKZ2RWRPpaYaBNbyYTgzp48T7xq5bzi3pyUa5d5GkXYhwM4AhdOoQbXUAcPCDzCM7zCm/VkvVjv1se0tWDlM7vwB9bnDwhplgg=</latexit><latexit sha1_base64="TRsC3i0iYgFrQJQAhSKDgpd3DFo=">AAACAnicbVBNS8NAEJ3Ur1q/ot70slgED1KSIqgHoejFYwVjC00pm+2mXbrZhN2NWELBi3/FiwcVr/4Kb/4bt20O2vpg4O17M+zMCxLOlHacb6uwsLi0vFJcLa2tb2xu2ds7dypOJaEeiXksmwFWlDNBPc00p81EUhwFnDaCwdXYb9xTqVgsbvUwoe0I9wQLGcHaSB17z+dY9DhF4TF66DDky+nzAlU7dtmpOBOgeeLmpAw56h37y+/GJI2o0IRjpVquk+h2hqVmhNNRyU8VTTAZ4B5tGSpwRFU7m9wwQodG6aIwlqaERhP190SGI6WGUWA6I6z7atYbi/95rVSHZ+2MiSTVVJDpR2HKkY7ROBDUZZISzYeGYCKZ2RWRPpaYaBNbyYTgzp48T7xq5bzi3pyUa5d5GkXYhwM4AhdOoQbXUAcPCDzCM7zCm/VkvVjv1se0tWDlM7vwB9bnDwhplgg=</latexit><latexit sha1_base64="TRsC3i0iYgFrQJQAhSKDgpd3DFo=">AAACAnicbVBNS8NAEJ3Ur1q/ot70slgED1KSIqgHoejFYwVjC00pm+2mXbrZhN2NWELBi3/FiwcVr/4Kb/4bt20O2vpg4O17M+zMCxLOlHacb6uwsLi0vFJcLa2tb2xu2ds7dypOJaEeiXksmwFWlDNBPc00p81EUhwFnDaCwdXYb9xTqVgsbvUwoe0I9wQLGcHaSB17z+dY9DhF4TF66DDky+nzAlU7dtmpOBOgeeLmpAw56h37y+/GJI2o0IRjpVquk+h2hqVmhNNRyU8VTTAZ4B5tGSpwRFU7m9wwQodG6aIwlqaERhP190SGI6WGUWA6I6z7atYbi/95rVSHZ+2MiSTVVJDpR2HKkY7ROBDUZZISzYeGYCKZ2RWRPpaYaBNbyYTgzp48T7xq5bzi3pyUa5d5GkXYhwM4AhdOoQbXUAcPCDzCM7zCm/VkvVjv1se0tWDlM7vwB9bnDwhplgg=</latexit><latexit sha1_base64="TRsC3i0iYgFrQJQAhSKDgpd3DFo=">AAACAnicbVBNS8NAEJ3Ur1q/ot70slgED1KSIqgHoejFYwVjC00pm+2mXbrZhN2NWELBi3/FiwcVr/4Kb/4bt20O2vpg4O17M+zMCxLOlHacb6uwsLi0vFJcLa2tb2xu2ds7dypOJaEeiXksmwFWlDNBPc00p81EUhwFnDaCwdXYb9xTqVgsbvUwoe0I9wQLGcHaSB17z+dY9DhF4TF66DDky+nzAlU7dtmpOBOgeeLmpAw56h37y+/GJI2o0IRjpVquk+h2hqVmhNNRyU8VTTAZ4B5tGSpwRFU7m9wwQodG6aIwlqaERhP190SGI6WGUWA6I6z7atYbi/95rVSHZ+2MiSTVVJDpR2HKkY7ROBDUZZISzYeGYCKZ2RWRPpaYaBNbyYTgzp48T7xq5bzi3pyUa5d5GkXYhwM4AhdOoQbXUAcPCDzCM7zCm/VkvVjv1se0tWDlM7vwB9bnDwhplgg=</latexit>

X = [x1 x2 · · · xN ]<latexit sha1_base64="36l9paWuaKl0+bBV4lmCwShAMJY=">AAACCXicbVDLSsNAFJ3UV62vqEs3o0VwVZIiqIhQdONKKhhbSEKYTCbt0MmDmYm0hK7d+CtuXKi49Q/c+TdO2iy09cDlHs65l5l7/JRRIQ3jW6ssLC4tr1RXa2vrG5tb+vbOvUgyjomFE5bwro8EYTQmlqSSkW7KCYp8Rjr+4KrwOw+EC5rEd3KUEjdCvZiGFCOpJE/f78ILaA89EzrncOg1i+bgIJFiKty4nl43GsYEcJ6YJamDEm1P/3KCBGcRiSVmSAjbNFLp5ohLihkZ15xMkBThAeoRW9EYRUS4+eSUMTxUSgDDhKuKJZyovzdyFAkxinw1GSHZF7NeIf7n2ZkMT92cxmkmSYynD4UZgzKBRS4woJxgyUaKIMyp+ivEfcQRliq9mgrBnD15nljNxlnDvD2uty7LNKpgDxyAI2CCE9AC16ANLIDBI3gGr+BNe9JetHftYzpa0cqdXfAH2ucPDJiXig==</latexit><latexit sha1_base64="36l9paWuaKl0+bBV4lmCwShAMJY=">AAACCXicbVDLSsNAFJ3UV62vqEs3o0VwVZIiqIhQdONKKhhbSEKYTCbt0MmDmYm0hK7d+CtuXKi49Q/c+TdO2iy09cDlHs65l5l7/JRRIQ3jW6ssLC4tr1RXa2vrG5tb+vbOvUgyjomFE5bwro8EYTQmlqSSkW7KCYp8Rjr+4KrwOw+EC5rEd3KUEjdCvZiGFCOpJE/f78ILaA89EzrncOg1i+bgIJFiKty4nl43GsYEcJ6YJamDEm1P/3KCBGcRiSVmSAjbNFLp5ohLihkZ15xMkBThAeoRW9EYRUS4+eSUMTxUSgDDhKuKJZyovzdyFAkxinw1GSHZF7NeIf7n2ZkMT92cxmkmSYynD4UZgzKBRS4woJxgyUaKIMyp+ivEfcQRliq9mgrBnD15nljNxlnDvD2uty7LNKpgDxyAI2CCE9AC16ANLIDBI3gGr+BNe9JetHftYzpa0cqdXfAH2ucPDJiXig==</latexit><latexit sha1_base64="36l9paWuaKl0+bBV4lmCwShAMJY=">AAACCXicbVDLSsNAFJ3UV62vqEs3o0VwVZIiqIhQdONKKhhbSEKYTCbt0MmDmYm0hK7d+CtuXKi49Q/c+TdO2iy09cDlHs65l5l7/JRRIQ3jW6ssLC4tr1RXa2vrG5tb+vbOvUgyjomFE5bwro8EYTQmlqSSkW7KCYp8Rjr+4KrwOw+EC5rEd3KUEjdCvZiGFCOpJE/f78ILaA89EzrncOg1i+bgIJFiKty4nl43GsYEcJ6YJamDEm1P/3KCBGcRiSVmSAjbNFLp5ohLihkZ15xMkBThAeoRW9EYRUS4+eSUMTxUSgDDhKuKJZyovzdyFAkxinw1GSHZF7NeIf7n2ZkMT92cxmkmSYynD4UZgzKBRS4woJxgyUaKIMyp+ivEfcQRliq9mgrBnD15nljNxlnDvD2uty7LNKpgDxyAI2CCE9AC16ANLIDBI3gGr+BNe9JetHftYzpa0cqdXfAH2ucPDJiXig==</latexit><latexit sha1_base64="36l9paWuaKl0+bBV4lmCwShAMJY=">AAACCXicbVDLSsNAFJ3UV62vqEs3o0VwVZIiqIhQdONKKhhbSEKYTCbt0MmDmYm0hK7d+CtuXKi49Q/c+TdO2iy09cDlHs65l5l7/JRRIQ3jW6ssLC4tr1RXa2vrG5tb+vbOvUgyjomFE5bwro8EYTQmlqSSkW7KCYp8Rjr+4KrwOw+EC5rEd3KUEjdCvZiGFCOpJE/f78ILaA89EzrncOg1i+bgIJFiKty4nl43GsYEcJ6YJamDEm1P/3KCBGcRiSVmSAjbNFLp5ohLihkZ15xMkBThAeoRW9EYRUS4+eSUMTxUSgDDhKuKJZyovzdyFAkxinw1GSHZF7NeIf7n2ZkMT92cxmkmSYynD4UZgzKBRS4woJxgyUaKIMyp+ivEfcQRliq9mgrBnD15nljNxlnDvD2uty7LNKpgDxyAI2CCE9AC16ANLIDBI3gGr+BNe9JetHftYzpa0cqdXfAH2ucPDJiXig==</latexit>

in general, we have a measurement matrix

Page 5: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Compressed sensing

X = [x1 x2 · · · xN ]<latexit sha1_base64="36l9paWuaKl0+bBV4lmCwShAMJY=">AAACCXicbVDLSsNAFJ3UV62vqEs3o0VwVZIiqIhQdONKKhhbSEKYTCbt0MmDmYm0hK7d+CtuXKi49Q/c+TdO2iy09cDlHs65l5l7/JRRIQ3jW6ssLC4tr1RXa2vrG5tb+vbOvUgyjomFE5bwro8EYTQmlqSSkW7KCYp8Rjr+4KrwOw+EC5rEd3KUEjdCvZiGFCOpJE/f78ILaA89EzrncOg1i+bgIJFiKty4nl43GsYEcJ6YJamDEm1P/3KCBGcRiSVmSAjbNFLp5ohLihkZ15xMkBThAeoRW9EYRUS4+eSUMTxUSgDDhKuKJZyovzdyFAkxinw1GSHZF7NeIf7n2ZkMT92cxmkmSYynD4UZgzKBRS4woJxgyUaKIMyp+ivEfcQRliq9mgrBnD15nljNxlnDvD2uty7LNKpgDxyAI2CCE9AC16ANLIDBI3gGr+BNe9JetHftYzpa0cqdXfAH2ucPDJiXig==</latexit><latexit sha1_base64="36l9paWuaKl0+bBV4lmCwShAMJY=">AAACCXicbVDLSsNAFJ3UV62vqEs3o0VwVZIiqIhQdONKKhhbSEKYTCbt0MmDmYm0hK7d+CtuXKi49Q/c+TdO2iy09cDlHs65l5l7/JRRIQ3jW6ssLC4tr1RXa2vrG5tb+vbOvUgyjomFE5bwro8EYTQmlqSSkW7KCYp8Rjr+4KrwOw+EC5rEd3KUEjdCvZiGFCOpJE/f78ILaA89EzrncOg1i+bgIJFiKty4nl43GsYEcJ6YJamDEm1P/3KCBGcRiSVmSAjbNFLp5ohLihkZ15xMkBThAeoRW9EYRUS4+eSUMTxUSgDDhKuKJZyovzdyFAkxinw1GSHZF7NeIf7n2ZkMT92cxmkmSYynD4UZgzKBRS4woJxgyUaKIMyp+ivEfcQRliq9mgrBnD15nljNxlnDvD2uty7LNKpgDxyAI2CCE9AC16ANLIDBI3gGr+BNe9JetHftYzpa0cqdXfAH2ucPDJiXig==</latexit><latexit sha1_base64="36l9paWuaKl0+bBV4lmCwShAMJY=">AAACCXicbVDLSsNAFJ3UV62vqEs3o0VwVZIiqIhQdONKKhhbSEKYTCbt0MmDmYm0hK7d+CtuXKi49Q/c+TdO2iy09cDlHs65l5l7/JRRIQ3jW6ssLC4tr1RXa2vrG5tb+vbOvUgyjomFE5bwro8EYTQmlqSSkW7KCYp8Rjr+4KrwOw+EC5rEd3KUEjdCvZiGFCOpJE/f78ILaA89EzrncOg1i+bgIJFiKty4nl43GsYEcJ6YJamDEm1P/3KCBGcRiSVmSAjbNFLp5ohLihkZ15xMkBThAeoRW9EYRUS4+eSUMTxUSgDDhKuKJZyovzdyFAkxinw1GSHZF7NeIf7n2ZkMT92cxmkmSYynD4UZgzKBRS4woJxgyUaKIMyp+ivEfcQRliq9mgrBnD15nljNxlnDvD2uty7LNKpgDxyAI2CCE9AC16ANLIDBI3gGr+BNe9JetHftYzpa0cqdXfAH2ucPDJiXig==</latexit><latexit sha1_base64="36l9paWuaKl0+bBV4lmCwShAMJY=">AAACCXicbVDLSsNAFJ3UV62vqEs3o0VwVZIiqIhQdONKKhhbSEKYTCbt0MmDmYm0hK7d+CtuXKi49Q/c+TdO2iy09cDlHs65l5l7/JRRIQ3jW6ssLC4tr1RXa2vrG5tb+vbOvUgyjomFE5bwro8EYTQmlqSSkW7KCYp8Rjr+4KrwOw+EC5rEd3KUEjdCvZiGFCOpJE/f78ILaA89EzrncOg1i+bgIJFiKty4nl43GsYEcJ6YJamDEm1P/3KCBGcRiSVmSAjbNFLp5ohLihkZ15xMkBThAeoRW9EYRUS4+eSUMTxUSgDDhKuKJZyovzdyFAkxinw1GSHZF7NeIf7n2ZkMT92cxmkmSYynD4UZgzKBRS4woJxgyUaKIMyp+ivEfcQRliq9mgrBnD15nljNxlnDvD2uty7LNKpgDxyAI2CCE9AC16ANLIDBI3gGr+BNe9JetHftYzpa0cqdXfAH2ucPDJiXig==</latexit>

in general, we have a measurement matrix

y = Xf<latexit sha1_base64="LHRD23RPIEfUyMZq5FCLL1bM3Ag=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCqYttKFstpt27WY37G6EEPofvHhQ8eoP8ua/cdvmoK0PBh7vzTAzL0w408Z1v53Syura+kZ5s7K1vbO7V90/aGmZKkJ9IrlUnRBrypmgvmGG006iKI5DTtvh+Hbqt5+o0kyKB5MlNIjxULCIEWys1MrQNepE/WrNrbszoGXiFaQGBZr96ldvIEkaU2EIx1p3PTcxQY6VYYTTSaWXappgMsZD2rVU4JjqIJ9dO0EnVhmgSCpbwqCZ+nsix7HWWRzazhibkV70puJ/Xjc10WWQM5GkhgoyXxSlHBmJpq+jAVOUGJ5Zgoli9lZERlhhYmxAFRuCt/jyMvHP6ld17/681rgp0ijDERzDKXhwAQ24gyb4QOARnuEV3hzpvDjvzse8teQUM4fwB87nD+8gjj4=</latexit><latexit sha1_base64="LHRD23RPIEfUyMZq5FCLL1bM3Ag=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCqYttKFstpt27WY37G6EEPofvHhQ8eoP8ua/cdvmoK0PBh7vzTAzL0w408Z1v53Syura+kZ5s7K1vbO7V90/aGmZKkJ9IrlUnRBrypmgvmGG006iKI5DTtvh+Hbqt5+o0kyKB5MlNIjxULCIEWys1MrQNepE/WrNrbszoGXiFaQGBZr96ldvIEkaU2EIx1p3PTcxQY6VYYTTSaWXappgMsZD2rVU4JjqIJ9dO0EnVhmgSCpbwqCZ+nsix7HWWRzazhibkV70puJ/Xjc10WWQM5GkhgoyXxSlHBmJpq+jAVOUGJ5Zgoli9lZERlhhYmxAFRuCt/jyMvHP6ld17/681rgp0ijDERzDKXhwAQ24gyb4QOARnuEV3hzpvDjvzse8teQUM4fwB87nD+8gjj4=</latexit><latexit sha1_base64="LHRD23RPIEfUyMZq5FCLL1bM3Ag=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCqYttKFstpt27WY37G6EEPofvHhQ8eoP8ua/cdvmoK0PBh7vzTAzL0w408Z1v53Syura+kZ5s7K1vbO7V90/aGmZKkJ9IrlUnRBrypmgvmGG006iKI5DTtvh+Hbqt5+o0kyKB5MlNIjxULCIEWys1MrQNepE/WrNrbszoGXiFaQGBZr96ldvIEkaU2EIx1p3PTcxQY6VYYTTSaWXappgMsZD2rVU4JjqIJ9dO0EnVhmgSCpbwqCZ+nsix7HWWRzazhibkV70puJ/Xjc10WWQM5GkhgoyXxSlHBmJpq+jAVOUGJ5Zgoli9lZERlhhYmxAFRuCt/jyMvHP6ld17/681rgp0ijDERzDKXhwAQ24gyb4QOARnuEV3hzpvDjvzse8teQUM4fwB87nD+8gjj4=</latexit><latexit sha1_base64="LHRD23RPIEfUyMZq5FCLL1bM3Ag=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCqYttKFstpt27WY37G6EEPofvHhQ8eoP8ua/cdvmoK0PBh7vzTAzL0w408Z1v53Syura+kZ5s7K1vbO7V90/aGmZKkJ9IrlUnRBrypmgvmGG006iKI5DTtvh+Hbqt5+o0kyKB5MlNIjxULCIEWys1MrQNepE/WrNrbszoGXiFaQGBZr96ldvIEkaU2EIx1p3PTcxQY6VYYTTSaWXappgMsZD2rVU4JjqIJ9dO0EnVhmgSCpbwqCZ+nsix7HWWRzazhibkV70puJ/Xjc10WWQM5GkhgoyXxSlHBmJpq+jAVOUGJ5Zgoli9lZERlhhYmxAFRuCt/jyMvHP6ld17/681rgp0ijDERzDKXhwAQ24gyb4QOARnuEV3hzpvDjvzse8teQUM4fwB87nD+8gjj4=</latexit>

and a measurement

N = Km log

✓d

m

<latexit sha1_base64="OEtNCMh+b8pW0pBIo2UDP5cqQQE=">AAACCXicbVBNS8NAEN34WetX1KOX1SLUS0lEUA9C0YsgSAVjC00om82mXbr5YHcilNCzF/+KFw8qXv0H3vw3btsctPXBwOO9GWbm+angCizr25ibX1hcWi6tlFfX1jc2za3te5VkkjKHJiKRLZ8oJnjMHOAgWCuVjES+YE2/fznymw9MKp7EdzBImReRbsxDTgloqWPu3eBzfB1hVyRdV7AQqm4oCcWBliTv9uCwY1asmjUGniV2QSqoQKNjfrlBQrOIxUAFUaptWyl4OZHAqWDDspsplhLaJ13W1jQmEVNePn5liA+0EuAwkbpiwGP190ROIqUGka87IwI9Ne2NxP+8dgbhqZfzOM2AxXSyKMwEhgSPcsEBl4yCGGhCqOT6Vkx7REcBOr2yDsGefnmWOEe1s5p9e1ypXxRplNAu2kdVZKMTVEdXqIEcRNEjekav6M14Ml6Md+Nj0jpnFDM76A+Mzx8vFpjl</latexit><latexit sha1_base64="OEtNCMh+b8pW0pBIo2UDP5cqQQE=">AAACCXicbVBNS8NAEN34WetX1KOX1SLUS0lEUA9C0YsgSAVjC00om82mXbr5YHcilNCzF/+KFw8qXv0H3vw3btsctPXBwOO9GWbm+angCizr25ibX1hcWi6tlFfX1jc2za3te5VkkjKHJiKRLZ8oJnjMHOAgWCuVjES+YE2/fznymw9MKp7EdzBImReRbsxDTgloqWPu3eBzfB1hVyRdV7AQqm4oCcWBliTv9uCwY1asmjUGniV2QSqoQKNjfrlBQrOIxUAFUaptWyl4OZHAqWDDspsplhLaJ13W1jQmEVNePn5liA+0EuAwkbpiwGP190ROIqUGka87IwI9Ne2NxP+8dgbhqZfzOM2AxXSyKMwEhgSPcsEBl4yCGGhCqOT6Vkx7REcBOr2yDsGefnmWOEe1s5p9e1ypXxRplNAu2kdVZKMTVEdXqIEcRNEjekav6M14Ml6Md+Nj0jpnFDM76A+Mzx8vFpjl</latexit><latexit sha1_base64="OEtNCMh+b8pW0pBIo2UDP5cqQQE=">AAACCXicbVBNS8NAEN34WetX1KOX1SLUS0lEUA9C0YsgSAVjC00om82mXbr5YHcilNCzF/+KFw8qXv0H3vw3btsctPXBwOO9GWbm+angCizr25ibX1hcWi6tlFfX1jc2za3te5VkkjKHJiKRLZ8oJnjMHOAgWCuVjES+YE2/fznymw9MKp7EdzBImReRbsxDTgloqWPu3eBzfB1hVyRdV7AQqm4oCcWBliTv9uCwY1asmjUGniV2QSqoQKNjfrlBQrOIxUAFUaptWyl4OZHAqWDDspsplhLaJ13W1jQmEVNePn5liA+0EuAwkbpiwGP190ROIqUGka87IwI9Ne2NxP+8dgbhqZfzOM2AxXSyKMwEhgSPcsEBl4yCGGhCqOT6Vkx7REcBOr2yDsGefnmWOEe1s5p9e1ypXxRplNAu2kdVZKMTVEdXqIEcRNEjekav6M14Ml6Md+Nj0jpnFDM76A+Mzx8vFpjl</latexit><latexit sha1_base64="OEtNCMh+b8pW0pBIo2UDP5cqQQE=">AAACCXicbVBNS8NAEN34WetX1KOX1SLUS0lEUA9C0YsgSAVjC00om82mXbr5YHcilNCzF/+KFw8qXv0H3vw3btsctPXBwOO9GWbm+angCizr25ibX1hcWi6tlFfX1jc2za3te5VkkjKHJiKRLZ8oJnjMHOAgWCuVjES+YE2/fznymw9MKp7EdzBImReRbsxDTgloqWPu3eBzfB1hVyRdV7AQqm4oCcWBliTv9uCwY1asmjUGniV2QSqoQKNjfrlBQrOIxUAFUaptWyl4OZHAqWDDspsplhLaJ13W1jQmEVNePn5liA+0EuAwkbpiwGP190ROIqUGka87IwI9Ne2NxP+8dgbhqZfzOM2AxXSyKMwEhgSPcsEBl4yCGGhCqOT6Vkx7REcBOr2yDsGefnmWOEe1s5p9e1ypXxRplNAu2kdVZKMTVEdXqIEcRNEjekav6M14Ml6Md+Nj0jpnFDM76A+Mzx8vFpjl</latexit>

Our goal is to make only N measurements

where K in some settings is 4 or max 20

Page 6: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Compressed sensingCompression: y takes N logN

<latexit sha1_base64="yBK2voB0QqZtCL5ROVK8BCSFm7w=">AAAB7nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lEUG9FL55KBWMLbSib7aZdutmNuxuhhP4JLx5UvPp7vPlv3LY5aOuDgcd7M8zMi1LOtPG8b6e0srq2vlHerGxt7+zuufsHD1pmitCASC5VO8KaciZoYJjhtJ0qipOI01Y0upn6rSeqNJPi3oxTGiZ4IFjMCDZWajdQl8sBavTcqlfzZkDLxC9IFQo0e+5Xty9JllBhCMdad3wvNWGOlWGE00mlm2maYjLCA9qxVOCE6jCf3TtBJ1bpo1gqW8Kgmfp7IseJ1uMksp0JNkO96E3F/7xOZuLLMGcizQwVZL4ozjgyEk2fR32mKDF8bAkmitlbERlihYmxEVVsCP7iy8skOKtd1fy782r9ukijDEdwDKfgwwXU4RaaEAABDs/wCm/Oo/PivDsf89aSU8wcwh84nz9o2Y8Y</latexit><latexit sha1_base64="yBK2voB0QqZtCL5ROVK8BCSFm7w=">AAAB7nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lEUG9FL55KBWMLbSib7aZdutmNuxuhhP4JLx5UvPp7vPlv3LY5aOuDgcd7M8zMi1LOtPG8b6e0srq2vlHerGxt7+zuufsHD1pmitCASC5VO8KaciZoYJjhtJ0qipOI01Y0upn6rSeqNJPi3oxTGiZ4IFjMCDZWajdQl8sBavTcqlfzZkDLxC9IFQo0e+5Xty9JllBhCMdad3wvNWGOlWGE00mlm2maYjLCA9qxVOCE6jCf3TtBJ1bpo1gqW8Kgmfp7IseJ1uMksp0JNkO96E3F/7xOZuLLMGcizQwVZL4ozjgyEk2fR32mKDF8bAkmitlbERlihYmxEVVsCP7iy8skOKtd1fy782r9ukijDEdwDKfgwwXU4RaaEAABDs/wCm/Oo/PivDsf89aSU8wcwh84nz9o2Y8Y</latexit><latexit sha1_base64="yBK2voB0QqZtCL5ROVK8BCSFm7w=">AAAB7nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lEUG9FL55KBWMLbSib7aZdutmNuxuhhP4JLx5UvPp7vPlv3LY5aOuDgcd7M8zMi1LOtPG8b6e0srq2vlHerGxt7+zuufsHD1pmitCASC5VO8KaciZoYJjhtJ0qipOI01Y0upn6rSeqNJPi3oxTGiZ4IFjMCDZWajdQl8sBavTcqlfzZkDLxC9IFQo0e+5Xty9JllBhCMdad3wvNWGOlWGE00mlm2maYjLCA9qxVOCE6jCf3TtBJ1bpo1gqW8Kgmfp7IseJ1uMksp0JNkO96E3F/7xOZuLLMGcizQwVZL4ozjgyEk2fR32mKDF8bAkmitlbERlihYmxEVVsCP7iy8skOKtd1fy782r9ukijDEdwDKfgwwXU4RaaEAABDs/wCm/Oo/PivDsf89aSU8wcwh84nz9o2Y8Y</latexit><latexit sha1_base64="yBK2voB0QqZtCL5ROVK8BCSFm7w=">AAAB7nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lEUG9FL55KBWMLbSib7aZdutmNuxuhhP4JLx5UvPp7vPlv3LY5aOuDgcd7M8zMi1LOtPG8b6e0srq2vlHerGxt7+zuufsHD1pmitCASC5VO8KaciZoYJjhtJ0qipOI01Y0upn6rSeqNJPi3oxTGiZ4IFjMCDZWajdQl8sBavTcqlfzZkDLxC9IFQo0e+5Xty9JllBhCMdad3wvNWGOlWGE00mlm2maYjLCA9qxVOCE6jCf3TtBJ1bpo1gqW8Kgmfp7IseJ1uMksp0JNkO96E3F/7xOZuLLMGcizQwVZL4ozjgyEk2fR32mKDF8bAkmitlbERlihYmxEVVsCP7iy8skOKtd1fy782r9ukijDEdwDKfgwwXU4RaaEAABDs/wCm/Oo/PivDsf89aSU8wcwh84nz9o2Y8Y</latexit>

Application: Hubble telescope

Has high-res camera in space, for less interference. But communication to Earth is expensive. With an initially random matrix X in space used as a mask, the images are compressed. Matrix X is known on Earth.

Data is often noisy, and have more attributes than the real underlying process. CS helps finding the true structure of the data.

Noisy Data

Page 7: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

How to recover the compressed signal?

One of the simplest ways: Orthogonal Matching Pursuit

Simple and greedy method

Greedy algorithm: makes the locally optimal choice at each stage, hoping to find a global solution

Example: reaching the path of largest sum

https://en.wikipedia.org/wiki/Greedy_algorithm

A greedy strategy is not guaranteed to reach the optimal solution.

Page 8: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Orthogonal Matching pursuit

Xj<latexit sha1_base64="6E9Oi48tZeFkKRsbB82UZln6sLc=">AAAB6XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWNFYwttKJvtpF272YTdjVBCf4IXDype/Ufe/Ddu2xy09cHA470ZZuaFqeDauO63s7S8srq2Xtoob25t7+xW9vYfdJIphj5LRKJaIdUouETfcCOwlSqkcSiwGQ6vJ37zCZXmibw3oxSDmPYljzijxkp3re5jt1J1a+4UZJF4BalCgUa38tXpJSyLURomqNZtz01NkFNlOBM4LncyjSllQ9rHtqWSxqiDfHrqmBxbpUeiRNmShkzV3xM5jbUexaHtjKkZ6HlvIv7ntTMTXQQ5l2lmULLZoigTxCRk8jfpcYXMiJEllClubyVsQBVlxqZTtiF48y8vEv+0dlnzbs+q9asijRIcwhGcgAfnUIcbaIAPDPrwDK/w5gjnxXl3PmatS04xcwB/4Hz+AJ8AjY0=</latexit><latexit sha1_base64="6E9Oi48tZeFkKRsbB82UZln6sLc=">AAAB6XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWNFYwttKJvtpF272YTdjVBCf4IXDype/Ufe/Ddu2xy09cHA470ZZuaFqeDauO63s7S8srq2Xtoob25t7+xW9vYfdJIphj5LRKJaIdUouETfcCOwlSqkcSiwGQ6vJ37zCZXmibw3oxSDmPYljzijxkp3re5jt1J1a+4UZJF4BalCgUa38tXpJSyLURomqNZtz01NkFNlOBM4LncyjSllQ9rHtqWSxqiDfHrqmBxbpUeiRNmShkzV3xM5jbUexaHtjKkZ6HlvIv7ntTMTXQQ5l2lmULLZoigTxCRk8jfpcYXMiJEllClubyVsQBVlxqZTtiF48y8vEv+0dlnzbs+q9asijRIcwhGcgAfnUIcbaIAPDPrwDK/w5gjnxXl3PmatS04xcwB/4Hz+AJ8AjY0=</latexit><latexit sha1_base64="6E9Oi48tZeFkKRsbB82UZln6sLc=">AAAB6XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWNFYwttKJvtpF272YTdjVBCf4IXDype/Ufe/Ddu2xy09cHA470ZZuaFqeDauO63s7S8srq2Xtoob25t7+xW9vYfdJIphj5LRKJaIdUouETfcCOwlSqkcSiwGQ6vJ37zCZXmibw3oxSDmPYljzijxkp3re5jt1J1a+4UZJF4BalCgUa38tXpJSyLURomqNZtz01NkFNlOBM4LncyjSllQ9rHtqWSxqiDfHrqmBxbpUeiRNmShkzV3xM5jbUexaHtjKkZ6HlvIv7ntTMTXQQ5l2lmULLZoigTxCRk8jfpcYXMiJEllClubyVsQBVlxqZTtiF48y8vEv+0dlnzbs+q9asijRIcwhGcgAfnUIcbaIAPDPrwDK/w5gjnxXl3PmatS04xcwB/4Hz+AJ8AjY0=</latexit><latexit sha1_base64="6E9Oi48tZeFkKRsbB82UZln6sLc=">AAAB6XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWNFYwttKJvtpF272YTdjVBCf4IXDype/Ufe/Ddu2xy09cHA470ZZuaFqeDauO63s7S8srq2Xtoob25t7+xW9vYfdJIphj5LRKJaIdUouETfcCOwlSqkcSiwGQ6vJ37zCZXmibw3oxSDmPYljzijxkp3re5jt1J1a+4UZJF4BalCgUa38tXpJSyLURomqNZtz01NkFNlOBM4LncyjSllQ9rHtqWSxqiDfHrqmBxbpUeiRNmShkzV3xM5jbUexaHtjKkZ6HlvIv7ntTMTXQQ5l2lmULLZoigTxCRk8jfpcYXMiJEllClubyVsQBVlxqZTtiF48y8vEv+0dlnzbs+q9asijRIcwhGcgAfnUIcbaIAPDPrwDK/w5gjnxXl3PmatS04xcwB/4Hz+AJ8AjY0=</latexit>

Find the measurement column

Note: we are not finding the row xi<latexit sha1_base64="AVmfx+YeFaOOyFtkiB8CIxCujYQ=">AAAB6XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWNFYwttKJvtpl262YTdiVhCf4IXDype/Ufe/Ddu2xy09cHA470ZZuaFqRQGXffbWVpeWV1bL22UN7e2d3Yre/sPJsk04z5LZKJbITVcCsV9FCh5K9WcxqHkzXB4PfGbj1wbkah7HKU8iGlfiUgwila6e+qKbqXq1twpyCLxClKFAo1u5avTS1gWc4VMUmPanptikFONgkk+Lncyw1PKhrTP25YqGnMT5NNTx+TYKj0SJdqWQjJVf0/kNDZmFIe2M6Y4MPPeRPzPa2cYXQS5UGmGXLHZoiiTBBMy+Zv0hOYM5cgSyrSwtxI2oJoytOmUbQje/MuLxD+tXda827Nq/apIowSHcAQn4ME51OEGGuADgz48wyu8OdJ5cd6dj1nrklPMHMAfOJ8/zh2NrA==</latexit><latexit sha1_base64="AVmfx+YeFaOOyFtkiB8CIxCujYQ=">AAAB6XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWNFYwttKJvtpl262YTdiVhCf4IXDype/Ufe/Ddu2xy09cHA470ZZuaFqRQGXffbWVpeWV1bL22UN7e2d3Yre/sPJsk04z5LZKJbITVcCsV9FCh5K9WcxqHkzXB4PfGbj1wbkah7HKU8iGlfiUgwila6e+qKbqXq1twpyCLxClKFAo1u5avTS1gWc4VMUmPanptikFONgkk+Lncyw1PKhrTP25YqGnMT5NNTx+TYKj0SJdqWQjJVf0/kNDZmFIe2M6Y4MPPeRPzPa2cYXQS5UGmGXLHZoiiTBBMy+Zv0hOYM5cgSyrSwtxI2oJoytOmUbQje/MuLxD+tXda827Nq/apIowSHcAQn4ME51OEGGuADgz48wyu8OdJ5cd6dj1nrklPMHMAfOJ8/zh2NrA==</latexit><latexit sha1_base64="AVmfx+YeFaOOyFtkiB8CIxCujYQ=">AAAB6XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWNFYwttKJvtpl262YTdiVhCf4IXDype/Ufe/Ddu2xy09cHA470ZZuaFqRQGXffbWVpeWV1bL22UN7e2d3Yre/sPJsk04z5LZKJbITVcCsV9FCh5K9WcxqHkzXB4PfGbj1wbkah7HKU8iGlfiUgwila6e+qKbqXq1twpyCLxClKFAo1u5avTS1gWc4VMUmPanptikFONgkk+Lncyw1PKhrTP25YqGnMT5NNTx+TYKj0SJdqWQjJVf0/kNDZmFIe2M6Y4MPPeRPzPa2cYXQS5UGmGXLHZoiiTBBMy+Zv0hOYM5cgSyrSwtxI2oJoytOmUbQje/MuLxD+tXda827Nq/apIowSHcAQn4ME51OEGGuADgz48wyu8OdJ5cd6dj1nrklPMHMAfOJ8/zh2NrA==</latexit><latexit sha1_base64="AVmfx+YeFaOOyFtkiB8CIxCujYQ=">AAAB6XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWNFYwttKJvtpl262YTdiVhCf4IXDype/Ufe/Ddu2xy09cHA470ZZuaFqRQGXffbWVpeWV1bL22UN7e2d3Yre/sPJsk04z5LZKJbITVcCsV9FCh5K9WcxqHkzXB4PfGbj1wbkah7HKU8iGlfiUgwila6e+qKbqXq1twpyCLxClKFAo1u5avTS1gWc4VMUmPanptikFONgkk+Lncyw1PKhrTP25YqGnMT5NNTx+TYKj0SJdqWQjJVf0/kNDZmFIe2M6Y4MPPeRPzPa2cYXQS5UGmGXLHZoiiTBBMy+Zv0hOYM5cgSyrSwtxI2oJoytOmUbQje/MuLxD+tXda827Nq/apIowSHcAQn4ME51OEGGuADgz48wyu8OdJ5cd6dj1nrklPMHMAfOJ8/zh2NrA==</latexit>

j is the index of f that better explains y

Step 1

Page 9: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Orthogonal Matching pursuit

Reconstruction residual between measurement and what is our estimate for the entry j of f

Step 2

Page 10: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Orthogonal Matching pursuit

this is what remains to be explained by other elements of f

Step 3

compute the residual

and return to step 1, until r is small (there is not that much to explain) oris small (further explanation is not useful)

�<latexit sha1_base64="+03BdUB6TWqLjuCfwSU3OIMjF3A=">AAAB7HicbVDLSgNBEOz1GeMr6tHLYBA8hV0R1FvQi8cIbhJIljA7mU3GzGOZmRXCkn/w4kHFqx/kzb9xkuxBEwsaiqpuurvilDNjff/bW1ldW9/YLG2Vt3d29/YrB4dNozJNaEgUV7odY0M5kzS0zHLaTjXFIua0FY9up37riWrDlHyw45RGAg8kSxjB1knN7gALgXuVql/zZ0DLJChIFQo0epWvbl+RTFBpCcfGdAI/tVGOtWWE00m5mxmaYjLCA9pxVGJBTZTPrp2gU6f0UaK0K2nRTP09kWNhzFjErlNgOzSL3lT8z+tkNrmKcibTzFJJ5ouSjCOr0PR11GeaEsvHjmCimbsVkSHWmFgXUNmFECy+vEzC89p1Lbi/qNZvijRKcAwncAYBXEId7qABIRB4hGd4hTdPeS/eu/cxb13xipkj+APv8wfzVo7p</latexit><latexit sha1_base64="+03BdUB6TWqLjuCfwSU3OIMjF3A=">AAAB7HicbVDLSgNBEOz1GeMr6tHLYBA8hV0R1FvQi8cIbhJIljA7mU3GzGOZmRXCkn/w4kHFqx/kzb9xkuxBEwsaiqpuurvilDNjff/bW1ldW9/YLG2Vt3d29/YrB4dNozJNaEgUV7odY0M5kzS0zHLaTjXFIua0FY9up37riWrDlHyw45RGAg8kSxjB1knN7gALgXuVql/zZ0DLJChIFQo0epWvbl+RTFBpCcfGdAI/tVGOtWWE00m5mxmaYjLCA9pxVGJBTZTPrp2gU6f0UaK0K2nRTP09kWNhzFjErlNgOzSL3lT8z+tkNrmKcibTzFJJ5ouSjCOr0PR11GeaEsvHjmCimbsVkSHWmFgXUNmFECy+vEzC89p1Lbi/qNZvijRKcAwncAYBXEId7qABIRB4hGd4hTdPeS/eu/cxb13xipkj+APv8wfzVo7p</latexit><latexit sha1_base64="+03BdUB6TWqLjuCfwSU3OIMjF3A=">AAAB7HicbVDLSgNBEOz1GeMr6tHLYBA8hV0R1FvQi8cIbhJIljA7mU3GzGOZmRXCkn/w4kHFqx/kzb9xkuxBEwsaiqpuurvilDNjff/bW1ldW9/YLG2Vt3d29/YrB4dNozJNaEgUV7odY0M5kzS0zHLaTjXFIua0FY9up37riWrDlHyw45RGAg8kSxjB1knN7gALgXuVql/zZ0DLJChIFQo0epWvbl+RTFBpCcfGdAI/tVGOtWWE00m5mxmaYjLCA9pxVGJBTZTPrp2gU6f0UaK0K2nRTP09kWNhzFjErlNgOzSL3lT8z+tkNrmKcibTzFJJ5ouSjCOr0PR11GeaEsvHjmCimbsVkSHWmFgXUNmFECy+vEzC89p1Lbi/qNZvijRKcAwncAYBXEId7qABIRB4hGd4hTdPeS/eu/cxb13xipkj+APv8wfzVo7p</latexit><latexit sha1_base64="+03BdUB6TWqLjuCfwSU3OIMjF3A=">AAAB7HicbVDLSgNBEOz1GeMr6tHLYBA8hV0R1FvQi8cIbhJIljA7mU3GzGOZmRXCkn/w4kHFqx/kzb9xkuxBEwsaiqpuurvilDNjff/bW1ldW9/YLG2Vt3d29/YrB4dNozJNaEgUV7odY0M5kzS0zHLaTjXFIua0FY9up37riWrDlHyw45RGAg8kSxjB1knN7gALgXuVql/zZ0DLJChIFQo0epWvbl+RTFBpCcfGdAI/tVGOtWWE00m5mxmaYjLCA9pxVGJBTZTPrp2gU6f0UaK0K2nRTP09kWNhzFjErlNgOzSL3lT8z+tkNrmKcibTzFJJ5ouSjCOr0PR11GeaEsvHjmCimbsVkSHWmFgXUNmFECy+vEzC89p1Lbi/qNZvijRKcAwncAYBXEId7qABIRB4hGd4hTdPeS/eu/cxb13xipkj+APv8wfzVo7p</latexit>

y = r<latexit sha1_base64="6oq/mpUf2eN9xBPKMOeoPhxeCvI=">AAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2lA22027dLMJuxOhhP4GLx5UvPqHvPlv3LY5aOuDgcd7M8zMC1MpDLrut1NaWV1b3yhvVra2d3b3qvsHjybJNOM+S2Si2yE1XArFfRQoeTvVnMah5K1wdDv1W09cG5GoBxynPIjpQIlIMIpW8sfkmuhetebW3RnIMvEKUoMCzV71q9tPWBZzhUxSYzqem2KQU42CST6pdDPDU8pGdMA7lioacxPks2Mn5MQqfRIl2pZCMlN/T+Q0NmYch7Yzpjg0i95U/M/rZBhdBrlQaYZcsfmiKJMEEzL9nPSF5gzl2BLKtLC3EjakmjK0+VRsCN7iy8vEP6tf1b3781rjpkijDEdwDKfgwQU04A6a4AMDAc/wCm+Ocl6cd+dj3lpyiplD+APn8wdWA43o</latexit><latexit sha1_base64="6oq/mpUf2eN9xBPKMOeoPhxeCvI=">AAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2lA22027dLMJuxOhhP4GLx5UvPqHvPlv3LY5aOuDgcd7M8zMC1MpDLrut1NaWV1b3yhvVra2d3b3qvsHjybJNOM+S2Si2yE1XArFfRQoeTvVnMah5K1wdDv1W09cG5GoBxynPIjpQIlIMIpW8sfkmuhetebW3RnIMvEKUoMCzV71q9tPWBZzhUxSYzqem2KQU42CST6pdDPDU8pGdMA7lioacxPks2Mn5MQqfRIl2pZCMlN/T+Q0NmYch7Yzpjg0i95U/M/rZBhdBrlQaYZcsfmiKJMEEzL9nPSF5gzl2BLKtLC3EjakmjK0+VRsCN7iy8vEP6tf1b3781rjpkijDEdwDKfgwQU04A6a4AMDAc/wCm+Ocl6cd+dj3lpyiplD+APn8wdWA43o</latexit><latexit sha1_base64="6oq/mpUf2eN9xBPKMOeoPhxeCvI=">AAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2lA22027dLMJuxOhhP4GLx5UvPqHvPlv3LY5aOuDgcd7M8zMC1MpDLrut1NaWV1b3yhvVra2d3b3qvsHjybJNOM+S2Si2yE1XArFfRQoeTvVnMah5K1wdDv1W09cG5GoBxynPIjpQIlIMIpW8sfkmuhetebW3RnIMvEKUoMCzV71q9tPWBZzhUxSYzqem2KQU42CST6pdDPDU8pGdMA7lioacxPks2Mn5MQqfRIl2pZCMlN/T+Q0NmYch7Yzpjg0i95U/M/rZBhdBrlQaYZcsfmiKJMEEzL9nPSF5gzl2BLKtLC3EjakmjK0+VRsCN7iy8vEP6tf1b3781rjpkijDEdwDKfgwQU04A6a4AMDAc/wCm+Ocl6cd+dj3lpyiplD+APn8wdWA43o</latexit><latexit sha1_base64="6oq/mpUf2eN9xBPKMOeoPhxeCvI=">AAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2lA22027dLMJuxOhhP4GLx5UvPqHvPlv3LY5aOuDgcd7M8zMC1MpDLrut1NaWV1b3yhvVra2d3b3qvsHjybJNOM+S2Si2yE1XArFfRQoeTvVnMah5K1wdDv1W09cG5GoBxynPIjpQIlIMIpW8sfkmuhetebW3RnIMvEKUoMCzV71q9tPWBZzhUxSYzqem2KQU42CST6pdDPDU8pGdMA7lioacxPks2Mn5MQqfRIl2pZCMlN/T+Q0NmYch7Yzpjg0i95U/M/rZBhdBrlQaYZcsfmiKJMEEzL9nPSF5gzl2BLKtLC3EjakmjK0+VRsCN7iy8vEP6tf1b3781rjpkijDEdwDKfgwQU04A6a4AMDAc/wCm+Ocl6cd+dj3lpyiplD+APn8wdWA43o</latexit>

do

Page 11: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Orthogonal Matching pursuit

r = y<latexit sha1_base64="BZQIY4Rd6Cwz0+aWq3TLHaYHud4=">AAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2lA22027dLMJuxOhhP4GLx5UvPqHvPlv3LY5aOuDgcd7M8zMC1MpDLrut1NaWV1b3yhvVra2d3b3qvsHjybJNOM+S2Si2yE1XArFfRQoeTvVnMah5K1wdDv1W09cG5GoBxynPIjpQIlIMIpW8jW5JuNetebW3RnIMvEKUoMCzV71q9tPWBZzhUxSYzqem2KQU42CST6pdDPDU8pGdMA7lioacxPks2Mn5MQqfRIl2pZCMlN/T+Q0NmYch7Yzpjg0i95U/M/rZBhdBrlQaYZcsfmiKJMEEzL9nPSF5gzl2BLKtLC3EjakmjK0+VRsCN7iy8vEP6tf1b3781rjpkijDEdwDKfgwQU04A6a4AMDAc/wCm+Ocl6cd+dj3lpyiplD+APn8wdV543o</latexit><latexit sha1_base64="BZQIY4Rd6Cwz0+aWq3TLHaYHud4=">AAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2lA22027dLMJuxOhhP4GLx5UvPqHvPlv3LY5aOuDgcd7M8zMC1MpDLrut1NaWV1b3yhvVra2d3b3qvsHjybJNOM+S2Si2yE1XArFfRQoeTvVnMah5K1wdDv1W09cG5GoBxynPIjpQIlIMIpW8jW5JuNetebW3RnIMvEKUoMCzV71q9tPWBZzhUxSYzqem2KQU42CST6pdDPDU8pGdMA7lioacxPks2Mn5MQqfRIl2pZCMlN/T+Q0NmYch7Yzpjg0i95U/M/rZBhdBrlQaYZcsfmiKJMEEzL9nPSF5gzl2BLKtLC3EjakmjK0+VRsCN7iy8vEP6tf1b3781rjpkijDEdwDKfgwQU04A6a4AMDAc/wCm+Ocl6cd+dj3lpyiplD+APn8wdV543o</latexit><latexit sha1_base64="BZQIY4Rd6Cwz0+aWq3TLHaYHud4=">AAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2lA22027dLMJuxOhhP4GLx5UvPqHvPlv3LY5aOuDgcd7M8zMC1MpDLrut1NaWV1b3yhvVra2d3b3qvsHjybJNOM+S2Si2yE1XArFfRQoeTvVnMah5K1wdDv1W09cG5GoBxynPIjpQIlIMIpW8jW5JuNetebW3RnIMvEKUoMCzV71q9tPWBZzhUxSYzqem2KQU42CST6pdDPDU8pGdMA7lioacxPks2Mn5MQqfRIl2pZCMlN/T+Q0NmYch7Yzpjg0i95U/M/rZBhdBrlQaYZcsfmiKJMEEzL9nPSF5gzl2BLKtLC3EjakmjK0+VRsCN7iy8vEP6tf1b3781rjpkijDEdwDKfgwQU04A6a4AMDAc/wCm+Ocl6cd+dj3lpyiplD+APn8wdV543o</latexit><latexit sha1_base64="BZQIY4Rd6Cwz0+aWq3TLHaYHud4=">AAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2lA22027dLMJuxOhhP4GLx5UvPqHvPlv3LY5aOuDgcd7M8zMC1MpDLrut1NaWV1b3yhvVra2d3b3qvsHjybJNOM+S2Si2yE1XArFfRQoeTvVnMah5K1wdDv1W09cG5GoBxynPIjpQIlIMIpW8jW5JuNetebW3RnIMvEKUoMCzV71q9tPWBZzhUxSYzqem2KQU42CST6pdDPDU8pGdMA7lioacxPks2Mn5MQqfRIl2pZCMlN/T+Q0NmYch7Yzpjg0i95U/M/rZBhdBrlQaYZcsfmiKJMEEzL9nPSF5gzl2BLKtLC3EjakmjK0+VRsCN7iy8vEP6tf1b3781rjpkijDEdwDKfgwQU04A6a4AMDAc/wCm+Ocl6cd+dj3lpyiplD+APn8wdV543o</latexit>

Xj = argmaxXj0

|hr,Xj0i|<latexit sha1_base64="2onlBeauZORnbY/TioPQ77F+beQ=">AAACM3icbVDLSgMxFM34tr6qLt0Ei+BCyowI6kIQ3QhuFKwWOmXIpLc1mmSG5I5YxvkoN36IGxFcqLj1H0wfC18HAodz7uXmnDiVwqLvP3sjo2PjE5NT06WZ2bn5hfLi0rlNMsOhxhOZmHrMLEihoYYCJdRTA0zFEi7i68Oef3EDxopEn2E3haZiHS3agjN0UlQ+rkdXdI+GiuFlkuYhwi3mzHQUuy2KKK9H+VWYGqGgKOgdDSXTHQnUbHwzQjMQ76Jyxa/6fdC/JBiSChniJCo/hq2EZwo0csmsbQR+ik13HgWXUJTCzELK+DXrQMNRzRTYZt4PXdA1p7RoOzHuaaR99ftGzpS1XRW7yV44+9vrif95jQzbO81c6DRD0HxwqJ1JigntNUhbwgBH2XWEcSPcXym/ZIZxdD2XXAnB78h/SW2zulsNTrcq+wfDNqbIClkl6yQg22SfHJETUiOc3JMn8krevAfvxXv3PgajI95wZ5n8gPf5BVz3rLY=</latexit><latexit sha1_base64="2onlBeauZORnbY/TioPQ77F+beQ=">AAACM3icbVDLSgMxFM34tr6qLt0Ei+BCyowI6kIQ3QhuFKwWOmXIpLc1mmSG5I5YxvkoN36IGxFcqLj1H0wfC18HAodz7uXmnDiVwqLvP3sjo2PjE5NT06WZ2bn5hfLi0rlNMsOhxhOZmHrMLEihoYYCJdRTA0zFEi7i68Oef3EDxopEn2E3haZiHS3agjN0UlQ+rkdXdI+GiuFlkuYhwi3mzHQUuy2KKK9H+VWYGqGgKOgdDSXTHQnUbHwzQjMQ76Jyxa/6fdC/JBiSChniJCo/hq2EZwo0csmsbQR+ik13HgWXUJTCzELK+DXrQMNRzRTYZt4PXdA1p7RoOzHuaaR99ftGzpS1XRW7yV44+9vrif95jQzbO81c6DRD0HxwqJ1JigntNUhbwgBH2XWEcSPcXym/ZIZxdD2XXAnB78h/SW2zulsNTrcq+wfDNqbIClkl6yQg22SfHJETUiOc3JMn8krevAfvxXv3PgajI95wZ5n8gPf5BVz3rLY=</latexit><latexit sha1_base64="2onlBeauZORnbY/TioPQ77F+beQ=">AAACM3icbVDLSgMxFM34tr6qLt0Ei+BCyowI6kIQ3QhuFKwWOmXIpLc1mmSG5I5YxvkoN36IGxFcqLj1H0wfC18HAodz7uXmnDiVwqLvP3sjo2PjE5NT06WZ2bn5hfLi0rlNMsOhxhOZmHrMLEihoYYCJdRTA0zFEi7i68Oef3EDxopEn2E3haZiHS3agjN0UlQ+rkdXdI+GiuFlkuYhwi3mzHQUuy2KKK9H+VWYGqGgKOgdDSXTHQnUbHwzQjMQ76Jyxa/6fdC/JBiSChniJCo/hq2EZwo0csmsbQR+ik13HgWXUJTCzELK+DXrQMNRzRTYZt4PXdA1p7RoOzHuaaR99ftGzpS1XRW7yV44+9vrif95jQzbO81c6DRD0HxwqJ1JigntNUhbwgBH2XWEcSPcXym/ZIZxdD2XXAnB78h/SW2zulsNTrcq+wfDNqbIClkl6yQg22SfHJETUiOc3JMn8krevAfvxXv3PgajI95wZ5n8gPf5BVz3rLY=</latexit><latexit sha1_base64="2onlBeauZORnbY/TioPQ77F+beQ=">AAACM3icbVDLSgMxFM34tr6qLt0Ei+BCyowI6kIQ3QhuFKwWOmXIpLc1mmSG5I5YxvkoN36IGxFcqLj1H0wfC18HAodz7uXmnDiVwqLvP3sjo2PjE5NT06WZ2bn5hfLi0rlNMsOhxhOZmHrMLEihoYYCJdRTA0zFEi7i68Oef3EDxopEn2E3haZiHS3agjN0UlQ+rkdXdI+GiuFlkuYhwi3mzHQUuy2KKK9H+VWYGqGgKOgdDSXTHQnUbHwzQjMQ76Jyxa/6fdC/JBiSChniJCo/hq2EZwo0csmsbQR+ik13HgWXUJTCzELK+DXrQMNRzRTYZt4PXdA1p7RoOzHuaaR99ftGzpS1XRW7yV44+9vrif95jQzbO81c6DRD0HxwqJ1JigntNUhbwgBH2XWEcSPcXym/ZIZxdD2XXAnB78h/SW2zulsNTrcq+wfDNqbIClkl6yQg22SfHJETUiOc3JMn8krevAfvxXv3PgajI95wZ5n8gPf5BVz3rLY=</latexit>

�j = argmin�

kr �Xj�k<latexit sha1_base64="7aTTb7M8M5uMDXtbsUbkY1QxtOM=">AAACKXicbVDLSsQwFE3H9/iqunQTHAQ3Dq0I6kIYdeNSwdGBaSlpJh3jJGlJbsWh9nvc+CtuXPja+iNmHgtfBwKHc+7l5pw4E9yA5707lYnJqemZ2bnq/MLi0rK7snpp0lxT1qSpSHUrJoYJrlgTOAjWyjQjMhbsKu6dDPyrW6YNT9UF9DMWStJVPOGUgJUi9yjoEilJdIMPcSAJXKdZEQC7g4LoruSqLKNiNFLi4B5jjbdxKypuypFotciteXVvCPyX+GNSQ2OcRe5z0ElpLpkCKogxbd/LILT3gFPBymqQG5YR2iNd1rZUEclMWAyjlnjTKh2cpNo+BXioft8oiDSmL2M7OUhjfnsD8T+vnUOyHxZcZTkwRUeHklxgSPGgN9zhmlEQfUsI1dz+FdNrogkF227VluD/jvyXNHfqB3X/fLfWOB63MYvW0QbaQj7aQw10is5QE1H0gJ7QC3p1Hp1n5835GI1WnPHOGvoB5/MLSmKnXw==</latexit><latexit sha1_base64="7aTTb7M8M5uMDXtbsUbkY1QxtOM=">AAACKXicbVDLSsQwFE3H9/iqunQTHAQ3Dq0I6kIYdeNSwdGBaSlpJh3jJGlJbsWh9nvc+CtuXPja+iNmHgtfBwKHc+7l5pw4E9yA5707lYnJqemZ2bnq/MLi0rK7snpp0lxT1qSpSHUrJoYJrlgTOAjWyjQjMhbsKu6dDPyrW6YNT9UF9DMWStJVPOGUgJUi9yjoEilJdIMPcSAJXKdZEQC7g4LoruSqLKNiNFLi4B5jjbdxKypuypFotciteXVvCPyX+GNSQ2OcRe5z0ElpLpkCKogxbd/LILT3gFPBymqQG5YR2iNd1rZUEclMWAyjlnjTKh2cpNo+BXioft8oiDSmL2M7OUhjfnsD8T+vnUOyHxZcZTkwRUeHklxgSPGgN9zhmlEQfUsI1dz+FdNrogkF227VluD/jvyXNHfqB3X/fLfWOB63MYvW0QbaQj7aQw10is5QE1H0gJ7QC3p1Hp1n5835GI1WnPHOGvoB5/MLSmKnXw==</latexit><latexit sha1_base64="7aTTb7M8M5uMDXtbsUbkY1QxtOM=">AAACKXicbVDLSsQwFE3H9/iqunQTHAQ3Dq0I6kIYdeNSwdGBaSlpJh3jJGlJbsWh9nvc+CtuXPja+iNmHgtfBwKHc+7l5pw4E9yA5707lYnJqemZ2bnq/MLi0rK7snpp0lxT1qSpSHUrJoYJrlgTOAjWyjQjMhbsKu6dDPyrW6YNT9UF9DMWStJVPOGUgJUi9yjoEilJdIMPcSAJXKdZEQC7g4LoruSqLKNiNFLi4B5jjbdxKypuypFotciteXVvCPyX+GNSQ2OcRe5z0ElpLpkCKogxbd/LILT3gFPBymqQG5YR2iNd1rZUEclMWAyjlnjTKh2cpNo+BXioft8oiDSmL2M7OUhjfnsD8T+vnUOyHxZcZTkwRUeHklxgSPGgN9zhmlEQfUsI1dz+FdNrogkF227VluD/jvyXNHfqB3X/fLfWOB63MYvW0QbaQj7aQw10is5QE1H0gJ7QC3p1Hp1n5835GI1WnPHOGvoB5/MLSmKnXw==</latexit><latexit sha1_base64="7aTTb7M8M5uMDXtbsUbkY1QxtOM=">AAACKXicbVDLSsQwFE3H9/iqunQTHAQ3Dq0I6kIYdeNSwdGBaSlpJh3jJGlJbsWh9nvc+CtuXPja+iNmHgtfBwKHc+7l5pw4E9yA5707lYnJqemZ2bnq/MLi0rK7snpp0lxT1qSpSHUrJoYJrlgTOAjWyjQjMhbsKu6dDPyrW6YNT9UF9DMWStJVPOGUgJUi9yjoEilJdIMPcSAJXKdZEQC7g4LoruSqLKNiNFLi4B5jjbdxKypuypFotciteXVvCPyX+GNSQ2OcRe5z0ElpLpkCKogxbd/LILT3gFPBymqQG5YR2iNd1rZUEclMWAyjlnjTKh2cpNo+BXioft8oiDSmL2M7OUhjfnsD8T+vnUOyHxZcZTkwRUeHklxgSPGgN9zhmlEQfUsI1dz+FdNrogkF227VluD/jvyXNHfqB3X/fLfWOB63MYvW0QbaQj7aQw10is5QE1H0gJ7QC3p1Hp1n5835GI1WnPHOGvoB5/MLSmKnXw==</latexit>

r = r �Xj�j<latexit sha1_base64="U5oaMI2JZkX4pYDqkd9mXrbG+us=">AAAB/XicbVBNS8NAEJ34WetXVDx5WSyCF0signoQil48VjC20Iaw2W7abXeTsLsRSij4V7x4UPHq//Dmv3Hb5qCtDwYe780wMy9MOVPacb6thcWl5ZXV0lp5fWNza9ve2X1QSSYJ9UjCE9kMsaKcxdTTTHPaTCXFIuS0EQ5uxn7jkUrFkvheD1PqC9yNWcQI1kYK7H2JrpBEJ6gZ5P1Ru4uFwEE/sCtO1ZkAzRO3IBUoUA/sr3YnIZmgsSYcK9VynVT7OZaaEU5H5XamaIrJAHdpy9AYC6r8fHL+CB0ZpYOiRJqKNZqovydyLJQaitB0Cqx7atYbi/95rUxHF37O4jTTNCbTRVHGkU7QOAvUYZISzYeGYCKZuRWRHpaYaJNY2YTgzr48T7zT6mXVvTur1K6LNEpwAIdwDC6cQw1uoQ4eEMjhGV7hzXqyXqx362PaumAVM3vwB9bnD+WLlGA=</latexit><latexit sha1_base64="U5oaMI2JZkX4pYDqkd9mXrbG+us=">AAAB/XicbVBNS8NAEJ34WetXVDx5WSyCF0signoQil48VjC20Iaw2W7abXeTsLsRSij4V7x4UPHq//Dmv3Hb5qCtDwYe780wMy9MOVPacb6thcWl5ZXV0lp5fWNza9ve2X1QSSYJ9UjCE9kMsaKcxdTTTHPaTCXFIuS0EQ5uxn7jkUrFkvheD1PqC9yNWcQI1kYK7H2JrpBEJ6gZ5P1Ru4uFwEE/sCtO1ZkAzRO3IBUoUA/sr3YnIZmgsSYcK9VynVT7OZaaEU5H5XamaIrJAHdpy9AYC6r8fHL+CB0ZpYOiRJqKNZqovydyLJQaitB0Cqx7atYbi/95rUxHF37O4jTTNCbTRVHGkU7QOAvUYZISzYeGYCKZuRWRHpaYaJNY2YTgzr48T7zT6mXVvTur1K6LNEpwAIdwDC6cQw1uoQ4eEMjhGV7hzXqyXqx362PaumAVM3vwB9bnD+WLlGA=</latexit><latexit sha1_base64="U5oaMI2JZkX4pYDqkd9mXrbG+us=">AAAB/XicbVBNS8NAEJ34WetXVDx5WSyCF0signoQil48VjC20Iaw2W7abXeTsLsRSij4V7x4UPHq//Dmv3Hb5qCtDwYe780wMy9MOVPacb6thcWl5ZXV0lp5fWNza9ve2X1QSSYJ9UjCE9kMsaKcxdTTTHPaTCXFIuS0EQ5uxn7jkUrFkvheD1PqC9yNWcQI1kYK7H2JrpBEJ6gZ5P1Ru4uFwEE/sCtO1ZkAzRO3IBUoUA/sr3YnIZmgsSYcK9VynVT7OZaaEU5H5XamaIrJAHdpy9AYC6r8fHL+CB0ZpYOiRJqKNZqovydyLJQaitB0Cqx7atYbi/95rUxHF37O4jTTNCbTRVHGkU7QOAvUYZISzYeGYCKZuRWRHpaYaJNY2YTgzr48T7zT6mXVvTur1K6LNEpwAIdwDC6cQw1uoQ4eEMjhGV7hzXqyXqx362PaumAVM3vwB9bnD+WLlGA=</latexit><latexit sha1_base64="U5oaMI2JZkX4pYDqkd9mXrbG+us=">AAAB/XicbVBNS8NAEJ34WetXVDx5WSyCF0signoQil48VjC20Iaw2W7abXeTsLsRSij4V7x4UPHq//Dmv3Hb5qCtDwYe780wMy9MOVPacb6thcWl5ZXV0lp5fWNza9ve2X1QSSYJ9UjCE9kMsaKcxdTTTHPaTCXFIuS0EQ5uxn7jkUrFkvheD1PqC9yNWcQI1kYK7H2JrpBEJ6gZ5P1Ru4uFwEE/sCtO1ZkAzRO3IBUoUA/sr3YnIZmgsSYcK9VynVT7OZaaEU5H5XamaIrJAHdpy9AYC6r8fHL+CB0ZpYOiRJqKNZqovydyLJQaitB0Cqx7atYbi/95rUxHF37O4jTTNCbTRVHGkU7QOAvUYZISzYeGYCKZuRWRHpaYaJNY2YTgzr48T7zT6mXVvTur1K6LNEpwAIdwDC6cQw1uoQ4eEMjhGV7hzXqyXqx362PaumAVM3vwB9bnD+WLlGA=</latexit>

gj = �j<latexit sha1_base64="kLC609921l1PgIXwWoQVISafiwg=">AAAB9HicbVBNS8NAEN3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2hgm20267W4SdjdKCf0fXjyoePXHePPfuG1z0NYHA4/3ZpiZF6ScKW3b31ZpaXllda28XtnY3Nreqe7u3askk4S6JOGJbAegKGcxdTXTnLZTSUEEnLaC4fXEbz1SqVgS3+lRSj0BUcxCRkAb6SHyB/gSdyMQAvyBX63ZdXsKvEicgtRQgaZf/er2EpIJGmvCQamOY6fay0FqRjgdV7qZoimQIUS0Y2gMgiovn149xkdG6eEwkaZijafq74kchFIjEZhOAbqv5r2J+J/XyXR47uUsTjNNYzJbFGYc6wRPIsA9JinRfGQIEMnMrZj0QQLRJqiKCcGZf3mRuCf1i7pze1prXBVplNEBOkTHyEFnqIFuUBO5iCCJntErerOerBfr3fqYtZasYmYf/YH1+QPrYpGv</latexit><latexit sha1_base64="kLC609921l1PgIXwWoQVISafiwg=">AAAB9HicbVBNS8NAEN3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2hgm20267W4SdjdKCf0fXjyoePXHePPfuG1z0NYHA4/3ZpiZF6ScKW3b31ZpaXllda28XtnY3Nreqe7u3askk4S6JOGJbAegKGcxdTXTnLZTSUEEnLaC4fXEbz1SqVgS3+lRSj0BUcxCRkAb6SHyB/gSdyMQAvyBX63ZdXsKvEicgtRQgaZf/er2EpIJGmvCQamOY6fay0FqRjgdV7qZoimQIUS0Y2gMgiovn149xkdG6eEwkaZijafq74kchFIjEZhOAbqv5r2J+J/XyXR47uUsTjNNYzJbFGYc6wRPIsA9JinRfGQIEMnMrZj0QQLRJqiKCcGZf3mRuCf1i7pze1prXBVplNEBOkTHyEFnqIFuUBO5iCCJntErerOerBfr3fqYtZasYmYf/YH1+QPrYpGv</latexit><latexit sha1_base64="kLC609921l1PgIXwWoQVISafiwg=">AAAB9HicbVBNS8NAEN3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2hgm20267W4SdjdKCf0fXjyoePXHePPfuG1z0NYHA4/3ZpiZF6ScKW3b31ZpaXllda28XtnY3Nreqe7u3askk4S6JOGJbAegKGcxdTXTnLZTSUEEnLaC4fXEbz1SqVgS3+lRSj0BUcxCRkAb6SHyB/gSdyMQAvyBX63ZdXsKvEicgtRQgaZf/er2EpIJGmvCQamOY6fay0FqRjgdV7qZoimQIUS0Y2gMgiovn149xkdG6eEwkaZijafq74kchFIjEZhOAbqv5r2J+J/XyXR47uUsTjNNYzJbFGYc6wRPIsA9JinRfGQIEMnMrZj0QQLRJqiKCcGZf3mRuCf1i7pze1prXBVplNEBOkTHyEFnqIFuUBO5iCCJntErerOerBfr3fqYtZasYmYf/YH1+QPrYpGv</latexit><latexit sha1_base64="kLC609921l1PgIXwWoQVISafiwg=">AAAB9HicbVBNS8NAEN3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2hgm20267W4SdjdKCf0fXjyoePXHePPfuG1z0NYHA4/3ZpiZF6ScKW3b31ZpaXllda28XtnY3Nreqe7u3askk4S6JOGJbAegKGcxdTXTnLZTSUEEnLaC4fXEbz1SqVgS3+lRSj0BUcxCRkAb6SHyB/gSdyMQAvyBX63ZdXsKvEicgtRQgaZf/er2EpIJGmvCQamOY6fay0FqRjgdV7qZoimQIUS0Y2gMgiovn149xkdG6eEwkaZijafq74kchFIjEZhOAbqv5r2J+J/XyXR47uUsTjNNYzJbFGYc6wRPIsA9JinRfGQIEMnMrZj0QQLRJqiKCcGZf3mRuCf1i7pze1prXBVplNEBOkTHyEFnqIFuUBO5iCCJntErerOerBfr3fqYtZasYmYf/YH1+QPrYpGv</latexit>

Set

Return

S1:

S2:

S3:

Page 12: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Examplef = [0, 0, 1, 0, 0, 1, 0, 0, 1, 0]

<latexit sha1_base64="orN/A16ZqzHYB4nfSe/ir16mQXA=">AAACCnicbVDLSgMxFL1TX7W+qi7dhBbBhZQZEdSFUHTjsoJjC9OhZNJMG5rJDElGKEP3bvwVNy5U3PoF7vwb0+kstPWQXA7n3EtyT5BwprRtf1ulpeWV1bXyemVjc2t7p7q7d6/iVBLqkpjHshNgRTkT1NVMc9pJJMVRwGk7GF1P/fYDlYrF4k6PE+pHeCBYyAjWRupVayG6RJ59jMxx8jpP/F61bjfsHGiROAWpQ4FWr/rV7cckjajQhGOlPMdOtJ9hqRnhdFLppoommIzwgHqGChxR5Wf5LhN0aJQ+CmNprtAoV39PZDhSahwFpjPCeqjmvan4n+elOjz3MyaSVFNBZg+FKUc6RtNgUJ9JSjQfG4KJZOaviAyxxESb+ComBGd+5UXinjQuGs7tab15VaRRhgOowRE4cAZNuIEWuEDgEZ7hFd6sJ+vFerc+Zq0lq5jZhz+wPn8AmbCT/Q==</latexit><latexit sha1_base64="orN/A16ZqzHYB4nfSe/ir16mQXA=">AAACCnicbVDLSgMxFL1TX7W+qi7dhBbBhZQZEdSFUHTjsoJjC9OhZNJMG5rJDElGKEP3bvwVNy5U3PoF7vwb0+kstPWQXA7n3EtyT5BwprRtf1ulpeWV1bXyemVjc2t7p7q7d6/iVBLqkpjHshNgRTkT1NVMc9pJJMVRwGk7GF1P/fYDlYrF4k6PE+pHeCBYyAjWRupVayG6RJ59jMxx8jpP/F61bjfsHGiROAWpQ4FWr/rV7cckjajQhGOlPMdOtJ9hqRnhdFLppoommIzwgHqGChxR5Wf5LhN0aJQ+CmNprtAoV39PZDhSahwFpjPCeqjmvan4n+elOjz3MyaSVFNBZg+FKUc6RtNgUJ9JSjQfG4KJZOaviAyxxESb+ComBGd+5UXinjQuGs7tab15VaRRhgOowRE4cAZNuIEWuEDgEZ7hFd6sJ+vFerc+Zq0lq5jZhz+wPn8AmbCT/Q==</latexit><latexit sha1_base64="orN/A16ZqzHYB4nfSe/ir16mQXA=">AAACCnicbVDLSgMxFL1TX7W+qi7dhBbBhZQZEdSFUHTjsoJjC9OhZNJMG5rJDElGKEP3bvwVNy5U3PoF7vwb0+kstPWQXA7n3EtyT5BwprRtf1ulpeWV1bXyemVjc2t7p7q7d6/iVBLqkpjHshNgRTkT1NVMc9pJJMVRwGk7GF1P/fYDlYrF4k6PE+pHeCBYyAjWRupVayG6RJ59jMxx8jpP/F61bjfsHGiROAWpQ4FWr/rV7cckjajQhGOlPMdOtJ9hqRnhdFLppoommIzwgHqGChxR5Wf5LhN0aJQ+CmNprtAoV39PZDhSahwFpjPCeqjmvan4n+elOjz3MyaSVFNBZg+FKUc6RtNgUJ9JSjQfG4KJZOaviAyxxESb+ComBGd+5UXinjQuGs7tab15VaRRhgOowRE4cAZNuIEWuEDgEZ7hFd6sJ+vFerc+Zq0lq5jZhz+wPn8AmbCT/Q==</latexit><latexit sha1_base64="orN/A16ZqzHYB4nfSe/ir16mQXA=">AAACCnicbVDLSgMxFL1TX7W+qi7dhBbBhZQZEdSFUHTjsoJjC9OhZNJMG5rJDElGKEP3bvwVNy5U3PoF7vwb0+kstPWQXA7n3EtyT5BwprRtf1ulpeWV1bXyemVjc2t7p7q7d6/iVBLqkpjHshNgRTkT1NVMc9pJJMVRwGk7GF1P/fYDlYrF4k6PE+pHeCBYyAjWRupVayG6RJ59jMxx8jpP/F61bjfsHGiROAWpQ4FWr/rV7cckjajQhGOlPMdOtJ9hqRnhdFLppoommIzwgHqGChxR5Wf5LhN0aJQ+CmNprtAoV39PZDhSahwFpjPCeqjmvan4n+elOjz3MyaSVFNBZg+FKUc6RtNgUJ9JSjQfG4KJZOaviAyxxESb+ComBGd+5UXinjQuGs7tab15VaRRhgOowRE4cAZNuIEWuEDgEZ7hFd6sJ+vFerc+Zq0lq5jZhz+wPn8AmbCT/Q==</latexit>

y = XfT<latexit sha1_base64="pRN4evLHOreRm5ER1zkFskUF844=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqB6EohePFRobaGPZbDft0s0m7m6EEPonvHhQ8erv8ea/cdvmoK0PBh7vzTAzL0g4U9q2v63Syura+kZ5s7K1vbO7V90/uFdxKgl1Scxj6QVYUc4EdTXTnHqJpDgKOO0E45up33miUrFYtHWWUD/CQ8FCRrA2kpehK+SFD+1+tWbX7RnQMnEKUoMCrX71qzeISRpRoQnHSnUdO9F+jqVmhNNJpZcqmmAyxkPaNVTgiCo/n907QSdGGaAwlqaERjP190SOI6WyKDCdEdYjtehNxf+8bqrDCz9nIkk1FWS+KEw50jGaPo8GTFKieWYIJpKZWxEZYYmJNhFVTAjO4svLxD2rX9adu/Na87pIowxHcAyn4EADmnALLXCBAIdneIU369F6sd6tj3lrySpmDuEPrM8fSfaPBA==</latexit><latexit sha1_base64="pRN4evLHOreRm5ER1zkFskUF844=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqB6EohePFRobaGPZbDft0s0m7m6EEPonvHhQ8erv8ea/cdvmoK0PBh7vzTAzL0g4U9q2v63Syura+kZ5s7K1vbO7V90/uFdxKgl1Scxj6QVYUc4EdTXTnHqJpDgKOO0E45up33miUrFYtHWWUD/CQ8FCRrA2kpehK+SFD+1+tWbX7RnQMnEKUoMCrX71qzeISRpRoQnHSnUdO9F+jqVmhNNJpZcqmmAyxkPaNVTgiCo/n907QSdGGaAwlqaERjP190SOI6WyKDCdEdYjtehNxf+8bqrDCz9nIkk1FWS+KEw50jGaPo8GTFKieWYIJpKZWxEZYYmJNhFVTAjO4svLxD2rX9adu/Na87pIowxHcAyn4EADmnALLXCBAIdneIU369F6sd6tj3lrySpmDuEPrM8fSfaPBA==</latexit><latexit sha1_base64="pRN4evLHOreRm5ER1zkFskUF844=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqB6EohePFRobaGPZbDft0s0m7m6EEPonvHhQ8erv8ea/cdvmoK0PBh7vzTAzL0g4U9q2v63Syura+kZ5s7K1vbO7V90/uFdxKgl1Scxj6QVYUc4EdTXTnHqJpDgKOO0E45up33miUrFYtHWWUD/CQ8FCRrA2kpehK+SFD+1+tWbX7RnQMnEKUoMCrX71qzeISRpRoQnHSnUdO9F+jqVmhNNJpZcqmmAyxkPaNVTgiCo/n907QSdGGaAwlqaERjP190SOI6WyKDCdEdYjtehNxf+8bqrDCz9nIkk1FWS+KEw50jGaPo8GTFKieWYIJpKZWxEZYYmJNhFVTAjO4svLxD2rX9adu/Na87pIowxHcAyn4EADmnALLXCBAIdneIU369F6sd6tj3lrySpmDuEPrM8fSfaPBA==</latexit><latexit sha1_base64="pRN4evLHOreRm5ER1zkFskUF844=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqB6EohePFRobaGPZbDft0s0m7m6EEPonvHhQ8erv8ea/cdvmoK0PBh7vzTAzL0g4U9q2v63Syura+kZ5s7K1vbO7V90/uFdxKgl1Scxj6QVYUc4EdTXTnHqJpDgKOO0E45up33miUrFYtHWWUD/CQ8FCRrA2kpehK+SFD+1+tWbX7RnQMnEKUoMCrX71qzeISRpRoQnHSnUdO9F+jqVmhNNJpZcqmmAyxkPaNVTgiCo/n907QSdGGaAwlqaERjP190SOI6WyKDCdEdYjtehNxf+8bqrDCz9nIkk1FWS+KEw50jGaPo8GTFKieWYIJpKZWxEZYYmJNhFVTAjO4svLxD2rX9adu/Na87pIowxHcAyn4EADmnALLXCBAIdneIU369F6sd6tj3lrySpmDuEPrM8fSfaPBA==</latexit>

y = [0, 0, 0, 1, 1,�2]T<latexit sha1_base64="8nLsPuAsSL+BjApHEaL/aihB+zA=">AAACAXicbVDLSsNAFJ3UV62vqCtxM1gEF1qSIqgLoejGZYXGFtJYJtNJO3QyCTMTIYTixl9x40LFrX/hzr9xmmahrYc7cDjnXu7c48eMSmVZ30ZpYXFpeaW8Wllb39jcMrd37mSUCEwcHLFIdHwkCaOcOIoqRjqxICj0GWn7o+uJ334gQtKIt1QaEy9EA04DipHSUs/cS+EldK1jOC07r5O6d9/qmVWrZuWA88QuSBUUaPbMr24/wklIuMIMSenaVqy8DAlFMSPjSjeRJEZ4hAbE1ZSjkEgvy08Yw0Ot9GEQCf24grn6eyJDoZRp6OvOEKmhnPUm4n+em6jg3MsojxNFOJ4uChIGVQQnecA+FQQrlmqCsKD6rxAPkUBY6dQqOgR79uR54tRrFzX79rTauCrSKIN9cACOgA3OQAPcgCZwAAaP4Bm8gjfjyXgx3o2PaWvJKGZ2wR8Ynz+gJJKm</latexit><latexit sha1_base64="8nLsPuAsSL+BjApHEaL/aihB+zA=">AAACAXicbVDLSsNAFJ3UV62vqCtxM1gEF1qSIqgLoejGZYXGFtJYJtNJO3QyCTMTIYTixl9x40LFrX/hzr9xmmahrYc7cDjnXu7c48eMSmVZ30ZpYXFpeaW8Wllb39jcMrd37mSUCEwcHLFIdHwkCaOcOIoqRjqxICj0GWn7o+uJ334gQtKIt1QaEy9EA04DipHSUs/cS+EldK1jOC07r5O6d9/qmVWrZuWA88QuSBUUaPbMr24/wklIuMIMSenaVqy8DAlFMSPjSjeRJEZ4hAbE1ZSjkEgvy08Yw0Ot9GEQCf24grn6eyJDoZRp6OvOEKmhnPUm4n+em6jg3MsojxNFOJ4uChIGVQQnecA+FQQrlmqCsKD6rxAPkUBY6dQqOgR79uR54tRrFzX79rTauCrSKIN9cACOgA3OQAPcgCZwAAaP4Bm8gjfjyXgx3o2PaWvJKGZ2wR8Ynz+gJJKm</latexit><latexit sha1_base64="8nLsPuAsSL+BjApHEaL/aihB+zA=">AAACAXicbVDLSsNAFJ3UV62vqCtxM1gEF1qSIqgLoejGZYXGFtJYJtNJO3QyCTMTIYTixl9x40LFrX/hzr9xmmahrYc7cDjnXu7c48eMSmVZ30ZpYXFpeaW8Wllb39jcMrd37mSUCEwcHLFIdHwkCaOcOIoqRjqxICj0GWn7o+uJ334gQtKIt1QaEy9EA04DipHSUs/cS+EldK1jOC07r5O6d9/qmVWrZuWA88QuSBUUaPbMr24/wklIuMIMSenaVqy8DAlFMSPjSjeRJEZ4hAbE1ZSjkEgvy08Yw0Ot9GEQCf24grn6eyJDoZRp6OvOEKmhnPUm4n+em6jg3MsojxNFOJ4uChIGVQQnecA+FQQrlmqCsKD6rxAPkUBY6dQqOgR79uR54tRrFzX79rTauCrSKIN9cACOgA3OQAPcgCZwAAaP4Bm8gjfjyXgx3o2PaWvJKGZ2wR8Ynz+gJJKm</latexit><latexit sha1_base64="8nLsPuAsSL+BjApHEaL/aihB+zA=">AAACAXicbVDLSsNAFJ3UV62vqCtxM1gEF1qSIqgLoejGZYXGFtJYJtNJO3QyCTMTIYTixl9x40LFrX/hzr9xmmahrYc7cDjnXu7c48eMSmVZ30ZpYXFpeaW8Wllb39jcMrd37mSUCEwcHLFIdHwkCaOcOIoqRjqxICj0GWn7o+uJ334gQtKIt1QaEy9EA04DipHSUs/cS+EldK1jOC07r5O6d9/qmVWrZuWA88QuSBUUaPbMr24/wklIuMIMSenaVqy8DAlFMSPjSjeRJEZ4hAbE1ZSjkEgvy08Yw0Ot9GEQCf24grn6eyJDoZRp6OvOEKmhnPUm4n+em6jg3MsojxNFOJ4uChIGVQQnecA+FQQrlmqCsKD6rxAPkUBY6dQqOgR79uR54tRrFzX79rTauCrSKIN9cACOgA3OQAPcgCZwAAaP4Bm8gjfjyXgx3o2PaWvJKGZ2wR8Ynz+gJJKm</latexit>

Page 13: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Compressed sensing: full reconstructionWe can fully reconstruct any sparse signal if it was compressed by a matrix satisfying the Restricted Isoperimetric Property (RIP)

RIP guarantees a low distortion of the norm of any sparse representable vector

Full reconstruction is done by a solving linear problem: polynomial time

A random matrix with the characteristics of X is likely to satisfy the RIP condition

Page 14: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

CS: problem formulation

f̂ 2 argming:Xg=y

kgk0<latexit sha1_base64="pmHqRjyfxPvrobBarcsT/QYPvc4=">AAACG3icbVDLSsNAFJ34rPVVdelmsAiuSqKCDxCKblxWsLbQlDCZTtKhk0mYuRFLmg9x46+4caHiSnDh3zh9LLT1wIXDOfdy7z1+IrgG2/625uYXFpeWCyvF1bX1jc3S1vadjlNFWZ3GIlZNn2gmuGR14CBYM1GMRL5gDb93NfQb90xpHstb6CesHZFQ8oBTAkbySkdul0AW5NjlErsRgW6cZC6wB8iICiMu89zLwvNmeNE3PYPQHXi2VyrbFXsEPEucCSmjCWpe6dPtxDSNmAQqiNYtx06gbRYAp4LlRTfVLCG0R0LWMlSSiOl2Nnoux/tG6eAgVqYk4JH6eyIjkdb9yDedw/P1tDcU//NaKQSn7YzLJAUm6XhRkAoMMR4mhTtcMQqibwihiptbMe0SRSiYPIsmBGf65VlSP6ycVZyb43L1cpJGAe2iPXSAHHSCquga1VAdUfSIntErerOerBfr3foYt85Zk5kd9AfW1w+rgKKO</latexit><latexit sha1_base64="pmHqRjyfxPvrobBarcsT/QYPvc4=">AAACG3icbVDLSsNAFJ34rPVVdelmsAiuSqKCDxCKblxWsLbQlDCZTtKhk0mYuRFLmg9x46+4caHiSnDh3zh9LLT1wIXDOfdy7z1+IrgG2/625uYXFpeWCyvF1bX1jc3S1vadjlNFWZ3GIlZNn2gmuGR14CBYM1GMRL5gDb93NfQb90xpHstb6CesHZFQ8oBTAkbySkdul0AW5NjlErsRgW6cZC6wB8iICiMu89zLwvNmeNE3PYPQHXi2VyrbFXsEPEucCSmjCWpe6dPtxDSNmAQqiNYtx06gbRYAp4LlRTfVLCG0R0LWMlSSiOl2Nnoux/tG6eAgVqYk4JH6eyIjkdb9yDedw/P1tDcU//NaKQSn7YzLJAUm6XhRkAoMMR4mhTtcMQqibwihiptbMe0SRSiYPIsmBGf65VlSP6ycVZyb43L1cpJGAe2iPXSAHHSCquga1VAdUfSIntErerOerBfr3foYt85Zk5kd9AfW1w+rgKKO</latexit><latexit sha1_base64="pmHqRjyfxPvrobBarcsT/QYPvc4=">AAACG3icbVDLSsNAFJ34rPVVdelmsAiuSqKCDxCKblxWsLbQlDCZTtKhk0mYuRFLmg9x46+4caHiSnDh3zh9LLT1wIXDOfdy7z1+IrgG2/625uYXFpeWCyvF1bX1jc3S1vadjlNFWZ3GIlZNn2gmuGR14CBYM1GMRL5gDb93NfQb90xpHstb6CesHZFQ8oBTAkbySkdul0AW5NjlErsRgW6cZC6wB8iICiMu89zLwvNmeNE3PYPQHXi2VyrbFXsEPEucCSmjCWpe6dPtxDSNmAQqiNYtx06gbRYAp4LlRTfVLCG0R0LWMlSSiOl2Nnoux/tG6eAgVqYk4JH6eyIjkdb9yDedw/P1tDcU//NaKQSn7YzLJAUm6XhRkAoMMR4mhTtcMQqibwihiptbMe0SRSiYPIsmBGf65VlSP6ycVZyb43L1cpJGAe2iPXSAHHSCquga1VAdUfSIntErerOerBfr3foYt85Zk5kd9AfW1w+rgKKO</latexit><latexit sha1_base64="pmHqRjyfxPvrobBarcsT/QYPvc4=">AAACG3icbVDLSsNAFJ34rPVVdelmsAiuSqKCDxCKblxWsLbQlDCZTtKhk0mYuRFLmg9x46+4caHiSnDh3zh9LLT1wIXDOfdy7z1+IrgG2/625uYXFpeWCyvF1bX1jc3S1vadjlNFWZ3GIlZNn2gmuGR14CBYM1GMRL5gDb93NfQb90xpHstb6CesHZFQ8oBTAkbySkdul0AW5NjlErsRgW6cZC6wB8iICiMu89zLwvNmeNE3PYPQHXi2VyrbFXsEPEucCSmjCWpe6dPtxDSNmAQqiNYtx06gbRYAp4LlRTfVLCG0R0LWMlSSiOl2Nnoux/tG6eAgVqYk4JH6eyIjkdb9yDedw/P1tDcU//NaKQSn7YzLJAUm6XhRkAoMMR4mhTtcMQqibwihiptbMe0SRSiYPIsmBGf65VlSP6ycVZyb43L1cpJGAe2iPXSAHHSCquga1VAdUfSIntErerOerBfr3foYt85Zk5kd9AfW1w+rgKKO</latexit>

If X is RIP

kfk0 m<latexit sha1_base64="am+yA3MWYdKVmF202w6Dt/5pJNA=">AAAB9HicbVBNSwMxEM3Wr1q/qh69BIvgqeyKoN6KXjxWcG2hu5ZsOtuGJtk1ySpl2//hxYOKV3+MN/+N6cdBWx8MPN6bYWZelHKmjet+O4Wl5ZXVteJ6aWNza3unvLt3p5NMUfBpwhPVjIgGziT4hhkOzVQBERGHRtS/GvuNR1CaJfLWDFIIBelKFjNKjJXug2EcDNsuDjg8YNEuV9yqOwFeJN6MVNAM9Xb5K+gkNBMgDeVE65bnpibMiTKMchiVgkxDSmifdKFlqSQCdJhPrh7hI6t0cJwoW9Lgifp7IidC64GIbKcgpqfnvbH4n9fKTHwe5kymmQFJp4vijGOT4HEEuMMUUMMHlhCqmL0V0x5RhBobVMmG4M2/vEj8k+pF1bs5rdQuZ2kU0QE6RMfIQ2eohq5RHfmIIoWe0St6c56cF+fd+Zi2FpzZzD76A+fzBxg4kco=</latexit><latexit sha1_base64="am+yA3MWYdKVmF202w6Dt/5pJNA=">AAAB9HicbVBNSwMxEM3Wr1q/qh69BIvgqeyKoN6KXjxWcG2hu5ZsOtuGJtk1ySpl2//hxYOKV3+MN/+N6cdBWx8MPN6bYWZelHKmjet+O4Wl5ZXVteJ6aWNza3unvLt3p5NMUfBpwhPVjIgGziT4hhkOzVQBERGHRtS/GvuNR1CaJfLWDFIIBelKFjNKjJXug2EcDNsuDjg8YNEuV9yqOwFeJN6MVNAM9Xb5K+gkNBMgDeVE65bnpibMiTKMchiVgkxDSmifdKFlqSQCdJhPrh7hI6t0cJwoW9Lgifp7IidC64GIbKcgpqfnvbH4n9fKTHwe5kymmQFJp4vijGOT4HEEuMMUUMMHlhCqmL0V0x5RhBobVMmG4M2/vEj8k+pF1bs5rdQuZ2kU0QE6RMfIQ2eohq5RHfmIIoWe0St6c56cF+fd+Zi2FpzZzD76A+fzBxg4kco=</latexit><latexit sha1_base64="am+yA3MWYdKVmF202w6Dt/5pJNA=">AAAB9HicbVBNSwMxEM3Wr1q/qh69BIvgqeyKoN6KXjxWcG2hu5ZsOtuGJtk1ySpl2//hxYOKV3+MN/+N6cdBWx8MPN6bYWZelHKmjet+O4Wl5ZXVteJ6aWNza3unvLt3p5NMUfBpwhPVjIgGziT4hhkOzVQBERGHRtS/GvuNR1CaJfLWDFIIBelKFjNKjJXug2EcDNsuDjg8YNEuV9yqOwFeJN6MVNAM9Xb5K+gkNBMgDeVE65bnpibMiTKMchiVgkxDSmifdKFlqSQCdJhPrh7hI6t0cJwoW9Lgifp7IidC64GIbKcgpqfnvbH4n9fKTHwe5kymmQFJp4vijGOT4HEEuMMUUMMHlhCqmL0V0x5RhBobVMmG4M2/vEj8k+pF1bs5rdQuZ2kU0QE6RMfIQ2eohq5RHfmIIoWe0St6c56cF+fd+Zi2FpzZzD76A+fzBxg4kco=</latexit><latexit sha1_base64="am+yA3MWYdKVmF202w6Dt/5pJNA=">AAAB9HicbVBNSwMxEM3Wr1q/qh69BIvgqeyKoN6KXjxWcG2hu5ZsOtuGJtk1ySpl2//hxYOKV3+MN/+N6cdBWx8MPN6bYWZelHKmjet+O4Wl5ZXVteJ6aWNza3unvLt3p5NMUfBpwhPVjIgGziT4hhkOzVQBERGHRtS/GvuNR1CaJfLWDFIIBelKFjNKjJXug2EcDNsuDjg8YNEuV9yqOwFeJN6MVNAM9Xb5K+gkNBMgDeVE65bnpibMiTKMchiVgkxDSmifdKFlqSQCdJhPrh7hI6t0cJwoW9Lgifp7IidC64GIbKcgpqfnvbH4n9fKTHwe5kymmQFJp4vijGOT4HEEuMMUUMMHlhCqmL0V0x5RhBobVMmG4M2/vEj8k+pF1bs5rdQuZ2kU0QE6RMfIQ2eohq5RHfmIIoWe0St6c56cF+fd+Zi2FpzZzD76A+fzBxg4kco=</latexit>

)<latexit sha1_base64="apOjzIa4RvBwcVe1EmrYuZsBwiw=">AAAB8XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWMVYwttKJvtpl26yYbdiVJCf4YXDype/Tfe/Ddu2xy09cHA470ZZuaFqRQGXffbWVpeWV1bL22UN7e2d3Yre/sPRmWacZ8pqXQrpIZLkXAfBUreSjWncSh5MxxeT/zmI9dGqOQeRykPYtpPRCQYRSu1O3eiP0CqtXrqVqpuzZ2CLBKvIFUo0OhWvjo9xbKYJ8gkNabtuSkGOdUomOTjciczPKVsSPu8bWlCY26CfHrymBxbpUcipW0lSKbq74mcxsaM4tB2xhQHZt6biP957QyjiyAXSZohT9hsUZRJgopM/ic9oTlDObKEMi3srYQNqKYMbUplG4I3//Ii8U9rlzXv9qxavyrSKMEhHMEJeHAOdbiBBvjAQMEzvMKbg86L8+58zFqXnGLmAP7A+fwB+1uRQQ==</latexit><latexit sha1_base64="apOjzIa4RvBwcVe1EmrYuZsBwiw=">AAAB8XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWMVYwttKJvtpl26yYbdiVJCf4YXDype/Tfe/Ddu2xy09cHA470ZZuaFqRQGXffbWVpeWV1bL22UN7e2d3Yre/sPRmWacZ8pqXQrpIZLkXAfBUreSjWncSh5MxxeT/zmI9dGqOQeRykPYtpPRCQYRSu1O3eiP0CqtXrqVqpuzZ2CLBKvIFUo0OhWvjo9xbKYJ8gkNabtuSkGOdUomOTjciczPKVsSPu8bWlCY26CfHrymBxbpUcipW0lSKbq74mcxsaM4tB2xhQHZt6biP957QyjiyAXSZohT9hsUZRJgopM/ic9oTlDObKEMi3srYQNqKYMbUplG4I3//Ii8U9rlzXv9qxavyrSKMEhHMEJeHAOdbiBBvjAQMEzvMKbg86L8+58zFqXnGLmAP7A+fwB+1uRQQ==</latexit><latexit sha1_base64="apOjzIa4RvBwcVe1EmrYuZsBwiw=">AAAB8XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWMVYwttKJvtpl26yYbdiVJCf4YXDype/Tfe/Ddu2xy09cHA470ZZuaFqRQGXffbWVpeWV1bL22UN7e2d3Yre/sPRmWacZ8pqXQrpIZLkXAfBUreSjWncSh5MxxeT/zmI9dGqOQeRykPYtpPRCQYRSu1O3eiP0CqtXrqVqpuzZ2CLBKvIFUo0OhWvjo9xbKYJ8gkNabtuSkGOdUomOTjciczPKVsSPu8bWlCY26CfHrymBxbpUcipW0lSKbq74mcxsaM4tB2xhQHZt6biP957QyjiyAXSZohT9hsUZRJgopM/ic9oTlDObKEMi3srYQNqKYMbUplG4I3//Ii8U9rlzXv9qxavyrSKMEhHMEJeHAOdbiBBvjAQMEzvMKbg86L8+58zFqXnGLmAP7A+fwB+1uRQQ==</latexit><latexit sha1_base64="apOjzIa4RvBwcVe1EmrYuZsBwiw=">AAAB8XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWMVYwttKJvtpl26yYbdiVJCf4YXDype/Tfe/Ddu2xy09cHA470ZZuaFqRQGXffbWVpeWV1bL22UN7e2d3Yre/sPRmWacZ8pqXQrpIZLkXAfBUreSjWncSh5MxxeT/zmI9dGqOQeRykPYtpPRCQYRSu1O3eiP0CqtXrqVqpuzZ2CLBKvIFUo0OhWvjo9xbKYJ8gkNabtuSkGOdUomOTjciczPKVsSPu8bWlCY26CfHrymBxbpUcipW0lSKbq74mcxsaM4tB2xhQHZt6biP957QyjiyAXSZohT9hsUZRJgopM/ic9oTlDObKEMi3srYQNqKYMbUplG4I3//Ii8U9rlzXv9qxavyrSKMEhHMEJeHAOdbiBBvjAQMEzvMKbg86L8+58zFqXnGLmAP7A+fwB+1uRQQ==</latexit>

f̂ = f<latexit sha1_base64="qLac7PGSar98RwOyckxNI39rl9k=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW21A22027dLMJuxOhhP4LLx5UvPpzvPlv3LY5aPXBwOO9GWbmhakUBl33yyktLa+srpXXKxubW9s71d29e5NkmnGfJTLR7ZAaLoXiPgqUvJ1qTuNQ8lY4up76rUeujUjUHY5THsR0oEQkGEUrPXSHFElELknUq9bcujsD+Uu8gtSgQLNX/ez2E5bFXCGT1JiO56YY5FSjYJJPKt3M8JSyER3wjqWKxtwE+eziCTmySp9EibalkMzUnxM5jY0Zx6HtjCkOzaI3Ff/zOhlG50EuVJohV2y+KMokwYRM3yd9oTlDObaEMi3srYQNqaYMbUgVG4K3+PJf4p/UL+re7WmtcVWkUYYDOIRj8OAMGnADTfCBgYIneIFXxzjPzpvzPm8tOcXMPvyC8/ENhtiPtA==</latexit><latexit sha1_base64="qLac7PGSar98RwOyckxNI39rl9k=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW21A22027dLMJuxOhhP4LLx5UvPpzvPlv3LY5aPXBwOO9GWbmhakUBl33yyktLa+srpXXKxubW9s71d29e5NkmnGfJTLR7ZAaLoXiPgqUvJ1qTuNQ8lY4up76rUeujUjUHY5THsR0oEQkGEUrPXSHFElELknUq9bcujsD+Uu8gtSgQLNX/ez2E5bFXCGT1JiO56YY5FSjYJJPKt3M8JSyER3wjqWKxtwE+eziCTmySp9EibalkMzUnxM5jY0Zx6HtjCkOzaI3Ff/zOhlG50EuVJohV2y+KMokwYRM3yd9oTlDObaEMi3srYQNqaYMbUgVG4K3+PJf4p/UL+re7WmtcVWkUYYDOIRj8OAMGnADTfCBgYIneIFXxzjPzpvzPm8tOcXMPvyC8/ENhtiPtA==</latexit><latexit sha1_base64="qLac7PGSar98RwOyckxNI39rl9k=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW21A22027dLMJuxOhhP4LLx5UvPpzvPlv3LY5aPXBwOO9GWbmhakUBl33yyktLa+srpXXKxubW9s71d29e5NkmnGfJTLR7ZAaLoXiPgqUvJ1qTuNQ8lY4up76rUeujUjUHY5THsR0oEQkGEUrPXSHFElELknUq9bcujsD+Uu8gtSgQLNX/ez2E5bFXCGT1JiO56YY5FSjYJJPKt3M8JSyER3wjqWKxtwE+eziCTmySp9EibalkMzUnxM5jY0Zx6HtjCkOzaI3Ff/zOhlG50EuVJohV2y+KMokwYRM3yd9oTlDObaEMi3srYQNqaYMbUgVG4K3+PJf4p/UL+re7WmtcVWkUYYDOIRj8OAMGnADTfCBgYIneIFXxzjPzpvzPm8tOcXMPvyC8/ENhtiPtA==</latexit><latexit sha1_base64="qLac7PGSar98RwOyckxNI39rl9k=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW21A22027dLMJuxOhhP4LLx5UvPpzvPlv3LY5aPXBwOO9GWbmhakUBl33yyktLa+srpXXKxubW9s71d29e5NkmnGfJTLR7ZAaLoXiPgqUvJ1qTuNQ8lY4up76rUeujUjUHY5THsR0oEQkGEUrPXSHFElELknUq9bcujsD+Uu8gtSgQLNX/ez2E5bFXCGT1JiO56YY5FSjYJJPKt3M8JSyER3wjqWKxtwE+eziCTmySp9EibalkMzUnxM5jY0Zx6HtjCkOzaI3Ff/zOhlG50EuVJohV2y+KMokwYRM3yd9oTlDObaEMi3srYQNqaYMbUgVG4K3+PJf4p/UL+re7WmtcVWkUYYDOIRj8OAMGnADTfCBgYIneIFXxzjPzpvzPm8tOcXMPvyC8/ENhtiPtA==</latexit>

Page 15: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

CS: convex relaxation is tight

We can prove, under mild conditions, that

f = argming:Xg=y

kgk0 = argming:Xg=y

kgk1<latexit sha1_base64="dcyv8us7XcNtJNtodiWFL7dFCYM=">AAACPXicjVA9S8RAEN347fl1ammzeAhWRyKCH3Ag2lgqGD24HGGzt8ktbjZhdyKGXH6Zjb/BztLGQsXW1r0zhZ4WPhh4vDfDzLwgFVyDbT9aE5NT0zOzc/O1hcWl5ZX66tqlTjJFmUsTkah2QDQTXDIXOAjWThUjcSDYVXB9MvSvbpjSPJEXkKesG5NI8pBTAkby626IW9iLCfSTtPCA3UJBVBRzWZZ+ER22o1ZeYm8QeQPf/nen49cbdtMeAf8mTkUaqMKZX3/wegnNYiaBCqJ1x7FT6JoFwKlgZc3LNEsJvSYR6xgqScx0txi9X+Ito/RwmChTEvBI/T5RkFjrPA5M5/B8Pe4Nxb+8TgbhfrfgMs2ASfq1KMwEhgQPs8Q9rhgFkRtCqOLmVkz7RBEKJvGaCcEZf/k3cXeaB03nfLdxdFylMYc20CbaRg7aQ0foFJ0hF1F0h57QC3q17q1n6816/2qdsKqZdfQD1scnGF2wgA==</latexit><latexit sha1_base64="dcyv8us7XcNtJNtodiWFL7dFCYM=">AAACPXicjVA9S8RAEN347fl1ammzeAhWRyKCH3Ag2lgqGD24HGGzt8ktbjZhdyKGXH6Zjb/BztLGQsXW1r0zhZ4WPhh4vDfDzLwgFVyDbT9aE5NT0zOzc/O1hcWl5ZX66tqlTjJFmUsTkah2QDQTXDIXOAjWThUjcSDYVXB9MvSvbpjSPJEXkKesG5NI8pBTAkby626IW9iLCfSTtPCA3UJBVBRzWZZ+ER22o1ZeYm8QeQPf/nen49cbdtMeAf8mTkUaqMKZX3/wegnNYiaBCqJ1x7FT6JoFwKlgZc3LNEsJvSYR6xgqScx0txi9X+Ito/RwmChTEvBI/T5RkFjrPA5M5/B8Pe4Nxb+8TgbhfrfgMs2ASfq1KMwEhgQPs8Q9rhgFkRtCqOLmVkz7RBEKJvGaCcEZf/k3cXeaB03nfLdxdFylMYc20CbaRg7aQ0foFJ0hF1F0h57QC3q17q1n6816/2qdsKqZdfQD1scnGF2wgA==</latexit><latexit sha1_base64="dcyv8us7XcNtJNtodiWFL7dFCYM=">AAACPXicjVA9S8RAEN347fl1ammzeAhWRyKCH3Ag2lgqGD24HGGzt8ktbjZhdyKGXH6Zjb/BztLGQsXW1r0zhZ4WPhh4vDfDzLwgFVyDbT9aE5NT0zOzc/O1hcWl5ZX66tqlTjJFmUsTkah2QDQTXDIXOAjWThUjcSDYVXB9MvSvbpjSPJEXkKesG5NI8pBTAkby626IW9iLCfSTtPCA3UJBVBRzWZZ+ER22o1ZeYm8QeQPf/nen49cbdtMeAf8mTkUaqMKZX3/wegnNYiaBCqJ1x7FT6JoFwKlgZc3LNEsJvSYR6xgqScx0txi9X+Ito/RwmChTEvBI/T5RkFjrPA5M5/B8Pe4Nxb+8TgbhfrfgMs2ASfq1KMwEhgQPs8Q9rhgFkRtCqOLmVkz7RBEKJvGaCcEZf/k3cXeaB03nfLdxdFylMYc20CbaRg7aQ0foFJ0hF1F0h57QC3q17q1n6816/2qdsKqZdfQD1scnGF2wgA==</latexit><latexit sha1_base64="dcyv8us7XcNtJNtodiWFL7dFCYM=">AAACPXicjVA9S8RAEN347fl1ammzeAhWRyKCH3Ag2lgqGD24HGGzt8ktbjZhdyKGXH6Zjb/BztLGQsXW1r0zhZ4WPhh4vDfDzLwgFVyDbT9aE5NT0zOzc/O1hcWl5ZX66tqlTjJFmUsTkah2QDQTXDIXOAjWThUjcSDYVXB9MvSvbpjSPJEXkKesG5NI8pBTAkby626IW9iLCfSTtPCA3UJBVBRzWZZ+ER22o1ZeYm8QeQPf/nen49cbdtMeAf8mTkUaqMKZX3/wegnNYiaBCqJ1x7FT6JoFwKlgZc3LNEsJvSYR6xgqScx0txi9X+Ito/RwmChTEvBI/T5RkFjrPA5M5/B8Pe4Nxb+8TgbhfrfgMs2ASfq1KMwEhgQPs8Q9rhgFkRtCqOLmVkz7RBEKJvGaCcEZf/k3cXeaB03nfLdxdFylMYc20CbaRg7aQ0foFJ0hF1F0h57QC3q17q1n6816/2qdsKqZdfQD1scnGF2wgA==</latexit>

More information in Shai Shalev-Shwartz, Compressed Sensing: Basic results and self contained proofs

Page 16: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Matrix sketching

Going back to a dense data matrix A

We are going to see a way to “improve” PCA and the SVD

Page 17: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Matrix sketching: the SVD

Jeff M. Phillips, University of Utah

Page 18: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Matrix sketching: the SVD

Jeff M. Phillips,

PCA:

Page 19: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Approximation of A using the SVD

Columns of V are linear combinations of features

Interpretability of V

What is a linear combination of purchased books?

What is a linear combination between number of calls and being male or female?

Computational efficiency

Computing the SVD demands loading all A into memory and requires time

What about really big data?

What about streamed data?

Page 20: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Row sampling

Interpretability: choose V so that columns of V are columns of A