• course / assignmnet overview
• basic image i/o (PPM, PGM images)
• typical image processing nested for loop
• 2D access to 1D array
• "wraparound indices"
• periodic image extension

• OpenMP multi-core parallelization
• basic convolution
• separable filters
• Sobel operators, see also: separable filters

• omp.tar.gz
• convolution.pdf
• numpy.convolve
• linear convolution in C

• Fourier Transform
• basis vectors and functions
• orthogonality and the dot product
• projection onto basis

• example "bar chart" representation by basis functions
• properties of basis functions
• solving for coefficients
• infinite support vs. compact support

• the Discrete Wavelet Tranform (DWT)
• scaling, dilating, shifting
• unit square example
• recipe for wavelet function from scaling function:
  \(\psi(x) = \sum_{k}(-1)^{k}c_{k+1}\phi(2x - k)\)
• orthogonality of \(\phi\) and \(\psi\):
  \(\langle\phi,\psi\rangle = 0\)
• recursive decomposition

• wavelets.pdf
• asg01 tarball

• schematic for 1D wavelet decomposition with FIR filters
• Haar wavelets
• quadrature mirror filters
• reconstruction
• numerical example
• schematic for 2D wavelet decomposition with FIR filters

• imaging pipeline
• when to normalize image

• dwt code, main.c
• pgm_idwt2d() call
• make_wavelets(HAAR)
• pgm_dwt2d() details (two-pass DWT)
• extra things to do for DWT assignment
  • MSE against original image
  • edge detection via modulus maxima
  • compression simulation via wavelet coefficient decimation
  • anisotropic filtering via wavelet coefficient decimation

• intro to video processing
• video files as containers (of streams)
• ffmpeg, ffplay, ffprobe
• MPEG encoding

• img.py
• cam.py
• vid.py
• vidcopy.py

• glcam.py

• short (e.g., 3 min) PowerPoint presentation
• show images produced
• show interesting stuff you implemented
• give feedback, e.g., lessons learned, what was fun, what wasn't, ...

Face detection

• Viola-Jones detector
• integral image representation, Haar-like feature sets
• machine learning approach
• cascaded combination of classifiers
• how to implement

Supplemental

• Viola, P. and Jones, M., Rapid Object Detection using a Boosted Cascade of Simple Features, in Proceedings of Computer Vision and Pattern Recognition, December 8-14, 2001, Kauai, HI.

Face detection and tracking

• frame coherence (more or less) patterened after particle systems
  • face object is born
  • face object grows old, i.e., decays, decay = \(e^{-ka}\) where \(k\) is some constant (e.g., \(0.05\)) and \(a\) is age
  • face object dies

• Duchowski, A. T., Gehrer, N. A., Schönenberg, M., and Krejtz, K., Art Facing Science: Artistic Heuristics for Face Detection, in Proceedings of the 4th Workshop on Eye Tracking and Visualization (ETVIS), June 25-28, 2019, Denver, CO.

• quick review of affine transoformations (rotations, translations)
• intro to homography
• solving the homography

• Camera Calib
• Homographies
• Transformations

• what is the homography matrix
• intrinsic parameters
• extrinsic parameters
• distortion parameters
• camera calibration in OpenCV

• Haar Cascades

Use: The ACM Publishing System (TAPS) Best Practices [templates]

DEMO DAY: Students present their face detection & tracking work

• short (e.g., 3 min) PowerPoint presentation
• show videos produced
• show interesting stuff you implemented
• give feedback, e.g., lessons learned, what was fun, what wasn't, ...

OpenCV camera calibration and pose estimation

• chessboard parameters: know its configuration (important)
• K, dist instrinsic parameters
• rvec, tvec rotation, translation
• pose estimation via drawAxes or projectPoints
• projection using OpenGL

More on camera calibration

• how to use arrays of objpoints = [] and imgpoints = []
• solving for rvec, tvec every frame with solvePnP()
• analogy to 2D calibration using minimization of least squares

OpenCV ArUco marker detection

• marker parameters: know its dictionary and size
• using aruco.detectMarkers() to find markers
• using aruco.estimatePoseSingleMarkers() to solve for rvec, tvec

• ellipse fitting
• KLT computation
• block coding

• Andrist, S., Gleicher, M., and Mutlu, B., Looking Coordinated: Bidirectional Gaze Mechanisms for Collaborative Interaction with Virtual Characters, in Proceedings of CHI 2017, May 6-11, 2017, Denver, CO.
• Garrido-Jurado, S., Munoz-Salinas, R., Madrid-Cuevas, F. J., and Marin-Jimenez, M. J., Automatic generation and detection of highly reliable fiducial markers under occlusion, Pattern Recgonition, 47, 2014.
• Romero-Ramirez, F. J., Munoz-Salinas, R., and Medina-Carnicer, R., Speeded Up Detection of Squared Fiducial Markers, Image and Vision Computing, 2018.
• Romero-Ramirez, F. J., Munoz-Salinas, R., and Medina-Carnicer, R., Fractal Markers: A New Approach for Long-Range Marker Pose Estimation Under Occlusion, IEEE Access, 2019.

Computational Photography

• Fredo Durand's EG 2017 keynote address

• bilateral filtering PPTX and example
• High-Dynamic Range (HDR) tone mapping
  • C code
  • OpenCV example
• Raman and Chaudhuri, Bilateral Filter Based Compositing for Variable Exposure Photography, EuroGraphics 2009.
• Paris and Durand, A Fast Approximation of the Bilateral Filter using a Signal Processing Approach, MIT Techreport, 2006.
• Durand and Dorsey, Fast Bilateral Filtering for the Display of High-Dynamic-Range Images, SIGGRAPH 2002.

• Santella, A. and DeCarlo, D., Robust Clustering of Eye Movement Recordings for Quantification of Visual Interest, ETRA, 2004.
• Fitzgibbon, A. W., Pilu, M., and Fisher, R. B., Direct Least Squares Fitting of Ellipses, IEEE PAMI, 21(5), May, 1999.
• Halir, R., Flusser, J., Numerically Stable Direct Least Squares Fitting of Ellipses, 1998.

• Stereo registration
• Canny edge detection
• SIFT
• Kalman filter

• hand in a short writeup and/or a web page

• basic image operations, i/o
• OpenMP parallelization
• edge detection, convolution, Finite Impulse Reponse (FIR) filters
• Sobel edge detection filters
• Fourier Transform (FT), basis vectors
• Discrete Wavelet Transform (DWT), wavelet filters
• basic video processing, ffmpeg
• OpenCV, video i/o
• face detection & tracking
• camera calibration, homography
• fiducial marker detection & tracking
• Karhunen-Loeve Transform (KLT) / Hotelling Transform
• Principle Component Analysis (PCA), ellipse fitting
• Computer Vision papers:
  • stereo registration
  • Canny edge detection
  • Kalman filter
• Compuattional Photography
• High Dynamic Range (HDR) tone mapping
• bilateral (Infinite Impulse Reponse, IIR) filter

• go through each others' projects before final presentations