Image Pyramids {#tutorial_pyramids} ============== @prev_tutorial{tutorial_morph_lines_detection} @next_tutorial{tutorial_threshold} Goal ---- In this tutorial you will learn how to: - Use the OpenCV functions **pyrUp()** and **pyrDown()** to downsample or upsample a given image. Theory ------ @note The explanation below belongs to the book **Learning OpenCV** by Bradski and Kaehler. - Usually we need to convert an image to a size different than its original. For this, there are two possible options: -# *Upsize* the image (zoom in) or -# *Downsize* it (zoom out). - Although there is a *geometric transformation* function in OpenCV that -literally- resize an image (**resize** , which we will show in a future tutorial), in this section we analyze first the use of **Image Pyramids**, which are widely applied in a huge range of vision applications. ### Image Pyramid - An image pyramid is a collection of images - all arising from a single original image - that are successively downsampled until some desired stopping point is reached. - There are two common kinds of image pyramids: - **Gaussian pyramid:** Used to downsample images - **Laplacian pyramid:** Used to reconstruct an upsampled image from an image lower in the pyramid (with less resolution) - In this tutorial we'll use the *Gaussian pyramid*. #### Gaussian Pyramid - Imagine the pyramid as a set of layers in which the higher the layer, the smaller the size. ![](images/Pyramids_Tutorial_Pyramid_Theory.png) - Every layer is numbered from bottom to top, so layer \f$(i+1)\f$ (denoted as \f$G_{i+1}\f$ is smaller than layer \f$i\f$ (\f$G_{i}\f$). - To produce layer \f$(i+1)\f$ in the Gaussian pyramid, we do the following: - Convolve \f$G_{i}\f$ with a Gaussian kernel: \f[\frac{1}{16} \begin{bmatrix} 1 & 4 & 6 & 4 & 1 \\ 4 & 16 & 24 & 16 & 4 \\ 6 & 24 & 36 & 24 & 6 \\ 4 & 16 & 24 & 16 & 4 \\ 1 & 4 & 6 & 4 & 1 \end{bmatrix}\f] - Remove every even-numbered row and column. - You can easily notice that the resulting image will be exactly one-quarter the area of its predecessor. Iterating this process on the input image \f$G_{0}\f$ (original image) produces the entire pyramid. - The procedure above was useful to downsample an image. What if we want to make it bigger?: columns filled with zeros (\f$0 \f$) - First, upsize the image to twice the original in each dimension, with the new even rows and - Perform a convolution with the same kernel shown above (multiplied by 4) to approximate the values of the "missing pixels" - These two procedures (downsampling and upsampling as explained above) are implemented by the OpenCV functions **pyrUp()** and **pyrDown()** , as we will see in an example with the code below: @note When we reduce the size of an image, we are actually *losing* information of the image. Code ---- This tutorial code's is shown lines below. @add_toggle_cpp You can also download it from [here](https://raw.githubusercontent.com/opencv/opencv/3.4/samples/cpp/tutorial_code/ImgProc/Pyramids/Pyramids.cpp) @include samples/cpp/tutorial_code/ImgProc/Pyramids/Pyramids.cpp @end_toggle @add_toggle_java You can also download it from [here](https://raw.githubusercontent.com/opencv/opencv/3.4/samples/java/tutorial_code/ImgProc/Pyramids/Pyramids.java) @include samples/java/tutorial_code/ImgProc/Pyramids/Pyramids.java @end_toggle @add_toggle_python You can also download it from [here](https://raw.githubusercontent.com/opencv/opencv/3.4/samples/python/tutorial_code/imgProc/Pyramids/pyramids.py) @include samples/python/tutorial_code/imgProc/Pyramids/pyramids.py @end_toggle Explanation ----------- Let's check the general structure of the program: #### Load an image @add_toggle_cpp @snippet cpp/tutorial_code/ImgProc/Pyramids/Pyramids.cpp load @end_toggle @add_toggle_java @snippet java/tutorial_code/ImgProc/Pyramids/Pyramids.java load @end_toggle @add_toggle_python @snippet python/tutorial_code/imgProc/Pyramids/pyramids.py load @end_toggle #### Create window @add_toggle_cpp @snippet cpp/tutorial_code/ImgProc/Pyramids/Pyramids.cpp show_image @end_toggle @add_toggle_java @snippet java/tutorial_code/ImgProc/Pyramids/Pyramids.java show_image @end_toggle @add_toggle_python @snippet python/tutorial_code/imgProc/Pyramids/pyramids.py show_image @end_toggle #### Loop @add_toggle_cpp @snippet cpp/tutorial_code/ImgProc/Pyramids/Pyramids.cpp loop @end_toggle @add_toggle_java @snippet java/tutorial_code/ImgProc/Pyramids/Pyramids.java loop @end_toggle @add_toggle_python @snippet python/tutorial_code/imgProc/Pyramids/pyramids.py loop @end_toggle Perform an infinite loop waiting for user input. Our program exits if the user presses **ESC**. Besides, it has two options: - **Perform upsampling - Zoom 'i'n (after pressing 'i')** We use the function **pyrUp()** with three arguments: - *src*: The current and destination image (to be shown on screen, supposedly the double of the input image) - *Size( tmp.cols*2, tmp.rows\*2 )* : The destination size. Since we are upsampling, **pyrUp()** expects a size double than the input image (in this case *src*). @add_toggle_cpp @snippet cpp/tutorial_code/ImgProc/Pyramids/Pyramids.cpp pyrup @end_toggle @add_toggle_java @snippet java/tutorial_code/ImgProc/Pyramids/Pyramids.java pyrup @end_toggle @add_toggle_python @snippet python/tutorial_code/imgProc/Pyramids/pyramids.py pyrup @end_toggle - **Perform downsampling - Zoom 'o'ut (after pressing 'o')** We use the function **pyrDown()** with three arguments (similarly to **pyrUp()**): - *src*: The current and destination image (to be shown on screen, supposedly half the input image) - *Size( tmp.cols/2, tmp.rows/2 )* : The destination size. Since we are upsampling, **pyrDown()** expects half the size the input image (in this case *src*). @add_toggle_cpp @snippet cpp/tutorial_code/ImgProc/Pyramids/Pyramids.cpp pyrdown @end_toggle @add_toggle_java @snippet java/tutorial_code/ImgProc/Pyramids/Pyramids.java pyrdown @end_toggle @add_toggle_python @snippet python/tutorial_code/imgProc/Pyramids/pyramids.py pyrdown @end_toggle Notice that it is important that the input image can be divided by a factor of two (in both dimensions). Otherwise, an error will be shown. Results ------- - The program calls by default an image [chicky_512.png](https://raw.githubusercontent.com/opencv/opencv/3.4/samples/data/chicky_512.png) that comes in the `samples/data` folder. Notice that this image is \f$512 \times 512\f$, hence a downsample won't generate any error (\f$512 = 2^{9}\f$). The original image is shown below: ![](images/Pyramids_Tutorial_Original_Image.jpg) - First we apply two successive **pyrDown()** operations by pressing 'd'. Our output is: ![](images/Pyramids_Tutorial_PyrDown_Result.jpg) - Note that we should have lost some resolution due to the fact that we are diminishing the size of the image. This is evident after we apply **pyrUp()** twice (by pressing 'u'). Our output is now: ![](images/Pyramids_Tutorial_PyrUp_Result.jpg)