Selection in 3D Graphics Environments
Christine Albert and Lindsey Press

Because this research is looking at the selection of objects, we next began to implement color-based picking. This process involved re-rendering the image into the frame buffer when the mouse clicks a point on the screen, and determining the color under the mouse. From there, the cube's unique color under the mouse is used to highlight that color, which entails the whole cube the user has clicked.

The changes made to the code to implement color-based picking are as follows:



Step 1: Added a MouseButton method to re-render the image into the frame buffer. The frame buffer is not swapped with the display buffer when this is done in order to preserve the fully colored image on the screen.

void MouseButton(GLFWWindow* window, int button, int action, int mods)
{
      double xpos, ypos;
      GLint viewport[4];
      GLubyte pixel[4] = {0,0,0,0};

      if (button == GLFW_MOUSE_BUTTON_LEFT && action == GLFW_PRESS)
      {
           glClearColor(0, 0, 0, 1); //black
           glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
           //bind the program (the shaders)
           gProgram->use();

           //bind the VAO (the triangle)
           glBindVertexArray(gVAO);

           for(int i = 0; i < NUMCUBES; i++)
           {
                gProgram->setUniform("model", cubes[i].model);
                gProgram->setUniform("color", GetColorByIndex(cubes[i].index));
                glDrawArrays(GL_TRIANGLES, 0, 6*2*3);
           }
           glfwGetCursorPos(window, &xpos, &ypos);
           //printf("%f %f\n", xpos, ypos);
           glReadPixels(xpos, SCREEN_SIZE.y-ypos, 1, 1, GL_RGBA, GL_UNSIGNED_BYTE, pixel);
           if(pixel[0] == 0 && pixel[1] == 0 && pixel[2] == 0)
                clickedindex = -1;
           else
                clickedindex = GetIndexByColor(pixel[0],pixel[1],pixel[2]);
           //printf("%x %x %x\n", pixel[0], pixel[1], pixel[2]);
           printf("index = %d\n", clickedindex);

           //unbind the VAO
           glBindVertexArray(0);

           //unbind the program
           gProgram->stopUsing();
           /* Do NOT swap the buffers so the image does not appear on the screen */
      }
}

This method, MouseButton, has parameters for a GLFW window, and integer values for button, action, and mods. The body of the code starts by stating that if the left mouse button is clicked, the rest of the code in the method will continue to execute. The glClearColor method is invoked and sets the color to black when the buffer is cleared. Following this, the glClear method is invoked to clear the buffers to the preset values. The next step is to bind the program shaders, and then to bind the VAO (vertex array object), which is the triangle. At this point, the for loop begins to execute. It runs from 0 to the NUMCUBES, which is set to 600 at the beginning of the program. For every cube, a uniform model is set by index and a uniform color is set by index. This uniform setting of color is displayed in the image at the bottom of the webpage. Because it is set by index, the first 255 cubes are varying shades of red, and the remaining of the cubes begin to have green added. This image is never seen by the user because the buffers are never swapped, and this underlying color is a method of setting the cube color based on the index and using that to select the cube that has been clicked. The glDrawArrays method is then invoked, still within the for loop for every cube, and renders primitivves from the array data. Here, it is rendering triangles starting at the array index 0 and it will render 6*2*3 indicies.
At this point, the code exits the for loop. The method GetCursorPos is used, and the address of the x and y positions are obtained from the window the user clicks. The pixels are then read, and if the 0, 1, and 2 pixel positions are set to 0, then the clickedindex is changed to -1, meaning the cube has not been clicked. Otherwise, the clickedindex is set to the index of the cube using the RGB values of the pixel location. This goes back to the GetIndexByColor method described in the previous version of code, and means the cube has been clicked. The index value is then printed, and the VAO is unbound, and after this the program is unbound. There is a comment to reinforce the importance of not swapping the buffers; this prevents this image of the red cubes from appearing on the screen, and the original image of colored cubes remains on display as all of this happens in the background.

Step 2: Modified the render method, and if a cube has been clicked it is drawn in red. The changes from the previous stage of code are in bold.

static void Render(){

...

for(int i = 0; i < NUMCUBES; i++)

{

      gProgram->setUniform("model", cubes[i].model);

      if(clickedIndex == i)

           gProgram->setUniform("color", vec3(1.0,0.0,0.0)); // draw selected cube in red

      else

           gProgram->setUniform("color", cubes[i].color); //d draw as normal

      glDrawArrays(GL_TRIANGLES, 0, 6*2*3);

}

...

}

The changes added to the Render method function to incorporate the mouse clicking into the rendering. This allows for the cube that is clicked to be drawn in red, 'highlighting' it when it is selected. The if/else statement added checks to see if the clickedindex is the value of the cube's index; if so, it has been clicked and the color is redrawn as red to highlight the selected cube. If the clickedindex is not the value of the index, so if it is -1, the cube color is drawn as normal when the frame is re-rendered. The final result of this addition is that any cube that has been clicked will appear highlighted as red, and all of the unclicked cubes will remain their original unchanged colors.

Step 3: Changes to the AppMain method to implement the clicking interaction with an open window. Changes from the previous version of code are in bold.

void AppMain() {

...

      glfwSetMouseButtonCallback(gWindow, MouseButton);

...

      //run while the window is open

      while((glfwGetKey(gWindow, GLFW_KEY_ESCAPE ) != GLFW_PRESS) && !glfwWindowShouldClose(gwindow)){

           //process pending events

           glfwPollEvents();

...

}

The line added underneath the comment to run while the window is open is an adjustment to match the newest version of glfw. It states that if the escape key is not pressed and if the window is not being closed, to continue processing. The change right above it, which is the SetMouseButtonCallback method, which sets the mouse button callback of the specified window. This means that when a mouse button is pressed or released, the program is notified before going on to render the image.

Underlying Red Cubes



Resources:

http://www.ntu.edu.sg/home/ehchua/programming/opengl/cg_basicstheory.html