Monday, February 22, 2010

Drawing a HUD Display in OpenGL ES

One thing that I've been asked by a number of people is how to do a "heads up display" (or HUD) when using OpenGL to display three-dimensional objects inside a perspective viewport. Many games and other programs need to present certain data to the user as if it were written right on the screen in front of the three-dimensional world. But, if you've used glFrustum() to set up your viewport it won't look like that, since any drawing done in a perspective viewport will experience some amount of distortion when filtered through the projection matrix.

The answer to this problem is relatively straightforward and simple. You just don't let the HUD drawing go through the same projection matrix as the three-dimensional objects. Under the hood, the project view matrix is just a 4x4 matrix of GLfloats, exactly like the model view matrix we use to transform objects in our virtual world. And, just like the model view matrix, we can make changes to the projection matrix between drawing calls.

To illustrate the point, I'm going to take the old icosahedron project from the OpenGL ES from the Ground Up series and do some two-dimensional drawing on top of the icosahedron, exactly the same way you would draw a HUD. To keep things simple, I'm just going to draw a few squares and a bit of text, like so:
Screen shot 2010-02-22 at 2.35.14 PM.png

The three squares and text will be drawn using an orthographic projection matrix, so no matter how the object behind it is transformed, moved, or projected, the three squares and the text will be drawn exactly the same with no distortion or movement. You can use this exact same technique to draw the player's score, the crosshairs of a gun, a spaceship's controls, or any other drawing that needs to appear as if it was on a piece of glass in front of the virtual world.

Before we begin, let's look at the setupView: function where we set up the viewport in the old icosahedron project:

GLfloat size;
const GLfloat zNear = 0.1,
zFar = 1000.0,
fieldOfView = 60.0;
size = zNear * tanf(DEGREES_TO_RADIANS(fieldOfView) / 2.0);
CGRect rect = self.view.bounds;
glFrustumf(-size, size, -size / (rect.size.width / rect.size.height), size /
(rect.size.width / rect.size.height), zNear, zFar);

glViewport(0, 0, rect.size.width, rect.size.height);
glClearColor(0.0f, 0.0f, 0.0f, 1.0f);
colors = [[NSMutableArray alloc] initWithCapacity:20];
for (int i =0; i < 20; i++)
[colors addObject:[UIColor randomColor]];

It's pretty much a bog-standard perspective viewport setup that will allow our three dimensional objects to get smaller as they move away from the viewer. Everything we draw will be distorted to represent perspective, which we don't want for our HUD elements. So… let's create a couple of convenience methods to get out of perspective mode and into orthographic mode and vice versa

First, to switch the projection to orthographic mode, we'll use this method:

glOrthof(0, self.view.bounds.size.width, 0, self.view.bounds.size.height, -5, 1);

This is pretty simple, but let's break it down line-by-line. The first line is important. We don't want depth testing if we're drawing in two dimensions because we want all of our drawing done on the same plane with no depth.

Next, we call glMatrixMode() to make the projection matrix active. After that, we push the existing projection matrix onto the matrix stack so that we can restore it later. When this gets called, the projection matrix contains a matrix that was generated by our call to glFrustum(), and we need to be able to get back to it later. We then load the identify identity and then set up an orthographic projection where every screen pixel equates to one OpenGL unit, which will make our two-dimensional drawing easier since each unit will equal one pixel.

After that, we load the identity matrix into our model view matrix so we start drawing our HUD with a clean slate, unaffected by any previously used transformations.

In addition to a method to get into orthographic mode, we also need a method to restore our previous projection matrix, the one that we created in setupView:. Since we pushed the existing projection matrix onto the matrix stack earlier, All we have to do here is re-eneable depth-testing and pop our old projection matrix off the matrix stack:


Now we have the ability to switch back and forth between perspective and orthographic mode, so let's do some drawing. Here's the drawing method from the icosahedron project with the new HUD drawing code drawn in bold. I've used Apple's Texture2D class to handle the text drawing. I've done this just for simplicity - it's far from the most efficient way to draw text in OpenGL ES. In fact, it's quite inefficient, but it's easy and free. Perhaps I'll focus on better ways to draw text in a future posting, but this post is about HUDs, not about text, so we can live with a little inefficiency.

- (void)drawView:(GLView*)view;

static GLfloat rico;
static const GLfloat icosahedronVertices[]= {
0, -0.525731, 0.850651,
0.850651, 0, 0.525731,
0.850651, 0, -0.525731,
-0.850651, 0, -0.525731,
-0.850651, 0, 0.525731,
-0.525731, 0.850651, 0,
0.525731, 0.850651, 0,
0.525731, -0.850651, 0,
-0.525731, -0.850651, 0,
0, -0.525731, -0.850651,
0, 0.525731, -0.850651,
0, 0.525731, 0.850651,
static const GLubyte icosahedronFaces[] = {
1, 2, 6,
1, 7, 2,
3, 4, 5,
4, 3, 8,
6, 5, 11,
5, 6, 10,
9, 10, 2,
10, 9, 3,
7, 8, 9,
8, 7, 0,
11, 0, 1,
0, 11, 4,
6, 2, 10,
1, 6, 11,
3, 5, 10,
5, 4, 11,
2, 7, 9,
7, 1, 0,
3, 9, 8,
4, 8, 0,
static const GLubyte icosahedronNumberOfFaces = 60;

// Icosahedron

glVertexPointer(3, GL_FLOAT, 0, icosahedronVertices);
for(int i = 0; i < icosahedronNumberOfFaces; i += 3)
UIColor *oneColor = [colors objectAtIndex:i/3];
[oneColor setOpenGLColor];

glDrawElements(GL_TRIANGLES, 3, GL_UNSIGNED_BYTE, &icosahedronFaces[i]);


static NSTimeInterval lastDrawTime;
if (lastDrawTime)
NSTimeInterval timeSinceLastDraw = [NSDate timeIntervalSinceReferenceDate] - lastDrawTime;
rico+=50 * timeSinceLastDraw;

// ------------------------------------------------
// Draw HUD ---------------------------------------
// ------------------------------------------------
[self switchToOrtho];

static const GLfloat squareVertices[] = {
5.0f, 150.0f,
5.0f, 250.0f,
100.0f, 250.0f,
100.0f, 150.0f

glColor4f(0.0, 0.0, 1.0, 1.0); // blue
glTranslatef(5.0, 0.0, 0.0);
glVertexPointer(2, GL_FLOAT, 0, squareVertices);

glDrawArrays(GL_LINE_LOOP, 0, 4);
glTranslatef(100.0, 0.0, 0.0);
glColor4f(1.0, 0.0, 0.0, 1.0); // Red
glDrawArrays(GL_LINE_LOOP, 0, 4);
glTranslatef(100.0, 0.0, 0.0);
glColor4f(1.0, 1.0, 0.0, 1.0); // Yellow
glDrawArrays(GL_LINE_LOOP, 0, 4);

glBlendFunc (GL_ONE, GL_ONE);
glColor4f(1.0, 1.0, 1.0, 1.0);
Texture2D *textTex = [[Texture2D alloc] initWithString:@"Text"
dimensions:CGSizeMake(100., 40.0)
font:[UIFont boldSystemFontOfSize:36.0]
[textTex drawAtPoint:CGPointMake(160.0, 440.0) depth:-1];

[self switchBackToFrustum];

lastDrawTime = [NSDate timeIntervalSinceReferenceDate];

We start off by calling our switchToOrtho method. We then define vertices for a square that will be drawn using GL_LINE_LOOP and submit those vertices using glVertexPointer(). After that, we call glDrawArrays() three times, changing the color and doing a translate transform between the calls so that each square gets drawn to the right of the previous one and in a different color.

After that, we use Texture2D to draw the word "Text" on the screen. Because this class essentially draws the text into a graphic context then creates a texture out of it, we have to enable (and then disable) a bunch of texture-related options.

After we're all done drawing, we call the other method we created, switchBackToFrustum, to switch back to perspective mode so that the next time our drawView: method is called, we're back in three-dimensional mode.

And that is all there is to drawing a HUD over a perspective viewport. You can download the Xcode project right here and play with it yourself.


C Harrison said...

Your tutorials (both on-line and in your books) are consistently well done and useful.

I could have used this technique in a previous project but I'm sure it will come in handy in the future too.

Charybdis said...

Useful post! I wasn't sure if the standard HUD drawing technique was to just draw at a higher z axis value, or to actually change to ortho mode.

On a somewhat related note, I'm wondering how to do menu overlays on an OpenGL background. Is it better to make a new UIView and put it on top of the OpenGL view, and place the buttons on the over-layed view, or is there a better method? In Doodle Jump it seems as if they used this method, but it isn't exactly clear.

I'd like to make a game with a score listed on top, and the Texture2D method looks quite simple, so I'll try that for now. Thanks for the tips!

a random John said...


Great post. I attended NSConf on Sunday and when you were mentioning setting up a viewing frustum I was going to mention that somebody had ported gluLookAt from Mesa and then I looked at where I had gotten the code from and it was you. :)

The HUD stuff is interesting. There's more than one way to do it depending on what you want to achieve. In my game Battle for Vesta I actually use two methods. One is the method you use here, drawing the scene first and then drawing again in ortho mode.

However in addition to a 2D overlay I wanted to be able to highlight certain objects and have it look like that is a part of the overlay. What I did was draw the scene in 3D, and then do a second pass through the objects that needed to be highlighted. I turned off depth detection and drew brackets around them. This seems simple initially, but it isn't as easy as just drawing some boxes as you have to know the viewer's up vector and also the dot product of the up vector and the forward vector in order draw in a plane that is perpendicular to the viewer no matter how the viewer is oriented.

Once all that is done I draw the boxes and an onscreen radar. Then I switch to ortho mode and draw the 2D stuff.

There's a video ans screenshots at the link above that shows footage from the game so you can see the effect I'm describing. I should probably do a blog post on it at some point and publish the relevant code.

warmi said...

John ... nice ... I like well executed yet simple graphically games on the iPhone ...looks like a lot of fun.
If you want your quads to live in "3d" space then billboarding was the right way to go but if you just wanted 2d objects overlayed on top of 3d objects then it would have been easier to project back your 3d object to viewport coordinates and draw your 2 shapes based on that.

a random John said...


Any links on how to project from an 3D projection set up using gluLookAt to a 2D one? I had considered that approach but frankly it was easier for me to draw it in 3D space and just make sure it was perpendicular to the viewer rather than run the transforms myself.

Also for some things I do want the quads to live in 3D. For instance if something is past where I've going to draw the 3D object I don't want to bother drawing the heads up either. That gets handled automatically for me as well as all the scaling and such, though I'm assuming that an appropriate transform would hand that as well.


Frederic said...

Hi Jeff,
cheers from France.

I really like your tutorials. Very well done! Now I have a question. I have a program running on PC in OpenGL, a simple camera moving on a sort of fractal terrain and I would like to do the same on iPhone. But it is in landscape mode, not portrait. Can you tell what would be the best way to do a landscape display, rather than portrait ? Would it be to swap X and Y at render time, or any other method ?


Dynamic Methods said...

Hello Jeff,

I using glpaint example in my project. can provide guideline for eraser functionality of drawing.


Dynamic Methods said...

Hello Jeff,

I using glpaint example in my project. can provide guideline for eraser functionality of drawing.


Edwin said...

scrub m65 kamagra attorney lawyer body scrub field jacket lovegra marijuana attorney injury lawyer

janani said...


I'm in an exactly opposite situation.I need to draw opengl drawings on top of the live camera view.I couldnt set the background for opengl to be transparent so that I can view the underlying camera.

brinkka said...

I recently came across your blog and have been reading along. I think I will leave my first comment. I don’t know what to say except that I have enjoyed reading. Nice blog. I will keep visiting this blog very often.
By :

SEO Services Consultants said...

Nice information, many thanks to the author. It is incomprehensible to me now, but in general, the usefulness and significance is overwhelming. Thanks again and good luck! Web Design Company

pedro velasquez said...

Bitmap fonts are ideally suited for scale anddominical costa rica tours rotation-independent labeling of items within a scene. Since bitmap fonts are by nature pre-rasterized, they render very quickly, making them a good choice where speed is important.
The location of a bitmap font is determined by

healthpharmacyrx1 said...

plavix Kaufen
cipro Rezeptfrei Kaufen
accutane Rezeptfrei Bestellen
cialis soft acheter
xenical Rezeptfrei Bestellen
proscar Ohne Rezept
kamagra oral jelly Bestellen Rezeptfrei
accutane Prix de Vente
vermox Ohne Rezept
Ohne Rezept viagra soft
strattera Rezeptfrei
flagyl Bestellen
achat proscar france
Rezeptfrei lovegra