Qt5 OpenGL Part 3b: Camera Control 13


As we learned in Part 2, there is something known as the Transformation Pipeline which all of our points move through. This pipeline allows us to translate points around to give the geometry the sense that it’s moving. In a similar sense, we can perform the same kind of math to approximate a camera. The idea is the same, we’re going to move points around such that they are with respect to another vector space. The only real difference is that how we calculate the final worldToCamera matrix will be a little different than what our custom Transform3D class does.

This will probably be a pretty big tutorial, because we still have to cover the Input class, which I alluded to the implementation we would use in Part 3a. It will contain a lot of dense sections of code. So get comfy, this is going to take a while.

The Camera Matrix

So we saw in Part 2 that the Transformation Matrix for any given object can be calculated by Identity * Translation * Rotation * Scale, and that the order is important because we are in row-major format. We didn’t really discuss the math behind these transformations, but for the average user this should be enough to understand that we are combining transformation information. What this does is translates things so that they are in respect to the world. That is why this is known as the modelToWorld matrix.

Cameras are a little different. Think about the modelToWorld transformation for a second; there are two things a point can be in-respect to in this transformation, the model, and the world. We are moving information from the model, to the world. So if we imagine a camera as an object, and we want to get the world to be in respect of the camera, but we are not really moving the object from it’s position in the world, to the position of the camera. What we realize is that this is not quite the same transformation as what we would compute for modelToWorld. Simply put: We can’t have just another Transform3D which represents the camera and use that. It wont put the world in respect to the camera, it would put the camera in respect to the world.

However, if we know a little Linear Algebra, we can infer the following:

modelToWorld^{-1} = worldToModel

Thus the simplest camera we can implement at this time is a Transform3D, which in the end, instead of just passing the matrix to the shader, does the following:

You can try it if you like, but in the end it’s better to opt for making a custom Camera3D class. This act of building and inverting a matrix that I mention is possible for Transform3D contains needless computation. Instead we should simply build the transformation matrix to form the worldToCamera instead of forming the cameraToWorld matrix.

Right-Handed / Left-Handed Coordinate Systems

The next important thing to consider is how we’re going to move this camera around. When dealing with a 3D environment, it’s all about how easy and convenient you’ve made your Transform3D class. Ours is handy, but yet incomplete. We forgot one of the most important bits of information that the Transform3D implicitly holds; the forward, up, and right vector.

The forward, up, and right vectors define what vector direction corresponds to the given values. Locally, what is the direction of forward generically for any object? And then if we want to find what direction forward is in terms of the world, we only need to transform our vector by the object’s rotation (since scale and translation don’t matter for directions). As you can imagine, this is very important information logically. We can say handy things like “Rotate about your up vector”. And no matter how that object is oriented, it will rotate properly.

But I’m getting ahead of myself, first we need to define our coordinate system. The common decision you have to make is between one of two coordinate systems: Right-Handed, and Left-Handed.

Arrows point towards positive values.

The reason for such names is because we can form the coordinate systems physically using our fingers. Take the above picture as an example, let’s practice by forming the Right-Handed coordinate system.

  • First, make a fist and hold your hand up with your palm facing towards your face.
  • Then extend your thumb so that it points out to the right. At this point you should be holding a side-ways thumbs-up.
  • After that, extend your index finger so that it is pointing as straight as possible upwards. Now your fingers should be forming an L shape.
  • Finally, point your middle finger towards yourself.

The middle finger is actually the most important bit of information. I always remember it because I find it funny that I’m “flicking myself off”. Your middle finger represents positive direction along the Z-axis (our forward vector). Given that information, it’s usually pretty easy to determine the rest.

Another important trick for making sense of our 3D world is using our hands to model 3D Quaternion rotation. If we consider that our right thumb is the axis of rotation, if we extend our fingers similarly to the right-handed coordinate system above, the way our index and middle fingers “wind” shows us what direction a positive rotation will go. So if you want to rotate an object about it’s up vector; make a right-handed coordinate system, point your thumb upwards, and you’ll see by following the directions of your index and middle finger what direction the rotation will go.

Note: It’s important to note that quaternions don’t really have “handedness”, so regardless of what coordinate system your object is in, it’s best to use our right-handed coordinate system to see what direction rotating about a vector will yield. (Simply because it’s easy to note that the rotation follows the winding of your fingers).

A Fly-Through Camera

Given the information above, it’s not too difficult to imagine how we would implement a fly-through camera. For an example of how a fly-through camera works, load up a program which supports it (say, Unity; hold right mouse button in to enable fly-through mode, and use WASDQE to move), and try moving around in the world. Which vectors do you think the camera is rotating about? What translations do you think are happening when you use WASDQE? In reality it’s pretty simple (this is what we’ll implement).

Hint: Our camera uses a Left-Handed coordinate system for translation. Rotations do not have “handedness”, so for testing rotations we can use our right hand. In Qt, the cursor position is mapped from <0,0> top-left of the screen, to <width, height> bottom-right of the screen. We can record deltas of mouse movement by recording current and previous cursor position.

Expand the spoiler below to find the answer:

Fly-Through Camera Logic

There are actually two rotations occurring based on the mouse movement.

  • The first rotation is about a constant upwards vector, and uses the mouse’s -x-delta.
  • The second rotation is about the object’s right vector, and uses the mouse’s -y-delta.
  • W and S keys use the object’s forward vector to move in the direction the object is facing.
  • A and D keys use the object’s right vector to strafe in the direction of the object’s side.
  • Q and E keys use the object’s up vector to raise and lower itself in the direction of the object’s up.

With this information, it should be trivial to implement a camera controller!


Camera Control

1. Create the Input Manager

So the first thing we need is an input manager. Start by creating a class named Input, and edit input.h to the following:

Nothing really important, but you can see that there is a lot of different static state information. Most of the state will be held in the source file. As you can tell, this is basically an extension of the Input Manager that we discussed creating in 3a, but we’ve got a few more functions to check state information.

But the header isn’t too interesting. Let’s look at input.cpp:

Since input.cpp is a little confusing, we’re going to build it in steps. Everything that I’m about to show you is in order. Comments denote that the code will continue after I mention something about the content of that section.

First we’ll have just our regular includes. As we saw in Part 3a, QVector and QList are significantly less efficient, so we’re going to opt for std::vector to keep querying and accessing more efficient. You will often have code that iterates over several key states checking to see if they’re pressed, released, or whatever. So we want this to be efficient.

Next we’re going to define some static helper types and data.

Whenever we work with the STL, it becomes important to typedef and consolidate code as much as possible. If we don’t we’ll have pretty ugly function calls with multiple complex declarations. Not to mention what if we find out that we need to use a different container type? I don’t want to change a bunch of lines of code just because I need to change a single type. All of these problems can be avoided if we just typedef.

Another kind of interesting C++-ism here is the template InputInstance. The reason we’re providing a template is because the functionality between checking if a key or a mouse button is down is pretty much the same. Instead of duplicating our efforts manually, let’s just instruct the compiler to do this with templates.

In the end, we won’t even have to write a for loop to look at our data, because we can just use std::find from the algorithms header – which does pretty much exactly what you’d expect it to do. Hence the operator== definition.

As for the globals. Yes they’re not good. We’ve basically got a singleton pattern going on here. In most cases you want to avoid singletons, as they tend to create complexities in the code that are hard to maintain. There are ways we can contextually make this an instance, but act like a singleton – but we’re just redefining the same problem. Ultimately, I’ve never seen a need to think really hard about the Input Manager not being a singleton. You need to access it from many different locations, so we’ll just have to code carefully. In most games, this will not pose a problem because our uses are simple.

Next up, some helper functions that will make our lives easier.

First we’re going to leverage the <algorithms> header for manipulating our vector. All we provide are a few simple functions which make it so that we don’t have to supply begin() or end() to the algorithm every call. UpdateStates and CheckReleased are both functions which have intended usage with STL; UpdateStatus updates the current state of the keys, and CheckReleased is a predicate to see if a key should be removed or not.

Finally in Update, we use these helpers to modify a templated container, we remove first – which is important – and then we update existing data. STL newbies might be a bit confused as to why we call std::remove_if followed by std::vector::erase. This technique is known as the Erase-Remove Idiom (please click the link if you want to know more).

The rest is pretty straight-forward if you’re use to working with the STL. The trickiest thing to understand about Input is that there are two modes for Input; listening and updating.

  • Listening state is when we are receiving input KeyEvents. The key needs to be marked as “should update”, but not immediately update. The reason for this is because our update pass will move states along as well – so we don’t want a key to skip InputTriggered and go directly to InputPressed in one pass.
  • Updating state happens just before any user logic occurs. This is when we actually take the keys which were marked for update and actually update them. This allows a key to go from InputRegistered -> InputTriggered, or from InputUnregistered -> InputReleased. This step is required and cannot be consolidated into the Listening state, because next loop the key needs to move from InputTriggered -> InputPressed.

That’s it! Using Input is as simple as calling Input::keyPressed(Qt::Key_<keyname>), there’s a little more setup involved (we have to register to receive key events, and we have to ask Input to update), but we will see that in a bit.

2. Create the Camera3D class

Next we need a Camera3D class – it’s very similar to Transform3D, but it is vastly simpler. Recall that I mentioned the need for Camera3D at the beginning of this tutorial. It’s not that we directly need such a class, but it would be nice to not perform extra, unneeded calculations such as inverting our matrix. The interface will be very similar to Transform3D, only we will not support scale (it’s not a common camera operation).

camera3d.h:

Lot’s of stuff! But like I mentioned, nothing super interesting. It’s very similar to Transform3D. The only real difference is the presence of forward(), up(), and right(), as well as LocalForward, LocalUp, and LocalRight. We will discuss these more in the source file.

camera3d.cpp:

Really there are only two parts to talk about.

The first is Camera3D::toMatrix(). As you can tell, multiplication order and the values we’re passing are different. Recall we’re building an inverse matrix. We’re building a matrix which will move us from being with respect of the world, to being with respect of the camera. We could form a matrix and then invert it; or we could just build it inverted!

The next thing is the addition of forward(), right(), and up(). As we talked about at the beginning of this tutorial, these have great significance to us. And they’re fairly simple to form. You take the vectors of the current object’s coordinate system, and you rotate them using our QQuaternion. If we wanted to add this same functionality to Transform3D, we would only need to change our Transform3D::LocalForward vector to be <0.0f, 0.0f, 1.0f> instead of what is defined for Camera3D <0.0f, 0.0f, -1.0f>. Recall that the camera’s coordinate system is Left-Handed, and our objects is Right-Handed. This is what will describe our coordinate system to outside objects.

I’ve added these same functions to Transform3D, but the changes are redundant to what I’ve explained above. Expand the spoiler below to view my changes:

Transform3D Changes

transform3d.h:

transform3d.cpp:

3. Add Camera3D and Input to Window

Next we need to actually update our Input Manager, as well as construct a camera which we will pass to the shader. We will be renaming some of our matrices as well, because we no longer just have a worldToView matrix, instead we will have a worldToCamera and a cameraToView matrix. The reason we called the previous matrix worldToView is because we’re assuming the camera is unchanged at the origin. In such a case it forms the identity matrix, which changes nothing about our cameraToView matrix. So if we assume the camera never moves, and is constantly at the origin, our cameraToView matrix is our worldToView matrix.

However, now that is not true, so we will have to split it into two parts.

window.h:

After that, we need to add some changes to the source.

window.cpp:

First you’ll have to include Input so we can access the input manager.

Next we need to cache the worldToCamera uniform location, and we’ll have to change worldToView to be cameraToView. Again, we can probably make this caching much nicer later on. But for now let’s keep things simple.

After that we will upload the new worldToCamera and cameraToView matrices to the GPU.

Update has the most changes to pre-existing code. This is where we will implement our Fly-Through camera. Like mentioned in the spoiler for Fly-Through Camera at the top of the page, we only need to apply two rotations, and 6 translations.

For rotation, we must apply each rotation successively. We cannot combine the two axes to form one single rotation axis which applies both rotations properly. So we must perform one rotation about one axis, and then a second rotation about a second axis.

Translation is a bit simpler. We can aggregate the translation values into one final vector, and then perform the whole translation in one go at the very end. This is handy because we will only have to multiply by the speed of the translation once instead of multiple times.

For now, the speed of these actions are simply hard coded at the top of the function. The units are fairly arbitrary, we simply state that we move 0.5f units per frame. Per frame movement operations are usually frowned upon, so later on we will make this rely on some delta time that the frame provides. At that time we will get more appropriate values for these speeds.

One question you may have is about the if (event->isAutoRepeat())  check. Essentially if a key is autorepeating, it’s because you are holding it down. Think about opening a text document and holding down the “a” key. Notice how it follows the format of <A>-onelongpause-<A>-<A>-<A>-… Those repeat keys are what we’re trying to ignore. Everything else seems pretty simple, we just pass information to Input so that we can have dynamic state information.

4. Update shader code

The only other change we need to make is to the Vertex Shader.

That’s it! Your final executable should allow you to use WASDQE to move, and mouse movement to rotate (when you are holding the Qt::MouseRight button). Nothing visually different, but this is enough to allow us to look at our object from many different angles. This will come in handy when we start loading assets in (next time)!

Not too different looking, but we can move around!

Not too different looking, but we can move around!

Summary

In this tutorial, we learned about the following topics.

  • What the camera matrix represents, and how to calculate it.
  • What a Fly-Through Camera was, and the math behind it.
  • How to create a rudimentary Input Manager for Qt, and how to register it.
  • How to create our Camera Matrix using the information/classes we already have.

View Code on GitHub

Cheers!


Leave a comment

Your email address will not be published. Required fields are marked *

13 thoughts on “Qt5 OpenGL Part 3b: Camera Control

  • George

    Wow, very nice post! I especially liked the clever use of stl in the Input Controller. Is there a difference in using the QMatrix4x4.lookAt() method instead of creating the camera matrix this way?

    • Trent Post author

      Hey George!

      Been a tad busy with other things (hence lack of update on my website), sorry this took so long. For this question, you really have to ask yourself what information your trying to show. A lookAt() may be great if you really have something you want to look at, and you have the necessary information handy (for example, a camera that is locked on to some target). Maybe in that case it might be better since there’s less math involved to produce the rotation matrix. But then again, a lookAt() also involves some normalization (sqrt calls). I’d really have to test both methods together to see if one would be better over the other, but ultimately my guess would be a quaternion would win for two reasons:

      1. The algorithm to produce a rotation matrix (though it looks complicated) may be less computationally expensive than keeping valid up/right/forward vectors for the lookAt().
      2. Many of the pros of using quaternions apply (no gimbal lock https://en.wikipedia.org/wiki/Gimbal_lock) and less memory overhead as well.

      Honestly, after you learn quaternions, you’ll really miss having them. A lookAt() is good for getting yourself oriented, I think – but when you’re rotating around some arbitrary axis, you can’t really beat quaternions. I’ve never really cared to keep up/right/forward vectors because it’s always just simpler to slap a quaternion in there.

      I guess I’m saying, this is just a quick assessment. Nothing truly beats testing it yourself. :)
      I hope to revisit graphics when Vulkan is released, and I’ll keep this in mind for something to experiment with when the time comes.

    • Trent Post author

      Hey Marcel,

      Yup! I entirely did this on purpose.

      Generally when you create a tool for manipulating objects in 3D space, you want some key or button to signify the user intends to move the camera. The reason for this is because otherwise they could accidentally move the camera just through normal usage. (Most common with mouse movement)

      Of course, it’s up to you if you want to move logic around, change this configuration, etc. This is just what I’ve found to be the most comfortable when I was working on this project.

  • Ryan

    Trent,

    This tutorial is amazing. I am trying to make a flight simulator, like “Pilot Wings”, in QT and this has been extremely helpful. I am trying to take the camera around with using yaw, roll, and pitch as well but I want to make sure my idea is along the right path. So, would I use QQuaternion::fromEulerAngles(p,y,r) that returns the Quat and then use that to change the camera view, and then translate and rotate according to WASDQE (or in my case: lift, speed, cross-wind)?

    • Trent Post author

      Hey Ryan,

      Glad you found this useful! :)
      You can apply quaternions to alter rotations, if that’s what you mean. That function will return a quaternion which – when applied to another quaternion, will apply it’s rotation on an existing rotation. I think that’s what you’re asking, but I’m not totally sure – hopefully that helps. Sorry for the insanely late response!

  • Jiaxin Liu

    Thanks a lot for a tutorial! I’ve got a problem that I run the GitHub code and nothing happens when I press WASD keys. I wonder what happens here. I use Qt 5.8 and Visual Studio 2015. Thank you! :-)