Learning CUDA has definitely been an interesting experience. As much as they make it sound like it’s simple to get started (and for the most part, it is), there are lots of little traps that can keep you frustrated for hours… or days. Here are four tips that stumped me during initial development of Haste (which is now on GitHub!) that might be helpful to you.

Long running kernels on a desktop workstation

In Linux, X’s driver watchdog will kill a process that leaves a driver hanging for too long, so to prevent that from happening you can’t launch a GPU kernel unless it returns within a couple milliseconds. (This happens in Windows, too, but I’m working mainly in Linux at the moment.) However, you might want to test kernels on your workstation. The way around this is to switch to a text-only terminal before running your CUDA program. On most Linux distributions, you can swap between terminals using Ctrl-Alt-F2 through Ctrl-Alt-F6, where each is a different terminal. If you hit Ctrl-Alt-F1 in Fedora 14, it will take you back to your X session (you’re still logged in and everything).

So, all you need to do is write code in your graphical desktop, compile, hit Ctrl-Alt-F2 to switch to a text-only terminal, then run your program for testing. When you want to go back to graphical mode to fix bugs, just Ctrl-Alt-F1 back and off you go.

Slow device info queries

If you’re doing doing development on a headless compute box (like our Tesla machine at Cal Poly), you might have noticed that querying device information takes a long time. This is compounded if it’s a multi-device machine. Our box at Poly has four Tesla GPUs, and Haste startup was frustratingly slow. All we did is query the device list once, then query each device individually using cudaGetDeviceProperties(). It usually take on the order of 30 to 45 seconds at program startup to get all the device information and allocate memory before we were off to the races launching kernels.

The problem is that the NVIDIA drivers normally maintain a lot of state about the GPUs in memory. However, this state is only there if there’s some resident process keeping it there, like X. If X is not running (or not even installed, like on our headless compute box), that state will need to get reinitialized every time you make a call that requires it. This can be excruciatingly slow, especially on multi-device machines.

The solution? Well, the easiest one is to just install and leave X running, even on a headless machine. Just make sure it’s not driving a display, or better yet switch it over to a text-only terminal with Ctrl-Alt-F2 to keep X around but not have it interfere with your kernels.

Printing debug info in device kernels

I must admit, while debuggers are neat, I tend to like printf() debugging. It’s not that I don’t see the value of debuggers; for some problems they’re really the only way to solve things. Maybe it has something do with the fact that cuda-gdb inexplicably crashes on every machine and kernel I try to run it on.

With the Fermi architecture, available in cards of compute capability 2.0 and higher, you can actually do printf()‘s directly from your device code now, without having to jump through any strange library hoops. Initially, however, I was never able to get it to work. I couldn’t find which CUDA header I needed to include to get things off the ground, and even when it seemed to compile it didn’t print anything.

Well, it sounds silly, but just #include <stdio.h> and away you go. I never tried this initially because I thought that didn’t make any sense. The C standard library doesn’t have CUDA device code! The best I can tell, nvcc is rewriting these standard calls from device code behind the scenes.

The device info’s maximumThreadsPerBlock lies!

This one really irks me. If you query a device’s properties, it reports the maximum number of threads per block in a cudaDeviceProp struct member called, shockingly, maxThreadsPerBlock. The problem is that this is not the actual number of threads you can launch. That depends entirely on your kernel’s occupancy, which you can figure out using the difficult-to-find occupancy calculator spreadsheet. You’ll also want to compile your kernel with the nvcc option --ptxas-options=-v to see the shared memory and register usage for your kernel. You’ll need it in the spreadsheet.

The occupancy limit doesn’t bug me so much as the fact that this is not mentioned anywhere in the documentation where maxThreadsPerBlock is mentioned. Once would think that would be a great place to throw up a warning flag, letting developers know that that number is purely speculative, and that they need to do some real benchmarking of their kernel to find the best occupancy and thread launch combination. Essentially, the maxThreadsPerBlock element is entirely superfluous, since it’s only real use would be in scaling kernel launch sizes by number of device threads available. However, instead we should apparently embed the Excel worksheet in our program and have the device properties chug through the macros to provide any runtime adjustments based on the hardware we’re running on. (</sarcasm>) Yeesh.

Hopefully these tips help you out. As I continue to bang my head against the wall and find new tidbits I’ll be keeping track of them on my GitHub wiki page. Happy holidays!

Getting a complete CUDA development environment up and running under Windows can be a bit… daunting. Between all the dev drivers, SDKs, toolkits, and other trimmings it can take several hours to get your workstation up and running. However, the results are pretty nice.

This guide will give you the following setup:

• Visual Studio 2010 (C/C++ compiler and IDE)
• CUDA Toolkit 3.2 (CUDA C compiler and runtime)
• GPU Computing SDK 3.2 (sample code and utility libraries)
• Parallel Nsight 1.5 (live debugging and performance analysis of CUDA code)
• Visual Assist X 10.6 (syntax highlighting and completion goodies)

There are, however, some drawbacks that you should be aware of:

• In addition to Visual Studio 2010, you need Visual Studio 2008. This is because the CUDA compiler (nvcc) only supports the VS 9.0 build tools at the moment. You can still develop in and compile from VS 2010 however.
• You need two CUDA-capable GPUs in your machine to do CUDA debugging with Parallel Nsight.
• Installation and setup will take the whole afternoon.

For reference, my machine is running Windows 7 Professional x64. Your mileage may vary.

Step 1: Get everything downloaded.

Make sure you have downloaded installers or installation disks handy for all of the following:

• Microsoft Visual Studio 2008, and the SP1 update (Express Edition won’t cut it)
• Microsoft Visual Studio 2010 (Express Edition won’t cut it)
• Visual Assist X 10.6.1833 or newer
• NVIDIA Developer Driver (Note: Don’t use the one posted on the CUDA page, use the newer version posted on the Parallel Nsight page instead.)
• NVIDIA CUDA Toolkit (CUDA C compiler and runtime)
• NVIDIA GPU Computing SDK (Sample code and utility libraries)
• Parallel Nsight Host Installer (The host is the machine running the development environment.)
• Parallel Nsight Monitor Installer (The monitor is the target machine running the CUDA code.)

Step 2: Install Visual Studio(s).

Both VS 2008 and 2010 can coexist side-by-side. Install VS 2008, then the SP1 update. Lastly install VS 2010. Be sure to launch Windows Update afterwords to pull down any patches to the dev tools.

Step 3: Install Visual Assist X.

If you haven’t used Visual Assist X before, you’ve been missing out. It has lots of refactoring and code exploration features, but its killer feature for me is how well it complements Visual Studio’s Intellisense. When you run the installer, you’ll be able to select if you want to install it for both versions of VS or just 2010. It’s up to you, but we only needed VS 2008 for the compiler so you can get away with just installing it for VS 2010.

Step 4: Install the CUDA tools.

First install the developer driver. Then the CUDA toolkit, and the GPU Computing SDK. The default installation paths are fine. Lastly, install the Parallel Nsight Host and Parallel Nsight Monitor.

Step 5: Configure Visual Assist X to know about CUDA.

To get full syntax highlighting and include support, we need to tell VAX about our CUDA libraries as well as the fact that it should treat CUDA files as C/C++ files.

Launch VS 2010 and open the Options screen (Tools > Options). Under Projects and Solutions > VC++ Project Settings add the following entries to the Extensions to Include item:

• .cu
• .cu.h (or whatever you use for your CUDA headers)

It should look something like this:

Now close VS 2010 and open up the registry editor (Start > regedit.exe). Browse to the following folder:

• HKEY_CURRENT_USER\Software\Whole Tomato\Visual Assist X\VANet10

Now look for the ExtHeader key and add .cu.h to the list. Make sure the whole line ends with a semicolon. It should look something like this:

Look a little further down and you should see the ExtSource key. Add .cu to the list in the same way. Again, make sure the line ends with a semicolon.

Now relaunch VS 2010 and open the VAX options (VAssistX > Visual Assist X Options). Open the Projects group on the left and select C/C++ Directories. Under Platform select Custom. Then, select Stable Include Files from the drop down on the right and add the paths to your CUDA toolkit includes and GPU Computing SDK includes. If you used the default installation directories, these are:

• C:\Program Files (x86)\NVIDIA GPU Computing Toolkit\CUDA\v3.2\include
• C:\ProgramData\NVIDIA Corporation\NVIDIA GPU Computing SDK 3.2\C\common\inc

In other words, your screen should look something like this:

Now switch the drop down to Source files and add the following paths:

• C:\Program Files (x86)\NVIDIA GPU Computing Toolkit\CUDA\v3.2\src
• C:\ProgramData\NVIDIA Corporation\NVIDIA GPU Computing SDK 3.2\C\common\src

Lastly, select the Performance item from the left and click the Rebuild Symbol Databases button.

You should be good to go now with Visual Assist X and CUDA.

Step 6: A bare bones CUDA project.

To take you through the process of setting up a new CUDA project in VS 2010, here’s a simple bare bones console application that adds two numbers on the GPU.

First, go to File > New > Project. Select Win32 Console Application from the Visual C++ category. Enter a location and a name and click OK. In the wizard, uncheck Precompiled Headers and check Empty Project.

Now in the Solution Explorer, right click on the name of your project and select Build Customizations. In the box that pops up, select CUDA 3.2 and click OK.

Now add a new source file to your project. Let’s call it hello.cu. Once it’s added, right click on it in the solution explorer and select Properties. Select the General item on the left and make sure the Item Type is set to CUDA C/C++.

Lastly, we need to make sure we’re including the GPU Computing SDK headers and linking to the CUDA runtime library, as well as tell Visual Studio to use the 2008 (9.0) version of the compiler.

Right click on your project and select Properties. From the Configuration drop down, select All Configurations. Under Configuration Properties > General, select v90 from the Platform Toolset item.

Under Configuration Properties > CUDA C/C++ > Common, add the GPU Computing SDK include path to Additional Include Directories. If you chose the default installer path, it will be:

• C:\ProgramData\NVIDIA Corporation\NVIDIA GPU Computing SDK 3.2\C\common\inc

Now under Configuration Properties > Linker > Input add cudart.lib to your Additional Dependencies.

Apply the settings and click OK. All we need now is to flesh out our hello.cu file with a sample program and we’re good to go. Here’s a sample program that adds two integers on the GPU and prints the result. (Note that this is not the default VS 2010 or VAX syntax highlighting, I’ve done some heavy customization.)

Hit the run button and away you go. For fun, (and to learn more about Parallel Nsight), try setting a breakpoint in your GPU code and debugging it.

So hopefully it doesn’t seem like we’ve jumped the shark before things have even gotten rolling, but today I’m going write about… printers.

“But Bob,” you ask, “this is a blog about graphics. Why are you writing about printers?”

Well, first of all, because my trusty HP PSC 1315 All-in-One is wheezing its way out of existence at the moment. I’m shocked that a $50 printer/scanner lasted me over 5 years, but it’s held up quite well.

Secondly, because this blog is about our graphics research and research means one thing… printing out boatloads of other people’s research to read through. Sure everything’s a PDF these days so I could read it on the screen, but screens mean internet access and internet access means hours wasted on Reddit, Facebook, or Minecraft instead of reading. Eventually you realize that while you just spent hours building TNT-powered sheep cannons in a pixelated voxel world, your PDF, sadly, did not read itself. I know that I have a much higher chance of actually reading something if I can kick back on my couch or bed away from the those horrible backlit time sinks and read something on a good ole’ stack of dead trees and ink.

Convinced yet? If not, then this post was written by Chris. If you are, check out the sweet laser printer I just ordered from Newegg.

Looks pretty humble, but check out the specs:

• 27 pages per minute (a page about every 2 seconds!)
• Wired and wireless network connectivity
• Auto-duplex (prints on both sides without having to refeed!)
• Only $150 shipped.

On top of that, the toner cartridges are only $57. That’s what it already costs me to replace my inkjet cartridges, except the toner cartridges will print about 2600 pages before needing to be replaced, and you can set the printer to toner-save mode to get even better mileage out of them.

Now, granted, it is only a monochrome printer, but I’m primarily just printing text these days anyway (research papers and whatnot). I’m particularly excited about the auto-duplexing, because I can save half my paper that way and I might actually have a fighting chance of getting a staple through some of these research papers. In fact, I’m kinda surprised at how excited I am over… a printer.

As a disclaimer, I have not received the Brother HL-2270DW yet, so I can’t comment on whether or not it’s actually all that and a bag of chips, but the Amazon and Newegg reviews are pretty positive. Assuming it doesn’t catch fire and burn my house down* when I plug it in, I think I’ll be happy with it.

* If it does catch fire and burn my house down, I reserve the right to edit this post with vicious commentary representative of only a single user’s poor experience with the HL-2270DW, neglecting everyone else’s glowing reviews. You know, like a real blogger would do.

EDIT: I’ve had the HL-2270DW for over a month now. Yes, it is all that and a bag of chips. Very happy with in, and Newegg has even run a few specials where you can pick it up for around $90. It’s an absolute steal at that price.

The version currently in the yum repositories has a broken init script, so if you try to sudo service murmur start you’ll get all sorts of nasty errors. The version currently in updates-testing works great, though. Install it from there like so:

sudo yum --enablerepo=updates-testing install murmur

That should do it. If you still have problems, try installing the redhat-lsb and qt-sqlite packages and see if that helps.

The good news is, it’s not lost — just frozen. You can “unstop” the terminal by hitting Ctrl-q. Good as new! But we can do more…

If you’re a Windows user who ssh’s into *nix boxes frequently, we can actually make Ctrl-s in vim save the file like you’re intending. First, add the following to your .bashrc file to disable terminal stopping:

stty stop ''

You’ll notice that now the Ctrl-s doesn’t lock up vim anymore, but it doesn’t do anything yet. Let’s add that functionality now with two mappings in our .vimrc file.

map <C-s> :w<CR>
imap <C-s> <Esc>:w<CR>i


Boom, done! Now when you hit Ctrl-s in vim, rather than locking up your terminal, it saves the file. In command mode, it just executes the traditional :w command, and in insert mode, it hits escape (to get to command mode), does the :w, and then hits “i” to get you back into insert mode where you left off.

I’m not a vim expert by any means, so if anyone has a better way to do it, I’m all ears.

There are already binary distributions out there, but I like to compile things myself, so I whipped up a Visual Studio 2010 project to compile the Lua static library (lua.lib) the command line interpreter (lua.exe) and the script compiler (luac.exe).

First things first. If you just want a zip file with those three binary files (the library is Visual Studio 2010 compatible), look no further:

• Lua 5.1.4 binaries compiled with VS 2010 (lua, luac, lua.lib, and necessary headers)

If you want to build Lua 5.1.4 yourself (like me) just grab the most recent copy of the 5.1.4 source code from the Lua website. Next, grab these Visual Studio 2010 project files and solution file:

• Lua 5.1.4 Visual Studio 2010 projects (to build the binaries yourself)

To use the project files, just unzip them into the root directory of your Lua source files (where the README, INSTALL, and COPYRIGHT files are). Then, load up lua-vs2010.sln in Visual Studio 2010. You should see three projects. The first, lualib will build the static library under lib/lua.lib. The second and third, lua and luac will build the Lua interpreter and compiler under bin/lua.exe and bin/luac.exe.

Enjoy!

If you need to install fonts accessible to all users on the system, you have to do some more complicated voodoo. I hate voodoo, so I’m not going to cover that.

If all you need is to install a font for your own user, it’s just this simple:

Put your myfont.ttf file in your home directory, under directory called /.fonts/. So in other words, your font lives at:

/home/youruser/.fonts/myfont.ttf

Create the directory if it doesn’t already exist. Finally, restart any application you want to use that font in, and you should see it show up. You’re good to go.

However, there is a nasty bug in the most recent version of VirtualBox (3.1.2) when combined with Fedora 12. After installing the Guest Additions kernel modules as per the user docs, the system boots to a black screen with a cryptic error message that looks like a SELinux labeling problem (it’s not).

type=1305 audit(12587840002.571:32444): audit_enabled=0 old=1 auid=4294967295 ses=4294967295 subj=system_u:system_r:readahead_t:s0 res=1

The problem is actually with the Guest Additions video driver, the one that gives you the nice resizable desktop window. Once the driver is built and installed, for some reason the X server can’t find any screens and refuses to start.

Until the bug gets fixed in the video driver, here’s how you can fix the system so that it will boot correctly, although you’ll lose the dynamic resizing ability. You’ll have to stick with fixed, predefined resolutions for now.

1. Mount a Fedora 12 ISO, such as the full or network install discs, and boot to it. Boot into Rescue Mode from the GRUB bootloader screen.
2. Breeze through the language and network options, but be sure to have it mount your hard disk image (it will mount under /mnt/sysimage).
3. Drop into a shell and change into your hard disk’s X11 config directory, so that would be:
   cd /mnt/sysimage/etc/X11
4. Edit your xorg.conf file… but wait! In Fedora 12, they switched to HAL for X configuration, so there is no xorg.conf file! Never fear, you just need to create one and it will override the HAL:
   vi xorg.conf
5. Now, use the following settings for the new xorg.conf file:

   Section "Device"
       Identifier "Configured Video Device"
       Driver "vboxvideo"
   EndSection
   
   Section "Monitor"
       Identifier "Configured Monitor"
   EndSection
   
   Section "Screen"
       Identifier "Configured Screen"
       Monitor "Configured Monitor"
       Device "Configured Video Device"
       SubSection "Display"
           Depth 24
           Modes "1440x900" "1680x1050"
       EndSubSection
   EndSection
   
   Section "InputDevice"
       Identifier "vboxmouse"
       Driver "vboxmouse"
       Option "CorePointer"
       Option "Device" "/dev/input/mice"
   EndSection
   
   Section "ServerLayout"
      Identifier   "Default Layout"
      Screen      "Configured Screen"   0 0
      InputDevice   "vboxmouse"
   EndSection
   

6. You’ll see I’ve defined two resolutions, 1440×900 and 1680×1050. What this allows me to do is work windowed at 1440×900 and if I want to go full screen (remember, dynamic resizing won’t work) I can hit the full screen shortcut in VirtualBox (Host+F) and change the resolution within Fedora to match my screen res.
7. Save from vi (:wq) and reboot the system. Remember to unmount the install disc! The system should boot correctly now, albeit without dynamic resizing.

A huge thanks goes out to Jits in the VirtualBox forums for this fix!

Frequently you have some class which is composed inside another class, but occasionally needs to access the class it’s composed inside of. Basically, the classes are composed inside each other, though the abstraction really only makes sense in one direction. Confused yet?

In this example, we’ll use a Refrigerator class which stores inside it an instance of a Cheese class. Why cheese, you ask? Because cheese is delicious. Also, our refrigerator is from the future and can slice and serve cheese just like the built in ice maker and water dispenser. It’s a pretty sweet fridge.

Now, we were all taught to keep our #includes in our header files, not the implementation files, so like good little programmers we construct our classes like so:

refrigerator.h

#include "cheese.h"

class Refrigerator {
private:
    Cheese *pCheese;
    int temp = 35;

public:
    void ServeCheese();
    int GetTemp();
};

refrigerator.cpp

#include "refrigerator.h"

void Refrigerator::ServeCheese() {
    printf("Now dispensing %s cheese!\n", pCheese->GetFlavor());
}

int Refrigerator::GetTemp() {
    return temp;
}

cheese.h

class Cheese {
private:
    Refrigerator *pFridge;
public:
    char *GetFlavor();
    void CheckTemp();
    void BeginMolding();
};

cheese.cpp

#include "cheese.h"

void Cheese::CheckTemp() {
    if (pFridge->GetTemp() > 45) {
        BeginMolding();
    }
}

char *Cheese::GetFlavor() {
    return "cheddar";
}

I’ve left out the constructors in this example for brevity, but let’s assume that they get the pointers set up correctly so that our instance of the Refrigerator class has a correct pointer to an instance of the Cheese class and vice versa.

Now, at this point you may be screaming that this needs to be refactored and reorganized. Yes, it probably does. But there are many instances where you simply can’t, and in fact the abstraction really only makes sense one way. The fridge has cheese in it, but the cheese certainly doesn’t have a fridge in it. We just need that pesky reference around so we can check the temperature of the fridge every so often.

(Yes, I am aware that the fridge could push it’s temperature down to all the items in it, ala the Observer Pattern. Yes I am aware that would be a better solution. But this is a contrived example anyway, so stick with me here.)

Now, the code given above doesn’t compile, because the Cheese class has no idea what the heck this Refrigerator class is, so we either need to include it or forward declare it. If we try to do this:

cheese.h

#include "refrigerator.h"

our compiler (more specifically, the preprocessor) is going to get very angry at us, depending on which order it decides to compile refrigerator and cheese. The solution, is a forward delcaration:

cheese.h

class Refrigerator;

class Cheese {
    // ...etc...
};

Basically what this does is tell the compiler, “Hey! There’s this class called Refrigerator that I might talk about, so here’s an empty declaration of it!”

The problem, though, is that this is rather limiting. Within the Cheese class, we can declare pointers to Refrigerator class, no problem. Pointers are of fixed size, so the compiler doesn’t much what care what it’s a pointer to, since it knows how much memory it needs to hold a pointer to it. When we try to access members of the class, though, like properties or methods, it falls apart because as far as the compiler knows, the class is empty. After all, we told it the Refrigerator class didn’t have anything in it.

So if we can’t #include it and forward declaring it doesn’t give us what we want, what can we do?

Well, we can do both. Kind of.

The solution is to forward declare in your header file, and #include in your implementation file. This will avoid the preprocessor headaches of of the chicken and egg #include, while allowing us to access the members of forward declared class in the implementation. In other words, here’s the fix:

cheese.h

class Refrigerator;

class Cheese {
    ...
};

cheese.cpp

#include "cheese.h"
#include "refrigerator.h"

Again, it goes without saying, the better solution is to refactor or rearchitect your code if you can. These kind of hacks can get really out of hand and are usually a good code smell that something needs to be fixed at a deeper level. However, if you’re working on a large codebase that you can’t change, this can help out a lot.

It’s well documented on how you go about spawning entities, but I couldn’t find a good place explaining how to remove spawned entities through code.

Don’t try meddling with the global entity list (gEntList) or calling its RemoveEntity() method. It doesn’t do what you want.

Instead, used one of the super-handy UTIL_* functions. Given a pointer to the entity you want to remove, simply use:

UTIL_Remove(pEntity);

Poof. Entity gone. Remember, entities are created and destroyed on the server side only. The server will automatically broadcast any changes to the entity list to its connected clients for you.