Learning CUDA has definitely been an interesting experience. As much as they make it sound like it’s simple to get started (and for the most part, it is), there are lots of little traps that can keep you frustrated for hours… or days. Here are four tips that stumped me during initial development of Haste (which is now on GitHub!) that might be helpful to you.

Long running kernels on a desktop workstation

In Linux, X’s driver watchdog will kill a process that leaves a driver hanging for too long, so to prevent that from happening you can’t launch a GPU kernel unless it returns within a couple milliseconds. (This happens in Windows, too, but I’m working mainly in Linux at the moment.) However, you might want to test kernels on your workstation. The way around this is to switch to a text-only terminal before running your CUDA program. On most Linux distributions, you can swap between terminals using Ctrl-Alt-F2 through Ctrl-Alt-F6, where each is a different terminal. If you hit Ctrl-Alt-F1 in Fedora 14, it will take you back to your X session (you’re still logged in and everything).

So, all you need to do is write code in your graphical desktop, compile, hit Ctrl-Alt-F2 to switch to a text-only terminal, then run your program for testing. When you want to go back to graphical mode to fix bugs, just Ctrl-Alt-F1 back and off you go.

Slow device info queries

If you’re doing doing development on a headless compute box (like our Tesla machine at Cal Poly), you might have noticed that querying device information takes a long time. This is compounded if it’s a multi-device machine. Our box at Poly has four Tesla GPUs, and Haste startup was frustratingly slow. All we did is query the device list once, then query each device individually using cudaGetDeviceProperties(). It usually take on the order of 30 to 45 seconds at program startup to get all the device information and allocate memory before we were off to the races launching kernels.

The problem is that the NVIDIA drivers normally maintain a lot of state about the GPUs in memory. However, this state is only there if there’s some resident process keeping it there, like X. If X is not running (or not even installed, like on our headless compute box), that state will need to get reinitialized every time you make a call that requires it. This can be excruciatingly slow, especially on multi-device machines.

The solution? Well, the easiest one is to just install and leave X running, even on a headless machine. Just make sure it’s not driving a display, or better yet switch it over to a text-only terminal with Ctrl-Alt-F2 to keep X around but not have it interfere with your kernels.

Printing debug info in device kernels

I must admit, while debuggers are neat, I tend to like printf() debugging. It’s not that I don’t see the value of debuggers; for some problems they’re really the only way to solve things. Maybe it has something do with the fact that cuda-gdb inexplicably crashes on every machine and kernel I try to run it on.

With the Fermi architecture, available in cards of compute capability 2.0 and higher, you can actually do printf()‘s directly from your device code now, without having to jump through any strange library hoops. Initially, however, I was never able to get it to work. I couldn’t find which CUDA header I needed to include to get things off the ground, and even when it seemed to compile it didn’t print anything.

Well, it sounds silly, but just #include <stdio.h> and away you go. I never tried this initially because I thought that didn’t make any sense. The C standard library doesn’t have CUDA device code! The best I can tell, nvcc is rewriting these standard calls from device code behind the scenes.

The device info’s maximumThreadsPerBlock lies!

This one really irks me. If you query a device’s properties, it reports the maximum number of threads per block in a cudaDeviceProp struct member called, shockingly, maxThreadsPerBlock. The problem is that this is not the actual number of threads you can launch. That depends entirely on your kernel’s occupancy, which you can figure out using the difficult-to-find occupancy calculator spreadsheet. You’ll also want to compile your kernel with the nvcc option --ptxas-options=-v to see the shared memory and register usage for your kernel. You’ll need it in the spreadsheet.

The occupancy limit doesn’t bug me so much as the fact that this is not mentioned anywhere in the documentation where maxThreadsPerBlock is mentioned. Once would think that would be a great place to throw up a warning flag, letting developers know that that number is purely speculative, and that they need to do some real benchmarking of their kernel to find the best occupancy and thread launch combination. Essentially, the maxThreadsPerBlock element is entirely superfluous, since it’s only real use would be in scaling kernel launch sizes by number of device threads available. However, instead we should apparently embed the Excel worksheet in our program and have the device properties chug through the macros to provide any runtime adjustments based on the hardware we’re running on. (</sarcasm>) Yeesh.

Hopefully these tips help you out. As I continue to bang my head against the wall and find new tidbits I’ll be keeping track of them on my GitHub wiki page. Happy holidays!

There are already binary distributions out there, but I like to compile things myself, so I whipped up a Visual Studio 2010 project to compile the Lua static library (lua.lib) the command line interpreter (lua.exe) and the script compiler (luac.exe).

First things first. If you just want a zip file with those three binary files (the library is Visual Studio 2010 compatible), look no further:

• Lua 5.1.4 binaries compiled with VS 2010 (lua, luac, lua.lib, and necessary headers)

If you want to build Lua 5.1.4 yourself (like me) just grab the most recent copy of the 5.1.4 source code from the Lua website. Next, grab these Visual Studio 2010 project files and solution file:

• Lua 5.1.4 Visual Studio 2010 projects (to build the binaries yourself)

To use the project files, just unzip them into the root directory of your Lua source files (where the README, INSTALL, and COPYRIGHT files are). Then, load up lua-vs2010.sln in Visual Studio 2010. You should see three projects. The first, lualib will build the static library under lib/lua.lib. The second and third, lua and luac will build the Lua interpreter and compiler under bin/lua.exe and bin/luac.exe.

Enjoy!

Frequently you have some class which is composed inside another class, but occasionally needs to access the class it’s composed inside of. Basically, the classes are composed inside each other, though the abstraction really only makes sense in one direction. Confused yet?

In this example, we’ll use a Refrigerator class which stores inside it an instance of a Cheese class. Why cheese, you ask? Because cheese is delicious. Also, our refrigerator is from the future and can slice and serve cheese just like the built in ice maker and water dispenser. It’s a pretty sweet fridge.

Now, we were all taught to keep our #includes in our header files, not the implementation files, so like good little programmers we construct our classes like so:

refrigerator.h

#include "cheese.h"

class Refrigerator {
private:
    Cheese *pCheese;
    int temp = 35;

public:
    void ServeCheese();
    int GetTemp();
};

refrigerator.cpp

#include "refrigerator.h"

void Refrigerator::ServeCheese() {
    printf("Now dispensing %s cheese!\n", pCheese->GetFlavor());
}

int Refrigerator::GetTemp() {
    return temp;
}

cheese.h

class Cheese {
private:
    Refrigerator *pFridge;
public:
    char *GetFlavor();
    void CheckTemp();
    void BeginMolding();
};

cheese.cpp

#include "cheese.h"

void Cheese::CheckTemp() {
    if (pFridge->GetTemp() > 45) {
        BeginMolding();
    }
}

char *Cheese::GetFlavor() {
    return "cheddar";
}

I’ve left out the constructors in this example for brevity, but let’s assume that they get the pointers set up correctly so that our instance of the Refrigerator class has a correct pointer to an instance of the Cheese class and vice versa.

Now, at this point you may be screaming that this needs to be refactored and reorganized. Yes, it probably does. But there are many instances where you simply can’t, and in fact the abstraction really only makes sense one way. The fridge has cheese in it, but the cheese certainly doesn’t have a fridge in it. We just need that pesky reference around so we can check the temperature of the fridge every so often.

(Yes, I am aware that the fridge could push it’s temperature down to all the items in it, ala the Observer Pattern. Yes I am aware that would be a better solution. But this is a contrived example anyway, so stick with me here.)

Now, the code given above doesn’t compile, because the Cheese class has no idea what the heck this Refrigerator class is, so we either need to include it or forward declare it. If we try to do this:

cheese.h

#include "refrigerator.h"

our compiler (more specifically, the preprocessor) is going to get very angry at us, depending on which order it decides to compile refrigerator and cheese. The solution, is a forward delcaration:

cheese.h

class Refrigerator;

class Cheese {
    // ...etc...
};

Basically what this does is tell the compiler, “Hey! There’s this class called Refrigerator that I might talk about, so here’s an empty declaration of it!”

The problem, though, is that this is rather limiting. Within the Cheese class, we can declare pointers to Refrigerator class, no problem. Pointers are of fixed size, so the compiler doesn’t much what care what it’s a pointer to, since it knows how much memory it needs to hold a pointer to it. When we try to access members of the class, though, like properties or methods, it falls apart because as far as the compiler knows, the class is empty. After all, we told it the Refrigerator class didn’t have anything in it.

So if we can’t #include it and forward declaring it doesn’t give us what we want, what can we do?

Well, we can do both. Kind of.

The solution is to forward declare in your header file, and #include in your implementation file. This will avoid the preprocessor headaches of of the chicken and egg #include, while allowing us to access the members of forward declared class in the implementation. In other words, here’s the fix:

cheese.h

class Refrigerator;

class Cheese {
    ...
};

cheese.cpp

#include "cheese.h"
#include "refrigerator.h"

Again, it goes without saying, the better solution is to refactor or rearchitect your code if you can. These kind of hacks can get really out of hand and are usually a good code smell that something needs to be fixed at a deeper level. However, if you’re working on a large codebase that you can’t change, this can help out a lot.

It’s well documented on how you go about spawning entities, but I couldn’t find a good place explaining how to remove spawned entities through code.

Don’t try meddling with the global entity list (gEntList) or calling its RemoveEntity() method. It doesn’t do what you want.

Instead, used one of the super-handy UTIL_* functions. Given a pointer to the entity you want to remove, simply use:

UTIL_Remove(pEntity);

Poof. Entity gone. Remember, entities are created and destroyed on the server side only. The server will automatically broadcast any changes to the entity list to its connected clients for you.

Google did not disappoint. I found this awesome web-based regex test tool built on Adobe Flex called RegExr. If you’ve got Flash installed, it runs right in your browser.

Not only does it show you what it’s matching live as you type, what your match groups contain, what each chunk of your regex means, etc., but it’s also got a phenomenal quick-reference sidebar for all those handy meta-characters. You can even save your favorite regexs and browse other people’s favorite submissions.

We need more tools like this. I was able to quickly and easily prototype a regex in no time at all, and the app took about 10 seconds to figure out how to use.

Thanks, gskinner.com! You guys are awesome!

Today at work a coworker received a large volume of joke emails telling them how bad their coding style was, with links to example code telling them to “learn how to program”. This was in response to what he thought was a simple 6-line change, but was actually a 22,000-line “oops”.

He opened up a file that’s roughly 25,000 lines (let’s not get into why it’s that long…) to make a simple change. After adding about 6 lines, he saved, quit, and committed the change to the repository. His editor of choice is vim (as is mine).

For those of you unfamiliar with vim, it’s a text-based editor that has a fairly steep learning curve. It takes years to truly master, but the power it offers you is incredible. In vim you start in command mode, which let you enter vim commands much like you would on a command line. You can switch to insert mode to actually insert text into your document, or other modes for formatting and whatnot.

Well in this case, the programmer accidentally brushed the escape key while he was typing, which brought him out of insert mode and into command mode. What he thought was being typed into a comment in his code was actually being typed into the vim command line. He must have brushed some other keys as well, because here’s the unfortunate key sequence that he ended up entering:

:% <<<<<<<<

For those of you not well-versed in vim commands, here’s an explanation. The colon starts a command, and the % symbol is like a wildcard for the command, applying it to the entire file instead of the current selection. Each < character shifts the current selection one tab stop to the left. Similarly, the > character shifts the current selection one tab stop to the right. It’s phenomenally handy for quickly adjusting the level that a block of code is indented.

However, what that above command will do is basically crush every level of indention against the left margin, and then apply that to the entire file. What you end up with is having every line of your beautifully indented code smashed down to column 0. No indention. None.

As vim was updating this change to the swap file (remember, there were 25,000 lines) he saved and quit. The file had been updated, but his screen had not, so he had no idea he’d accidentally just nuked the indention of the entire file.

It was his commit to the repository that threw up the red flag. This was supposed to be a small bug fix. However, the commit email, sent out to the entire mailing list, contained a 22,000 line diff. Woops.

After looking at the changes and chuckling over the fact that the file was completely unreadable now, many people on the coding team sent him joke emails about “what crappy coding style” he had, with links to “learn how to program”.

Now, naturally this took only about 2 seconds to fix, since the code was under source control. But the point still stands. Not only learn your editor, but watch it with a careful eye when you’re working with it, lest all those features that are there to increase productivity turn against you.

Before I explain why I think this is another example of Microsoft spending their time doing something incredibly useless instead of innovating, let me explain that all these blasted _s functions are one of the reasons I detest the Windows API so much.

I had the unfortunate “pleasure” of digging rather deep into the Windows API for a project I was working on this past spring quarter. For those of you who haven’t ventured into the Windows API, let me say this: It’s so incredibly confusing that it doesn’t even look like C anymore.

Almost everything uses custom types, even when there’s no logical reason to do so. The Linux API does this to some extent, but not nearly as bad as Microsoft.

Secondly, there seems to be no rhyme or reason as to what these types are named. Some are named as ALL_CAPITALS_TYPE, others _use_this_strange_underscore_prefix, and some use the standard type_t. If you start to use almost any standard library C function, you’ll inevitably be told by the compiler that you’re doing it wrong, and should use strcpy_s(), or _strcpy_s(), or _s_t_r_c_p_y_s_(). Seriously, their API has got more underscores than Bill Gates has dollar bills.

What this gives you is this strange, alien language that vaguely resembles C, but is so ugly and hideous that you’re afraid to touch it. Apple has Objective-C. Microsoft has Franken-C.

So let me back up and explain this blog post I mentioned earlier. I’m a bit behind on this one (I’ll admit I’m not often found venturing into the MSDN blogs) but back in May the SDL announced that they were adding memcpy() to their banned functions list, to join strcpy(), strcat(), strncpy(), strncat(), gets(), and others.

They announced it’s replacement, memcpy_s() (soon to be replaced by _memcpy_s() and _m_e_m_c_p_y_s_() I’m sure), which takes one additional argument: the size of the destination buffer.

This is aimed make usages of memcpy() more secure, by only copying up to the size of the destination buffer bytes, even if that’s less than the length of the bytes you want to copy. You go from using this:

memcpy(dst, src, len);

to using this:

memcpy_s(dst, sizeof(dst), src, len);

This sounds reasonable, except most Windows programmers will just do this:

memcpy_s(dst, len, src, len);

which makes your “secure” version useless.

The problem here is not that memcpy() doesn’t check the size of the destination buffer, but rather that some programmers are using it without thinking. A 50 caliber sniper rifle is a very powerful tool in the hands of a marksman, but in a cage full of chimpanzees, the results could be disastrous.

If nothing else, memcpy_s() makes you think about the size of the target buffer.

I suppose, unless you’re one of the mindless programmers using memcpy() unsafely before, in which case you’ll learn the new and improved mindless version memcpy_s(dst, len, src, len) and continue on your merry way.

My point here is that banning functions that are the common source of security vulnerabilities doesn’t fix the problem, because the problem isn’t with the functions. These functions are well documented and we know exactly how they work and what their dangers are. The problem is with the programmers.

You’ve got to teach your programmers how to use these functions securely, or at least evaluate when they should ask for someone to review their code. If training isn’t an option, there’s a better option than banning these functions.

Ban programmers who use them wrong. Yes, banish them to the land of C# and other fluffy dynamic languages with garbage collectors and infinite buffers. They’ll do far less harm there.

The key to doing memory management correctly (which includes using memcpy(), strcpy(), etc. in safe ways) is to completely engage your brain when you’re doing it. You cannot zone out when writing memory managing code. Although given the quality of code coming out of Redmond, I would not be surprised if most of the programmers have their brains permanently switched off.

As classic-Microsoft as this blog post was, the best line was last one.

I wonder when Larry, Steve and Linus will start banning strcpy() in their products?

Words cannot express the hilarity that ensued when I read this line. Maybe, just maybe, the reason they haven’t found the need to ban them is because they’re using them correctly. Perhaps if Microsoft tried that every once in a while, they would churn out more secure products themselves without having to resort to Franken-C.

One of the other corps members was absolutely destroying us. He clearly knew what he was doing when it came to fictional property management. He let me in on a little secret, though. Not all properties are created equal.

What he meant was that certain properties are statisically more likely to be landed on than others. It made sense when he explained it, but I wasn’t sure just how much of an actual difference it made. When I got home from the summer tour, I sat down and wrote some code to find out.

Thankfully in the game of Monopoly, a player’s movement around the board is very much decoupled from their financial transactions. What I mean by that is that they continue to move around the board in the same way, regardless of which properties they chose to buy, how much they have to pay to other players, or which properties they trade. The only exception is when a player goes bankrupt, and ceases to circle the board.

What this allowed me to do is write a very simple simulation in a few hours without having to worry about realistic AI. The players didn’t have to make financial transactions at all. They merely had to roll the dice and move according to the rules of Monopoly. I did need to account for the actions of the Chance and Community chest cards, but only when it impacted their position on the board or their future turns.

Certain spaces on the board can also affect the player’s position and future, so that had to be accounted for as well. For example, landing on Go to Jail will send a player directly to the jail square, and a player in jail has to go through a series of dice rolls or use a Get of Jail Free card to move again.

The game board used in a game of monopoly.

As I was finishing up writing the simulation, I began to suspect that the advice I had received on the train was correct. The claim was that the orange properties (St. James Place, Tennessee Avenue, and New York Avenue) were the most lucrative on the board because players were more likely to land on them. This indeed makes sense, because several Chance cards send you directly to jail and so does the Go to Jail square. Thus, the most common starting point for a dice roll is Jail. Since the most common dice roll for 2 six-sided dice will be in the 6-8 range, it only makes sense that those are the most commonly landed-on properties.

Indeed, this is exactly how it plays out. My simulation played 2000 games with 6 players and 1000 turns per game, and dumps the “landed-on” percentages out for each space along with a heat map overlayed onto the above Monopoly board. Let’s take a look at the results.

Raw results from the Monopoly simulation.

Squares landed on more often are "hotter" (tinted red) and squares landed on less often are "cooler" (tinted blue).

The orange property tract is certainly the most lucrative. But when you look at the raw percentages, it’s a little disappointing. The most landed-on property on the board (Community Chest #2, exactly 7 spaces away from Jail) is less than 1% more trafficked than the least landed-on property (Mediterranean Avenue).

Keep in mind though, that’s when it’s averaged out over 1000 turns, which is a massively long game of Monopoly. On top of that, those results are averaged over 2000 games. This smooths out the heat map quite a bit.

In a single game of Monopoly, the percentage swings will be much higher. The Orange properties could be red hot. They could also be colder than Mediterranean. However, given that in the long run they are hotter than the other spaces, it doesn’t hurt to try to score the orange properties. More often than not, they will see more action than any other tract on the board.