Know your text editor

By Bob Somers | Published: July 22, 2009

Let’s examine why it’s important as a programmer to really know your text editor, but in a different light that usual. Traditionally programmers are concerned with knowing all the tricks of their editor so that they can make use of them. Instead, let’s look at an example of why you should know what your editor does so it doesn’t screw you over.

Today at work a coworker received a large volume of joke emails telling them how bad their coding style was, with links to example code telling them to “learn how to program”. This was in response to what he thought was a simple 6-line change, but was actually a 22,000-line “oops”.

He opened up a file that’s roughly 25,000 lines (let’s not get into why it’s that long…) to make a simple change. After adding about 6 lines, he saved, quit, and committed the change to the repository. His editor of choice is vim (as is mine).

For those of you unfamiliar with vim, it’s a text-based editor that has a fairly steep learning curve. It takes years to truly master, but the power it offers you is incredible. In vim you start in command mode, which let you enter vim commands much like you would on a command line. You can switch to insert mode to actually insert text into your document, or other modes for formatting and whatnot.

Well in this case, the programmer accidentally brushed the escape key while he was typing, which brought him out of insert mode and into command mode. What he thought was being typed into a comment in his code was actually being typed into the vim command line. He must have brushed some other keys as well, because here’s the unfortunate key sequence that he ended up entering:

:% <<<<<<<<

For those of you not well-versed in vim commands, here’s an explanation. The colon starts a command, and the % symbol is like a wildcard for the command, applying it to the entire file instead of the current selection. Each < character shifts the current selection one tab stop to the left. Similarly, the > character shifts the current selection one tab stop to the right. It’s phenomenally handy for quickly adjusting the level that a block of code is indented.

However, what that above command will do is basically crush every level of indention against the left margin, and then apply that to the entire file. What you end up with is having every line of your beautifully indented code smashed down to column 0. No indention. None.

As vim was updating this change to the swap file (remember, there were 25,000 lines) he saved and quit. The file had been updated, but his screen had not, so he had no idea he’d accidentally just nuked the indention of the entire file.

It was his commit to the repository that threw up the red flag. This was supposed to be a small bug fix. However, the commit email, sent out to the entire mailing list, contained a 22,000 line diff. Woops.

After looking at the changes and chuckling over the fact that the file was completely unreadable now, many people on the coding team sent him joke emails about “what crappy coding style” he had, with links to “learn how to program”.

Now, naturally this took only about 2 seconds to fix, since the code was under source control. But the point still stands. Not only learn your editor, but watch it with a careful eye when you’re working with it, lest all those features that are there to increase productivity turn against you.

Posted in Code, Tools | Leave a comment

Ban programmers, not functions

By Bob Somers | Published: July 7, 2009

So my daily travels around the intertubes landed me on a very interesting blog post by Microsoft’s Security Development Lifecycle team (which they call SDL, not to be confused with the arguably more useful Simple DirectMedia Layer library). The post centered around them adding memcpy() to the banned functions list in favor of their more “secure” variant, memcpy_s(), which takes and checks the size of the destination buffer.

Before I explain why I think this is another example of Microsoft spending their time doing something incredibly useless instead of innovating, let me explain that all these blasted _s functions are one of the reasons I detest the Windows API so much.

I had the unfortunate “pleasure” of digging rather deep into the Windows API for a project I was working on this past spring quarter. For those of you who haven’t ventured into the Windows API, let me say this: It’s so incredibly confusing that it doesn’t even look like C anymore.

Almost everything uses custom types, even when there’s no logical reason to do so. The Linux API does this to some extent, but not nearly as bad as Microsoft.

Secondly, there seems to be no rhyme or reason as to what these types are named. Some are named as ALL_CAPITALS_TYPE, others _use_this_strange_underscore_prefix, and some use the standard type_t. If you start to use almost any standard library C function, you’ll inevitably be told by the compiler that you’re doing it wrong, and should use strcpy_s(), or _strcpy_s(), or _s_t_r_c_p_y_s_(). Seriously, their API has got more underscores than Bill Gates has dollar bills.

What this gives you is this strange, alien language that vaguely resembles C, but is so ugly and hideous that you’re afraid to touch it. Apple has Objective-C. Microsoft has Franken-C.

So let me back up and explain this blog post I mentioned earlier. I’m a bit behind on this one (I’ll admit I’m not often found venturing into the MSDN blogs) but back in May the SDL announced that they were adding memcpy() to their banned functions list, to join strcpy(), strcat(), strncpy(), strncat(), gets(), and others.

They announced it’s replacement, memcpy_s() (soon to be replaced by _memcpy_s() and _m_e_m_c_p_y_s_() I’m sure), which takes one additional argument: the size of the destination buffer.

This is aimed make usages of memcpy() more secure, by only copying up to the size of the destination buffer bytes, even if that’s less than the length of the bytes you want to copy. You go from using this:

memcpy(dst, src, len);

to using this:

memcpy_s(dst, sizeof(dst), src, len);

This sounds reasonable, except most Windows programmers will just do this:

memcpy_s(dst, len, src, len);

which makes your “secure” version useless.

The problem here is not that memcpy() doesn’t check the size of the destination buffer, but rather that some programmers are using it without thinking. A 50 caliber sniper rifle is a very powerful tool in the hands of a marksman, but in a cage full of chimpanzees, the results could be disastrous.

If nothing else, memcpy_s() makes you think about the size of the target buffer.

I suppose, unless you’re one of the mindless programmers using memcpy() unsafely before, in which case you’ll learn the new and improved mindless version memcpy_s(dst, len, src, len) and continue on your merry way.

My point here is that banning functions that are the common source of security vulnerabilities doesn’t fix the problem, because the problem isn’t with the functions. These functions are well documented and we know exactly how they work and what their dangers are. The problem is with the programmers.

You’ve got to teach your programmers how to use these functions securely, or at least evaluate when they should ask for someone to review their code. If training isn’t an option, there’s a better option than banning these functions.

Ban programmers who use them wrong. Yes, banish them to the land of C# and other fluffy dynamic languages with garbage collectors and infinite buffers. They’ll do far less harm there.

The key to doing memory management correctly (which includes using memcpy(), strcpy(), etc. in safe ways) is to completely engage your brain when you’re doing it. You cannot zone out when writing memory managing code. Although given the quality of code coming out of Redmond, I would not be surprised if most of the programmers have their brains permanently switched off.

As classic-Microsoft as this blog post was, the best line was last one.

I wonder when Larry, Steve and Linus will start banning strcpy() in their products?

Words cannot express the hilarity that ensued when I read this line. Maybe, just maybe, the reason they haven’t found the need to ban them is because they’re using them correctly. Perhaps if Microsoft tried that every once in a while, they would churn out more secure products themselves without having to resort to Franken-C.

Posted in Code, Microsoft, Security | Leave a comment

SSL detection by PHP scripts run through FastCGI on nginx

By Bob Somers | Published: July 3, 2009

How’s that title for acronym soup?

I ran across this issue when playing around with nginx. I was trying to set up phpMyAdmin for SQL administration, but ran into a rather peculiar issue. To explain the problem, let me give you some context.

I’m running nginx only on port 443, using SSL for everything. As I’m going through the setup for phpMyAdmin, imagine my surprise when it alerts me that I’m not using an SSL connection. In fact, it’s impossible for me not to use SSL, because there’s no regular HTTP server running on port 80. I continued with the setup anyway, checking the ForceSSL option which requires all phpMyAdmin requests to be done over SSL. When I finished installing it and tried to log in, I got a Firefox error that it was stuck in a redirect loop.

Much Google searching later, I still couldn’t find the problem. This was when I remembered that PHP is configured differently on nginx that is typically done with Apache. With Apache, many people use the mod_php module that compiles PHP support directly into the server. With nginx, you generally process PHP requests using FastCGI.

I wondered if perhaps the fact that the connection was taking place over SSL wasn’t being passed through to the FastCGI process. If that was the case, the phpMyAdmin setup script wouldn’t know it was being invoked over HTTPS, and when you tried to log in it would try to forward you to the HTTPS url, which is the same page you had just requested. That would push you into an infite redirect loop.

In fact, that’s exactly what was happening. You can fix this with a simple addition to your nginx.conf file:

server {
    listen 443;
    ... more config here, include SSL ...
    location ~ \.php$ {
        ... FastCGI config here ...
        fastcgi_param HTTPS on;
    }
}

That fastcgi_param HTTPS on; line does the trick. Now the PHP script knows it’s being invoked over SSL and doesn’t try to infinitely redirect you. Awesome.

Posted in Linux, Scripting, Security | 1 Comment

Fedora, you need some slimfast

By Bob Somers | Published: June 30, 2009

Let me get this out of the way up front. I absolutely love Fedora to death.

It’s my favorite distribution of linux by far, and I’ve played with a lot of different distros. I love the way way it works, I love how it’s on the bleeding edge, I love how things are laid out and they just make sense. I love the massive repository of almost every piece of software I want to run right at my fingertips. I love being able to type

# yum install package

and it just works. Absolutely fantastic. I’ve been a user since Fedora Core 3, and the team just pushed out version 11. Good job, guys.

But there’s one thing nagging me about the distro. I almost never notice it, but I’ve been running a lot fresh installs of Fedora lately, so it’s become pretty obvious to me. If she asks me if she looks fat in her Leonidas dress, I’m going to be honest and say yes.

Now let’s give it a fair shake here, it is, after all, a distro designed for desktop use. It’s aimed to be usable to folks who might be trying linux for the first time, and it’s stellar for that. But that comes at an expense to us power users on occasion.

About half the time or more I use it as a GUI-less server, so I’ll be the first to admit I’m probably using the wrong tool for the job, but I just can’t give it up. She makes a great server distro too…

Except for the fat.

I just did a fresh install of Fedora 11, and I challenged myself to configure it with the absolutely minimal set of packages I needed. No desktops, no windowing system, no crazy text-based packages either. I unchecked all the unnecessary WiFi drivers (all of them, I don’t have wireless on this machine) and really slimmed it down to the bare, bare essentials.

Or so I thought… it still installed a whopping 600 packages and takes up almost 2 gigs worth of space. I realize 2 gigs might seem like a small number, but I’m provisioning an old box that uses a 20 GB hard drive. I’d like to see a ~500 MB install rather than have it take up 10% of the disk for the OS alone.

As the package installs were flying by, I tried to get a glimpse of what on earth was taking up so much space. One package that caught my attention was the Leonidas wallpaper pack. Really Fedora? I didn’t even install a desktop or windowing system and you’re installing wallpapers? What am I going to use those for?

My guess is that they’re just standard packages included with every installation, but that doesn’t make much since if it offers me the ability to install without a desktop or X Windows. Perhaps some more intelligent package sorting is in order.

Thankfully I’m not the only one that’s noticed.

One of the planned features for Fedora 11 was a new Minimal Platform, which would install the bare minimum package set to get up and running, allowing you to bring up exactly what you needed with yum. When I originally read about it, I was ecstatic.

But alas, after realizing how much package culling it would require, dispute over where the option should appear in Anaconda (the graphical installer), and a general feeling that it wasn’t critical, it was pushed to Fedora 12.

Crap.

I guess I’ll stick with her… for now. Maybe I’ll hit the gym and see if she gets the hint. It’s just not good for your health Fedora, you’ve gotta drop some of that heft.

Posted in Linux | 1 Comment

Using a file as input for iptables-restore

By Bob Somers | Published: June 25, 2009

I deal with headless linux boxes a lot, and one of the first things you always want to do is configure your firewall. The general rule of thumb says to deny all traffic and only poke holes where you need them. It’s an awful pain to configure iptables one rule at a time from the command line. It’s also a pain if you need to open one more port before that deny-all at the end of a chain, because that involves running through the whole chain again.

Now there’s probably some clever “iptables ninja” way to do it, but I prefer simplicity, so I use the iptables-save and iptables-restore commands. The first will dump your current firewall rules to standard out. The second will read rules from standard in into the firewall table.

For reference, my distro of choice is Fedora, so these commands are Red-Hat-centric.

Using iptables-save to dump your rules to a file is simple enough. Just redirect standard out to a file. Don’t forget to run it as root.

# sudo iptables-save > ./firewall.rules

Open up that file in vim (or a less worthy text editor… take that emacs!) and you’ll see it’s just a list of iptables commands. Perfect. Just drop in the new rule where you wanted it.

Reloading the firewall table from this file is a little more tricky, though. Specifically, iptables-restore takes input from standard input, but mysteriously doesn’t work when you redirect standard in from a file with the < redirector.

The solution? Call the plumber! Using the piped output from cat works just fine. Don’t forget to flush the firewall rules before you read them in again, and run both the iptables commands as root.

# sudo iptables -F
# cat ./firewall.rules | sudo iptables-restore

Now double check to make sure that the firewall configuration is really what you think it is.

# sudo iptables -L

Lastly, save the firewall configuration so that it persists after a reboot. If you skip this step, your old configuration will come back when the iptables service starts next time.

# sudo service iptables save

For good measure, I always like to restart the iptables service to verify that it will come back up using the config that I expect.

# sudo service iptables restart
# sudo iptables -L

All done! Not to painful, right? This is also a great way to backup your firewall config. If anything bad happens, you need to reinstall iptables, or you’re provisioning a duplicate server, just run the back half of this process with iptables-restore and your backed up firewall.rules file.

Posted in Linux | 1 Comment