New Fun Blog – Scott Bilas

Sitting at a bus stop in Lima, compiler running in the background, got a few minutes to kill. Quickie time!

I came across this trick for qsort in some code at work that August wrote. Cool trick, simple, can’t believe I haven’t seen this before. It’s probably in K&R and a million FAQ’s and so on, but I missed it anyway. In a Google for “qsort compare function sample” you get a mix of both tricks that come up.

Here’s a basic qsort compare proc that I pulled from one of the samples I found:

int sortFunction(const void *a, const void *b)
{
    int intOne = *((int*)a);
    int intTwo = *((int*)b);
    if (intOne < intTwo)
        return -1;
    if (intOne == intTwo)
        return 0;
    return 1;
}

Pretty straightforward, but here’s a smaller and quicker version!

int sortFunction(const void *a, const void *b)
{
    int intOne = *((int*)a);
    int intTwo = *((int*)b);
    return intOne - intTwo;
}

No branching is required. This is much faster as well as being shorter. This is great when you are sorting by multiple keys at once – you only need to test for 0 before moving on to the next sub-key, otherwise return the difference you already have.

Some important notes:

• This obviously doesn’t work on non-numeric types like a string. Still have to do those the old way.
• In C++ if you are using the std library you should use std::sort, which only requires a single < compare (unlike qsort’s troika of <0, 0, or >0 returns). Does not use callbacks, and inlines your comparison proc. Much faster than qsort.
• If you do use this trick put a comment in the code mentioning it’s on purpose! At first it will look like a bug to a casual observer.

UPDATE: (10/10/2009)

Commenter Arseny points out that this can be dangerous! Put a big comment on the sort function noting that the caller needs to keep in mind the range of data being used.

It’s a good idea to put in a simple assert that tests for this.

I’d still not shy away from the technique. The boundary conditions are very far out, and unlikely for most scenarios. An assert would put my mind at ease.

I have an ongoing war with bad Windows installers. I just installed an Intel video driver that re-added its resident hotkey app to auto-start and it got me wanting to write about this.

(I touched on this a little in a previous post, but now I’m fired up about it! Rarr!)

Disclaimer: Many apps I use have very well-behaved installers. They don’t try to install spyware toolbars. They leave my startup alone. They don’t install DLL’s to system32 or add new system fonts. They sometimes even offer to install in a portable form so I can run off a flash drive. These are written by good, underappreciated people and this article does not apply to them. You know who you are.

The Problem

This article is about arrogant programmers and marketroids who think that their app is so important that it’s got to install itself in every part of the system. The Internet is filled with complaints about this. Sticking icons every little place, putting themselves in every possible context menu in Explorer, creating multiple Start Menu entries, adding scheduled tasks, running “agents” that make sure all of this sticks… More and more ways every day. And I’m not even talking about malware.

The installers for these apps never ask permission to add these things to your system, aside from the trivial “add to desktop/quick start” type checkboxes. They just ask for UAC permission to get admin access, which is required by the system anyway to copy into Program Files. So you have no choice. You must approve the UAC dialog to use the app at all (partially defeating the purpose of UAC).

Once they have full admin permission, they’re free to install services, DRM drivers, and tray apps, mess with your file associations, add icons and toolbars, and so on. All without your knowledge or approval.

To make matters worse, a lot of these things are poorly programmed as well. They always assume they’re the only thing running on the system, and not fighting with all the other apps for access to CPU, hard disk, and memory. They never take into account hibernate/resume or system bootup time. They often crash when an unexpected condition occurs (like the network going down) and don’t correctly handle corrupted files after such a crash. And because they were installed with admin access, a badly programmed service or driver can easily blue-screen your system (hello Toshiba!).

The Doghouse

The big companies tend to be the worst because they have the most resources, version upgrades, and itchy marketroids available to dedicate to invading your system. Apple, Google, Microsoft, ATI, nVidia, Intel, Corel, Sun, Real, Sony and friends – you all suck.

RealNetworks used to be the gold standard by which these things were measured. I’ll never forget the shock I got when I installed their media player only to find a tray app was spamming me with popup advertisements. After a period of stubbornly refusing to change while their reputation took a plunge, they learned and stopped doing this (as much).

So now Apple is the new King of Startup Suck on Windows, which sure is ironic (or perhaps strategic?). And like the old Real, they simply ignore complaints like this from their users. Each new version of iTunes adds some new service, some new system task, some new file association. Not to mention it re-enables whatever you had disabled properly through prefs, such as the QuickTime tray app.

So iTunes is currently up to: three startup tasks, a toolbar plug-in, an Explorer bar, two services, a couple drivers, and a partridge in a pear tree. All that, just to install iTunes, whether you have an iPhone or not, just in case you buy one. And a virus-like auto-updater that tries very hard to get you to install Safari and MobileMe.

Even “do no evil” Google is in the doghouse. I’m not sure what I installed (Picasa? Chrome?) but it never asked me if it was ok to add a system task (GoogleUpdateTaskMachine) to do updates. And a service too for good measure, the randomly-named gupdate1cas728374af, apparently to prevent scripted removal. This is assuming you disable their system watcher that tries to keep other apps from stealing their system hooks.

Microsoft deserves the blame for a lot of this. Their Office division invented or at least popularized a lot of these things. Things like tray balloons, the “fast startup helper app” that Adobe happily copied, and arrogantly installing the New and Open commands at the very top of the Start Menu.

With Windows 7 it seems that Microsoft has started to see the light and is finally starting to do something about this problem. We’ll see. I don’t think things will improve via slow-moving OS releases.

The Mac Gets It Right

Despite Apple being among the Worst Companies In The World on the Windows platform, OS X almost totally avoids the problem. I’m envious. Stuff like this makes me want to try again to Switch To Mac. If only they had the tiniest bit of respect for keyboard users…

Anyway, on the Mac, installing an app is typically done by dragging it out of its .dmg file into the Applications folder. That’s it. No special permissions required. It doesn’t even have to copy a billion little files because of the Mac’s smart package file system. The app is going to end up per-user and not shared, but does that matter? Not with most apps, which are small. Only with the biggest stuff or system hooks will an actual installer be required.

Microsoft, please learn from this. Mounting image files as drives, read-only executable package files, and drag-to-install. Please…

So, in almost all cases, when you get a new app for your Mac, you can be pretty sure it’s not going to be doing any evil. Only when it requires an installer and asks you for an admin password will you have to check to see if it added anything.

On Windows, installers that make you reboot are usually a good sign that something is afoot at the Circle K and it’s time to search and destroy the evil.

What To Do?

Well, there are a couple options on removal:

1. Go through every dialog and right-click menu and options screen trying to find where the stupid option is to stop the tray app from loading…if the option even exists.
2. Use a tool like Autoruns.

Obviously, I use Autoruns! (This article even has it in the title.)

The Solution: Autoruns

Autoruns is a tool by the wonderful Sysinternals guys (purchased a while ago by Microsoft) that has the following key features:

• It finds every possible way that an app can hook into the startup or login sequence of Windows.
  • Run, RunOnce, AppInit, GINA, Sidebar, Scheduled Tasks, Drivers, and many more. It’s very complete.
  • Incidentally, this is how I discovered the trick that Dexpot uses to bypass the UAC dialog on session start.
• It lets you delete the links to them so they don’t start up any more!
• Even more importantly, it lets you disable them, regardless of the type of link.
  • The link is squirreled away so the system won’t start it up, yet you can undo it later.
  • This is a key feature for me. More on this in a bit.
• The UI is set up for easy diagnosis.
  • A Jump-To feature to go to the location of the infestation.
  • There’s a column for the Publisher. This makes it a lot easier to seek out the developer you’re looking for or look for things that don’t belong (particularly subcontracted DRM drivers).

Autoruns Setup

Autoruns comes as a zip file with no installer, like all Sysinternals utilities. You can also run it directly from their web site (I prefer minimizing my online dependence, as I never know where I’ll be day to day).

So unzip it to some folder and make a shortcut to it. I have a c:/users/scott/programs folder where I put stuff like this. That way, it gets picked up by my minimal backup and is easy to migrate to other machines. [Hmm, now that I think of it, I might start using Windows Live Sync to keep this in sync across all my machines.]

I’d also set a couple options to make it easier to use Autoruns.

Require Elevation

I recommend marking Autoruns as requiring elevation by default. While Autoruns is Vista-aware, and will offer to re-launch itself with a UAC dialog if needed, in practice I’ve found that I always need it in admin mode.

There are a couple ways to do this:

• Right-click on autoruns.exe and go to Properties. Select the Compatibility tab and check the box for “Execute this program as administrator” (as in the picture on the right).
• You can also do this with a shortcut. Maybe make two shortcuts, one with admin and one without. Again, I don’t see the point of the non-admin version.

Hide Microsoft Entries

I also recommend selecting the Options | Hide Microsoft Entries option.

You are running Autoruns on a Microsoft OS. So there is an enormous amount of noise from everything they have installed as part of the OS. It really gets in the way.

These days, Microsoft is pretty good about not installing auto-run stuff, and if they do, it’s always straightforward to turn it off. And it almost always (with the notable exception of MSN Messenger) stays off. I do occasionally flip this bit back on to look for anything I may have missed. But typically I leave it off.

Note that when you flip the bit you have to refresh (F5).

Using Autoruns

There’s two ways I use Autoruns: killing new infestations, and re-killing old infestations back from the dead.

New Infestations

Pretty easy: after installing some new app, or an update, or a driver, run Autoruns and look for whatever’s new. I also look for entries with publisher names I don’t recognize in case something weird got through that I didn’t notice previously.

Note to Sysinternals: it would be convenient if Autoruns remembered what it saw the last time so it knows when something has changed.

For each new entry, you’ve got to decide if you need it or not. This is the tough part. More on this later. But as the diagram on the right shows, I’m certain that I don’t need QuickTime Task, the Java updater, hotkeys or a tray app for my video driver, and a bunch of other things below the fold.

Now, I don’t delete these entries, I only disable them. This lets me easily reenable if it turns out disabling was a bad idea. But more importantly, it helps with killing entries that riiiiiise from the dead.

Undead Infestations

Some installers, particularly those from Apple, like to reset certain things to default, just in case. For years I’d disable QuickTime Task the correct way, using the config dialog they give me. And on the next iTunes update, that stupid app would be right back in my tray. No more!

The better way to do it is use Autoruns and disable the entry. Then, after an updater runs, check for duplicates, as shown in the image to the right. They really stick out, don’t they? In this example, I’ve installed the newer Intel drivers and they’ve put back a couple things without asking me. The disabled duplicates tell me that in the past I had already gone through the trouble of finding out if it’s ok to disable those or not.

So, after verifying that the image path and link locations are the same, I just delete the new entries. Easy.

Use Extreme Caution

Be really careful about what to disable. There are a lot of unscientific “tweak” sites out there recommending disabling this-and-that service, saying things like “worked for me so it’s safe”. Oh really? And tested how? On a corporate LAN or a notebook that needs to talk to that LAN? VPN still works? Can still talk to that Linux NAS you have in the closet? Does Maya still run? Backups of your system still working properly?

My guess is that the audience for these sites is primarily amateur home users (i.e. kids) trying to overcl0ak their systems. More power to them. Just don’t listen to their advice if you’re doing anything other than playing games and surfing the web.

Disabling services may cause certain apps to crash or not work right. You really have to be absolutely certain first.

Say you disable a service in January, test thoroughly, and your system is fine. Yet in September you install some app that requires this disabled service. It’s definitely not expecting this, and so it crashes in odd ways. You get very angry at the developer and post nasty things about them on the Internet. And then the nice developer wastes hours of time with you in email trying to diagnose the problem. All because you forgot that you disabled some service from starting up 8 months previously.

Primarily I use Autoruns on the nasty stuff in the “Logon” category. That’s where most of the infestations occur.

What About Updates?

Ok so now that I’ve gone and disabled all these ridiculous auto-updaters, how do I know when and how to update? I don’t want to lose security updates or new features, I just want to do it on my own terms. Many apps such as SnagIt will offer to check on startup, which is great. What about the rest? When a new Java runtime comes out with security patches I definitely want to update.

The solution is easy: use FileHippo Updater.

It’s one single, tiny, fast tool that replaces all these other ridiculous auto-updaters. It finds updates and links directly to downloads of everything I may want. Ironically, it has a tray app, but that is straightforward to configure.

Any Other Tools?

I’m aware of tools like SpyBot Search & Destroy and the PC Decrapifier, and I’m sure they’re fine, but ultimately I just don’t trust tools to automatically keep my system clean. Maybe it goes back to when I used to manually optimize my config.sys in DOS but this is just something I have to do myself. [Although it looks like SpyBot S&D has come a long way since I last looked at it. Hmm…might have to give that a try after posting this.]

I also tried Windows Defender for a little while but it didn’t add anything of value. All it seemed to do was sit in my tray wanting me to update it every day, slowing down my system, and having me confirm things I don’t care about. It’s also not comprehensive like Autoruns is. So I used Autoruns to disable it.

There’s also MSConfig that comes with the OS. Again, it’s not as comprehensive as Autoruns, plus it doesn’t have a view to list every category at once (which is much faster). Though, given that Microsoft owns Autoruns now, we can hope that they integrate it into MSConfig in Windows 8 in 2048.

The Future Brings More Evil

This problem is only going to continue to get worse in the future. More and more apps are inserting themselves into the system in creative ways and it’s an arms race to keep the OS running smoothly and popup-free.

Just go to any computer store and check out one of the HP’s they have for sale. There are at least 10-15 little “helper” apps running in the tray, all happily monitoring for updates and sending notices and messages. It’s becoming common for installers for ordinary apps to install a tray app, a startup update checker, and possibly a service (often for DRM).

Tools like Autoruns are our only hope. But it needs improvement.

Autoruns Wishlist

Autoruns nails the startup problem. But thinking about the problem abstractly, it’s really a hook detector and disabler, with a focus on the startup and login sequence.

It’s missing runtime places that things can hook in, and that I’d love to see added to the “Everything” list:

• Right-click Send To.
• Right-click New (i.e. templates).
• File extension handlers (.doc –> Word, .txt –> Notepad).
• System and user paths in the environment (evil installers sometimes change these too).
• Hotkeys including Start item shortcut hotkeys (I’m talking to you, Windows Media Player).
• System event-triggered startups (does mobsync.exe really need to start up when Wifi is turned on, and also start wmplayer.exe??).
• Application-specific hooks (add-ins to Outlook, Firefox, Live Writer, Google Desktop, iTunes).

Another good feature would be a “permanently disable” feature that set permissions on the file or registry key or whatever to prevent installers that even have admin access from making changes. You can do this by simply removing all permissions from the link. This is what I did to keep mobsync.exe from auto-launching when Wifi turns on. Kind of a pain to set/restore but Autoruns could easily automate this.

Maybe RunAlyzer?

Autoruns is a pretty simple app but unfortunately closed-source. And now that it’s owned by Microsoft I expect it to mostly languish while they spend five years rolling it into Windows Super Defender or whatever.

Perhaps RunAlyzer from the SpyBot people will do this. I just discovered it, and I’ll be trying this out soon. They sound like good people. They’re at least as pissed off about invasions of control and privacy in the modern Windows world as I am.

This is the article I’ve been waiting for that I didn’t know was coming. Anand has done a thorough analysis of performance in the ways that really matter.

Recommended reading!

Finally, Real-World Testing

What the article talks about is the two things that actually matter in real-world desktop and notebook scenarios: small block latency and change in performance over time. I’m really tired of seeing SSD stats about throughput and only print the lowest possible latency numbers. Let’s just assume all SSD’s are way faster than hard drives in synthetic tests. This is the fatal flaw with most online reviews.

So thanks to AnandTech for getting it right!

JMicron: Not So Bad

I have a Patriot Warp 2 drive that I got in December 08 when prices were a lot higher and pickings were slim.

The Warp 2 is based on the JMicron B series that the article says performs so terribly that it shouldn’t even be on the market. Based on my own experience, I think this judgment is way too harsh. This SSD absolutely destroys the old hard drive that was in my notebook. Everything is faster for me – boot times, loading apps (particularly gigantic dev tools that pull in 1000 dll’s), surfing the web, browsing pics with Picasa…

I do see Anand’s point about consistent performance being better than high performance and tend to agree. That’s why console games so often run at 30 fps: they couldn’t get their 60 to be absolutely solid. And I definitely am seeing the 10-20 second system lockups. Though rare, it’s super frustrating when one of those kicks in right in the middle of me typing an instant message to someone. And when I’m doing things that I know do a lot of small writes, particularly builds of our game and tools, performance goes into the toilet once the write cache is overwhelmed. Overall write performance has gotten worse over time too, though it’s levelled off since a few months ago.

All that said, performance is still vastly better than that old hard drive. So much that I’ve even turned my page file back on and have gone back to my old habits of routinely running 20 apps at once plus a lot of stuff in the tray. I had it turned off in the past with my hard drive because I’d get system lockups from the flurry of small block reads when resuming from hibernate. This is a non-issue with the Patriot. When it comes out of hibernate it’s simply ready.

This is why I disagree with the article in completely disregarding JMicron-based drives. Especially considering how horrifyingly bad ordinary notebook hard drives perform.

I continue to watch how angry and frustrated my wife gets with her brand-stinkin’-new notebook, all due to the hard drive. She doesn’t even hibernate the thing any more. For her, it’s faster to boot from scratch and load all the apps again than to deal with that flurry of page file activity after a resume. If she leaves Outlook or Photoshop open, and doesn’t resume the machine for a few days, we’re looking at 5-10 minutes (I’m not exaggerating) of solid hard drive lock while the stupid thing settles down and becomes responsive enough to do anything. I can’t turn the page file off on her machine or Photoshop won’t work. She has 3G of RAM on there but Photoshop is a fat hog.

The problem is that the performance of rotational media doesn’t scale linearly with more simultaneous jobs run on it. SSD’s do. There’s no waiting for the head to move or the disk to rotate into position. Hard drives appear to scale on some kind of exponential curve of badness, to the point of a total lockup after enough is going on. And with Vista, that point is easy to hit.

On my machine, this problem is totally a thing of the past. I have some frustrating things that happen on here from the Patriot’s terrible small block write performance, but I’m never going back to a real hard drive in a notebook. The 128GB Patriot drive is down to $260 on Newegg. Now that’s damn cheap for an SSD. For an older notebook I wouldn’t think twice before putting one of those in. It will significantly boost performance.

The OCZ Vertex: Win

After reading Anand’s article, and seeing the new stuff coming from these ex-Samsung engineers, I think the OCZ Vertex or another Barefoot-based SSD is what I’ll be putting in Ally’s notebook.

The performance of the Vertex looks right on with what I’m after. I don’t need the extreme performance of the Intel, but that problem with small block writes has to be solved. And the Vertex does it. Bonus: the price is already lower than what I had paid for the Patriot. By the time we get back in the States it will be even lower. I’d buy one today for her if we wouldn’t get bitch-slapped by the Peruvian government with a duty as high as the cost of the drive.

AnandTech has an update on new Vertex firmware that increases performance even further. I’m sold.

My VAIO’s Future

As for my own notebook, I can’t see changing out the Patriot. I have very specific performance problems on here that I think I can solve another way without buying new hardware. My only issue with this thing is when doing builds of large projects like our game.

I’m going to try pulling out the CDROM I never use on here and swapping in the old hard drive that the Patriot replaced. Then I’ll symlink over to that drive all the folders that get hit with these types of writes. For our builds, they’re all redirected underneath a single ‘temp’ folder. I’ll also do my profile’s %temp% folder and some other things based on monitoring with ProcMon. I might even turn back on desktop search and put the index there.

The drive is back in the States so I’ll have to wait till August before I can try this out. As it’s old, it’s gotten a little loud. I might have to pack it in some kind of sound deadening material. Lots of space in the CD bay for that. I don’t care much about weight.

But I definitely won’t spread the page file onto this other drive. Then I’ll be back where I started!