Silver's Simple Site - Weblog - All Posts

Oh dear oh dear.

The world is at an end, I have a weblog. Things can only go downhill from here...

Permalink | Author: | Tags: Weblog | Posted: 02:38PM on Thursday, 02 June, 2005 | Modified: 02:54PM on Thursday, 02 June, 2005 | Comments: 0

And so it begins.

My time is slowly wasting away. Silly paid work. Still, money is money. Kinda.

New PC is still in bits, unfortunately, due to a rather important lack of PSU. There'll be some harsh words about this tomorrow!

Permalink | Author: | Tags: PC | Posted: 04:27PM on Thursday, 02 June, 2005 | Modified: 06:00PM on Thursday, 02 June, 2005 | Comments: 0

Sane Posting

Aaaah, a web-based posting system always feels so much better. Next stop, web-based editing...

Edit Ok, so the preview form is screwing up the <textarea>. *sigh*

Permalink | Author: | Tags: Weblog | Posted: 05:59PM on Thursday, 02 June, 2005 | Modified: 06:48PM on Thursday, 02 June, 2005 | Comments: 0


Boy do they have a lot to answer for. The company I work for uses it, and believe me, it makes a right mess of the code. Not to say other HTML editors don't, but when I'm trying hand edit some code (so it, you know, works in non-IE browsers) and I see this:

   <div align="left"><strong></strong></div>
   <div align="right">

I really start to get annoyed.

Permalink | Author: | Tags: Work, HTML | Posted: 04:12PM on Friday, 03 June, 2005 | Comments: 0

Users Who Don't Want Help

I get the feeling sometimes users don't really want any help, they just want to annoy those trying to help. You suggest they try X or Y, and they say that they copied some example from place Z. So you ask for a URL, and they give you one... you point out the major bit they didn't copy correctly, and they claim they are "just following the examples" and ignore anything you suggest again. Times like these I wonder why I ever tried to help in the first place...

Permalink | Author: | Tags: Mozilla, Users | Posted: 07:03PM on Friday, 03 June, 2005 | Comments: 0

RDF Debugging


Permalink | Author: | Tags: Mozilla, RDF | Posted: 09:14PM on Friday, 03 June, 2005 | Comments: 0


For those wondering, yes, commenting will be added to this weblog at some point. Not sure when, depends how much work I want to do... but it shouldn't be too hard.

Permalink | Author: | Tags: Weblog | Posted: 02:49AM on Saturday, 04 June, 2005 | Comments: 0

Linking libXUL

My build box actually ran out of virtual memory trying to link libxul yesterday. Scary. Luckly my new box (still in bits) will have somewhat more than 256MB RAM so it shouldn't even need VM to link!

Permalink | Author: | Tags: Mozilla | Posted: 01:24PM on Saturday, 04 June, 2005 | Comments: 0

RSS Feed

Ah, that was a fun exercise indeed. I've now got an RSS feed of the last 10 posts, which your browser should pick up automatically.

Permalink | Author: | Tags: Weblog | Posted: 02:41PM on Saturday, 04 June, 2005 | Comments: 0


This is a test weblog post which I will be using for testing some new features on over the next few days. Please don't be alarmed if new features appear here first!

Permalink | Author: | Tags: Weblog | Posted: 04:55PM on Saturday, 04 June, 2005 | Comments: 0

ChatZilla's Munger

It's in need of some slight attention right now. The main problem is that it always prefers munger entries that were enumerated first, which equates to added first in Spidermonkey (Mozilla's JS Engine).

I filed the bug on it a while ago now, but it recently came to my attention that Wikipedia actually had a script that broke stuff because of it. I've corrected the script with version 1.2 so it doesn't completely break normal links, but the real bug still stands.

Permalink | Author: | Tags: Mozilla, ChatZilla, Bugs | Posted: 03:32PM on Sunday, 05 June, 2005 | Comments: 0

File, Edit and View

You don't use those menus, do you? Good, because now you can't in ChatZilla on current Mozilla and Firefox builds! The menus vanished mysteriously one morning, after DOM Inspector got a menu hack-attack. If it weren't for a slight problem with :empty, it'd be ok, but it's not.

Edit: bsmedberg had a good idea to hack around the :empty problem, which I've done into a patch. Hopefully this will be fixed soon in trunk (and will be in

Permalink | Author: | Tags: Mozilla, ChatZilla, Bugs | Posted: 05:05PM on Sunday, 05 June, 2005 | Modified: 05:41PM on Sunday, 05 June, 2005 | Comments: 0

Cairo Burns

So, I added --enable-svg and --enable-canvas to my Mozilla Firefox build config the other day. Apparently that forces me into building Cairo, which, um, doesn't build.

m:\source-HEAD\mozilla\gfx\cairo\cairo\src\cairo-win32-private.h(43) : warning C4005: 'WINVER' : macro redefinition
       command-line arguments :  see previous definition of 'WINVER'
../../../../../../source-HEAD\mozilla\gfx\cairo\cairo\src\cairo-win32.h(66) : fatal error C1189: #error :  Cairo was not compiled with support for the win32 backend

make[4]: Leaving directory `/cygdrive/m/tree-firefox-debug/objdir/gfx/cairo/cairo/src'
make[3]: Leaving directory `/cygdrive/m/tree-firefox-debug/objdir/gfx/cairo'
make[2]: Leaving directory `/cygdrive/m/tree-firefox-debug/objdir/gfx'
make[1]: Leaving directory `/cygdrive/m/tree-firefox-debug/objdir'
make: Leaving directory `/cygdrive/m/tree-firefox-debug/objdir'

Update: It seems that, on way or another, I ended up with cairo-features.h in the srcdir without it being listed in CVS\Entries, so it never got removed. Thanks to vlad who suggested looking for that file in other locations. Let's hope it builds now.

Permalink | Author: | Tags: Mozilla, Firefox | Posted: 09:51PM on Sunday, 05 June, 2005 | Modified: 01:28AM on Monday, 06 June, 2005 | Comments: 0

It's Monday. I can tell.

You know how you get the feeling of what day it is by how well things go? Well, today has definately been a Monday - and a bad one. Not so much bad as in hardware blowing up, or injuries or anything, just people out to get me. All of them.

Permalink | Author: | Tags: Life | Posted: 08:44PM on Monday, 06 June, 2005 | Comments: 0

Last Minute Panic

Dontcha just love 'em? My work suddenly seemed to decide that everything you could think of needed changing/doing with the project, and with the deadline at 1pm! Oh, and we had some fun with rsync not setting the times on the synced files for part of the day. I've done over 18 hours work over yesterday and today, and man I'm worn out.

And my PSU hasn't arrived yet. europc--

Permalink | Author: | Tags: Work, PC | Posted: 09:25PM on Tuesday, 07 June, 2005 | Comments: 0



I'm off down to London again, so don't expect any bug fixin' 'til Friday!

Permalink | Author: | Tags: Work | Posted: 10:28AM on Wednesday, 08 June, 2005 | Comments: 0


Filtering stuff is one of those things everyone wants to do, no-one wants to write the code, and even fewer debug it when it breaks. Luckly, this doesn't stop some intrepid people from adding message filtering to ChatZilla.

Of course, the killer questions are:

  1. what to filter with, and
  2. what to do when a filter matches?

The first question is relatively easy, we only have a limited number of things known about a message anyway.

  • Actual message text, of course
  • The type of message (PRIVMSG, NOTICE, JOIN, etc.)
  • Source (e.g. the user object)
  • Destination (e.g. the channel object)
  • Handler source. This one is the fun one - it is where the message was displayed 'from', e.g. what code handled it. It may sound useless, except that it will be the default destination if no filters match, and will mean messages for a particular channel will appear on that channel (at least by default!).

That was easy. So what about the actions? They are more fun, as this is where the real flexibility will lie in the system. What I see being included:

  • display on source view
  • display on destination view
  • display on handler view
  • display on handler's parent view
  • set message priority (superfluous, activity, attention)
  • make sound (beep or WAV file)
  • activate/flash window
  • run some JS?

Update: Draft spec is here:

Permalink | Author: | Tags: Mozilla, ChatZilla, RFE | Posted: 04:13PM on Friday, 10 June, 2005 | Modified: 09:57AM on Tuesday, 14 June, 2005 | Comments: 0

null exceptions

Ever had the feeling the program is simply mocking you? Try doing addEventListener("foo", null, false) in a recent Mozilla or Firefox and you might understand.

It doesn't just throw an error (which seems a fairly sensible thing to do, given I'm providing bogus data), it throws a null error. That's not an error which says only "null", the actual error object itself is null! What gives?

With my basic knowledge of how XPConnect & co. work I can't work out how this can happen normally. The following three cases are what normally happens:

  • If it can't convert the input data to make the wrapped call, you get a huge long error about "arg 1" or whatever.
  • If the native code returns a failure code, you get an exception generated by XPConnect (which is just as excessive as the argument error one).
  • If the native code returns a success code, you get the _retval, and not an exception.

This leaves only one option I can think of: the native code is returning a failure code, but also setting the XPConnect exception object explicitly to null (which allows native code to return nice exceptions to JS and other scripted languages, while still returning the 'nice' NS_ERROR code to native callers). I just can't find out where.

Permalink | Author: | Tags: Mozilla, XPCOM | Posted: 10:38PM on Sunday, 12 June, 2005 | Comments: 0




Anyone home?


The world is out to get me.

Permalink | Author: | Tags: Life | Posted: 08:30PM on Friday, 17 June, 2005 | Comments: 0

Away 20th - 24th

I will be off down to Cov for the entire week, to generally socialise and stuff. Since multiple companies are sucking heavily with regards to something as simple as getting a PSU to me, the computer I am taking may or may not actually have a PSU in it when I take it. I'm willing to bet it wont. The upshot of this is that I may have significantly limited access to anything Internet-like for a few days, but damn well better be on-line by Wednesday.


Since I wont be taking my laptop, will stay up and fully functioning (bearing with issues like ADSL going pop), and all the usual contact methods will work as good as they ever do.

Note: It seems has died as of 12:22AM UK BST, suspected to be heatstroke. That wont be back until last morning Monday I suspect.


I have a mostly working line marker! It's not perfect (when it appears, it doesn't scroll the line into view, meaning it is really hard to tell when it is occuring to the unfocused view), but it works for the most part. Still got some issues with scrollDown and force to sort out...

Permalink | Author: | Tags: Fun, ChatZilla | Posted: 12:19AM on Sunday, 19 June, 2005 | Comments: 0


It's finally arrived! Wooyay wooyay!

Not that I'm there, of course, but it means me new box will work when I get home.

Permalink | Author: | Tags: PC | Posted: 04:55PM on Thursday, 23 June, 2005 | Comments: 0

Speshul Networking

My new box (GALINSTAN) is up and working great - I'm writing this from it. However, it is running Windows XP 64bit (which is great fun) and I lack wireless cards that have network drivers (from what I can find out, only one or maybe two companies have got 64bit drivers currently, which is a shame). So I decided to simply route the small physical LAN in my room through one box that can use the wireless. Here's a summary of before:

Computer Wireless ID Physical ID OS Spec
BRONZE [DHCP] 40 Win2K Pro 200MHz P2, 32MB RAM
GOLD [DHCP] 41 Win2K Pro 1GHz P3, 384MB RAM
SILVER [DHCP] 42 Win2K Pro 750MHz P3, 192MB RAM

Nice and simple - each box is on both the wireless (main) LAN using DHCP and also on my physical mini-LAN by fixed IP. I added Galinstan thus:

Computer Wireless ID Physical ID OS Spec
GALINSTAN 44 WinXP 64bit 2GHz AMD64, 1GB RAM

Then came the fun bit. I needed one of the three original boxes to route between the networks. So I just enable IP routing, right?

A few hours later, I finally found the key - literally.

 IPEnableRouter: 0x00000001

Set that and reboot. At least ipconfig /all agreed with me now. Of course, it's all very well having the box route, but everyone else needs to know this...

This was easy, luckly, and just involved setting the gateway on each box (BRONZE, SILVER, GOLD and GALINSTAN) to 41 (i.e. GOLD), which I'd picked to be the router. That almost works - packets go flying out onto the wireless network, but of course the replies don't know how to come back! So a single persisten route was added to PLATINUM (the wireless router box) to route anything for the physical mini-LAN to GOLD's wireless IP (which I'd now reserved in the DHCP server so it was effectively static).


It worked.

I now have 4 boxes in my room, all with 'net access, and all accessible from the house router (meaning I can forward public ports to them for services as nessessary).

Permalink | Author: | Tags: Home, Network | Posted: 03:14AM on Sunday, 26 June, 2005 | Modified: 03:14AM on Sunday, 26 June, 2005 | Comments: 0

I Never Could Get the Hang of Thursdays

- Arthur Dent, The Hitchhiker's Guide To The Galaxy, by Douglas Adams

You all know what I mean.

On a slightly better note, finally decided to start telling developers what is going on. About time, too.

Permalink | Author: | Tags: Life, Mozilla | Posted: 07:57PM on Friday, 08 July, 2005 | Comments: 0

What's in a Menu?

Everything: you use them almost all the time when using the mouse, and pretty often when using the keyboard - though it varies depending on the app.

ChatZilla's menus have been evolving over time since they first appeared, probably at least 5 years ago now, and are a in a bit of a mess. As part of fixing this, I've drawn up the current menus in the Mozilla Wiki. If anyone wants to move stuff around, into the "right" place, and add notes, they're welcome to, after all, it's a Wiki.

Permalink | Author: | Tags: Mozilla, ChatZilla, Bugs | Posted: 04:45PM on Tuesday, 12 July, 2005 | Comments: 0

It Broke

Begining build of ChatZilla
 WARNING: output XPI will be overwritten.
 Checking XPI structure.....            done
 Checking JAR structure..               done
 Updating Firefox Extension files.....  done
 Updating Mozilla Extension files....   done
 Constructing JAR package..C:\Program Files (x86)\Cygwin\bin\perl.exe (3972): *** recreate_mmaps_after_fork_failed
Cannot determine cygwin mount points. Exiting.
     3 [main] perl 3972 fixup_mmaps_after_fork: WARNING: VirtualProtectEx to return to previous state in parent failed for MAP_PRIVATE address 0x1ED0000, Win32 error 87
   538 [main] perl 3972 fixup_mmaps_after_fork: WARNING: VirtualProtect to copy protection to child failed forMAP_PRIVATE address 0x1ED0000, Win32 error 487
   680 [main] perl 3972 fixup_mmaps_after_fork: ReadProcessMemory (2nd try) failed for MAP_PRIVATE address 0x1ED0000, Win32 error 487
C:\Program Files (x86)\Cygwin\bin\perl.exe (3972): *** recreate_mmaps_after_fork_failed
     6 [main] perl 2916 fork_parent: child 3972 died waiting for dll loading

And thus ended my Cygwin Perl installation. No matter of rebaseall-ing or complete Cygwin reinstalls could fix it, and I don't even understand precicely what the error is.

So a short while later, I had myself an ActiveState ActivePerl install, (a wrapper I wrote to use ActivePerl in places expecting Cygwin Perl), and some tweaks to to let me specify the Perl engine (save me doing path & symlink hacks). And thus it did build! Yay!

Permalink | Author: | Tags: ChatZilla, Cygwin | Posted: 07:40AM on Tuesday, 19 July, 2005 | Comments: 0

Running Late

So much for early-June. Well, the tree is still frozen, which doesn't help at all. Plus reviews are slower than expected. We're also not going to get 0.9.69 out any time soon at all - the next release will be, hopefully not too far away now.

We are, however, making slow but steady inroads into the bug list, and have a few people outside the 'core' (Robert Ginda, Samuel Sieb and myself) who are doing patches and generally helping a lot. It sounds like we're going to get a better (ish) reconnect system shortly, too.

Permalink | Author: | Tags: ChatZilla | Posted: 12:30PM on Thursday, 21 July, 2005 | Comments: 0

Mozilla Update FTS

FTS? Whatever, Mozilla Update is not doing itself any favours. Apparently (according to the developers' page) version 1.0.1 of my extension was denied, despite it being a simple bug-fix over 1.0.0 (which was approved). Naturally there is no indication as to the reason. I would check my mail, but I can't. I've waited two (or is it three now?) days to get this far, and it sucks.

What really worries me is that it may have been denied just because it was a different reviewer. "How does that work?" I hear you ask. Simple. My extension, while small at only 5KB, does a lot of poking at the Windows Registry to do its job. This is not trivial stuff, and it is known to not always work (I can't find any platform/setup locally that fails, however, which doubly-sucks). Version 1.0.1 is better at working that 1.0.0 - it actually fixes a very specific bug for Firefox 1.0.x users, for example.

Yet all it takes is one reviewer who has a system where it happens to not work and all the users lose out. Bring on the multi-reviewer system - oh, and a few times more reviewers too.

Permalink | Author: | Tags: Mozilla, UMO | Posted: 03:12AM on Monday, 01 August, 2005 | Modified: 03:12AM on Monday, 01 August, 2005 | Comments: 0

Branching on Friday?

So Gecko 1.8 is branching on Friday, and there are only 126 bugs to fix before tomorrow night. Good luck!

Update: Oops, seems they're using bug 300860 to track blocking for the branch, so only two bugs left. This will be very amusing, however, when they realise they have a branch with 120 bugs to fix on it.

Permalink | Author: | Tags: Mozilla | Posted: 12:44PM on Wednesday, 10 August, 2005 | Modified: 03:29PM on Wednesday, 10 August, 2005 | Comments: 0

RSS is dead, long live Feedview

So the RSS button is gone from the statusbar, and now appears inside the URLBar. Yes, you heard me (and saw, too). So we now have a less-discoverable, less accessible, less flexible solution (no popup menu with a list of feeds any more!). And to top it all off, they are viewed in a hacked-up badly-integrated version of the Feedview extension (note: I have no idea if the Feedview extension suffers from any of the same issues as Firefox does, and don't mean to imply such), meaning you get suck and more suck, including such gems as bug 302749.

Permalink | Author: | Tags: Mozilla, Firefox | Posted: 09:34PM on Friday, 12 August, 2005 | Comments: 0

To be themed or not? That is the question

Since the menu change in Firefox to make the menus look like WinXP's themed style on all versions of Windows, rather a lot of people have been upset. Quite understandable, really, as the change has just made Firefox's menus look seriously lame on non-themed systems.

So much is the annoyance, that I've gone and started hacking nsNativeThemeWin.cpp to try and make it support theming menu components on Windows. It ain't pretty, but I've managed to fix some bugs with the toolbox and toolbar appearances already (which help a lot), and have got basic themed menupopup and menuitem support going. It lacks the classic appearance currently but, now I know my way around, that should be easier.

Permalink | Author: | Tags: Mozilla, Theming | Posted: 03:10PM on Saturday, 10 September, 2005 | Comments: 0


Well, it's taken a few days of work, but Firefox's finally look right in the Luna XP, Windows XP and Windows Classic themes. Woo.

Permalink | Author: | Tags: Mozilla, Theming | Posted: 09:19PM on Sunday, 11 September, 2005 | Comments: 0

It'll Never Make It

In case you hadn't guessed, I don't believe there is any way my work to fix Toolkit's toolbar and menu appearances on Windows will make it into Firefox 1.5. The patch has been ready and waiting for review since 2005-09-18, yet it'll still completely miss. All it does is demonstrate the problems with the review process. :-(

Also, I've stopped working on the menu shadows bug due to yet more politics (drivers, again).

Permalink | Author: | Tags: Mozilla, Theming | Posted: 01:02AM on Monday, 03 October, 2005 | Comments: 0

Drivers Pull Stupid Stunt

They've gone and done it folks. Without any warning, they have suddenly decided that my branch-only ban-aid needs to land on trunk for baking. WTF?

Permalink | Author: | Tags: Mozilla, Theming | Posted: 01:11PM on Thursday, 06 October, 2005 | Comments: 0

One Regression

It seems that only one regression has so far turned up from the unexpected trunk landing. That's a bit of surprise itself, but there you go.

I'm still expecting drivers to say "actually, no" to the whole thing landing on branch, even though Asa did set blocking1.8rc1+ on the 5th, but we'll see. Asa himself is already going and minus-ing quite a lot of stuff, so things don't look good.

Permalink | Author: | Tags: Mozilla, Theming | Posted: 11:04PM on Monday, 10 October, 2005 | Comments: 0

There Is No Headline That Adequately Explains This

I have now totally run out of patience with drivers, and I honestly don't know what I'll do the next time the screw me or any of 'my' bugs about.

Yes, they forced the theming patch on to branch, but they also screwed it up, in two ways. It is both impressive and unbelievably embarrassing.

Permalink | Author: | Tags: Mozilla, Theming | Posted: 01:00PM on Wednesday, 12 October, 2005 | Comments: 0

It Died

Well, that's it folks. The patch is no longer in the branch, and the menus in Firefox 1.5 will look like crap unless you use the Windows XP default 'Luna' theme.

I wont go in to any more detail than I did in the bug, though I will say that I will continue with this work on trunk for 2.0, due next millennium.

Permalink | Author: | Tags: Mozilla, Theming | Posted: 07:31PM on Friday, 14 October, 2005 | Comments: 0

Building the Builder

Well, I was bored one night and as usual, the result is something useful, but only to me. Having 4 computers here makes things weird enough, but now I can check the status of my build box (and will soon be able to queue up commands) from the web browser on my main desktop - all via my web server box. Wheee!

Permalink | Author: | Tags: Mozilla | Posted: 10:46PM on Saturday, 22 October, 2005 | Comments: 0

Native Theming on Windows

This is just to outline my plans for working on native theming for menus and toolbars, and related bugs:

  • First priority is to get the pure-CSS appearance in winstripe right. The target appearance is Windows XP Classic. This is being worked on in bug 313388. This is currently going well, and is nearing completion.
  • One single native bit will be written with the pure-CSS version - -moz-MenuBarHoverText, which will be (at this stage) implemented (on platforms which use winstripe, or maybe all) to work exactly like the CSS colour chosen for the hover text colour.
  • The above work will be checked in to trunk CVS (after reviews, etc.) and will remove the -moz-appearance properties currently there. This will mean everyone will see the pure-CSS appearance, and this is exactly what is intended.
  • Any problems found with the pure-CSS appearance (excluding, obviously, "it's not themed!" which I know a few idiots will file) will be fixed at this point, as once the native code is re-enabled, only odd groups of people will ever see it again (OS/2 users, for example).
  • At this point, there will be an expected regression with the Firefox Bookmarks toolbar, which will be fixed (the exact method is yet to be decided).
  • Work will then begin on the native code necessary to support the theme engine in Windows XP. This may involve some minor tweaks to the CSS, but the ideal result is that only -moz-appearance properties are added. -moz-MenuBarHoverText will be adjusted (on Windows only) to follow the correct rules for menu bar text with themes.
  • This will be reviewed and checked in, and that will be it done.
  • Regressions will be fixed here, of course. :-)

It may seem like a long-winded way of doing things, but it will mean you get a top-quality result, as each key part will be written separately, tested separately, and checked in separately (for regression spotting).

Permalink | Author: | Tags: Mozilla, Theming | Posted: 12:56AM on Wednesday, 02 November, 2005 | Comments: 0

Weblog Issues

I've noticed a few bugs and general issues with my weblog software today, so I'll just write it down so you know and I don't forget to fix it on Friday or at the weekend.

  • RSS feed for comments is needed (both for each post, and globally).
  • RSS feed for tag list, and for posts in each tag needed.
  • RSS feed for each date grouping allowed, maybe.
  • If you preview a comment post for a second time, the URL gains an extra "/", which messes up the stylesheet loading and the page title.
  • Link to next month is broken when next month is December (the page title is right, but it is using month 00, instead of 12).
  • Link to next/previous month doesn't change year if wrapping is involved.

Permalink | Author: | Tags: Weblog | Posted: 02:20PM on Wednesday, 09 November, 2005 | Comments: 0

Mysterious Build Errors

Ever since bsmedberg landed bug 313309 part 2, I've been getting the following build error:

Building deps for ....../../source-HEAD/mozilla/xpcom/glue/nsIInterfaceRequestorUtils.cpp
m:\tree-firefox-main\objdir\dist\include\xpcom\nsIProgrammingLanguage.h(35) : warning C4003: not enough actual parameters for macro 'NS_DEFINE_STATIC_IID_ACCESSOR'
m:\tree-firefox-main\objdir\dist\include\xpcom\nsIClassInfo.h(36) : warning C4003: not enough actual parameters for macro 'NS_DEFINE_STATIC_IID_ACCESSOR'
../../dist\include\xpcom\nsIInterfaceRequestor.h(41) : warning C4003: not enough actual parameters for macro 'NS_DEFINE_STATIC_IID_ACCESSOR'
m:\tree-firefox-main\objdir\dist\include\xpcom\nsISupportsUtils.h(202) : error C2039: 'GetIID' : is not a member of 'nsIInterfaceRequestor'
       ../../dist\include\xpcom\nsIInterfaceRequestor.h(38) : see declaration of 'nsIInterfaceRequestor'

This is the same error that the 'creature' tinderbox was getting, but unlike that machine, it didn't go away after all the bustage-fixes.

After nuking objdir/dist and objdir/xpcom/base - neither of which helped - I just nuked objdir/xpcom and it finally looks like it might work, having got twice as far into the build as the error point (though it would not surprise me if I needed to nuke other totally random directories later).

The build system is just broken.

It's getting to the point where I'm tempted to make an auto-build-fixer, that just keeps nuking random objdir folders until it works. It would be a more productive use of my time than fixing all these damn problems by hand.

Permalink | Author: | Tags: Mozilla, XPCOM | Posted: 02:26AM on Saturday, 12 November, 2005 | Comments: 0

Weblog Issues - Fixed

I've gone through and fixed all but one of the items listed previously. The only item remaining is the RSS for tags and the tag listing itself.

If you find anything is not working, please do comment, and I'll fix it. Please note that the comments feed is not linked from anywhere, as it is only useful for me monitoring new comments.

Permalink | Author: | Tags: Weblog | Posted: 06:44PM on Saturday, 12 November, 2005 | Comments: 0

Progress Bars

Continuing from my previous post about my Mozilla build tracker, I've added progress bars to the display while it is building. Naturally, it's all done with smoke and mirrors [1], and (perhaps surprisingly) doesn't mess up the display on Internet Explorer at all, though it doesn't show the progress bar.

Actually, thanks to the IE DOM Explorer, part of the IE Developer Toolbar (download from here), I've just fixed the progress bar to work in IE. Woo!

[1] Unofficially, it is all done with CSS, DIVs and PNGs, but we don't like to talk about them...

[2] Worryingly, it is more usable than Mozilla/Firefox's DOM Inspector, for me. I think it's partly that the select-by-click in the IE DOM Explorer totally trumps the same feature in Mozilla/Firefox.

Permalink | Author: | Tags: Mozilla, IE | Posted: 03:23AM on Friday, 18 November, 2005 | Comments: 0

ChatZilla 0.9.69

The next ChatZilla release will be called 0.9.69, mostly due to the excess of changes that have occurred on trunk since the last release. This will be the codebase merged to trunk, plus some extra compatibility updates; this means all the trunk changes since then will be included - and there are a lot of them!

Look forward to it!

Permalink | Author: | Tags: ChatZilla | Posted: 12:27PM on Wednesday, 30 November, 2005 | Comments: 0

It's Out!

I suppose I probably should mention that ChatZilla 0.9.69 has been released. Woo.

We're currently working through a couple of big issues, but the release is great. The main problem anyone is likely to encounter is bug 319066, which should only occur if you select the top user in the userlist, and then switch tabs.

Permalink | Author: | Tags: ChatZilla | Posted: 04:48PM on Monday, 05 December, 2005 | Comments: 0


It's been released, but may take a few days to appear on This update fixes the userlist/tab switching issue mentioned previously, as well as some less interesting things, like adding support for /dcc-accept with non-regexp parameters.

As ever, the revisions list has the gory details, including bug links and download link for those too impatient for

Permalink | Author: | Tags: ChatZilla | Posted: 02:14AM on Saturday, 17 December, 2005 | Comments: 0

Weblog URL updates

Well, I finally got around to nuking that annoying ".pl" in the weblog's URLs - all the URLs should now not include it. Naturally, I've put a redirect from to weblog, so no links should break.

It took quite a bit of thinking, but it works - not showing file extensions is not one of IIS' strong points, however, it has more than enough power to do it if you know what you're doing. :-)

Please comment here if you find any bugs or other issues with this change, or indeed any part of the weblog system.

Permalink | Author: | Tags: Weblog | Posted: 03:23AM on Saturday, 17 December, 2005 | Comments: 0

Feed Icons

It's nice to see a good bit of collaboration over something as visually important to users as icons. Since this is all good for the user, Feed Icons has sprung up to help everyone follow suit, and so my weblog now has those funky little orange icons in the top-right corner of every page with a feed.

Permalink | Author: | Tags: Weblog | Posted: 06:16PM on Saturday, 24 December, 2005 | Comments: 0

Contributor Relations

The Mozilla Foundation are really trying my patience with two issues right now:

  • No tinderbox graphs.
  • No despot (user accounts thing).

In both cases, the relevant pages have simply been blocked, returning error 403 or 500 and no information whatsoever has been provided. The closest I got was after filing a bug for the first one, where I got a comment from one admin of "That's on purpose. The script has issues.". The only information on the Despot issue is "security" - fat lot of good that is!

I'm am going to assume that both things are currently exploitable, which means that they should be fixing them, which they do not appear to be doing. If they know they can't fix them fast (or don't care), as appears to be the case, where's the bloody message saying so? It's absolutely appalling.

Permalink | Author: | Tags: Mozilla | Posted: 07:51PM on Saturday, 07 January, 2006 | Comments: 0

Why Peer Pressure Sucks

It may seem obvious to some, but the level it sucks at is surprisingly high, and causes some rather bad results.

Let's start with school; for reasons not entirely known to me, by about age 12 or 13 I'd already been labeled a "geek", in so much as I was "always right". Sure, I was probably the best in my year at maths, and not bad a physics, but quite how it all extrapolated I'll never know. Anyway, this led to someone always laughing if I answered a question wrong, which in turn meant by year 10 and 11 (age 15) I just didn't answer anything unless I was really sure. That kinda sucked. It also resulted in most of my shyness.

At collage (age 16 - 18 here in the UK), things were actually good; very little of the teacher asking questions, lots of just doing the work. Plus for various things, some people actually came to me for help.

(This is all going somewhere, promise.)

University (18 - 21) was even better, as there was entire groups of people with the same (and better) knowledge of things I'm interested in, and (probably much more importantly with regard to enjoying my time at Uni) people who liked me for being me, not because I knew the answer to something.

Since then (2 and a bit years), I've stayed in the IRC channel for the Computing Society (of which I was the webmaster for my time at Uni), and that has brought about another change. Firstly, I'll just say that IRC is probably the single worst way to have conversation, discussion, argument, whatever, with more than two people (it sucks pretty hard even for two people when it is an argument).

Anyway, one of the things that happens on IRC is the "mob effect", where one person says something that a lot of people disagree with, and they get mobbed. Often so much so that they simply don't get any chance to explain, defend or otherwise expand on their initial comment. The result of this is that I suspect a number of people, myself included, simply don't bother saying what they think sometimes, because they know that a) no one will actually listen, and b) they wont get a chance to explain anyway.

This has been exemplified over the past few days, as the IT people at Uni had some issues on Friday, and members of the IRC channel have been steadily decrementing their karma (karma works by saying Thing++ or Thing-- to show what you think of Thing), and although I don't want to defend what happened on Friday, I do object to a number of the reasons people have used for the decrementing, as well as the sheer volume of it, but I wont say anything because of the issues above.

Finally, to bring a side to this whole story; I simply don't file bugs when things break here with my own builds any more, even though it is obvious that it's nothing to do with me. The reason: because certain people have (incorrectly) blamed things on the fact my builds are depend builds and other things relating to my setup. Either depend builds are fucking supported or they are not. My conclusion is that they are (unofficially only) most definately not, and that's simply because the entire build system is a pile of crap that breaks if you look at it funny.

And this is why peer presure is bad - it makes people keep things to themselves when they really shouldn't.

Permalink | Author: | Tags: Life | Posted: 03:39AM on Tuesday, 24 January, 2006 | Comments: 0

Instrumentation Data

We're actually getting some interesting data coming it from the instrumentation "startup ping".

In particular, you can see that people updated from Firefox 1.5 to almost the moment it was released.

We can also estimate that there are approximately 1100 new or updated ChatZilla users a week (calculation: sum up new installs [the numbers on the pages linked, 40], divide by the number of days [25], multiply by 700 [1% and we want per-week]).

Permalink | Author: | Tags: ChatZilla, Firefox | Posted: 03:55PM on Monday, 06 February, 2006 | Comments: 0

My First Places

So I tried building Firefox with Places a couple of days ago. It ain't pretty. It's definitely good that it isn't being released for a while yet.

  • Importing my history and bookmarks took 15 minutes and have no UI what so ever. I only didn't kill the Firefox process using 100% CPU because I saw some sqlite files in my profile being created and modified.
  • The imported data is 74% bigger than the original. This seems like a rather big increase.
  • My personal toolbar is now empty, and I can't add stuff to it. (The folder in Places for this has the right stuff on, though.)
  • The URL bar does not update when changing tabs or if the page loads. This means it only ever displays what I last typed.
  • The URL bar has no autocomplete, either as-you-type or using the drop-down button.
  • All links appear unvisited, despite clearly being in history.
  • None of the navigation toolbar works either, so no back/forward/reload/stop. Not even the shortcuts work.
  • The browser no longer opens my homepage on start, even though it's still configured the same. It just loads about:blank, and if I click the Home button it loads the Google start page!
  • Add Bookmark menu item does nothing.
  • Selecting either the History or Subscriptions items in Places itself hangs Firefox for 5 - 10 seconds. I think this is the calendar, rather than the data, as searching the history only takes 1s second.
  • There is no way to select all dates in the calendar, even though it is (apparently) the default when going to a view.
  • Expanding or collapsing a folder in Places' top-left box takes 5 - 10 seconds if Subscriptions is selected.
  • Switching folders keeps the search string, but doesn't search the newly selected folder.
  • Searching "Current Collection Only" doesn't do anything.
  • Adding a search filter then can't be removed.
  • Filters only allow "starts with", "is" and "is on site" for location matching. Where's "contains"? Have to use "Containing the text" for that, which includes the title.
  • No title search option! Have to search title and URL together.
  • The icons for items are not fixed, so appear badly inconstant, as they show the favicon.
  • Properties context-menu item does nothing, at all, anywhere in Places.
  • Quitting Firefox closes all windows, then spends at least 15 seconds using 100% CPU (saving something I think).

Please don't bother pointing out that Places isn't finished yet - I know that, and it sure shows.

Permalink | Author: | Tags: Firefox, Places | Posted: 11:38PM on Sunday, 05 March, 2006 | Comments: 0

SUSE Net-install

So, here I am, 1am and nothing (interesting) to do. Time for something completely random, I think! Let's do a network install SUSE.

  • plus Like the graphical "thing" that starts right away from the CD.
  • minus "Press Esc for more information" it says. Do that, and you're dumped to text-mode - which is ok - but there's no way to get back to graphical-land (that I could find).
  • minus In the default graphical-land, the download of the network installer (all 65MB of it!) happens with no progress indication. The text-mode loader does show the percentage, though.
  • minus After all that pretty graphics (that did very little, mind), it imediately dumps you into text-mode YaST. Not fun!
  • minus Navigation in the text-mode YaST is a real bitch. It just seems to not follow any sort of convention. Esc never seems to close things, even the Help window which says it does! (Thank Bob that F1 also closes that window.)
  • minus It asks me whether I want KDE or GNOME. How the f**k should I know? So it helpfully explains how my decision will affect things:
    • KDE is a powerful and intuitive desktop environment that uses Kontact as its mailer and Konqueror as its browser and file manager.
    • GNOME is a powerful and intuitive desktop environment that uses Evolution as its mailer, Firefox as its browser, and Nautilus as its file manager.

    Joy of joys. They're the same. Why tell me what apps they use, when I've no idea what the difference in them are? If I didn't already know what these apps are like, or that KDE is supported better by SUSE, I'd be completely fucked at this point - it's anybody's guess which would be the better option. Oh, and there's an Other button too!

  • minus Now it analyses my system, and starts "Evaluating package selection..." (all I did was pick KDE), which really means let me download some more data with no progress indication. Judging by the time taken and my network graphs, I'd say it is probably downloading about 30MB.
  • minus And it's at this point (2:33am) that YaST just hangs entirely. Woohoo. Can't "click" (use the shortcut Alt-<letter>) any buttons, bring up help or anything, and it still says "Analysing your system..." under Software and Language.
  • AHA! 8 minutes later (!) it finally is done and ready for me to use it again. That was really bad.
  • minus Doing anything in the "Installation Settings" screen seems to take a decade to work, even opening the list of things to change (4 items)! Click accept button (2:46am).
  • Two minutes later, is asks if I'm sure I want to install! And now it starts formatting the disk.
  • minus By 3am, it's failed on the formatting. "An error occured during the installation." Joy.

And that's it. Can't get it to continue from this point - it spams something to the screen that looks like an error, but overwrites it with the error dialog almost instantly, and Ctrl-Alt-<number> doesn't work so I can't get at any logs or anything. Sucky.

Permalink | Author: | Tags: Linux, SUSE | Posted: 04:46AM on Thursday, 09 March, 2006 | Modified: 04:47AM on Thursday, 09 March, 2006 | Comments: 0

Technology vs. Common Sense

If you ever thought e-mail <--> newsgroup gateways were a good idea, think again. I think the following quote sums it up absolutely perfectly:

This is horrible. It feels like I’m playing some 21st century Web 2.0 version of “telephone”.

Permalink | Author: | Tags: Mozilla, Newsgroups | Posted: 12:36PM on Thursday, 30 March, 2006 | Comments: 0

And So It Begins

Here we go again, and this time I'm not in the mood for any shit. I just want to warn everyone that if I get messed about with this again, I'm just going to stop. They can find some other deluded soul to fix their mistakes next time.

Permalink | Author: | Tags: Mozilla, Theming | Posted: 04:09AM on Sunday, 09 April, 2006 | Comments: 0

The Ghost Buster now supports Thunderbird

That's right, we've finally got Thunderbird 1.5 support plugged in to the JavaScript Debugger. This also fixes the -venkman command-line flag to work with all toolkit-based apps (Firefox, Thunderbird, etc.).

Permalink | Author: | Tags: Venkman, Thunderbird | Posted: 10:28PM on Thursday, 13 April, 2006 | Comments: 0

Purty Pictures

Ever wondered what download rate the top extensions on Mozilla Addons get? How much it varies by hour/day? Even which are rising and which are falling?

Well, now you can.

(Yes, I was bored. Very bored.)

Permalink | Author: | Tags: Firefox, UMO, Fun | Posted: 07:35PM on Sunday, 23 April, 2006 | Comments: 0

Anger Rising

It's not a good day, and there's plenty to piss me off:

  • Someone broke the authentication for Trac (wants-to-be-Bugzilla when it grows up thing) for the IRC bot I work on.
  • Possibly same someone put a disrespectful message on the test version of the new Computing Society website.
  • Still no-one wants to review the patch which is blocking my theme work.
  • My ISP is dropping my ADSL connection an average of 15 times a day for the last two weeks.

Permalink | Author: | Tags: Mozilla, Theming, Bugs | Posted: 02:14PM on Saturday, 20 May, 2006 | Comments: 0

Firefox's Anti-Phishing

So I thought I'd try it out, as it's supposedly worth using. Ha. It suffers from a few rather fundamental problems here:

  • It waited three seconds before claiming the test page was actually a phishing page, even though it's set to use a locally stored list.
  • It changed my browser's configuration just to display the warning - it turned on the navigation toolbar, which would have been fine if it then hid it again after, but it didn't.
  • It positions its "disabled webpage" XUL code horribly, which causes the following extra statusbar row:

    To change into:

    In other words, it inserted itself between extensions and other code usually attached to the statusbar, and the statusbar itself, causing them to all end up at the top of the screen. If you don't see how silly this is, look what happens with the find toolbar:

  • Finally, it completely fails to display the "disabled" webpage. I expect this is Canvas and Cairo being shit, but it none the less makes the entire feature rather sucky.

Permalink | Author: | Tags: Firefox, Phishing | Posted: 01:32PM on Monday, 29 May, 2006 | Comments: 0

BFL - June 19th to 23rd

So it's that time of year again, when I disappear for an entire week to spend it in a room full of 55 other computer 'freaks' and generally have a good time (that's play games and socialise, before anyone gets any weird ideas!).


The past few days I've been building up my own PPM (ActiveState Perl Package Manager) repository, which basically involves downloading CPAN packages, building them, testing, then packaging up the built code. It's really not fun, as plenty of CPAN packages fail to build, or fail tests, and generally cause problems. Still, it's getting there. The aim, what there is of one, is to have all the packages needed for Task::Catalyst and the new CompSoc website project. I think the website will be finished before my repo. is. :(

Permalink | Author: | Tags: Fun, CompSoc | Posted: 04:09PM on Friday, 16 June, 2006 | Modified: 04:26PM on Friday, 16 June, 2006 | Comments: 0

Debugging Hung Scripts

There's some good fun work out on in bug 341764 allowing you to just smack "Debug" on the ultimate in annoying dialogs. I'm going to love that when it's done.

(Only two days to go!)

Permalink | Author: | Tags: Mozilla, Venkman | Posted: 01:49PM on Saturday, 17 June, 2006 | Comments: 0

ChatZilla 0.9.74

It is coming, eventually. The plan for last Sunday didn't quite work out, but we're now ready to go and should be rolling it out this coming weekend instead. To come in this version:

  • /who now accepts multiple parameters (useful for some of the more extended syntaxes some servers support).
  • A small leak for plugins has been fixed.
  • Sending files with DCC closes the file handle in more cases so it shouldn't stay locked after.
  • When midnight happens, the messages about the logs changing have been suppressed.
  • If your connection to a server drops, ChatZilla will now try to reconnect to the same server first, before moving on to the next one in the list.

Permalink | Author: | Tags: Mozilla, ChatZilla | Posted: 03:43PM on Wednesday, 21 June, 2006 | Comments: 0

On Life And Books

I ordered some books from Amazon last Sunday and, much to my surprise, they turned up on Tuesday morning (despite being scheduled for dispatch on Wednesday). The result is that I've been reading book one in Robin Hobb's Farseer trilogy, "Assassin's Apprentice".

Assassin's Apprentice, and I suspect the others in the trilogy, are written in a slightly odd 1st person recollection of events. Took a little while to get used to it, but it actually works very well for the most part - once you actually get to know the main character who is doing all the reciting of events. I'm wondering if there will be any significance to this way of telling the story, but for now it's just different.

It's a good read (so far), though some bits brought back some rather painful memories and regretted times for me. Not that that stops me reading a good book! I'm currently about three quarters through book one, so we'll see what happens over the weekend - hopefully I wont finish the entire trilogy in 3 days, like I did with some previous books (it's not that I don't want to read more - quite the opposite - but reading them all in one go both uses up a lot of free time and leaves you suddenly with a huge void when you finish).

Permalink | Author: | Tags: Life | Posted: 08:17PM on Friday, 30 June, 2006 | Comments: 0

The Art of the Proxy Configuration

ChatZilla doesn't support HTTP proxies. Well, I say "HTTP proxies" but this particular thing apparently has far too many names:

  • HTTP proxy [X-Chat]
  • SSL proxy [Firefox]
  • CONNECT proxy
  • URL proxy
  • Secure proxy
  • (incorrectly) HTTPS proxy
  • "Proxy" protocol [mIRC]

Personally, most of them are completely meaningless terms. "HTTP proxy" is what everyone (except Ben C) calls it.

The problem that we face is relatively simple from the top, but a real pain to actually fix: we ask Necko (Mozilla's network layer) to pick the proxy to use.

The result is that it will ask PAC if so configured, just return the SOCKS v4/v5 proxy, or return no proxy. The user has no way to configure Necko to give IRC the HTTP proxy.

There are a few things we can do, none of which seem that good:

  • Add a single option to force ChatZilla to use Necko's HTTP proxy.
  • Add an option to pick between none, HTTP proxy and SOCKS proxy.
  • Add an option to override Necko, and also add host/port settings.

The real problem is that adding any options will be creating two locations for proxy configuration. Could we automatically pick when to ignore Necko's choice? Maybe. Should we? Probably not.

If only I got paid for doing this...

Permalink | Author: | Tags: Mozilla, ChatZilla | Posted: 03:48PM on Tuesday, 04 July, 2006 | Comments: 0

What Value Tinderboxes

The Mozilla Foundation did a major colo move on Saturday. Naturally, all the tinderbox trees were closed from Friday evening so that no-one tried to do anything silly during the move.

Now, it is Monday and the current tree has the following tinderboxes:

  • Linux comet Dep release
  • Linux lhasa Dep release (gtk2+xft)
  • Linux nye Dep bloat
  • MacOSX Darwin 7.9.0 planetoid Dep (temporary)
  • MacOSX Darwin 8.7.0 bm-xserve02 Dep Universal release

And the only reason the tree is currently still closed is this:

"The tree is CLOSED until we get luna or btek tinderbox up, for reliable Tp coverage."

Both luna and btek are Linux tinderboxes. That means that no-one cares that there are no Windows build boxes currently up and running properly. Seriously. That is just lame.

There is no way I am checking a single thing in until we have a decent set of builds testing the most-used tier 1 platform.

Permalink | Author: | Tags: Mozilla | Posted: 07:46PM on Monday, 17 July, 2006 | Comments: 0

That's It

I've had way more than enough of the fucking attitude from people. I am no longer going to any work on anything related with the exception of my continuing commitment to ChatZilla and Venkman (the JavaScript Debugger).

Have a nice day. You bitches.

Permalink | Author: | Tags: Mozilla | Posted: 09:01AM on Wednesday, 23 August, 2006 | Comments: 0

NVidia: 1 point for, 427 against

Last Sunday, I decided to start playing Oblivion again. Turns out I hadn't uninstalled it from last time, so that was easy. Except for the video playback problem - this was a problem originally, clearly hadn't fixed itself, and consists of all videos (opening sequence, main menu background, etc.) to be static images from apparently uninitialised memory. Which is fun.

Begin Oblivion Video Fixing Sequence, Take 2 (I'd tried before when I first got the game).

  1. Download and install latest NVidia drivers for Windows XP 64bit (91.31).
  2. Run Oblivion, and find the videos work!

Well that was shockingly easy. That is the one and only point for NVidia, though.

The first and most obvious downside to updating my NVidia drivers was the new "NVidia Control Panel", which does a good job of not quite matching Explorer is every way. It's got a UI consistency you could only otherwise have found on 1995 shareware, too. Horrible.

The looks of the new control panel 'thing' are not the only problems it has, oooh no. If you're running as a Standard User Account (LUA), as everyone does (right?), it fails to save any of the application-specific 3D settings. Better than that, it looks as if they were saved when they weren't! (The Apply button disappears in a fit of horrible UI design and there's no error message at all.)

Then there is the "NVidia Display Driver Service" (nvsvc64.exe), which sits in the background doing (apparently) nothing except leaking. It was leaking Paged Pool, Non-paged Pool, Commit and Handles earlier, although currently it only seems to be leaking Non-paged Pool and Handles. The 3 memory values were leaking at a combined rate of (approximately) 1.8MB/hour, and the handles at (approximately) 1700/hour. Yummy.

Finally, we come to the actual driver itself. The main deal. Which leaks entire processes through a really bizarre bug.

For this to make sense, I'll explain a few simple facts about the Windows Kernel:

  • It has an Object Manager that tracks all objects in kernel-space and user-space.
  • All objects have a "Handle Count" and "Pointer Count" - the former is for (obviously) any open handles to the object, which is mostly for user-space code, and the latter is for kernel code that simply has a pointer (it's a reference counter).
  • When both counts reach zero, non-permanent (i.e. most) objects are removed and cleaned up.

When you start a new process, naturally there enters into existence a kernel "Process" object (along with all the shenanigans that go with that). I started the NVidia Control Panel for this test.

lkd> !process fffffadfb2fe2750 1
PROCESS fffffadfb2fe2750
   SessionId: 0  Cid: 14c4    Peb: 7fffffd4000  ParentCid: 0230
   DirBase: 9546c000  ObjectTable: fffffa80009c0580  HandleCount: 189.
   Image: nvcplui.exe
   VadRoot fffffadfb1996b30 Vads 202 Clone 0 Private 4739. Modified 240. Locked 0.
   DeviceMap fffffa800249dc10
   Token                             fffffa80077cbcf0
   ElapsedTime                       00:00:48.515
   UserTime                          00:00:00.000
   KernelTime                        00:00:00.000
   QuotaPoolUsage[PagedPool]         1287904
   QuotaPoolUsage[NonPagedPool]      16720
   Working Set Sizes (now,min,max)  (8402, 50, 345) (33608KB, 200KB, 1380KB)
   PeakWorkingSetSize                8698
   VirtualSize                       657 Mb
   PeakVirtualSize                   658 Mb
   PageFaultCount                    15774
   MemoryPriority                    BACKGROUND
   BasePriority                      8
   CommitCharge                      5247

lkd> !object fffffadfb2fe2750
Object: fffffadfb2fe2750  Type: (fffffadfb5ab86c0) Process
   ObjectHeader: fffffadfb2fe2720
   HandleCount: 2  PointerCount: 74

Most of the above is not too important, but the Image: and two counts from !object are - notice it starts with 2 handles and 74 pointers (2 of which will be the 2 handles). These are all constant while I look around the control panel. Then I go to the "Adjust image settings with preview" page, which has a real live 3D animation. Big mistake! Only moments after going to it, the object has:

   HandleCount: 2  PointerCount: 2029

And it keeps going up, even after switching to another view! It was going up at something like 1000 pointers/second, although I don't have timestamps for my debugging log. By the time I closed the application, it was:

   HandleCount: 0  PointerCount: 21516

Notice that there's no handles - nothing in user-space cares about it any more. There's still over 21,000 pointers to it in kernel-space, though. Or so the Object Manager is lead to believe. One last look at the process object in detail gives:

lkd> !process fffffadfb2fe2750 1
PROCESS fffffadfb2fe2750
   SessionId: 0  Cid: 14c4    Peb: 7fffffd4000  ParentCid: 0230
   DirBase: 9546c000  ObjectTable: 00000000  HandleCount:   0.
   Image: nvcplui.exe
   VadRoot 0000000000000000 Vads 0 Clone 0 Private 253. Modified 769. Locked 0.
   DeviceMap fffffa800249dc10
   Token                             fffffa80077cbcf0
   ElapsedTime                       00:01:31.953
   UserTime                          00:00:11.593
   KernelTime                        00:00:02.734
   QuotaPoolUsage[PagedPool]         0
   QuotaPoolUsage[NonPagedPool]      0
   Working Set Sizes (now,min,max)  (6, 50, 345) (24KB, 200KB, 1380KB)
   PeakWorkingSetSize                11123
   VirtualSize                       80 Mb
   PeakVirtualSize                   670 Mb
   PageFaultCount                    27942
   MemoryPriority                    BACKGROUND
   BasePriority                      8
   CommitCharge                      0

Interesting points on this are that the ObjectTable, VadRoot and CommitCharge are now all zero. This means that the process' virtual address space has been cleaned up entirely. The process is not even in the session process table (list of processes for the logged in session), although it is in the overall kernel process table (which nothing in user-space can see - Task Manager can't see it).

So what's happened? Almost certainly, a driver (most likely the NVidia one, since this only happens with applications that use 3D acceleration and only since the driver upgrade) is adding a reference count to the process it is handling but not releasing it. Thus, leaking hundreds of reference counting points (there's unlikely to be any actual leaked pointers). A few of my Oblivion processes have over 3 million PointerCounts.

Excellent work, NVidia. You've managed to leak in such a special way that no-one will even notice. Except me and my wonderful friend windbg.

Permalink | Author: | Tags: NVidia | Posted: 02:30AM on Friday, 01 September, 2006 | Comments: 0


   <vcard>  BEGIN:VCARD   FN:Author   EMAIL;   END:VCARD   </vcard>

'Nuff said.

Permalink | Author: | Tags: Work, XML | Posted: 08:05PM on Monday, 18 September, 2006 | Comments: 0

Start Shortcuts

So I was bored, and reading Faux's weblog archive, and found this one on mouseless use.

That got me thinking - just how many shortcuts does Windows have, that most people don't know? I'd wager it's more than you think.

The most well-known set are <Win> plus a letter. So here's the list for Windows XP Professional x64:

  • <Win> - Start menu
  • <Win>+B - Focus Shell Notification Area
  • <Win>+D - Show Desktop (toggle)
  • <Win>+E - Open Explorer
  • <Win>+F - Open Find for Files
  • <Win>+<Control>+F - Open Find for Computers
  • <Win>+L - Lock Workstation
  • <Win>+M - Minimise all windows (<Win>+<Shift>+M to undo)
  • <Win>+R - Run dialog
  • <Win>+U - Run Utility Manager (accessibility stuff lies this way)

<Win>+B took me by surprise, so I figure not everyone knows all of the above list, however common some of them are.

The others, people only seem to know a few, and I suspect there will be at least one new one here for everyone.

  • <Win>+F1 - Open Help and Support Center
  • <Win>+<Break> - System Properties
  • <Win>+<Tab> - Focus and select taskbar programs (try this - you'll like it)
  • <Control>+D - Delete in Explorer
  • <Control>+<Esc> - Start menu for luser keyboards
  • <Control>+<Shift>+<Esc> - Task Manager
  • <Shift>+<Menu> - Extra context items (like Run As and Open With)
  • <Shift>+F10 - Context menu (<Shift>+<Menu> for luser keyboards)
  • <Shift>+[No] on Explorer dialogs - No to all

Those are all the ones I can remember and/or find right now. I am certain that there are more, but they will have to be documented another day.

The reason for all this was because Faux's description of using <Win>+D then <Tab> to get to the start menu button seemed excessive, and has the side-effect of minimising all running programs. <Win>, <Esc> is much easier to focus the button, but for what he's suggesting, you just need to hit <Win> and use the arrow keys. Or, if you want to play with your open applications, <Win>, <Esc> then <Tab> to the taskbar, though I prefer <Win>+<Tab> to do that directly.

Now if only my weblog had a shortcut for [Post]...

Permalink | Author: | Tags: Windows | Posted: 02:26PM on Saturday, 30 September, 2006 | Comments: 0

Fast User Switching, the Welcome Screen and &lt;Control&gt;+&lt;Alt&gt;+&lt;Delete&gt;

Fast User Switching, the Welcome Screen and <Control>+<Alt>+<Delete>

First of all, let's start with a brief explanation of Windows' multi-layered sand-boxing and separation of logins and the user's desktop.

The basic structure is thus:

  • Windows kernel and Object Manager
  • "Sessions"
  • Window Stations
  • Desktops
  • Your desktop 'n' stuff.

I.e. the kernel can maintain any number of "sessions", all of which can have any number of window stations, all of which can have any number of desktops, all of which can have, well, the windows themselves.

Certain things are sand-boxed at different levels. Named kernel objects (e.g. events, critical sections, etc.) are stored in a session-specific location. Atoms and the clipboard, for example, are part of the Window Station, where as hooks are part of the Desktop. You can only send messages between processes with the same desktop, too.

All the sessions use the same setup:

  • WinSta0 (Window Station)
    • Winlogon (Desktop) AKA "secure desktop"
    • Default (Desktop)

The login screen, locked dialog and <Control>+<Alt>+<Delete> dialog all run in WinSta0\Winlogon. All your applications run in WinSta0\Default.

It should be noted that applications can create their own Window Stations and Desktops; indeed, my main system currently has 5 extra Window Stations (created by services, like Task Scheduler).

Windows NT and 2000

With Windows NT and 2000, things were simple. Session 0 was the one and only local "interactive" session (the one attached to the physical display device, physical input devices, etc.). Remote Desktop created other sessions for each remote login, as you might expect.

Windows XP

With Windows XP, two things were done to make life easier for the home user - Fast User Switching, and the Welcome Screen.

As you might have guessed, Fast User Switching utilises the "sessions" layer, allowing completely isolated user sessions. Adding this local disconnect/connection required enough work on the session system for an entire book, but it's not relevant here - what is, though, is that any session may be connected to the local physical devices.

Now that the system can do this magic, a UI is needed. Enter the Welcome Screen. The Welcome Screen (logonui.exe) runs in the secure desktop (WinSta0\Winlogon) when necessary, such as when a user logs off or locks the computer. It is interesting to note that the actual process is started by these events and does not run all the time.

  • Initial system
    • Active desktop: \Sessions\0\WinSta0\Winlogon
    • Welcome Screen running. No user's logged in.
  • Initial login
    • Active desktop: \Sessions\0\WinSta0\Default
    • No secure desktop processes.

At this point, I will point out that Start > Log off > Switch Users does exactly the same thing as <Win>+L (Lock).

  • Locking initial login
    • Active desktop: \Sessions\0\WinSta0\Winlogon
    • Welcome Screen running. One user logged in.
  • Second login
    • Active desktop: \Sessions\1\WinSta0\Default
    • Welcome Screen creates new session and connects local system to it. No secure desktop processes afterwards.
  • Locking second login
    • Active desktop: \Sessions\1\WinSta0\Winlogon
    • Welcome Screen running. Two users logged in.

This is where things get more interesting. The Welcome Screen can unlock either session, but is currently still connected to the second login. This has a bearing on, for example, sound - which is still connected, so your music will keep playing when you lock the computer (as it always has done). If you unlock the initial login, however, it will be disconnected.

  • Unlock initial login
    • Active desktop: \Sessions\0\WinSta0\Default
    • Welcome Screen connects to session 0, and unlocks. No secure desktop processes afterwards.

Just for fun, you can also disconnect the local session (from Task Manager, f.e.), which starts a new session just for the Welcome Screen:

  • Disconnect local session
    • Active desktop: \Sessions\2\WinSta0\Winlogon
    • Welcome Screen running. Two users logged in.

There are two UI points that change other than the login screen:

  • Security (<Control>+<Alt>+<Delete>) dialog.
  • Locked Workstation dialog.

Most people will see that the Locked Workstation dialog is counter-productive when Fast User Switching is enabled, and thus you get the Welcome Screen instead of it.

The Security dialog is a slightly different story. Winlogon uses GINA modules show various login UI. This includes all the classic login dialogs, the locked workstation dialog and the security dialog. The GINA module can be replaced with a 3rd-party one, such as NetWare's login screens and there is no telling what a 3rd-party security dialog might do. This means that the <Control>+<Alt>+<Delete> behaviour must still exist for non-local situations, and it makes sense (for consistency) to keep showing it when you also have the classic login UI.

In other words, although being in a domain or other network environment does mean sacrificing the Welcome Screen, there is no hard and fast reason for not having the <Control>+<Alt>+<Delete> dialog when using the Welcome Screen. It's just nicer not too. ;-)

Windows Vista

With Windows Vista, things change again. GINA modules are gone, presumably replaced with something else. This is what allows it to present the Welcome Screen for domain setups, and a pretty version of the classic security dialog has been added too. Nice one.

Disclaimer: all reasons behind design decisions, why X does Y, etc. are all based on 2nd-hand data, debugging and intelligent guesswork. I make no claims to them being strictly accurate.

Permalink | Author: | Tags: Windows | Posted: 08:00PM on Tuesday, 10 October, 2006 | Comments: 0

Windows' Memory Management

I was recently reading a weblog post on Linux and measuring memory used by individual applications, and it reminded me just how complex most OSes' memory management really is. Just when you think you've got it, another spanner hits you on the shin. So, let's have a look at Windows' memory management:

  1. Non-paged pool
  2. Paged pool
  3. Driver code
  4. Kernel code
  5. System cache
  6. Applications
  7. "Available"

All physical memory (available to the OS - we're ignoring anything that the BIOS is owning here) is used by one of the 7 categories above. Applications and "available" are the most interesting ones, and there is a reason I'm quoting "available", which I'll come to later.

Items 1-5 are all kernel memory, and the remaining two user memory.

1 Non-paged pool

Drivers are allowed to allocate memory from two pools; this is the lesser-used one, and for good reason - all memory allocated from the non-paged pool is, not surprisingly, never paged out. It is useful for operations that require this, such as DMA transfers, but not a very good thing to use for normal driver allocations.

You can see the size of this pool in Task Manager (Kernel Memory, Nonpaged) or with Performance Monitor (counter \\.\Memory\Pool Nonpaged Bytes).

2 Paged pool

The normal driver memory pool; all this memory can be paged out at the kernel's whim, and is ideal for storing driver structures that aren't used by hardware.

You can see the total allocated size of this pool in Task Manager (Kernel Memory, Paged) or with Performance Monitor (counter \\.\Memory\Pool Paged Bytes). You can see the currently paged-in size with Performance Monitor (counter \\.\Memory\Pool Paged Resident Bytes).

3 Driver code

This is the pageable memory being used to hold drivers, both their code and data pages.

You can see the size of this with Performance Monitor (counters \\.\Memory\System Driver Total Bytes and \\.\Memory\System Driver Resident Bytes).

4 Kernel code

This is the pageable memory being used to hold the kernel's own code (e.g. nsoskrnl.exe, hal.dll, boot drivers and boot file systems).

You can see the size of this with Performance Monitor (counters \\.\Memory\System Code Total Bytes and \\.\Memory\System Code Resident Bytes).

5 System cache

This is the area of kernel memory used for caching, mostly the file cache. This memory is never paged out, it is just freed (paging out a cache would be rather pointless).

You can see the size of this with Performance Monitor (counters \\.\Memory\System Cache Resident Bytes).

6 Applications

This area is defined by one book as "total memory minus the other 6 components", which is hardly surprising - it is a really hard number to track, and for various reasons there is no Performance Monitor counter for this. |\.\Process(_Total)\Working Set will usually come close, but can just as easily be larger than the real applications usage as it can smaller.

The reason it is so hard to calculate this value is simple: shared memory. This is the same reason that ps is "wrong" on Linux, as explained pretty well in the article I mentioned at the start of this post. For exactly the same reasons, Task Manager and Performance Monitor will "lie" about the memory usage.

Shared memory can be an executable (EXE, DLL, etc.) loaded into more than one process space, or specially allocated "shared memory" which is sometimes used for inter-process communications.

Each process has a working set (counter \\Process(*)\Working Set) which reports the total amount of physical memory used by that process. Task Manager calls this column "Mem Usage". Some of this memory is almost certainly shared with other processes, and thus the total of all working sets will be greater than the actual amount used.

Each process also has private bytes (counter \\Process(*)\Private Bytes) which reports the amount of allocated memory specific to this process, e.g. malloc and other direct allocations. Task Manager calls this column, rather confusingly, "VM Size". None of this memory is shared, but you also cannot tell how much of the working set is private bytes, and how much is shared bytes. (Note that a DLL loaded into a single process is still 'shared' memory, and not private bytes, because it can be shared.)

Process Explorer tries to help things by listing "WS Private", "WS Shareable" and "WS Shared", which it does by collecting data from all processes via a kernel driver. In this case, WS Private + WS Shareable = Working Set. WS Shared is just how much of the memory that can be shared is actually being shared.

I should also note that drivers can allocate memory which falls into this group, and is impossible to track as far as I know. Very few can, or do; the most notable example is Virtual PC, which allocates the VMs' memory like this.

7 "Available"

You can see the size of this group in Task Manager (Physical Memory, Available) or with Performance Monitor (counter \\.\Memory\Available Bytes).

It is not, however, as simple as it appears. Not all of the "available" memory is free memory, and therein lines the subtlety.

"Available" memory is made up 3 types:

  • Standby List
  • Free List
  • Zeroed List

In Windows Vista, there are some new Performance Monitor counters to measuring the size of these three types, \\.\Memory\Standby Cache Core Bytes, \\.\Memory\Standby Cache Normal Priority Bytes, \\.\Memory\Standby Cache Reserve Bytes and \\.\Memory\Free & Zero Page List Bytes).

When Windows want to trim a process' working set, the trimmed pages are moved (usually) to the Standby List. From here, they can be brought back to life in the working set with only a soft page fault (much faster than a hard fault, which would have to talk to the disk). If a page stays in the standby List for a long time, it gets freed and moved to the Free List.

In the background, there is a low priority thread (actually, the only thread with priority 0) which takes pages from the Free List and zeros them out. Because of this, there is usually very little in the Free List.

All new allocations always come from the Zeroed List, which is memory pages that have been overwritten with zeros. This is a standard part of the OS' cross-process security, to prevent any process ever seeing data from another. If the Zeroed List is empty, Free List memory is zeroed and used or, if that is empty too, Standby List memory is freed, zeroed, and used. It is because all three can be used with so little effort that they are all counted as "available".

Observed Behaviours

With all that above information, you might now be able to guess what it going on when you minimize an application and see "Mem Usage" in Task Manager drop by a large amount, and why restoring the application soon after doesn't usually involve any disk paging.

Permalink | Author: | Tags: Windows | Posted: 04:03PM on Thursday, 26 October, 2006 | Comments: 0

Discussions Belong on Newsgroups

Different people have different opinions on the whole Fora (forums) vs. mailing lists vs. newsgroups thing. Here I lay out my reasons.


  • Have topic and sub-topic groups.

Mailing Lists

  • Archived off-line.
  • Archived on-line, often.
  • Real "From" contacts (i.e. can look up people in your address book, etc.).
  • Real threading (usually; some lists fuck this up).
  • Individual message "read" state.


  • Full tree hierarchy of topics.
  • Archived off-line, if configured.
  • Archived on-line, for most public groups.
  • Real "From" contacts (i.e. can look up people in your address book, etc.).
  • Real threading.
  • Individual message "read" state.
  • Ignore and watching of threads.
  • Subscription is all done in the client, and thus more reliable.

Now, before you get all flaming toasty on my arse, I will point out that not all these points are always restricted to the items I've put them under. For example, some fora do do 'real' threading, but I've yet to see one that does it as well as an off-line client.

The Result

As you can see, I rate both mailing lists and newsgroups much better than fora. It's a bit of a toss-up which of those two win, though I prefer newsgroups for discussion (pull), and mailing lists for notifications (push) generally.

Permalink | Author: | Tags: Newsgroups, Mailing lists, Fora | Posted: 12:21AM on Sunday, 19 November, 2006 | Comments: 0

NVidia in Bug-Fixing Shocker

You may remember my previous post about NVidia's (then) latest drivers have some issues; in particular, the leaking of kernel-space objects being the most serious.

Since then, I'd upgraded to 91.47 to no avail, but today I upgraded to 93.71 and it is no longer leaking them!

Start NVidia control panel:

lkd> !object fffffadfb4fbc060
Object: fffffadfb4fbc060  Type: (fffffadfb5ab86c0) Process
   ObjectHeader: fffffadfb4fbc030
   HandleCount: 2  PointerCount: 87

Close it again:

lkd> !object fffffadfb4fbc060
fffffadfb4fbc060: Not a valid object (ObjectType invalid)


Permalink | Author: | Tags: NVidia | Posted: 11:40PM on Sunday, 03 December, 2006 | Comments: 0

Venkman Fork

This is just an informational post to say that I have created my own fork of Venkman so that I can actually get some damn work done on it. More information will follow; for now you may rant and rave in ignorance.

Permalink | Author: | Tags: Venkman | Posted: 11:52PM on Sunday, 18 March, 2007 | Comments: 0

Venkman Fork Update

Since the fork just over a week ago, the following issues have been fixed:

  • Bug 270298 - Fix profiler to show proper ranges.
  • Bug 295275 - Fix long-standing bug with floating views going blank. Woo.
  • Bug 298736 - Don't get button-related code execution when pausing.
  • Sync some changes from ChatZilla to Venkman. Source views work better after this.
  • Bug 356267 - Fix componentPath to work with toolkit's array of paths. Causes extensions' components to count, for example.
  • Final part of script filtering - preserving chrome, search and recent exclusions; everything works as expected now.
  • Bug 213494 - Sort properties using numeric and non-numeric blocks.
  • Prevent default operations when using -venkman.

More things will be fixed over the coming weeks; any suggestions for which bugs in Bugzilla are causing the most problems are welcome.

Now for the important bit - how to access all this goodness:

cvs -d checkout venkman

Note that, at least for the time being, the fork will continue to have the same extension ID and name as the original and I will not be releasing any XPIs generated from this code (this is very likely to change).

Permalink | Author: | Tags: Venkman | Posted: 02:58PM on Monday, 26 March, 2007 | Comments: 0

Time for a New Roadmap

Now that it has been officially stated that XULRunner is dead, it is time for a ChatZilla Roadmap Change™.

The original XULRunner and ChatZilla plan was to wait until the 1.9 version of XULRunner was officially distributed - 1.8 was never going to have the necessary shared UI or support - and then produce a ChatZilla package that works on it. Ideally, we would use the same XPI as for Firefox, SeaMonkey, etc. installs, although there were/are still some pretty critical hurdles to overcome for that to work. Even if it was a separate XPI, it would be a normally-size ChatZilla XPI, and could be on Firefox Add-ons.

So much for that.

After getting over the initial "WTF?" reaction, we need to decide on the way forward for ChatZilla on XULRunner. Thanks to tH's wonderful ChatZilla on XULRunner page (which include built nightlies from the appropriate dates), we do actually have a measurable percentage of our user base running this configuration. A small percentage, but not zero.

  • Do we build XULRunner ourselves? This would require a lot of effort on our part, and really should be shared with other projects if possible.
  • Do we only make available our ChatZilla package, and tell users to find their own XULRunner (or link to someone else who is making them)?
  • Do we give up trying to support XULRunner entirely?

Right now, I don't know, but the last option is looking disappointingly good.

Permalink | Author: | Tags: Mozilla, ChatZilla, XULRunner | Posted: 03:19PM on Monday, 14 May, 2007 | Comments: 0

Friday Pains

I'm sure browser plugins come with their own little Quantum Failure Generators or something. I've spent most of today trying to narrow down certain sizing issues with the Windows Media Player plugin across various OS versions and browsers. It was not (and still isn't) fun.

At one point, even Virtual PC 2007 itself crashed.

Then, starting openSUSE 10.2, it decided that it needed to fsck each and every partition just because it hadn't done it for a while. This would have been fine, except that it blocks the boot process for upwards of an hour with no indications on the graphical boot screen.

Now, I'm about to go install some beta software as part of a private beta test program; this can't possibly go wrong, can it?

Permalink | Author: | Tags: Firefox, IE, Linux, SUSE, Windows, WMP | Posted: 04:58PM on Friday, 08 June, 2007 | Comments: 0

ChatZilla: Notify on All Messages

ChatZilla users have for a while wanted to get a notification on all messages, but we've never considered it important enough to warrant fixing individually, instead opting to recommended some workarounds:

  • Add all vowels/letters to the Stalk List, thus triggering the normal stalk matching on (almost) every message.
  • Write a simple script, but never providing one ourselves.
  • Wait for the new message filtering which will be able to do it, among many other variations.

Today, another user asked for this feature, and I caved in and wrote a script to do it. To install the script:

  1. Find the scripts folder. You can use either /pref profilePath or /pref initialScripts since the latter defaults to the "scripts" subdirectory of the former. You want the "scripts" subdirectory, if that wasn't obvious.
  2. Create a new directory for this script. My suggested name is "message-notify", just to match the script's ID, but ChatZilla doesn't care.
  3. Save the script itself into the new directory, making sure to name it "init.js".
  4. Reload all scripts with "/eval processStartupScripts()". This will reload all the scripts found in the subdirectories of the "scripts" directory - including the new one.

The following message categories trigger the script, but only when the window is inactive (nothing happens if the window is active):

  • channel:privmsg
  • channel:action
  • channel:notice
  • channel:join
  • channel:part
  • channel:quit
  • channel:kick

The script has two preferences:

  • sound - specifies what sound to play when a message arrives. Defaults to "beep", and can accept the same things as the standard ChatZilla sound preferences (namely, a file: URL, "beep" or "none").
  • attention - specifies whether to grab the user's attention, which usually means flashing the window but can vary by platform. Defaults to "true".


Permalink | Author: | Tags: ChatZilla, Scripts | Posted: 09:09PM on Friday, 08 June, 2007 | Comments: 0

Summer Break

It's that time of year again. I will be away from Monday (25th) to Friday (29th), enjoying the annual mega LAN.

Permalink | Author: | Tags: CompSoc, Fun | Posted: 06:27AM on Sunday, 24 June, 2007 | Comments: 0

How Not to Migrate a Server

  1. E-mail people at 6PM saying the server will be offline as of 10AM the next day.
  2. The next day, find out that the new server doesn't want to work with the disks from the old server.
  3. Overwrite the partition table on one of the disks.
  4. Spend hours backing up the data and rewriting the partition table by hand.
  5. The next day, copy data and install new server into limited-access server room.
  6. While remotely migrating setup, remove /etc/passwd file (thus preventing all remote access).
  7. Wait 3 days to gain physical access to the machine.
  8. Slowly start setting up everything.
  9. Have users find out that some of the services are now running different software and didn't have their data migrated over.

Things which could have been better:

  • Advanced notice of migration. Server migration is never simple, advertising it the night before is seriously inadequate for most people.
  • Notice of which services, if any, will not be migrated exactly as-is, and how the decision was made.
  • Setting up existing services on the new server prior to the swap.

Even now, the server still isn't running all services correctly, but hopefully it will be soon. I look forward to receiving mail via the new server, 4 days after it was supposed to be all working. ;-)

Permalink | Author: | Tags: CompSoc | Posted: 04:51PM on Monday, 09 July, 2007 | Comments: 0

Windows Vista Shortcuts for Japanese IME

General Language and Keyboard Shortcuts

Shortcut Action
<Left Alt> + <Shift> Switch input language
<Control> + <Shift> Switch keyboard layout

These can be changed from Control Panel, in Regional and Language Options, Keyboard and Languages tab, "Change keyboards...", Advanced Key Settings tab. Alternatively, right-click the Language Bar and select "Settings...".

Japanese IME Shortcuts (summary)

Normal Mode (not in the middle of entering a word/phrase)
<Control> + <Caps Lock> Switch to Hiragana input
<Alt> + <Caps Lock> Switch to full-width Katakana input
<Shift> + <Caps Lock> Toggle half-width alphanumeric (English) input and Hiragana input
<Alt> + <`>
<Control> + <Shift> + <Caps Lock> Toggle kana and romanji input
<Shift> + <Space> Enter half-width alphanumeric (English) space
Conversion Mode (when entering word/phrase via IME)
<Shift> + <Left> Use less characters for conversion
<Shift> + <Right> Use more characters for conversion
<F6> Convert to Hiragana Each of these will flip through a number of possibilities on each press.
<F7> Convert to Katakana
<F8> Convert to half-width Katakana
<F9> Convert to alphanumeric
<F10> Convert to half-width alphanumeric (English)

These can be changed from the Japanese IME itself, which can be accessed from the Language Bar (right-click and select "Settings...", then select the IME and click "Properties..."). On the Editing tab, there is a drop-down of key templates and the option to edit. The above keys are all for the default key template ("Microsoft IME").

Permalink | Author: | Tags: Windows, Vista, IME | Posted: 05:39PM on Saturday, 21 July, 2007 | Comments: 0

Mozilla Error Lookup now has LDAP and CMS

For many years, the code behind the Mozilla Error Lookup has been unable to correctly include the LDAP and CMS error codes, as the C++ definitions used the IDL constants and the script is just not smart enough for that.

It is now.

That means that all the error code definitions it can find are now correct and can be found from their numeric forms. Enjoy.

Permalink | Author: | Tags: Mozilla | Posted: 11:58PM on Friday, 03 August, 2007 | Comments: 0

Windows Live Writer support: part 1

I am currently in the progress of implementing the necessary features and components in my weblog software to allow the use of a standalone client to post entries (and ultimately edit, as well). As I'm already beta testing a bunch of other Live tools, I thought I'd start with Windows Live Writer.

The current beta version supports:

  • Blogger
  • TypePad
  • LiveJournal
  • Movable Type
  • WordPress (inc. 2.2+)
  • Community Server
  • dasBlog
  • Radio Userland

Additionally, it supports two raw APIs on anything that implements them:

  • Metaweblog API
  • Movable Type API

Not bad. So, I've chosen the Movable Type API for mine, but I'm jumping ahead - there's something else, something really cool, that I should mention.

Really Simple Discovery, RSD, is a method by which a tool can identify information about a page and its supported editing schemes. Including a link in the main weblog page with rel="EditURI" to point to the RSD file (which is XML), and that indicates which editing APIs are supported and their respective URIs.

Windows Live Writer checks for and uses the RSD information, which results in a 3-entry configuration: weblog URL, username and password. It doesn't get much simpler than that.

Still to come: the Movable Type API, Blogger, categories and more.

Permalink | Author: | Tags: Live, Weblog, Windows, Writer, RSD | Posted: 12:48PM on Wednesday, 08 August, 2007 | Comments: 0

Media Center Video Stretching Tip

I find Microsoft Windows Vista Media Center is great in so many ways, but nothing's perfect (more's the pity).

One thing which has been irritating me for a while is how it was using the "shrink-to-fit" method of sizing local video playback instead of "expand-to-fit". This results in any wide-screen videos losing the left and right edges, as I only have a 4:3 monitor (shame on me). I believe it'd be causing equal problems with non-wide-screen videos if I had a wide-screen monitor, though.

Until a few days ago, I'd been using ffdshow's capable suite of options to force output into 4:3 (while preserving the aspect ratio, i.e. adding evil black bars). This really eats CPU, even on the fastest resize mode, increasing usage by approximately 30%, which was actually causing my AMD64 3200+ (2.0GHz) to run out of cycles on some videos.

What I found a few days ago, though, was the real solution: time to start the Registry Editor, folks. Locate the following key:

HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Media Center\Settings\VideoSettings

Under this key there are a large set of options, most of which you don't want to touch. In this case, however, one particular one is crucial - ZoomMode_Video. Set that to "0". Done.

Footnote: There are 3 other ZoomMode_ options (ATSC, DVD, TV) and all 3 were already set to "0" here. ZoomMode_Video, however, was set to "1". I'd be very interested to hear how many, if any, other people have any of these ZoomMode_ options not set to "0" and which ones.

Permalink | Author: | Tags: Vista, Windows, Media Center | Posted: 09:24AM on Friday, 23 November, 2007 | Comments: 0

Windows Vista Start Menu "Bye" Options

Although I don't really like Joel on Software, in this case he has the most appropriate rant to give the context for this post.

"Every time you want to leave your computer, you have to choose between nine, count them, nine options: two icons and seven menu items."

Ignoring the fact that I only have six menu items (no Hibernate), he has entirely missed the fact you only need the two buttons in all normal circumstances.

Sleep (left button)

By default, the left button is sleep, and it is the new hybrid sleep added in Vista. This does a combination of hibernate and suspend from previous versions - saving your work to disk, but only putting the computer into a low power state. This lets you resume work really fast and protects from power-loss.

Lock (right button)

The right button locks the computer. But wait, it also gives you access to switch user - using that giant blue button labeled "Switch User" on the locked screen. You can even shut down (or restart, or sleep - again) from the Switch User screen.

Redundant Options?

In his rant, Joel suggests that you don't need these at all:

  • Switch User
  • Log Off
  • Restart

While they are not usually useful, it's still useful sometimes to log off or manually restart, and switch user is a shortcut to lock+switch user which you might not remember or know.


In fact, there is only one option you might regularly need: shutdown. If you need this often, you're unlikely to need sleep, so the obvious thing to do is replace the sleep button with a shutdown one:

  1. Control Panel > Hardware and Sound > Power Options.
  2. Change when to turn off the display.
  3. Change advanced power settings.
  4. Power buttons and lid > Start menu power button > Setting.
  5. Select shutdown from the drop-down list.
  6. Done.

Permalink | Author: | Tags: Vista, Windows | Posted: 08:04PM on Tuesday, 15 January, 2008 | Comments: 0

ChatZilla Static Release Console

Some people know about the ChatZilla Release Console, a slightly unstable and very dynamic page which displays the bugs planned for the next ChatZilla release or two (as well as an archive of bugs fixed in older versions). It's been around a while, and works in IE 7 and Firefox 2, but it is pretty heavy on the dynamics, and some people have had it completely hang Firefox on occasion.

To address this, I have created the ChatZilla Static Release Console, which is entirely static HTML (and CSS), created by Perl on my server rather than JavaScript on the client. It doesn't even have a META refresh. You can also get it to view a specific release.

Permalink | Author: | Tags: ChatZilla, Perl | Posted: 04:27PM on Sunday, 20 January, 2008 | Comments: 0

Compatibility Administrator Meets Train Simulator

The original Microsoft Train Simulator was a great game at the time, and continues to be good fun (I'll save why it's better than Kuju's newer Rail Simulator for another time). It does, however, not entirely work by default on Windows Vista. There are a couple of issues:

  • Resolution changes cause windows to be rearranged.
  • On exit, it tries to create C:\FileIn.cns and C:\FileOut.cns and gets stuck in a loop because it can't.

You could work around the latter problem by running it as administrator, but that's clearly a substandard "fix". However, a much better solution which can solve both issues is the Compatibility Administrator in the Microsoft Application Compatibility Toolkit.

The Compatibility Administrator lets you tap in to a powerful collection of fixes already written by Microsoft for making other applications compatible with new versions of Windows. It also lets you view the fixes applied to each and every application in its system AppCompat database.

After installing the Application Compatibility Toolkit, you can launch the Compatibility Administrator as a standard user. For the purposes of this post, I'll create an AppCompat database just for fixing Train Simulator, but you can fix any number of programs with a single database, which simplifies deployment and installation.

  1. Since it starts with a new database by default, just rename it by right-clicking and select "Rename". It really doesn't matter what you call it; I called mine "James Ross" in case I ever fix anything else with it.
  2. Select the database, select the "Database" menu, "Create New", "Application fix...". This opens the wizard which will configure all the fixes we want for a single application.
  3. Enter the details on the first page:
    • Name of the program to be fixed: "Train Simulator" (it's only for display).
    • Name of the vendor for this program: "Microsoft" (also only for display).
    • Program file location: browse to "train.exe" (e.g. "C:\Program Files\Microsoft Games\Train Simulator\train.exe"). This will be used later.

  4. Now we get to pick from some standard fixes, including the commonly-used version lies. We don't want to select anything here, so ensure "None" is selected for the "Operating System Modes" and move on.

  5. This is where the fun happens! There is a long list of possible fixes here, many of which can also take a command-line with options to control or limit the code that applies the individual fix. For the relatively simple problems with Train Simulator, we want to use two fixes:
    • CorrectFilePaths

      With this, we can redirect CreateFile API calls (among other file APIs) to actually access somewhere other than where the program passes. We use the following command-line:

      -b "C:\FileIn.cns;%UserAppData%\FileIn.cns" "C:\FileOut.cns;%UserAppData%\FileOut.cns"

      The -b (bare) turns off the standard mapping (which covers some system files that have moved over Windows versions), and then we specify two file corrections. Important to note that the double-quotes go outside the semi-colon separated pair of file paths. Also, we make use of one of the replacements (%UserAppData%), but note that these are not environment variables. There is a good weblog post about using CorrectFilePaths as well as some TechNet documentation.

    • ForceTemporaryModeChange

      This one is much simpler, and probably obvious from its name! It simply ensures that any resolution change is flagged as temporary, which prevents the window manager from shuffling things around.

    As you may have noticed, the wizard has "Test Run..." buttons on each page of options. You need to be an administrator to use them (as it installs an AppCompat database temporarily to do it), but they can be very handy in some cases.

  6. Now we must tell Windows how it is supposed to identify that this set of fixes should be applied to a launching application. The wizard will automatically add the executable selected at the start, and tick some properties. In our case, there's no need for most of them and I unselected everything except PRODUCT_NAME.

    You could, if needed, add references to other files and decide which properties from them matter too; the fixes will still only be for the original executable, but it provides finer control over when to apply the fixes.

  7. That's the AppCompat database created. You should now save it somewhere you'll remember (and be able to access from your administrator account).
  8. Close Compatibility Administrator, and re-open it as administrator.
  9. Open your AppCompat database, right-click it and select "Install".
  10. Done!

Now, any time you run Train Simulator, the two fixes will be applied and joy will ensue.

Permalink | Author: | Tags: Compatibility Administrator, Train Simulator, Vista, Windows | Posted: 11:17PM on Tuesday, 29 January, 2008 | Comments: 0

Windows Calendar and Pre-Patch Tuesday Thursday

I'm sure most people know that Microsoft designates the second Tuesday of each month as "Patch Tuesday" - the standard day for releasing all kinds of updates (both security and non-security) to Windows and other Microsoft products and technologies. Marking this in Windows Calendar is a trivial operation:

However, less well known is that on the Thursday before Patch Tuesday, Microsoft release an "Advance Notification" which indicates how many security patches are planned for release on the following Tuesday, what products they affect, and their severity. But how would you add that to Windows Calendar?

You can't do it via the GUI - it's simply too complex a rule. Wait, how would you describe it in any calendar anyway? Windows Calendar, like the majority of calendaring applications, uses the iCalendar format (RFC 2445). You can't specify that an event is positioned relative to another event, but iCalendar is still very flexible, as it allows you to combine restrictions on a repetition rule (see section 4.3.10 of the RFC for the gory details).

The simplest way I found to get the correct Thursday is to restrict the event using two criteria: month day must be 3 - 9 inclusive and it must be a Thursday. This works because only one day in the range 3 - 9 can be a Thursday each month, and it is five days ahead of the range of possible Patch Tuesday days (8 - 14). The rule is thus: "RRULE:FREQ=MONTHLY;BYMONTHDAY=3,4,5,6,7,8,9;BYDAY=TH".

To add such an event:

  1. Start by adding a normal monthly repeating event and give it a unique summary.
  2. Close Windows Calendar.
  3. Find all the calendars in C:\Users\<username>\AppData\Local\Microsoft\Windows Calendar\Calendars.
  4. Open the correct calendar file (they use the name set in the GUI).
  5. Find the correct event by searching for the summary of the event.
  6. Just above the summary is the "RRULE", which should be replaced with "RRULE:FREQ=MONTHLY;BYMONTHDAY=3,4,5,6,7,8,9;BYDAY=TH".
  7. Re-open Windows Calendar, and it should appear on the Thursday before the 2nd Tuesday of every month.

Done! You can even edit anything about the event (except the recurring) from the GUI. Windows Calendar even knows it can't edit the more advanced recurrence rule:

Permalink | Author: | Tags: Vista, Windows, Calendar | Posted: 10:31PM on Wednesday, 27 February, 2008 | Comments: 0

Windows Vista: "Why so fat, winsxs?"

It's well-known that the \Windows\winsxs directory in Windows Vista is large and can take up a significant portion of disk space on smaller devices. Less well-known is what it actually does, why it is huge or why it's not entirely a waste of space. I'll try and answer all three of these today.

To help dig in to this, I've written a small C++ application that will report certain information about the files in a particular directory - or individual files - and will use this to help explain the answers. Binaries and source code will be provided at the end.

What does winsxs do?

At the highest level, it is a collection of "packages" - lots of small groups of files. Each package (group of files) is identified by:

  • Architecture ("x86", "amd64", "ia64", "wow64" (32bit on 64bit) and "msil")
  • Name
  • Signing Key
  • Version
  • Locale ("none", "en-us", etc.)
  • Package Hash

It does more than just store the packages, though; applications can bind themselves to particular packages and specify what versions are acceptable. This is most evident with the Microsoft Visual C++ runtime files which, along with the Common Controls version 6 package, are more or less all you'll find in \Windows\winsxs on Windows XP. There are other features, like package redirections, which are in Windows XP as well.

Why is winsxs huge?

For Windows Vista, however, the main change is not so much how it works (although that was improved and made more robust as well) but in what it's used to store. In Windows Vista, all the operating system components are in packages inside \Windows\winsxs. That's all the core features (like Explorer) and everything you can enable in "Turn windows features on or off" (like IIS). That's a lot of stuff!

A consequence of putting system components in \Windows\winsxs is that they can be uninstalled reliably and with less dependence on order in Windows Vista - Windows XP keeps a load of \Windows\$NtUninstall... directories to manage it, but it has issues on ordering of installs and uninstalls.

Back in Windows 2000, you could install all the data from the CD-ROM onto the hard disk to avoid needing the CD when turning features on or off. I used it, but few people knew about it as you had to set it all up by hand (some OEMs did bother, though). This was a much simpler version of what \Windows\winsxs is in Windows Vista today. If you've ever turned on East-Asian support in Windows XP, you'll probably know how annoying it is to go and find the CD just to copy some fonts and locale support - this is just one of many places where Windows Vista already has the data available and it will never need you to rummage for the install DVD.

Why is winsxs not entirely a waste of space?

I'm sure you can appreciate that not needing to rummage for a DVD when changing features is good, but you probably don't do that often. Windows Vista might have a lot of data in \Windows\winsxs, but what about the features that are enabled? Don't they use up twice their size in disk space?

Here's the clever bit.

Installed features only consume space for their data, if they have any, but not for any of the code. How does that work? Hardlinks, a little-known feature of NTFS since Windows 2000 that allow the same file (and the same data) to be available from multiple locations in the directory hierarchy. The important part of measuring disk usage is that the vast majority of tools - Windows Explorer included - don't know or don't bother to know about hardlinks, which is partially why I wrote my own tool to explore this further.

Let's look at an example, C:\Windows\system32\shell32.dll ("Hardlink Scanner.exe" /F C:\Windows\system32\shell32.dll):

Breakdown for "C:\Windows\system32\shell32.dll"
 Unique ID:    8e0000000a1534
 Hardlink count:            2
 Naive file size:  25,794,560 bytes
 Unique file size: 12,897,280 bytes
 Kind of file:         normal

In this example, it's clear this is a 64bit system as it is the "amd64" architecture that is linked into \Windows\system32. More important is that although both locations appear to have their own 12MB shell32.dll, using nearly 25MB in total, there is actually only one file and only 12MB of disk space is used.

Another example, the entire directory C:\Windows\SysWOW64 ("Hardlink Scanner.exe" C:\Windows\SysWOW64):

Hardlinks:    Files              Unique Files
 <total>          4,278              4,275 ( 99%)
 1                  394 (  9%)         394 (  9%)
 2                3,101 ( 72%)       3,101 ( 72%)
 3                  725 ( 16%)         725 ( 16%)
 4                   53 (  1%)          50 (  1%)
 5                    1 (  0%)           1 (  0%)
 7                    4 (  0%)           4 (  0%)

Here, about 90% of the files in the directory have at least one other hardlink that is outside this directory (you can work this out by comparing the "Files" and "Unique Files"; if the hardlinks were all inside this directory, the unique counts would be significantly lower). The higher-than-2 hardlink counts also bring up another clever feature of the Windows Vista \Windows\winsxs directory. Let's have a look at the 32bit MPEG 2 decoder ("Hardlink Scanner.exe" /F C:\Windows\SysWOW64\

Breakdown for "C:\Windows\SysWOW64\"
 Unique ID:     100000004f2ab
 Hardlink count:            4
 Naive file size:     278,528 bytes
 Unique file size:     69,632 bytes
 Kind of file:         normal

Although there are three different versions of this package, didn't change between versions and so \Windows\winsxs has used hardlinks between the packages as well as to the final destination. That's reduced the space consumption to just 25% for this file.

What does the saving look like for the whole of \Windows? ("Hardlink Scanner.exe" C:\Windows):

Hardlinks:    Files              Unique Files
 <total>         93,670             76,585 ( 81%)
 1               61,354 ( 65%)      61,354 ( 80%)
 2               22,750 ( 24%)      12,467 ( 16%)
 3                6,117 (  6%)       2,143 (  2%)
 4                  552 (  0%)         138 (  0%)
 5                1,365 (  1%)         273 (  0%)
 6                  796 (  0%)         133 (  0%)
 7                  329 (  0%)          47 (  0%)
 8                  200 (  0%)          25 (  0%)
 9                   18 (  0%)           2 (  0%)
 10                  30 (  0%)           3 (  0%)

Naive file size:        25,266,466,330
Unique file size:       19,881,302,264 ( 78%)
Difference:              5,385,164,066 ( 21%)

That's more than 5GB saved by using hardlinks within \Windows. Scanning my entire drive ("Hardlink Scanner.exe" C:), to include \Program Files (Internet Explorer and Media Player, for example) gives an overall difference of 5.9GB.

So, just takes those computed sizes from Windows Explorer (and other apps) with a little pinch of salt when dealing with Windows Vista's system files.

Code: Hardlink Scanner (x86), Hardlink Scanner (x64) and Hardlink Scanner source (Visual Studio 2008 Solution).

Permalink | Author: | Tags: Vista, Windows | Posted: 11:26PM on Friday, 31 October, 2008 | Comments: 0

Searching Games Explorer from the Start Menu

A feature new to Windows Vista is the Games Explorer, a folder dedicated to the games installed on the computer; games are required to exclusively add themselves there as part of the Games for Windows technical requirements, and Microsoft provide details of hundreds of pre-Vista games which appear automatically as a result.

Unfortunately, you can't search this folder directly from the Start Menu. While this is an annoying problem, it's not insurmountable. In fact, it's pretty easy to manually rectify with a short bit of scripting.

Update Games Explorer is a short Command Script to enumerate the contents of your Games Explorer and copy the shortcuts into the Games folder in the Start Menu, thus rendering them indexed and searchable. The script can be run directly or as a scheduled task, whichever suits best.

Permalink | Author: | Tags: Vista, Windows, Games, Games for Windows | Posted: 10:50PM on Saturday, 03 January, 2009 | Comments: 0

Empty Windows' System File Cache

Reading Startup, part 1, where Vladimir Vukićević is measuring warm and cold start times for Firefox, he mentions that he is doing the work on Mac OS X rather than Windows because there is a simple tool to empty the system's file cache to test cold starts without actually restarting the OS. I like challenges.

There isn't, it seems, a nicely documented API for doing this on Windows - it certainly isn't something the majority of applications would want to do, though, either. However, there is an undocumented API for it: NtSetSystemInformation(SYSTEM_INFORMATION_CLASS, PVOID, ULONG) (it has a slightly documented sibling for getting information, NtQuerySystemInformation(SYSTEM_INFORMATION_CLASS, PVOID, ULONG, PULONG)). These are actually fairly generic wrappers around a whole load of kernel things; the undocumented information class of interest here is "SystemCacheInformation" (value 0x15), which lets you get and set information about the system file cache.

Each information class for NtSetSystemInformation/NtQuerySystemInformation has a single structure, which goes into the 2nd argument (and its size into the 3rd). For 32bit (applications; not OS) the cache information structure can be defined like this:

   typedef struct _SYSTEM_CACHE_INFORMATION {
       unsigned long CurrentSize;
       unsigned long PeakSize;
       unsigned long PageFaultCount;
       unsigned long MinimumWorkingSet;
       unsigned long MaximumWorkingSet;
       unsigned long Unused1;
       unsigned long Unused2;
       unsigned long Unused3;
       unsigned long Unused4;

Along very similar lines to SetProcessWorkingSetSize, specifying the minimum and maximum working set values as 0xFFFFFFFF signals to flush as much as possible out of the system file cache, which gets it down to less than 100KB by the time I can check it afterwards.

Simple, really.

The rest of the code I wrote is to enable the SE_INCREASE_QUOTA privilege Administrator users have, which is necessary to call the API. I believe the executable will work on anything from Windows 2000 and up, with no external dependencies; I've tested it on Windows XP 32bit and Windows 7 64bit. You will obviously need to run it elevated (run as Administrator).

Executable and Code: purge.exe and purge.cpp.

Permalink | Author: | Tags: Cache, File, Windows | Posted: 12:28AM on Wednesday, 29 July, 2009 | Comments: 0

Simis File Format

Microsoft Train Simulator was made by Kuju Entertainment, formed by a management buyout of the Simis game studio in 1998, according to Wikipedia. Many of the files that form part of the game are in one of a number of formats bearing the "SIMISA" signature. For the purposes of discussing the file formats, I will only label the outer-most format the "Simis file format"; inner formats will be identified separately, later.

The Simis file format is identified a 16 byte or longer signature and header, with the remainder of the file being an inner format/data. The formats can all be identified by reading the first 8 bytes.

Uncompressed Simis File Format

00000000   53 49 4D 49  53 41 40 40  40 40 40 40  40 40 40 40   SIMISA@@@@@@@@@@

Uncompressed files are simple; 16 fixed bytes ("SIMISA@@@@@@@@@@"), followed by the inner format/data.

Compressed Simis File Format

00000000   53 49 4D 49  53 41 40 46  .. .. .. ..  40 40 40 40   SIMISA@F....@@@@
00000010   78 9C                                                x?

Compressed files are slightly more interesting; 8 fixed bytes ("SIMISA@F"), followed by a 4 byte unsigned integer containing the uncompressed size of the remaining data, and another 4 fixed bytes ("@@@@"). The data itself is compressed using zlib's DEFLATE algorithm, which is identified by the two bytes following the 16 bytes header; all subsequent data is the raw DEFLATE stream.

Unicode Text Simis File Format

00000000   FF FE 53 00  49 00 4D 00  49 00 53 00  41 00 40 00   ÿþS.I.M.I.S.A.@.
00000010   40 00 40 00  40 00 40 00  40 00 40 00  40 00 40 00   @.@.@.@.@.@.@.@.
00000020   40 00                                                @.

This format is a variation on uncompressed, but for a specific type of inner format: Unicode text encoded with UTF16-LE. In this situation, two things are done to the header:

  • It is encoded as UTF16-LE characters like the inner format/data.
  • A UTF16-LE byte order mark is pre-pended.

Inner Formats

Whether the Simis file is compressed or not does not affect the inner formats allowed; any format may be compressed or uncompressed with no differences beyond that compression and the header indicating as much. It is unclear, however, how the Unicode text format would be compressed so, as I have not found any examples within Microsoft Train Simulator, I am considering them distinct at this level.

Permalink | Author: | Tags: Format, Games, Kuju, Microsoft, Simis, Train Simulator | Posted: 12:53AM on Friday, 09 April, 2010 | Comments: 0

Simis Jinx Unicode Text File Format

This is one of the 2nd-level (inner) formats for the Simis file format used by Microsoft Train Simulator.

As I mentioned last time, the 1st-level (outermost) format has an adjusted header when working with this 2nd-level format; in particular, the file starts with a UTF16-LE byte order mark and the header itself is Unicode text encoded as UTF-16LE. Here it is again:

00000000   FF FE 53 00  49 00 4D 00  49 00 53 00  41 00 40 00   ÿþS.I.M.I.S.A.@.
00000010   40 00 40 00  40 00 40 00  40 00 40 00  40 00 40 00   @.@.@.@.@.@.@.@.
00000020   40 00                                                @.

The 2nd-level format identifies itself with its own header, unsurprisingly, starting with the text "JINX0", two values indicating the 3rd-level (inner inner) format, the text "t" indicating the Unicode text variety of this format, some padding (6 underscores) and a newline.

00000020         4A 00  49 00 4E 00  58 00 30 00  .. .. .. ..     J.I.N.X.0.....
00000030   74 00 5F 00  5F 00 5F 00  5F 00 5F 00  5F 00 0D 00   t._._._._._._...
00000040   0A 00                                                ..

Those 4 missing bytes are the two characters (a letter then a number) that identify the 3rd-level format used. As text, the header of these files looks like the following line:


Now follows the actual data...

Simis Jinx files all store data in a tree-like structure, where each tree node has a type and optional name, and is intermixed with values. In other words, each node's children can be both nodes and values, including a mixture. The structure is marked out by parentheses ("(" and ")") for blocks, with values being raw or quoted strings (generally speaking, any value with no whitespace and no significant symbols - quotes, parentheses - can be left unquoted). Let's get straight to an example from Microsoft Train Simulator, the file GLOBAL\gui.txt:


io_dev ( KEYB 0
    io_map ( T                 "sounddialog"        ALL_UP SHIFT_DOWN )
    io_map ( ESCAPE            "escape"             ALL_UP )
    io_map ( F1                "Help"               ALL_UP )

The header has identified this as containing 3rd-level format "I0"; for now, though, let's focus on the 2nd-level format. There is a single root node of type "io_dev" (and it has no name) which contains:

  • The value "KEYB".
  • The value "0".
  • Three nodes of type "io_map", each containing a selection of values but no further nodes.

This is just a simple example, but the format is pretty easy to read (although a little tricky to parse correctly); the 3rd-level format actually defines which node types and what nesting of them is allowed and which values should be where.

Permalink | Author: | Tags: Format, Games, Microsoft, Simis, Train Simulator | Posted: 04:48PM on Thursday, 22 April, 2010 | Comments: 0

Simis Jinx Binary File Format

This is the second of the 2nd-level (inner) formats for the Simis file format used by Microsoft Train Simulator.

Unlike the text format, the binary format has a simple binary header - but it should look rather familiar. The binary header has the same 16 characters as the text header, but this time they're encoded as single bytes and one of the characters is notably different: the 8th character is "b" for binary, rather than "t" for text (if there existed a single-byte text format this would have been the only difference with this format in the headers).

00000010   4A 49 4E 58  30 .. .. 62  5F 5F 5F 5F  5F 5F 0D 0A   JINX0..b______..

Just like in the text format, the 2 missing bytes in the middle are the two characters (a letter then a number) that identify the 3rd-level format used. We will get to those next time.

Note: If the 1st-level format is using compression, this header is the first item within the compressed stream; for simplicity, all my examples will be for uncompressed files.

Now follows the actual data...

As I started talking about last time, Simis Jinx files are basic trees; the binary format is nothing more than an alternative representation of the same data. It is, however, more of a challenge to read and write correctly - something the 3rd-level formats will help deal with.

Each node in the tree has an 8 byte header, consisting of a 4 byte unsigned integer identifying the node's type and a 4 byte unsigned integer specifying the length of the contents. The contents consist of an optional name - 1 byte for length plus UTF16-LE characters - for the enclosing node and the child values and nodes.

There are a number of common types of value included:

  • Unsigned integer (4 bytes).
  • Signed integer (4 bytes).
  • Floating-point number (4 bytes).
  • String (2 bytes for length plus UTF16-LE characters).

Let's have a look at an example, GLOBAL\capview.iom, but remember that to correctly parse this I am using the 3rd-level format:

00000000   53 49 4D 49  53 41 40 40  40 40 40 40  40 40 40 40   SIMISA@@@@@@@@@@

The standard 1st-level header...

00000010   4A 49 4E 58  30 69 30 62  5F 5F 5F 5F  5F 5F 0D 0A   JINX0i0b______..

2nd-level header indicating a binary version of 3rd-level formal 'i0'.

00000020   63 00 00 00  64 02 00 00                             c...d...

Node type is 99 (0x63), contents size is 612 bytes (0x264).

00000020                             00                                 .

Node has no name.

00000020                                01 00 00  00 00 00 00            .......
00000030   00                                                   .

Two unsigned integer values: 1 and 0.

00000030      64 00 00  00 35 00 00  00 00                       d...5....

Node type is 100 (0x64), contents size is 53 bytes (0x35) and there's no name.

00000030                                   CB 00  01 00                   ....

Unsigned integer value: 65,739 (0x100CB).

00000030                                                15 00                 ..
00000040   6E 00 75 00  64 00 67 00  65 00 5F 00  63 00 61 00   n.u.d.g.e._.c.a.
00000050   62 00 63 00  6F 00 6E 00  74 00 72 00  6F 00 6C 00   b.c.o.n.t.r.o.l.
00000060   5F 00 6C 00  65 00 66 00  74 00                      _.l.e.f.t.

String value: length of 21 (0x15) plus 21 UTF16-LE characters "nudge_cabcontrol_left".

00000060                                   00 00  00 00                   ....

Unsigned integer value: 0.

The 1 byte for no name, 8 bytes for two unsigned integer values plus 44 bytes for the string mean the total contents are up to 53 bytes - that means it is the end of this node type 100.

00000060                                                64 00                 d.
00000070   00 00 37 00  00 00 00                                ..7....

Node type is 100 (0x64), contents size is 55 bytes (0x37) and there's no name.

Writing the above in text format would give:

<node type 99> (
    <node type 100> (
    <node type 100> (

It's clear that we're missing the node type names found in the text files, and there's no indication whether some 4 bytes are a new node, an integer or float - some heuristics can work for this some of the time, I found, but in the end this is what the 3rd-level format is for.

Other 2nd-level formats

I've covered the two main Simis 2nd-level formats, but there are others; most notably, the texture files (.ace) are wrapped in a 1st-level Simis header with their own format inside. I won't be covering these other formats soon, as there are already tools that can handle these files sufficiently for Microsoft Train Simulator's needs, and the 3rd-level Simis formats are more interesting anyway.

Permalink | Author: | Tags: Format, Games, Microsoft, Simis, Train Simulator | Posted: 11:07PM on Sunday, 09 May, 2010 | Comments: 0

Simis Jinx 3rd Level File Formats

The Simis file format with the 2nd-level Unicode text and binary Jinx formats are a pretty generic set of formats; they contain an arbitrarily nested tree structure with strings, integer and floating point numbers at any level. To actually interpret and describe the contents, a 3rd level of formats is needed.

As mentioned in both Simis Jinx Unicode Text File Format and Simis Jinx Binary File Format, this 3rd level of formats is identified by a letter and a number - and there are quite a lot of them. To actually define these formats in a useful way, though, we need to use another format - Backus-Naur Form (BNF). The exact format I've used is a variant of the standard Backus-Naur Form derived from the BNF files that shipped with Microsoft Train Simulator itself (in the UTILS\FFEDIT directory).

Train Simulator Backus-Naur Form

The BNF files are text; new lines have no significance; any of ASCII, UTF-8 and UTF-16 character encodings can be used, provided a byte order mark is included to identify UTF-8 and UTF-16. The files are made up of a number of definitions and productions - in any order - and a special termination marker.

Definitions specify a shared or standalone expression. Any other expression can reference it and has their reference expanded to the expression on the right-hand side of the equals ("=").

Productions specify, through the expression on the right of the arrow ("==>"), what is allowed/expected inside the block identified by the name on the left.

The expressions in both definitions and productions contain a space-separated list of items, each of which can be:

  • A string literal, e.g. "Activity".
  • A pre-defined data type, e.g. :sint. Available data types:
    • uint
    • sint
    • dword
    • string

    Data types can additionally be named, by including a comma and identifier after the type, e.g. :sint,TileX.

  • Another production or definition, e.g. :Tr_Activity.

There are three operators allowed within expressions:

  • Square brackets, denoting an optional section, e.g. [:Description].
  • Curly brackets, denoting a repeatable section (1 or more times), e.g. {:UiD :SidingItem}.
  • Pipe symbol, denoting a choice between sections, e.g. :Engine|:Wagon.

The choice operator (pipe) binds tighter than whitespace. Therefore, the expression :foo :bar|:baz means "foo followed by either bar or baz".

The end of an expression is denoted by a period (".").

Comments can be placed anywhere whitespace is allowed and use the common multi-line comment syntax of "/*" to start and "*/" to finish.

Termination of the BNF is indicated by the identifier "EOF". Everything after this is completely ignored.

3rd-level Format BNFs

Here's the current route car spawn.bnf as an example:

/* File format information */
FILE                          = :uint,Count [{:CarSpawnerItem}] .
FILE_NAME                     = "Route Car Spawn" .
FILE_EXT                      = "carspawn.dat" .
FILE_TYPE                     = "v" .
FILE_TYPE_VER                 = "1" .

/* Base types */
CarSpawnerItem                ==> :string :uint .

/* Format types */

EOF                           /* End of file */

All BNFs for the tools are required to have the five definitions shown above, so that the various programs can use them. FILE_TYPE and FILE_TYPE_VER are the letter and number (both as strings) used in all Simis Jinx files. FILE_EXT is either a file extension (e.g. "act") or a filename (e.g. "carspawn.dat") which selects which files can contain this format. FILE_NAME is a name suitable for displaying to the user. FILE is an expression representing the root of the file - the base of all parsing.

Binary Block Type Names

While the BNFs define what is allowed where, there is still one remaining problem for the Simis Jinx Binary format - each block type is identified by a number, not a string. For this, we can turn to some other files included with the original Train Simulator - the files in UTILS\FFEDIT.

  • sidn.txt defines a few base IDs, including "core" and "train" (0 and 4 respectively).
  • coreids.tok contains a list of all core "tokens" - i.e. block type names - in order of the numerical value.
  • appids.tok is a C header which includes forms.hdr and loadstr.hdr with a token defined before and after each inclusion.
  • forms.hdr and loadstr.hdr contain lists of all MSTS tokens in numerical order.

To construct the 32bit unsigned number used in the Simis Jinx Binary file format, the base ID and the token ID (from its position) are combined with the base forming the most significant 16bits and the token the least significant 16bits. E.g. the 7th "train" token would be 0x00040007.


Together with the BNFs, the number-block type name mapping completes the picture for loading and saving Simis Jinx files. However, as the BNFs are of my own construction, they are necessarily incomplete and possibly still inaccurate in some areas. This has improved a lot over the past few months, and will continue to do so, providing a good, solid and generic reading and writing capability for most Simis Jinx files.

Permalink | Author: | Tags: Format, Games, Microsoft, Simis, Train Simulator | Posted: 11:55PM on Sunday, 23 May, 2010 | Modified: 12:02AM on Monday, 24 May, 2010 | Comments: 0

Simis Editor - Feedback class

Feedback makes everything better, eventually. Getting or sending feedback is, however, not always simple or usable; users need to be able to bang out simple comments easily, with no forms to fill in, whilst still providing proper context and technical information if the feedback is the result of the application malfunctioning. Feedback should also be anonymous if the user wishes. The Feedback class in the next release of Simis Editor is attempting to do this; here I'm going to outline its user-facing functionality and the back-end implementation.

Entry Points

There are two different ways the feedback process can be started:

  • From the user: a "Send Feedback..." menu item under "Help".
  • From the application: anywhere in the application that catches exceptions.

While both routes show the same dialog, the latter case collects a load more contextual information to go with the report - most obviously, the exception, but it can also take anything the catch code wants to include.

Instanciation Code

The Feedback class is really simple to use, for both cases:

   try {
       new Feedback().PromptAndSend(ownerForm);
   } catch (SomeException e) {
       new Feedback(e, "sending feedback").PromptAndSend(ownerForm);

The ownerForm is used for showing the dialog modally. The class switches mode based on the arguments: none means "user feedback", Exception (exception) and String (operation) mean "application failure"; there is also a third mode where the caller provides the feedback type, operation and an IDictionary<string, string> of details.

User Dialog

The dialog is mostly the same for the two cases; the biggest change is the "faces" and introductory text. For user feedback, the introduction just explains when to include your e-mail address, as it is entirely optional.

In the application failure case, this dialog is the first thing the user sees when an operation fails, so it must explain that something's gone wrong and then why you should send the feedback at all.

As the purpose of the feedback dialog is to collect as many reports as possible, it attempts to ensure all users (or a maximum of users) are happy to send the reports by allowing the user to view all the data collected for sending. As shown below, this includes the full exception details (obviously) as well as some general system information. It also includes a user ID, which is randomly generated the first time the application intends to send feedback and which is not shared between applications (i.e. two applications that a user has installed that use this feedback system will each send a different user ID).

If the user is happy to send the report and clicks the button, an XML document is constructed, serialised and POSTed to the feedback server. The user is then given a message showing the success or failure of the feedback as a clear completion of the process.

Feedback Format

The feedback is sent as XML to make handing the data as easy as possible. This is an example of an application failure report, but user feedback reports are basically the same - just without the <details>.

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<report version="1.0" uid="ipejGfrUIt5gAZ3Y" time="2010-05-31T22:13:56.4276545+01:00" type="ApplicationFailure" email="">
   <os version="6.1.7600.0">Microsoft Windows NT 6.1.7600.0</os>
   <processor cores="4" />
   <clr bits="64" version="2.0.50727.4927" />
 <application version="">Simis Editor</application>
 <source file="C:\Users\James\Documents\Visual Studio 2008\Projects\JGR MSTS Editor\Simis Editor\Editor.cs" line="185" column="5">SimisEditor.Editor.OpenFile</source>
 <details>C:\Program Files (x86)\Microsoft Games\Train Simulator\ROUTES\JAPAN2\carspawn.dat

> From 0x00000122 - data preceding failure:
>   wnerItem( "Jp1van.s" 6 )
>   CarSpawnerItem( "Jp1van2.s" 6 )
>   )
> From 0x000001A2 - data following failure:
> > BNF has completed.
> >
> > Available states: .
> > Current rule: <none>.
> > Current state:
> >
> >    at Jgr.Grammar.BnfState.LeaveBlock() in C:\Users\James\Documents\Visual Studio 2008\Projects\JGR MSTS Editor\JGR.Grammar\BNF.cs:line 175
> >    at Jgr.IO.Parser.SimisReader.ReadToken() in C:\Users\James\Documents\Visual Studio 2008\Projects\JGR MSTS Editor\JGR.IO.Parser\SimisReader.cs:line 181
>    at Jgr.IO.Parser.SimisReader.ReadToken() in C:\Users\James\Documents\Visual Studio 2008\Projects\JGR MSTS Editor\JGR.IO.Parser\SimisReader.cs:line 196
>    at Jgr.IO.Parser.SimisFile.ReadStream(Stream stream, SimisFormat& simisFormat, SimisStreamFormat& streamFormat, Boolean& streamCompressed, SimisTreeNode& tree) in C:\Users\James\Documents\Visual Studio 2008\Projects\JGR MSTS Editor\JGR.IO.Parser\SimisFile.cs:line 74
>    at Jgr.IO.Parser.SimisFile..ctor(String fileName, SimisProvider simisProvider) in C:\Users\James\Documents\Visual Studio 2008\Projects\JGR MSTS Editor\JGR.IO.Parser\SimisFile.cs:line 32

  at Jgr.IO.Parser.SimisFile..ctor(String fileName, SimisProvider simisProvider) in C:\Users\James\Documents\Visual Studio 2008\Projects\JGR MSTS Editor\JGR.IO.Parser\SimisFile.cs:line 37
  at Jgr.IO.Parser.MutableSimisFile.Read() in C:\Users\James\Documents\Visual Studio 2008\Projects\JGR MSTS Editor\JGR.IO.Parser\MutableSimisFile.cs:line 28
  at SimisEditor.Editor.OpenFile(String filename) in C:\Users\James\Documents\Visual Studio 2008\Projects\JGR MSTS Editor\Simis Editor\Editor.cs:line 185</details>

One thing which this does not show is "attachments" - where the code calling the Feedback class specifies arbitrary extra data to include; these are sent as additional details but each with a name: <details name="extra stuff">...</details>.

Permalink | Author: | Tags: Editor, Feedback, Simis, XML | Posted: 10:42PM on Monday, 31 May, 2010 | Comments: 0

Simis Editor v0.4

I've just released the latest version of my Microsoft Train Simulator tools: Simis Editor v0.4 with the usual documentation. Some highlights for this release:

  • Open and Save dialogs support full filename filters from BNFs (e.g. "tsection.dat") in addition to extension filters.
  • Support for adding new blocks to the tree via context menu with 4 groups of operations:
    • Insert previous siblings.
    • Insert next siblings.
    • Insert before children.
    • Insert after children.
  • Problems loading *.bnf files and loading or saving Simis files are all offered for reporting online (via the Feedback class).
  • Added a status bar and help text for menu items.

Permalink | Author: | Tags: Editor, Simis, Train Simulator | Posted: 10:42PM on Sunday, 06 June, 2010 | Comments: 0

Media Foundation, Matroska and MP3

I have a Matroska (.mkv) file with the following tracks (data streams):

Tracks : 2
Track 1 : Video
  - Codec : (V_MPEG4/ISO/AVC)
Track 2 : Audio
  - Codec : MPEG Audio 1, 2, 2.5 Layer III (A_MPEG/L3)

Nothing particularly special there; I have the following relevant DirectX Media Objects (DMOs), DirectShow and Media Foundation codecs installed:

  • Haali Media Splitter: a DirectShow splitter filter for Matroska files, among other container formats (analogous to the AVI Splitter for .avi and others).
  • ffdshow: a DirectShow decoder filter for just about anything, including MPEG-4 Video and MPEG-1 Audio Layer 3 (MP3).
  • Windows 7's in-box DMOs decoder filters for MPEG-4 Video and MP3. These can be used by both DirectShow and Media Foundation.

Question: What happens if this Matroska file is played in Windows Media Player or Windows Media Center?

Answer: No video and audio stutters a lot.

Question: Why?

Answer: Both will try to use Media Foundation first and DirectShow second. As Media Foundation has no preferred splitter for Matroska files (either in-box or that I've installed), it hunts for a supported transform (similar to DirectShow's filters) with which to play the file; the MP3 transform duly indicates that it can play the file.

I believe this is because the MP3 decoder ignores the data at the start of the file which it doesn't understand (to allow for ID3 tags) and then picks up the first frame of the audio stream inside the file. The stuttering is most likely it attempting to play back the video frames of data (the two data streams are interleaved within the container).

The solution: Amazingly simple; the only thing that matters is that they're trying to use Media Foundation first, so set one registry key to indicate .mkv files prefer to be handled by DirectShow and it works great in both players.

   Runtime    REG_DWORD    0x7


Permalink | Author: | Tags: DirectShow, MP3, Matroska, Media Center, Media Foundation, Microsoft, WMP, Windows | Posted: 10:27PM on Sunday, 13 June, 2010 | Comments: 0

Simis Editor v0.5

Download Simis Editor v0.5 and read the documentation. Release highlights:

  • New format support: Cameras, GUI, GUI Bitmaps, GUI Screens, Route Forests, Route Gantry Sets, Route Speed Post Sets, Route Tile Definition, Route Tile Definition Low, Route Track Types, Signals.
  • Updated format support: Activity, Route Reference, Route Train Path, Shape, Train Consist, Train Engine, Train Wagon, World.
  • Bug fixes and general improvements to underlying libraries.
  • Thanks to Jeffrey Kraus-Yao for many of the format additions and updates.

Permalink | Author: | Tags: Editor, Simis, Train Simulator | Posted: 10:38PM on Sunday, 19 September, 2010 | Comments: 0

Simis Editor v0.6 Preview - ACE Support

Microsoft Train Simulator uses a custom file format for its textures and other in-game images - ACE (no, not the data compression format). They are stored as a 2nd-level format inside the Simis file format, which means they can be compressed with zlib's DEFLATE algorithm transparently.

I'm not going to document the format publicly just yet, but instead show a preview of loading ACE images in the next version of Simis Editor:

I have also implemented a DXT1 compression algorithm which, together with the decoder, allows Simis Editor to load and save ACE files in all three of their "formats":

  • Uncompressed (A)RGB.
  • Zlib compressed (A)RGB.
  • DXT1 compressed.

While Simis Editor will not become an image editor in its own right, I will be supporting importing/exporting as standard image types (BMP, PNG, etc.) and the new "ImageFile" command-line tool will provide for easy bulk conversions. Stay tuned for more details.

Permalink | Author: | Tags: ACE, Editor, Games, Microsoft, Simis, Train Simulator | Posted: 04:20AM on Wednesday, 05 January, 2011 | Comments: 0

Open Rails Tips and Tricks

With the release of Open Rails 0.7.0, now seems a good time to document some of the hidden settings and features:

  • LoggingFilename (string setting) Specifies a static or generated name for the log file. Defaults to "OpenRailsLog.txt". This is a standard .NET format string, with the following replacements available: (New in 0.7.0)
    1. "Open Rails".
    2. Version number normally, build stamp if compiling yourself.
    3. Version number.
    4. Build stamp.
    5. Current date/time.
  • LoggingPath (string setting) Specifies an absolute path to save the log file when running the game. The default is your desktop.
  • ScreenshotPath (string setting) Specifies an absolute path to save in-game screenshots (using Print Screen). The default is the "Open Rails" folder in your pictures folder. (New in 0.7.0)
  • ShadowAllShapes (boolean setting) Specifies whether to generate shadows from all objects in the game, instead of only those which the route creator has specifically set. This can often make a route more visually pleasing, at a small performance cost.
  • VerticalSync (boolean setting) Specifies whether to limit the frame-rate to the monitor's refresh rate.
  • ViewingFOV (integer setting) Specifies the field-of-view for the display in degrees, vertically. The default is 45, which is equivalent to 60 horizontally in a 4:3 ratio. Because Open Rails measures this vertically, you'll automatically see more on widescreen resolutions. (New in 0.7.0)
  • Print Screen Takes a screenshot and saves it in the ScreenshotPath above in the PNG format. (New in 0.7.0)
  • Alt Hold down the "Alt" key and click on a switch to throw it (change which path is taken).
  • U Hold down the "U" key and click on a coupling to uncouple part of the player train.
  • Control - E Switches between locomotives in the player train.
  • Shift - Control - F Switches the direction of the player locomotive. This is a temporary substitute for the lack of MSTS Bin reverse cab support.
  • Alt - F1 Dumps the command list with keys to "keyboard.txt" and a high-resolution keyboard image (suitable for printing) to "keyboard.png".
  • Control - Alt - Page Up/Page Down/Home Controls the game speed, increasing, reducing and resetting it respectively.
  • Control - Alt - F11 Toggles the dispatcher/signalling visualisation, which highlights the selected route ahead of the player and AI trains, with labels for switches and signals.

To edit hidden settings

All Open Rails settings are stored in the Registry, under the key "HKEY_CURRENT_USER\Software\OpenRails\ORTS". The usual disclaimers about editing the registry apply.

  • To edit any setting, open Registry Editor (regedit.exe) and navigate the left side to "HKEY_CURRENT_USER\Software\OpenRails\ORTS".
  • Some settings will appear already, some have to be created.
  • To create a setting, either use the menu "Edit" > "New" or right-click the right side and use the menu "New".
    • String settings should be created as "String Value".
    • Integer and boolean settings should be created as "DWORD (32-bit) Value".
  • To edit any setting, double-click its name on the right.
    • Strings can just be edited as-is.
    • Integer settings should be edited with the "Base" option set to "Decimal" in the edit window.
    • Boolean settings should use the value "1" for true and "0" for false.

Permalink | Author: | Tags: Games, Open Rails, Train Simulator | Posted: 06:00PM on Wednesday, 14 December, 2011 | Comments: 0

Windows Live Mesh remote connections and changing monitor configurations

Windows Live Mesh has a remote connections feature, similar to Remote Desktop, which can be accessed from the Windows Live Mesh client and the Windows Live Devices website. I've used it frequently when I needed to access my home machine from work - it connects despite no ports being forwarded in the router and it doesn't mess up my multi-monitor configuration at home (normal Remote Desktop will change the resolutions and such).

That was until I changed my multi-monitor configuration at home.

Old arrangement: 1280x1024 / 1600x1200* / 1280x1024 (* primary)

New arrangement: 1280x1024 / 2560x1440* / 1600x1200 (* primary)

Then, every time Windows started, shortly after the login screen appeared, all the monitors would jumble themselves up in to something very closely resembling the old arrangement: 1280x1024 / 1280x1024 / 1600x1200* (* primary). It seemed clear something was attempting to put the monitors into the old arrangement based on some saved information, but it wasn't obvious what that something was.

To diagnose the issue, I carefully timed when the monitors first started showing signs of rearrangement after boot and consulted the list of processes and their start times (from "wmic process") to see what was starting at or just before the rearrangement.

That's when I spotted "C:\Program Files\Windows Live\Mesh\wlcrasvc.exe", the Windows Live Mesh remote connections service, starting just seconds before the rearrangement.

So up came Registry Editor (regedit.exe) and a look through HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft until I found HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Live Mesh. Hiding under there, at HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Live Mesh\Remote Desktop\DisplayDevices, is a key for each display device on the machine and some monitor arrangement data (x, y, width, height).

I stopped the Windows Live Mesh remote connections service and deleted the entire HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Live Mesh key, since it didn't appear to contain anything except the display devices information. Upon restarting the service, some parts of the key came back but no monitors got rearranged and, upon connecting from another machine, the display devices keys came back, but with my new arrangement. I restarted the service again to be sure, and nothing rearranged itself.


Permalink | Author: | Tags: Microsoft, Windows, Live, Remote Desktop | Posted: 06:00PM on Wednesday, 21 December, 2011 | Comments: 0

HTTP Archive Difference tool

When measuring website performance, comparing two different page loads is a common and useful operation. However, there isn't much available to do it; some tools, like Fiddler, let you export the timeline waterfall as an image, allowing a visual inspection of two page loads to be done, but this lacks the ability to easily compare things such as when a particular request was made in each page.

A good intermediate solution is to save HTTP Archive files of each page load; these are JSON files containing the request and response timing information, headers, cookies and (depending on tool) contents. HTTP Archive (HAR) files can be exported natively from Chrome and Fiddler, and via the NetExport extension for Firebug in Firefox. Internet Explorer can export an XML transcription of the official JSON HAR format.

But what about comparing the exported HAR data? There is an online HAR Viewer but nothing to compare two HAR files. This particular issue came to light for me a few months back, when I was working on page load performance at work, but it really hit home when, at the WebPerfDays on Friday, "HAR diff" appeared on a list of tools that web performance people wanted.

Putting the two together, I have put my HAR difference tool online for all to use. Just paste in two HAR files (or just one to view it alone) and hit "Compare!". The tool summarises the differences, both in which requests were made, their timing information, and how the browser utilised its connections. The precise level of details depends on whether a proxy (like Fiddler) or browser tool (like Firebug) was used to collect the information, but most of the information is generally available from anything that exports HAR files.

Permalink | Author: | Tags: HAR, Tools | Posted: 08:30PM on Monday, 08 October, 2012 | Comments: 0

Getting good MiniDumps from processes WER has already grabbed

If you've got a crashed process that WER has handled but you either know the JIT debugger is set wrong or you haven't got one set up at all (so no "Debug" button on the crash dialog), you can leave the dialog open and attach any debugger of your choice to the suspended, crashed process.

However, there's a problem. By attaching the debugger to the suspended process, the process' last-exception information has been lost; this is because the way the debugger sucessfully attaches to a suspended (or hung) process is to inject a new, running thread and have that thread crash. So your debugger, and any minidumps it generates with .dump, will not be able to find the original program's exception information. Oops!

The solution is very similar to Raymond Chen's post on Sucking the exception pointers out of a stack trace. The difference is how we find the pointers and what we do with them.

Starting with the stack trace from kb:

RetAddr           : Args to Child                                                           : Call Site
000007f9`d88212d2 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : ntdll!NtWaitForMultipleObjects+0xa
000007f9`dae8d20e : 00000000`02300000 000007f6`96e75000 00000000`00000000 00000000`000001dc : KERNELBASE!WaitForMultipleObjectsEx+0xe5
000007f9`dae8cfd2 : fffff8a0`00000080 00000000`80004005 00000000`000001ec 00000000`00000000 : KERNEL32!WerpReportFaultInternal+0x1fa
000007f9`d889fc87 : 00000000`00000000 00000000`00195c30 00000000`00000030 00000000`00195c30 : KERNEL32!WerpReportFault+0x42
000007f9`db56a43d : 00000001`00000000 00000000`00196990 00000000`0019f620 00000000`003441d0 : KERNELBASE!UnhandledExceptionFilter+0x1d7
000007f9`db4b4ad4 : 000007f9`d8eeabdd 00000000`00196990 00000000`00000000 00000000`00000000 : ntdll!LdrpLogFatalUserCallbackException+0x4d
000007f9`db4d464d : 00000000`00000000 00000000`00195de8 00000000`00195da0 000007f9`d8f7f000 : ntdll!KiUserCallbackDispatcherHandler+0x20
000007f9`db4d567c : 000007f9`d8ee0000 00000000`0019f5c0 000007f9`0000e4f0 00000000`00000000 : ntdll!RtlpExecuteHandlerForException+0xd
000007f9`db4b4bba : 00000000`00000000 00000000`00000000 000007f6`96e7e300 00000000`00000000 : ntdll!RtlDispatchException+0x392
000007f9`db4b2acd : 000007f9`d8bbc36c 00000000`00360f00 00000000`00360f00 00000000`00360f00 : ntdll!KiUserExceptionDispatch+0x2e
000007f9`d8bbc36c : 00000000`00360f00 00000000`00360f00 00000000`00360f00 00000000`00000000 : ntdll!memcpy+0x21a

The EXCEPTION_POINTERS structure address seems[1] to be passed as arguments 2 and 4 of the call to "KERNEL32!WerpReportFault". From this, the exception and context records can be displayed - but we only really need to include the EXCEPTION_POINTERS address in the minidump, since adding this one address will allow the !analyze command to work.

Create the minidump using the command ".dump /ma /xp 00195c30 dump.dmp" and you're done. (Apart from all of the actual debugging - have fun!)

[1] This is on Windows 8 64bit running the 64bit WinDbg debugging a 32bit executable. YMMV.

Permalink | Author: | Tags: Windows, WinDbg, Debugging | Posted: 09:00PM on Monday, 25 February, 2013 | Comments: 0

Mixing jQuery and Underscore.js nicely

Working on a new project, with both jQuery and Underscore.js, I found myself with a problem. Using Underscore.js on jQuery objects felt somewhat dirty/messy:

   // With $ and _ both in scope...
   // Typical code:
       .each(function(element) {
           element = $(element);
           // Run some jQuery on 'element'.
       .filter(function(element) {
           element = $(element);
           // Test some jQuery on 'element'.
       .each(function(element) {
           element = $(element);
           // Run some jQuery on 'element'.

The two key problems I had were:

  1. The initial "_($('..." and "_.chain($('..." bugged my sense of clean code.
  2. I had to re-wrap every element in jQuery inside each filter() or each() or any other Underscore.js iterator function.

The solution I found is pretty simple and also in two parts:

  1. Add a new mix-in to Underscore.js: the jQuery $() function.
  2. Use the map() method from Underscore.js directly with jQuery's $() function. (This also meant an increased use of the chain() method.)

   // With $ and _ both in scope...
   // One-time set-up:
   _.mixin({$: $});
   // Typical code:
       .$()     // This uses jQuery to select the elements matching the selector.
       .map($)  // This uses Underscore.js to wrap every matched element in jQuery.
       .filter(function(element) {
           // Test some jQuery on 'element'.
       .each(function(element) {
           // Run some jQuery on 'element'.

Much cleaner!

Addendum: It was pointed out to me that you can avoid the chain() methods of Underscore.js by replacing that library with Lo-Dash, which has much the same API, but with better performance and more intuitive method chaining (i.e. more like jQuery). Thanks tom!

Permalink | Author: | Tags: JavaScript, jQuery, Underscore.js | Posted: 06:00PM on Thursday, 31 October, 2013 | Comments: 0

Cloning a git-svn repository without re-downloading everything

This is a problem I've been having for a few weeks now, since I started using git and the git-svn plug-in to operate on a Subversion repository. Today I figured out the solution.

When you git clone a repository which has been used with Subversion and git-svn, none of the extra metadata is cloned with it. That's expected, so you have to do git svn init again to configure git-svn in this new repository. However, this does not save you from git-svn then downloading everything from Subversion! This can easily take hours, if not days, on a large repository.

The trick is simple enough: git update-ref refs/remotes/git-svn master. Note that you might need to use a different branch name in place of "master", depending on your git repository's set up. All this does is tell git-svn that the latest Subversion commit is the latest git commit on branch "master".

Now when you do git svn fetch, the worst it'll do is "Rebuilding ...rev_map..." which should only take a minute or two and is entirely local.

$ git clone
normal clone stuff

$ git svn init

$ git update-ref refs/remotes/git-svn master

$ git svn fetch
Rebuilding .git/svn/refs/remotes/git-svn/.rev-map.guid ...
r1 = sha1
Done rebuilding .git/svn/refs/remotes/git-svn/.rev-map.guid


Permalink | Author: | Tags: Git, Subversion, Version control | Posted: 08:45PM on Tuesday, 22 December, 2015 | Comments: 0

Offline London Underground routing on your phone

There are probably a number of apps available for this as well, but none that I've found for Windows Phone are any good. Even desktop routing via Google Maps or Bing Maps has not been very good at estimating journey times for me.

So I made a website which does it, just how I like it, with the following features:

  • Uses Application Cache to remain available even with no data connection
  • Browse lines, stations and routes
  • Show all possible routes with duration and stops
  • Entirely front-end code, using Angular JS

Offline Tube:

Permalink | Author: | Tags: Offline, Tube | Posted: 08:00PM on Sunday, 07 February, 2016 | Comments: 0

ChatZilla Plugin: Network Trace


This plugin for ChatZilla allows you to see the incoming and outgoing network messages easily.


  • /network-trace [<toggle>]

    Without any arguments, displays the tracing status for the current network.

    With the toggle argument, controls the tracing status for the current network. "true", "on", "yes" and "1" will enable tracing. "false", "off", "no" and "0" will disable tracing.


[<<] PING
[>>] PONG
[>>] JOIN #channel
[<<] :Nickname!chatzilla@hostname JOIN :#channel
[<<] 353 Nickname = #channel :@Nickname
[<<] 366 Nickname #channel :End of /NAMES list.
[>>] MODE #channel
[<<] 324 Nickname #channel +nt
[<<] 329 Nickname #channel 1454711109
[>>] PART #channel :
[<<] :Nickname!chatzilla@moz-eif.ga4.155.217.IP PART #channel :""


If you have ChatZilla build 0.9.92 2016-02-08 19:00 or later, you can do:


Otherwise, due to a bug, you will need to download the master archive and extract it in to a new subfolder of the "scripts" folder. This can be found inside the path shown by the command "/pref profilePath".


[INFO] Preference “profilePath” is “C:\Users\James\AppData\Roaming\Mozilla\Firefox\Profiles\\chatzilla”.

means the best place to extract the archive is


Permalink | Author: | Tags: ChatZilla, Scripts | Posted: 09:30PM on Sunday, 14 February, 2016 | Comments: 0

ChatZilla Plugin: Topic Diff


This plugin for ChatZilla shows you the changes made to a channel's topic.


=-= Nickname has changed the topic to “ChatZilla topic diff test”
=-= Nickname has changed the topic to “ChatZilla diff test”
=-= Nickname deleted "topic ".
=-= Nickname has changed the topic to “ChatZilla diff testing!”
=-= Nickname added "ing!".
=-= Nickname has changed the topic to “ChatZilla topic diffing”
=-= Nickname added "topic ", deleted " test", deleted "!".

Permalink | Author: | Tags: ChatZilla, Scripts | Posted: 08:00PM on Sunday, 21 February, 2016 | Comments: 0

ChatZilla Plugin: Activity in Title


This plugin for ChatZilla shows activity in channels and private messages in the title bar.

Display - Text

The textual display shows up to three groups, each containing a list of views in that state.

  • <...> indicates items with stalk events (e.g. mentions of your nickname, stalk words, and private messages).
  • [...] indicates items with chat events (e.g. all talking in channels).
  • (...) indicates items with superfluous events (e.g. joins, parts, topic changes).

Display - Symbol

The symbolic display shows a single symbol for each view, in the same order as the textual display groups.

  • ! indicates an item with stalk events (e.g. mentions of your nickname, stalk words, and private messages).
  • + indicates an item with chat events (e.g. all talking in channels).
  • - indicates an item with superfluous events (e.g. joins, parts, topic changes).


  • display.symbol boolean (default false)

    When enabled, the display is just status symbols (!, + and -). Otherwise, the display is (possibly truncated) names.

  • display.crop integer (default 0)

    Specifies how much of the view name to include, in characters. A value of 0 turns off truncation.

  • titlebar.event boolean (default false)

    When enabled, the title bar includes superfluous events.

  • boolean (default true)

    When enabled, the title bar includes chat events.

  • titlebar.stalk boolean (default true)

    When enabled, the title bar includes stalk events.


  • <UserA, UserB> [#channelA] (#channelB) ChatZilla!

    In this example, UserA and UserB have stalk events, #channelA has chat events, and #channelB has superfluous events. Superfluous events are hidden by default.

  • !!+-ChatZilla!

    This is the same as the example above, but with display.symbol set to true.

Permalink | Author: | Tags: ChatZilla, Scripts | Posted: 09:00PM on Sunday, 28 February, 2016 | Comments: 0

ChatZilla Plugin: Input Keys


This plugin for ChatZilla lets you type most text formatting commands with keyboard shortcuts.


  • Control-B for bold
  • Control-U for underline
  • Control-K for color
  • Control-O for reset (original)

Permalink | Author: | Tags: ChatZilla, Scripts | Posted: 09:00PM on Sunday, 13 March, 2016 | Comments: 0

Open Rails 1.1

Open Rails 1.1 has been released!




  • Refuelling from water troughs added
  • Activity restricted speed zones added
  • Switching control to another train (AI and static) added
  • Auxiliary tender support added
  • Blended dynamic brakes support added
  • Improved sound cut-off distances
  • Additional sound triggers added
  • Multiple freight animations added
  • Loading/unloading animations for freight animations added
  • Join/split functionality with player and AI trains improved


  • Code has been re-factored to ease future developments
  • 3D cabs loaded by default when present
  • Quick access to documentation from the menu
  • Activity events can now dynamically change the weather
  • AI trains can now get permission to pass signals at danger
  • Super-elevation updated and harmonic-rocking improved
  • Manual signalling mode can be engaged while moving
  • Compressor information now shown in HUD
  • Additional locomotive's brake status now shown in HUD
  • Engine brakes extended to work with tender
  • Keyboard commands for cutting off power and setting brakes off added
  • EP brake partial release support added
  • Brake pipe pressure propagation improved
  • Bailoff on twin pipe airbrakes improved
  • Engine and independent brakes improved
  • 3D cab digital gauges improved
  • Train lights condition for "service" improved
  • Sound looping improved
  • Stereo/mono cab sounds improved compatibility
  • Gear cab view control added
  • Projected speed cab view control added
  • Friction braking cab view control improved
  • Night-switching of cab view in tunnels improved
  • "Variable1" in gearless diesels improved
  • "Variable2" in electrics improved
  • "Variable3" in diesels and electrics improved
  • Multiplayer can now connect to domain names in addition to IPs
  • Options to limit safe speed through curves, with penalties for breaches
  • Options for improved weather visuals on supported systems
  • Optimizated selection of track sounds
  • Correction of faulty engine parameters added


  • Activity event popup is larger and more responsive to lots of text
  • Messages for AI trains improved
  • Train Operations window updates at reversal points
  • Signal lights reliability improved
  • Save/Resume after train reversal improved
  • Fuel consumption calculations improved
  • Doors open correctly after reversal
  • "Include" directive for content improved

Important notes:

  • Open Rails develops features and bug fixes in public at
  • Open Rails is distributed under the GNU GPL v3 free software license

Permalink | Author: | Tags: Open Rails | Posted: 07:30PM on Sunday, 27 March, 2016 | Comments: 0

Powered by the Content Parser System, copyright 2002 - 2017 James G. Ross.