Alex Hopmann’s Blog

The White House has a blog now which is pretty cool. My only complain would be that their RSS feed only has summaries so you have to click through to read the whole thing. I can understand some for-profit blogs that do that so they can get the click count & ads, but for this kind of thing its really annoying.

I had previously mentioned a cool Casio camera that could shoot high frames per second, and now it appears that
Casio is coming out with $300-$400 cameras that can do 1000fps for super slow motion. That plus 720P video (although I’d assume not both at once) might be too much to resist. Again, I’d want to try this out before just buying one which can be tricky nowadays. But the camera market had seemed a little stagnant (just adding more MP every year) and they have finally started latching on to some of the issues that make a difference.

Not sure if that other one was ever released?

Leading up to MacWorld there were a ton of rumors about a new MacMini, an iPhone Nano, and updates to Snow Leopard. Then the announcements during the actual show were a bit underwhelming(for most people- Chris was really excited about the new MacBook Pro).

My theory is this is all related to the reason that Apple is pulling out of MacWorld in the first place. It is a show they don’t control, IDC presumably wouldn’t let them webcast the keynote so they would rather introduce important things at their own events where they can get more focused press (on a schedule they control, not the middle of the CES mess and where they can time them so the products are ready), with a webcast and broadcast into all the Apple stores.

So they had already signed up for MacWorld and needed to announce something, but they kept it to the fairly tame iLife, iWork updates and the new MacBook Pro which is certainly a product for the enthusiast segment. The theory is that on good PR grounds they are holding the bigger more general audience announcements for events where they can control them better and make a bigger splash.

I’m assuming in a few weeks we are going to hear about invitations heading out for some other Apple event where they will start rolling out the next stuff (I’m hoping for an update to the MacMini myself, and betting that we won’t hear about iPhone updates until around June which appears to be when they refresh that product line).

Cliff Mass is a meteorologist at University of Washington and frequent contributer to KUOW our local public radio station and has been publishing an excellent blog about weather, especially focused on the weather of the Pacific NW. I’ve been very interested in weather lately, mostly stemming from needing to understand it as a pilot, although its often useful to help guess whether a given day will be a good one for skiing. Most weather reporting tends to be far too dumbed down for me, but Cliff appears to be blogging about the real details (while presenting it in an approachable way), plus the focus on the phenomena of the NW is really helpful.

At this point I’m pretty burned out on the Canon TX1. The form factor just sucks, and its starting to show some serious wear (the mode selector switch which is also really inconvenient is broken sometimes too and won’t go into certain modes).

Lucky for me its that time of year and there are some new options out. The Samsung HZ10W looks really cool- I love that it combines 10x zoom like the Canon but also does wider angle than typical small cameras all in what appears to be a fairly small form factor (and its only $300). It is supposed to have both optical and digital image stabalization, which can be critical with the higher zoom factor cameras- but I’d still want to try it out and see how well it works, I’ve often had problems with the TX1 getting it stable enough with even 5-6x zoom.

Olympus has the SP-590UZ that zoomes to 26x but its in a bigger form factor than I like.

Ever had that experience where you go do some work and then stop yourself and think “There has got to be an easier way to do this”?

I’ve been building some stuff that is mostly exposed as a web-service. I wanted to call it from PowerShell so I made a nifty set of Cmdlets that call it. But really each of them is just marshaling pretty much directly into the underlying SOAP service with no logic at all. I’ve got to assume someone already came up with a generic solution for this?

So here is a classic database design quandary. Lets say you are developing a service with accounts and each account has a set of parameters associated with it- have they bought the service, how much storage do they have, how many user licenses, etc.

Now, the set of things you want to store is going change over time for sure. So option 1 is to design a completely flexible system for storing key-value pairs-
CREATE TABLE [AcctParams1] (
[AcctId] [int] NOT NULL PRIMARY KEY,
[Key] [nvarchar] (16) NOT NULL,
[Value] [int] NOT NULL
)
CREATE UNIQUE CLUSTERED INDEX [AcctParamsIdx] ON [AcctParams1] ([AcctId], [Key])

A couple of notes before I move on. First of all its very important to create the clustered index on AcctId. You want to make sure that all the rows for the given account are grouped together on disk. If not, when the database goes to load just one account it might have to do IOs all over the place just to get the scattered AcctParam values.

As I said above, this approach is completely flexible, ideally involving no database schema changes ever. But the storage is somewhat less efficient and you are going to have to do multiple queries to retrieve everything you need about an account (one for the account itself, another for params).

The other way to do it is to just build out a table of the explicit columns for these parameters in classic database way-

CREATE TABLE [AcctParams2] (
[AcctId] [int] NOT NULL IDENTITY,
[Purchase] [int] NOT NULL,
[UserLimit] [int] NOT NULL,
[StorageLimit] [int] NOT NULL
)
With this approach you need to change the database schema every time you want to add some new param. However I think sometimes people get too freaked out by database schema changes (often because of too painful “upgrades” in the past). For example if you just needed to do this-

ALTER TABLE [AcctParams2] ADD [TransferLimit] [int] NOT NULL DEFAULT 5

On modern databases this kind of thing tends to execute pretty quickly but most importantly if your code is written carefully you can run this on your SQL box while your service is still online without having to simultaneously update the code of the service. I did some examples and create the above table and put a million rows of random data in it (which took 229 seconds on my test box). The above ALTER TABLE took 7 seconds to execute.

Guess what? We can do much better. Try this one-
ALTER TABLE [AcctParams2] ADD [TransferLimit] [int] NULL

By making it a NULL column it took… 0 seconds to execute. And again, if your app is built the right way, existing code will continue to work unchanged. So you can use a database schema that really defines the parameter names and types (in effect, SQL is handling the key-value stuff for you), but still has minimal upgrade impact on the uptime of your service.

A third approach is to have something like-

CREATE TABLE [AcctParams3] (
[AcctId] [int] NOT NULL IDENTITY,
[Params] [text] NOT NULL,
)

And stuff XML in the text column. Again, lots of flexibility, but you need to deal with all the XML parsing and it tends to be difficult to get the database to help you with any queries or analysis of that data. I’ve also seen multiple efforts that went down this path and the XML parsers ended up inflexible enough that they actually introduced tons of inflexibility and upgrade hassle. If you need to touch every row to upgrade the XML on this approach you just made yourself a huge problem.

One last note that applies to either approach 2 or 3- you can combine these params in the base [Account] table. Whether you want to do this or not depends on the access patterns. If 90%+ of the time when you want to access one, you want to access the other (IE, you are always writing SELECT * from [Account] join [AcctParams] on [Account].[AcctId] = [AcctParams].[AcctId] ), you might as well combine them so that its always just one I/O. On the other hand if you often want just one or the other AND they start to get large (lots of values, especially to the point where it gets close to the page size), it can make sense to split them out.

As will all optimization, there are no hard and fast rules- just guidelines and good places to test alternatives.

Dare embarked on a holiday project too, building some stuff using Azure. I’ve been meaning to get around to that one too myself. In any case, some great observations there, I’ll have to write up my likes and dislikes about PowerShell myself shortly.