Alex Hopmann’s Blog

Uploading any files more than a few hundred K from a browser has been a problem for years. The UI available in the browser is very limited and relying on a single HTTP request that might take minutes or hours (and that you have to start over from scratch if it fails) often turns into a huge source of user frustration. There is also an extra flaw in that the TCP connection can fail before the whole file is transmitted but depending on the circumstances the server might not be able to tell if the whole file was actually received. There are a bunch of sites that use various ActiveX or Java controls but those have typically been a pain to install and/or flakey.

I just discovered that Silverlight can be used to create much more functional upload controls. Here is one for example in the Codeplex Code library. Granted, users need to have Silverlight already installed, but once they do it becomes much easier to have a good user interface, while having the actual process send chunks of the file that can be resumed if any piece fails, etc. Combine that with the Azure Blob chunked-PUT mechanism and you can build a very robust storage mechanism right in the browser. I’m looking forward to trying it out.

One last thought- it would be useful to define a standard protocol for uploading content in chunks (and yes, this is distinct from an HTTP PUT/POST with chunked encoding). Something along the lines of what the Azure Blob store does but defined as a standard that various controls and services can all interoperate.

As an old HTTP guy I often get nervous about new services. They tend to violate all sorts of key HTTP architecture concepts and just take advantage of the flexibility to do whatever.

I’ve been really happy to see that the Azure Blob storage actually gets this stuff right. They have valid REST semantics with a good URL namespace, support GET and PUT with the right kind of range headers, etag and conditional operation support. They have a smart design for uploading a large blob in multiple pieces (which works around one of the bigger flaws in the older WebDAV support), and all. Anyway, its great to see a team do all their homework and get these details right- I suspect this will really payoff over the long lifespan of a service as it fits in cleanly with rest of the web services world. (note- I’m not saying other competing products aren’t also doing these things right, I haven’t researched those details lately).

Dare Obasanjo posted an excelent description of the essence of the REST architectural style. Very good write up and it includes some good history and theory.

The newspapers today are reporting that Cisco has acquired Pure Networks. Congrats to the team, and to Cisco/Linksys which is getting itself a fine group of people and some great technology. Pure was always in a complicated marketplace but it makes a ton of sense to me that an industry leader like Linksys would see Software as a great advantage in making Networking easier for their customers. Plus it makes me feel like my decision to buy that Linksys WRT600N a few weeks ago was the right one.

Speaking of which- so far the WRT600N is performing really well, and I’m also using it with a WGA600N which is a dual-band N bridge that I’ve got to hook up equipment downstairs (the XBox, the Wii and the TV). There are two main problems I’ve had so far with the WRT600N. The first is that I named both my 5.4ghz and 2.4ghz networks with the same SID and its sometimes really unpredictable which network a given device has joined (and usually difficult to tell which one its connect to). This is partly a problem with the devices which don’t really communicate well which band they are on, but the router could help a lot here too. Its really confusing to figure out which devices are going to work best on which bands (between trying to balance distance, penetration through walls and media-playback performance). Messing around with it sometimes my TV (which acts as a media extender) works with awesome HD capability and sometimes its just crap. This seems like one of those things some intelligent home network management software could help with (hint hint).

The bigger problem is that for some reason its tunneling isn’t working. I love to use the remote access client to connect in to my home machines and I just can’t get that to work with this router. I can’t tell if its bugged or I’m doing something wrong, but the interface to set it up is actually less intuitive than normal (which is saying something).

I’ve had a Gigabit Ethernet network for quite some time but have mostly been using older Cat5 cables. They looked like they worked fine so why mess with them, right?

Lately I’ve been upgrading them with newer Cat6 cables. I just noticed a file-transfer that appears to be averaging 60megabytes per second or 60% utilization on the Ethernet. I’ve never gotten anywhere close to this performance before- it looks like the Cat6 stuff does make a big difference.

One of my home machines has terrible network performance. Its an older Shuttle SB51G but it appears to mostly be in good shape running Vista with all the latest driver updates. It tends to run about 100k bytes/sec on a 100mbps Ethernet connection to a local server that normally gives much much better results. The bandwidth usage (as show in taskman) jumps all over but never gets more than 2%.

It is plugged into my gigabit ethernet switch which has been handling traffic just fine including 100mbps connections. I tried switching the cables used to connect it. I’m pretty sure this machine used to have decent network performance. Doing a NETSTAT -E shows only 8 ethernet “errors” which doesn’t seem abnormally high. NETSTAT -S shows ~1000 TCP retransmits and ~1800 UDP errors, but its been running for a long time and again those aren’t especially high.

Any suggestions?

I’ve been using VMWare workstation to run some virtual machines lately. Its very helpful to be able to have a WinXP IE6 box, WinXP IE7, Vista, Linux, etc, all at once, not to mention other nice uses of VMWare.

So yesterday I’m doing some big file copies and noticing that my network transfer performance on my gigabit Ethernet is running a solid 6%. Not a blip over or under, as if something were hard-limiting it. I also notice there are these two extra network adapters that VMWare has installed which each think they are 100mbps connections.

So I try disabling them (since with a little poking around I find out they are only necessary if you are doing on-machine NAT or internal networks) and like magic my network transfers hop up to 9.5%. These get installed by default but for most people you will never need them, so get rid of them.

Now to figure out what the next bottleneck is. The machines involved have plenty of CPU, I was testing on multiple disks so disk IO should not have been limiting, and I’m pretty sure everything is hooked up with nice high quality cables on my gigabit switch.