All Things Distributed - Handheld Edition
Updated 01:51 AM May 25, 2005

Two Recommended Essayists
01:51 AM May 25, 2005 - There are two online technology essay writers who I truly enjoy reading: Scott Berkun and Paul Graham. Paul is probably the more well known of the two given his Hacker and Painters book, but I think that both have a unique insights in the software industry and the development process. Both are excellent writers who take a deeper, philosophical look at human nature and the specifically the traits of professionals, they investigate the fundamentals of human collaboration, and the ways that it impacts our daily lifes Paul's background is in Lisp programming, while Scott's is in program management and UI design. If you want to sample Paul Graham I suggest you start with "Made in the USA", for Scott Berkun I suggest his recent "Why Smart People Defend Bad Ideas". What triggered this posting is that Scott Berkun now also has a book out: "The Art of Project Management" which I think is very much worthwhile reading for anyone involved in project management. There is a lot of useful food for thought in this book as well as many pratical suggestions, and this week it will land on the desks of the people that work for and with me as "suggested reading". A sample chapter can be found at Scott's book site, if you are interested in sampling before you buy.
comments (2) trackback(2)

Advice needed: Dual Opteron Boards
05:06 PM May 23, 2005 - Lazyweb: I need some suggestions for a dual proc Opteron board. My Tyan Thunder K8W went silent this weekend and I don’t seem to be able to revive it. The new board needs to be able to handle dual socket 940 processors with at least 4 GB per proc. It needs support for 4xSATA in RAID, and if possible have a second SATA RAID controller for a set of raptor boot disks.
comments (0) trackback(0)

The Future of the US Computer Science Research
07:20 PM May 10, 2005 - Computer Science has this strange symbiosis with the computer industry: without the computer industry there would be no computer science and without computer science there would be no computer industry. Computer Science research has been the breeding ground for many multi-billion dollar corporation and many of these research projects were funded through US government programs, mainly through DOD. Computer Science Research funding is dropping to a level that puts the field into crisis, and many believe that it will have a significant long term negative impact on the competitiveness of the US computer industry. Ed Lazowsky and Dave Patterson, both members of the President’s Information Technology Advisory Committee, are sounding the alarm bell in a very clearly written editorial in this month’s Science: Where will the next generation of groundbreaking innovations in IT arise? Where will the Turing Awardees 30 years hence reside? Given current trends, the answers to both questions will likely be, “not in the United States.” This is required reading for anyone who has a stake in the future of the US computer industry.
comments (1) trackback(1)

Bandwidth at 37,000 feet
05:30 AM Apr 19, 2005 - I am on the SAS flight from Seattle to Copenhagen, somewhere north of Iceland. I am using the onboard wireless internet service. Ironically the airplane is a Airbus 340, but the wireless service is offered by Boeing. $30 for the whole flight or $0.25/minute with a minimum of $10. The service is not too bad; high latency, but sufficiently downstream bandwidth to get email in. Not much bandwidth for sending. Latency: 750 millisecond to amazon.com Bandwidth: 287 kbps downlink, 27 kbps upstream Using a desktop feedreader would probably be effective, but the bloglines experience surprisingly does not suffer too much from the high latency. BTW the plane also has nice 110V sockets. I am on my way to the UK to speak about Amazon.com Web Services at a UK government sponsored event.
comments (0) trackback(5)

Stories from a Sailor
11:37 AM Apr 18, 2005 - I came to Microsoft ... to build the biggest computer you can imagine, by just using components you can find at Radio Shack. There are two hours of video interview with Jim Gray online at Channel 9 that have captured Jim at his best (part1, part 2, via dbox). Reviewing the past and future of scalable distributed computing, operating systems and databases, Jim shines as the formidable story teller he truly is. About ten years ago I hooked up with Jim in working on the Scalable Servers project. At that moment the debate was still raging what would be better, to scale up or to scale out. SGI and Sun where still heavily investing in building very advanced multi-processor boxes, but theses machines were expensive and tools for failure isolation in multi-processors were still minimal. Scale-out appeared the way to overcome these obstacles and we were advocates that the basis for this should be true commodity components. Together with Thorsten von Eicken (now chief architect at Citrix) I started working on the intra-node communication obstacles. At that time we believed that to make distributed systems work on clusters of scalable low-end components you needed to cut-away as much of the intra-node communication latency as possible. This meant removing the biggest latency introducer: the operating system. Big commercial clusters such as the IBM SP2 already had direct user-level communication, but in a fashion that didn't allow more than one application to use the interconnect directly. Our work resulted in the U-Net Architecture which allowed for delivery of data directly into application memory. It was published in the 1995 SOSP. What set this work apart from earlier approaches was that it was available for standard workstations and operating systems and that it provided full isolation such that multiple applications could access the interconnect concurrently. At that time the approach was considered a major breakthrough to enable the scaling of commodity clusters. The U-Net approach as well as technologies such as VMMC by Kai Li at Princeton were the precursors to what later became the Virtual Interface Architecture standard. VIA has been largely abandoned as source of hardware design in favor of Infiniband which also centers around direct user-level access. In "The Evolution of the Virtual Interface Architecture" Thorsten and I reviewed the transition of the research prototypes into industry standard. The U-Net work was made very popular by Matt Welsh, who added dynamic memory management, ported it to Linux and added support for Fast-Ethernet (Matt's old u-net project pages). Thorsten and I built the first prototype on the SunOS platform, but in 1995 Sun pulled the plug on the operating system in favor of Solaris. Access to the OS source code was necessary to understand the complex workings of device and user-level address mappings, and the new Solaris license was a nightmare for Universities. We didn't consider Linux as an option, because it was an old-fashioned monolithic operating system that didn't sport any of the advantages of the new operating systems world. E.g. the network buffer management was even worse than in SunOS. Microsoft saw the potential of working with Universities on these issues and produced a source code license that allowed faculty and students to work with the Windows OS source code, without signing their lives away. We were happy to work on Windows as it meant that our research could now potentially reach millions of real computer users. Jim Gray, visionary as always, had this figured out long before anybody else and was actively promoting the use of Windows source by academics. Thorsten and I were guinea pigs as we showed up at Jim's office in downtown San Francisco to work on porting U-Net to Windows, as the first access to Windows source code was only on-site at Microsoft offices. The port took us 4 days and we returned homed with sufficient material to make U-Net work on the Windows platform. In those 4 days Jim made a deep impression on me with his insight into computing, industry and lives as a technologists. After that week I would return frequently to Jim for guidance in research and have never been disappointed. The video interview give a good sense of his breadth and depth. The other passion of Jim we became familiar with that first week was sailing. Since that time I have made the after-work trip on Jim's boat to Sausalito for dinner and good stories several times. I like to believe that Jim's story telling talents are rooted in his sailors heart, and there is never a dull moment with him around. Jim got recognized for his many contributions to modern computing with a Turing Award in 1998.
comments (1) trackback(0)


Copyright © 2003 Werner Vogels