The Eclectic Technologist

Doing anything at the last minute is usually not a good idea. That seems obvious, but it’s something I had to learn the hard way.

Years ago, I used to work in the aerospace industry. It’s a lot of fun, but there’s also a lot of tedium and waiting. High dollar, single quantity products such as spacecraft, are not easy to build, test, and deliver. The problem is that everything has to be right the first time. This also applied to the systems used to test the flight systems.

There are many stages of testing, one of which is integration with a larger package, in our case the “ride”, or the satellite our instrument was piggybacking on. The satellite was built by GE Astrospace division, formerly RCA. This particular satellite system was quite old, it was primarily used by NOAA to measure cloud cover using infrared imaging.

Our instrument was bolted to the satellite after which GE would go through their automated test procedures. There are a lot of systems on a satellite, so the test takes a long time — usually hours. The test is monitored by GE and NASA staff in what is called the NAGE room. Our lead systems engineer was located there as well to monitor the test. Myself and another colleague were located in the NAGE annex. This was a small room far way from the NAGE room proper. We had a direct data link from the satellite telemetry stream which we would use to monitor our instrument. Our only communication was through that data link and a telephone.

The data link was a proprietary, synchronous serial stream designed in the 1960s. Nothing existed off shelf to interface to that stream, so we built our own interface. It was basically a couple of shift registers feeding a FIFO interfaced to a PC ISA bus. We had no idea if that interface would even work, this was the first time it had been used with a live system.

We were told that it would be several hours before our instrument was to be powered on and we would see data. In the mean time, we thought we might add a few features to our data capture and display software. It sounded like a good use of time. It didn’t really matter, we recorded everything on DAT tape in real time anyway.

We were given regular updates through phone calls from our systems engineer in the NAGE room. Eventually he called and told us we were about to be powered on. It was absolutely critical for our ground support (GSE) system to record all data during power up. If something went wrong, we could go back through the data and track it down. We sat patiently waiting, waiting, waiting for the bits to come down that cable in the wall.

Finally, it showed up. To our amazement, the instrument powered up normally, went through it’s built in test sequence and delivered good data. Our GSE interface worked flawlessly also. It was like the Eagle landing on the moon! We were jumping for joy along with the systems engineer in the NAGE room.

I thought I would switch screens to look at different telemetry streams. I hit one of the function keys. That was a mistake. The screen froze and the PC no longer responded. The tape drive write LED no longer flashed every half second indicating data was not recording.

Our GSE software was written in C, primarily by yours truly. As any C programmer knows, the C runtime does not have any runtime type safety. Simply put, a bad pointer caused the PC to crash. These were the DOS days, no hardware memory protection. Apparently, this was one of “features” I added while we were waiting in the annex. Things were not good, nothing was being recorded, bits were spilling on the floor. We were in shock. I handed the phone to my colleague who didn’t know what to do either. He hung up the phone without saying anything as we frantically tried to reboot as fast as possible.

Our systems engineer immediately called back, we felt like we were in the control room of Three mile island with the core temperature rising. Our systems engineer was very understanging, he tried to calm us down and said he was not going to yell at us, just to tell him what was going on. Eventully, we rebooted, started recording again and didn’t touch anything until the test was over.

Fortunately for us, nothing went wrong with the instrument. Everything we smoothly, but it was a hard learned lesson.

I like cryptograpy. The math is interesting. More importantly, crytography has become easily accessible to the average developer. The devil is in the details, like file formats. Faced with another potentially time consuming challenge, a simple and elegant solution was buried in the docs.

Back in the early 1990’s, I worked at an engineering firm that developed very sophisticated communications receivers for surveillance applications. The projects were managed by electrical engineers, they ran the show. As the receivers became more advanced, the embedded control software within the receivers increased in size and scope. This led to an interesting conflict between engineers designing the hardware and software developers writing the software.

In general, engineering responsibilities for a given project were broken down along physical lines. For one project I worked on, there were four physical circuit boards in the product:

• an RF board that contained the antenna input, mixers, and filters
• a frequency synthesizer board that generated mixing signals and CPU clocks
• an analog board for demodulation and power conditioning
• a digital board with a high performance floating point signal processor, 32 bit microprocessor, and an Ethernet interface

Four engineers were tasked to that project, each working on one board. I was responsible for the digital board and some of the frequency synthesizer. One of the engineers acted as the lead, making sure all the pieces fit together. The software side was a different story. First they started with two software developers, then three, then five, then as many eight. The engineers could not understand why it took so many software developers when only four engineers were needed for the same project. There were claims by the engineers that the software guys were poorly managed, incompetent, or otherwise did not know what they were doing. The animosity was exacerbated by rapid improvements in hardware sophistication and complexity in each new hardware design. Engineers skillfully managed this complexity, whereas the software developers appeared to flounder.

I spent some time with the software guys. I soon came to realize that they were not at all incompetent, in fact they were quite intelligent. Why did the software require so many more resources and take longer to develop? Part of the problem has to do with the maturity of each discipline. Electrical engineering is more than 100 years old, computer science is on the order of a human generation. The engineering development process is well established, following older, but similar traditions such as mechanical and civil engineering. Civil engineering is thousands of years old, and they still slip schedules. Why would a very young discipline, like software development fare any better?

Engineering disciplines have had time to develop effective processes and standards. Wheels are reinvented, but the sizes are standardized. In software, the few widely adopted standards that exist are often poorly implemented and suffer from fragile interoperability. Automated testing is only now becoming a recommended practice, but still largely overlooked.

Another key difference is the disparity in formal education. You cannot hold the title of engineer in most firms unless you have at least a four year degree in engineering. As a physics major, I was permitted to enter the ranks, but with some explaining. In software, computer science degrees are often a rare commodity. Some even believe that a computer science degree is not practical or even necessary. Formal education takes a back seat to practical experience. In engineering firms, if you didn’t have a degree, the best you could do is be a technician.

Recently, I’ve been working on applications for PocketPC and Palm handheld devices. In both cases, new devices are available in the market every month, but the software and development tools just don’t keep up. On several occasions, a particular device we were planning to deploy on a large scale reached end of life and was discontinued by the time the necessary support software and our applications were ready.

A colleague recently told me that one engineer can keep several software developers busy. Our demand for greater functionality and interoperability will probably continue to fuel that maxim. Software is like the wild west, anything goes and the best shooters win. Unfortunately, it’s hard to achieve high productivity when you don’t have electricity or running water.

There has a been a raging debate lately about static vs. dynamic typing. This is due in large part to the increase in webshare for Ruby and Ruby on Rails, and increase in criticism against J2EE. For the past several months, I have been trying to get to the bottom of the value proposition of each, and determine if one or the other is clearly superior.

One of the first salvos was fired by Bruce Tate, who I interviewed back in October of 2005. A former Java proponent, Bruce moved firmly into the dynamically typed, Ruby camp. I pressed him for specific, objective reasons why he felt dynamic typing was superior. He made some points in terms of expressiveness, but it wasn’t clear to me that static typing was dead.

James Gosling in his blog was flamed relentlessly by dynamic typists for his opinions on the value proposition of static typing. Much of the criticism came when he presented some of the weaknesses of dynamically typed languages. His most compelling argument lay in what he termed the inevitability of scale, meaning that dynamically typed languages ultimately suffer from performance limitations at some point. The comments on his blog became almost religious and partisan. They did little to advance the art or even expand the dialog. Some arm chair computer scientists even questioned Goslings competence. It devolved into a rediculous spectacle.

In my recent interview with Smalltalk enthusiast Ward Cunningham, I asked his opinion. Ward admitted that he once believed that dynamic typing was the only way to go, but with the availability of high quality IDEs like Eclipse, the burden of declaring types is not as great as it once was. After a long conversation, Ward made an even more interesting observation: the question is not whether static or dynamic typing is better, it is more about who you are writing code for. If you are writing code for yourself, it really doesn’t matter what you do or how you do it. If you are writing code for someone else or with a large group, it’s a different story.

Wards observation is another dimension of scale, not of computational performance, but of the audience for code. Based on my research, dynamic typing is far from a perfect solution for all problems. Tragically, little has changed since the introduction of Smalltalk 80. In the end, I have come to the conclusion that static typing won’t go away any time soon.

Anyone familiar with Smalltalk or the work of Xerox PARC between 1970 and 1980 has heard of Alan Kay. Kay has to be one of the greatest minds and greatest contributors to modern technology in our time.

My recent interest in Smalltalk lead me to a video presentation by Alan Kay in 1987. That time seems like the plasticine era, just as Kay remarked about the 1960s. The video, “Doing with Images Makes Symbols”, highlights the developments of early CAD systems, origins of object oriented programming, human interface devices, and culminates with absolutely fantastic developments at Xerox PARC. Kays quips about the state of technology in 1987 underscores the monumental gap between research and commercialization, something that is just as true today.

I strongly encourage anyone with any interest in technology to watch this video. The streaming version is of low quality, so I would suggest downloading the entire 256 kb MPEG4 file and viewing offline. It’s about 120 MB and runs 40 minutes or so.

There is a second part to this video, which is slightly longer, and includes several examples of interdisciplinary learning applied to computer interface design. If you have every wanted to learn to play tennis, this is a quick introduction! Further, interviews with Alan Kay expose key ideas, such as the importance of the aesthetic in almost any field, particularly science and engineering.

After watching this video, I can’t help but wonder if Steve Jobs was heavily influenced by Alan Kay and his ideas. Clearly the aesthetic has been central to the Macintosh and more recently the wildly succesful iPod.

I just ran into this prediction of Smalltalk adoption:

“The latest figures I’ve heard are that Smalltalk has between 15% and 25% of the entire OO market, but is growing at about 60% per year, while C++ is only growing at about 30% per year. If these trends continue (of course, they never *exactly* do), Smalltalk should overtake C++ by the end of the century!” 

This was posted to comp.lang.smalltalk on May 11, 1995. Last time I checked, the 20th century ended, C++ was still going strong and Smalltalk was blindsided by Java. What does that say about future predictions?

Apache web server is awesome! I just added URL rewriting which makes URLs look the way you want. Load balancing is also a nice feature.

All this and it’s free! It’s hard to beat…

If you haven’t guessed, I just installed WordPress. I didn’t time it, but the install could not have taken more than five minutes claimed by the instructions. All I can say is that there are some really, really good developers out there that understand the meaning of quality.

WordPress was recommended to me by Shahid Shah.