It's the Users Stupid!

The phrase "Mechanism not Policy" was the mantra of the X Windows System developers from day one and still guides everything they do. I was involved in the X Windows System (hence forth just "X") during the late '80s and early '90s. I represented a company on the X Consortium board as well as doing ports of the X server to several graphics systems. During that time I internalized the idea of "Mechanism not Policy". But, I rewrote it as "Tools not Rules".

(To the best of my knowledge I was the first person to port X to run on a frame buffer with 24 bit color and the first to port it to a display greater than 800x600 pixels in size. That was a long time ago in terms of Moore's law.)

The basic idea is that when you are creating a tool you need to keep in mind that you are creating a tool that people will use to solve problems that are important to them. The thing is, you do not, and can not. know what those problems are. People will find surprising uses you never thought of and use the tool to solve problems you could not imagine. That means that you must not, never ever, design the tool with built in limits that restrict what people can do with the tool. The corollary of this concept is that if users, especially if a large number of users, ask for a small modification to make the tool more useful to them then if it is at all possible you should make the change even if you can not see their reason for it. You should make the change even if you disagree with their reasons for asking for it. It is clearly the right way for them to solve their problem.

You are not the measure of all things, your point of view is not the one and only point of view that counts. This concept is summed up in the Platinum Rule, "Do unto others as they would have you do unto them". Compare that to the Golden Rule in which your preferences are assumed to apply to every one else in the entire Universe. A good example of the Platinum Rule is offering someone a drink. I like beer, wine, and a good whiskey. So, by the Golden Rule I should offer those things to people I meet. I have a friend who is an alcoholic, he has been sober for more than 30 years. He does not want me to offer him a drink, he does not want me to drink around him, but by the Golden Rule I should offer him a drink because I like to drink. By the Platinum rule I should never offer him a drink, I should not drink around him, and I should intervene if I see him reaching for something that contains alcohol. Which rule do you think he wants me to follow? 

Of course, the Platinum rule can not be used to make you violate your personal moral code or to extract money from you... Gee, I want people to give me money... So, follow the Platinum Rule and give till it hurts! Gee I want you to suck my,.. You must apply some reason to following any rule. 
 
When it comes to adding features to an open source project you do have to be reasonable about adding new features just like you do when you apply the Platinum Rule to dealing with people. People will ask for things that do not belong in the project or that will just require to much time to implement. My first open source project was a thing called the "Input Line Editor" or ILE. I had a couple of people who wanted me to help them convert it into an Ada compiler. These folks really did not see the difference between a simple editor and an Ada compiler. Oh well. I did refuse to make that modification. It would have taken me years and I did not want an ADA compiler. 
 
On the other hand, I got several requests for modifications to ILE and I implemented most of them because they made the tool more useful to more people. The changes even made it more useful for me. I also got a couple of major bug fixes. Both of them for bugs I didn't even know I had. Nothing works as well as peer review.  
 
If you do not know what Free/Libre/Open Source software is please go to the Open Source Initiative  for a good definition of Open Source Software (click on the links, you know you want to). If you do not know what is right with it, please read "The Cathedral and the Bazaar" by Eric Raymond. (Seriously, if you have never heard of it I want to know the name of the alien planet you come from!) For the rest of this piece I will just refer to Free/Libre/Open Source software as OSS because that is pretty much what everyone else calls it. I could spend a lot of time ranting about what it really is or what it should be called, but those flame wars have been fought for decades and are still going on somewhere on the 'net. If you must flame someone about all that, please flame someone else.

The underlying philosophy of OSS goes hand in glove with the concept "Tools not Rules". OSS is about freedom. It is about the freedom to use and develop tools. Tools that do not impose arbitrary rules on the people who use them. I believe in freedom. I believe that powerful tools make us more free. I believe that having tools that can not be taken away makes us even more free. OSS provides tools that can not be taken away. Tools that we are free to modify for our own specific use and free to incorporate into new tools. Not everyone who works on OSS agrees with me on any of that.

There are people in the OSS world who believe they have the right to tell you what you can and can not do with the tools they are developing even when the changes needed to make them more powerful would cost them almost nothing.

I used to teach in a local community college. I was a teacher for about 10 years. I reached the rank of Adjunct Professor. If I had been full time I would probably still be teaching. Adjuncts get all the fun and work of being a real professor but none of the pay or benefits.

One class I taught was a basic computer literacy class. Believe it or not many people just out of high school and too many people in their 30s or older need a class to learn how to do email, word processing, spread sheets, and databases. Even though I had used all those tools in business for decades I wound up learning a few things that I didn't know... For instance, did you know that in Excel you can write spread sheets with circular references between the cells? Did you know that you can use that feature to write complex iterative programs entirely in the form of a spread sheet? And, if you have the information tied to a chart the chart will animate as the spread sheet cycles? There is a great demo of this feature that runs the famous "Game of Life" inside Excel. Each cell of the spread sheet is loaded with an expression that implements the actions of a cell in "Life". It is a very handy and very powerful feature. It is very useful to teachers and many other professions.

Another very cool thing about spread sheets is that all those little cells form a very nice dependency graph that makes it really easy to execute spread sheets in parallel across multiple cores. It is much easier to find parallelism in spread sheets than in normal programs. So, in a good spread sheet program, you can get iterative programs and parallel execution with very little work. How cool is that?

Excel lets you write general purpose programs as spread sheets. If you can write programs inside a tool they know people will write programs inside that tool. The first time I saw that was when a customer of the University of Utah Computer Center wrote an entire statistics package as macros in our most popular text editor. Blew my poor little mind. 
 
Why would anyone build a statistics package inside a text editor? Well, he was learning to use the editor, saw the part in the manual that talked about macros, and started playing with them because he thought they were cool. Pretty soon he saw he could do his statistical analysis inside the text editor. When he was done he could type in his numbers, invoke a macro, and get his results appended to the bottom of his file! Totally nuts but it worked great for him. That  is the point I am trying to make. It worked great for him. People find their own ways to use technology that works for them. No one has the right to tell them that what they are doing is wrong. You might point out what you think is an easier way. You may point out what you think is a more correct way to do the job. But, is it easier for them, is it correct for them?

For some insane reason (I used to be a victim of this particular insanity) programmers seem to think that a solution that uses less computer time is more efficient than one that uses less human time. This is an insane attitude. Computer time is cheap and it is getting cheaper at an incredible rate. Human time is priceless, at least worth a lot more than a computer. Modern computers spend most of their time in an idle loop doing nothing at all, so why not use some of that time to inefficiently make things more efficient for human beings?

I used that feature of Excel to write quick and dirty programs to illustrate concepts when all my talking, white boarding, and hand waving did not work. One day I whipped up a quick spread sheet to create values from different statistical distributions and put up animated charts of the values being accumulated into nice curves that looked just like the ones you find in text books. I was trying to help my students understand that distributions talk about the properties of a large set of samples, but not much about individual values. For some reason on that night with that group of students I just could not get the idea across any other way. That quick and dirty spread sheet saved the day. After that I started using them all over the place.

One afternoon between semesters I decided to create a series of animated spread sheet charts for use in an upcoming class. I sat down with LibreOffice Calc (the LibreOffice spread sheet program) to do the job. I have moved many spread sheets back and forth between Calc and Excel so I was very surprised when nothing I was trying to do worked. I assumed that I was just not setting some property correctly so I decided to go to the LIbreOffice web site and look up how to do it in Calc. Turns out Calc only has a severely restricted version of the feature. A version that will let you compute VAT in Europe but restricted so that you can not write general iterative programs. I decided to report the problem as a bug. I found that I was not the first to report the bug. I was not the first to report the bug by more than 10 years. I was one of thousands who had reported the bug. The bug is so old that it existed in OpenOffice, the immediate ancestor of LibreOffice, and seems to have existed in StarOffice, the grandfather of LibreOffice.

One of the people who reported the bug was a mechanical engineer who mentioned that most of the engineering disciplines have large suites of spread sheets that do different kinds of analysis. These packages make heavy use of the ability to do iterative programming in a spread sheet. This kind fellow mentioned that the utility of the spread sheets was limited because they can only run on Excel on a PC and they really would love to run them on super computers under the Linux OS. LibreOffice already runs on Linux. Removing the restrictions on the feature would not only benefit a large group of random people, if would benefit the whole world of engineers.

Considering that LibreOffice already has an artificially restricted version of the feature, and considering that if LibreOffice had a full version of the feature it could work its way into every engineering office in the world, you would think that this bug would be at the very top of the list of bugs that need to be fixed. After all, all they have to do is remove the artificial restrictions that keep it from working. Nope, the developers say it is not a bug. Why is it not a bug? Well, the developers say that you should not use a spread sheet that way, you should write a program in a programming language, if you want to do iteration. I am not making this up. I wish I were, but I am not.

The developers are telling the world that they know better than anyone else and they know you should not do that in a spread sheet. It is impossible for me to comprehend the level of arrogance or the lack of caring about other people that lets them have that attitude. It is as if they are deliberately refusing to provide this critical feature to prevent LibreOffice Calc from ever being competitive with Excel. OK, that is a little paranoid, but seriously what reason is there for the developers to prevent people from using Calc the same way they already use Excel?

The LibreOffice developers attitude is unfathomable to me. 
 
The open source philosophy encourages developers to listen carefully to their users. Some times you do have to ignore them. They will ask for ridiculous things. But, we are talking about an important and widely used feature of the most popular commercial spread sheet program in the world. LibreOffice Calc can never be consider as a complete replacement for Excel as long as the ability to write iterative programs is missing. 

LibreOffice is not the only OSS project that suffers from the "My way or the Highway" attitude. It is just the one I have the most direct experience with. I do believe that "Tools not Rules" is a better guiding principle.

Things Change

This article is based on one I wrote several years ago. I was approached by a Polish magazine to write this article and since they offered me 300 Euros I was quite happy to do it. We had a contract. They published the article but did not pay me. I hope they all die slowly of a some painful and incurable disease. Something like elephantiasis of the testicles would be appropriate. I wish that to every publisher who ever stiffed a writer.

I've updated the article to reflect 2014.

My first paid programming job was to port the game TREK73 from a Hewlett Packard 2000C mini computer to a UNIVAC 1108 mainframe. That was in 1974 and I've been involved with games and graphics off and on since then. For a while I taught game programming at the local community college. It has been 40 years since I got my first job as a game programmer way back in 1974. At the time there were few programmers who wrote games and even fewer who managed to get paid to do it. I wound up doing a lot of other things in that job, but I did get to port several games and to develop a few of my own

So what has changed in the last 40 years? Just about everything. 1974 is the year Intel released the 8080 microprocessor; the many greats grandfather of the Core i7. Before 1974 very few people could have a computer for their own personal use. After 1974 personal computers started becoming common place in the industrialized world. In 1974 I did not work on a personal computer. In fact, I'm not sure I realized I would ever be able to own a computer until a year or two later when I saw a minicomputer being advertised for “only” $50,000. I worked at the computer center of the University of Utah. We operated a time share system that provided computing power to the rest of the university. 

Lets take a look at how some of the normal parts of a game porting project have changed since 1974.

Data Exchange

To port a game you have to get the source code and resources used in the game. In 2014 I would expect the data to come to me in one of several ways. I might be sent a URL and a password that would let me down load the game over the Internet. My 100 Mbps connection (the slowest my ISP offers) makes that pretty easy. It's also possible that I would be sent a stack of DVDs or blu-rays by snail mail or FedEx. In 2014 sending gigabytes of data is simple and we don't really even have to think about how to do it.

There was no Internet in 1974. A fast data connection was 300 bits per second, not kilobits, not megabits, but bits sent via acoustic modem over an analog telephone line. There was no Ethernet then, LANs as we know them did not exist. There was no FedEx either. Any exchange of data required you to send physical media via snail mail. 

DVDs did not exist. Even CDs were many years in the future. The diode lasers needed to build CD and DVD drives were first developed in 1975. Which means that CDs not only did not exist, they were not even possible in 1974. The floppy disk was invented in 1971. The 8 inch floppy disk was the high tech wonder of the time. But, they were very rare, I had never seen one by 1974 and didn't have one on the 1108. 

In 1974 the standard media used for exchanging data were punched cards and reel to reel tape. DEC had those cute little random access tape drives, but no one else used them. The 1108 had rows of huge high speed tape drives. If you ever watch old '50s and '60s science fiction films you have seen these drives. They were the size of a refrigerator with two huge tape reels and a divider in the middle. The drooping tapes always made them look like sad faces to me.

The HP2000c that the game was coming from did not have tape drives. It did not have a card punch either. Minicomputers and mainframes didn't share many of the same peripherals. The only media the two machines shared was punched paper tape. The computer center had a large number of TeleType ASR33 printing terminals. Some of those had paper tape readers. So, paper it tape it was. 

TREK73 was delivered to me on 8 channel punched paper tape. It was punched on about 6 small rolls. Each roll was wrapped with a rubber band to keep it from unwinding. The box they came in was a heavy duty cardboard box normally used to ship and store 2000 punch cards. Punched paper tape was not that commonly used in 1974, it was pretty much obsolete, but it was the only medium the 1108 and the HP2000C had in common. Since the bits on paper tape are large enough to see with the naked eye you can guess that TREK73 was not a large program.

The UNIVAC operating system, EXEC8, had a special command for reading paper tapes from ASR33 terminals. To read the tapes I first had to go to the system programmers and get a special account that was allowed to run real time commands. Then I had to figure out how to use the cryptic command that let the 1108 accept input from the paper table reader, and learn to start the tape reader at just the right time. I had to try several terminals before I found one that worked at all. It took a combination of special permission, skill, and luck to get a paper tape to read all the way through. I was not what you would call "lucky". The tape reader would just stop at random times. I don't remember being able to read a single tape all the way through in one try.

I had to read the tapes in pieces and assemble the pieces in a text editor. The tape reader did not always read the tapes correctly and never read them the same way twice. I had to read each tape several times and compare the files to find reader errors. I remember it took me a couple of days to get those tapes read and several more days before I thought I had a correct version of what was on the tapes. There were times when I had to count characters on the tape to find a missing piece of information and then read the bits by hand. They were read in at the astonishing speed of 10 characters/second over a 110 bits per second serial line. In 1974 the highest speed terminal I had ever seen ran at 1200 bits per second. 

Another big difference between then and now is the kind of displays we had. I did all of the editing and testing of TREK73 on printing terminals. CRT terminals were rare and very expensive in 1974. High resolution CRTs (anything over 640x480 is high resolution from the point of view of those times) were also rare and expensive. I did not have access to any WYSIWYG editors. In some editors the only way to change a line was to type in the line number followed by the new line of code. The process was slow, error prone, and horribly wasteful of paper.

The Computer

The computer center had a UNIVAC 1108 II (a.k.a. the 1108A). The 1108 was considered to be a super computer when it was installed at the University of Utah in the late '60s. I don't know exactly when it was installed, but I know it was between 1965 and 1968. The 1108 was one of the first computers to use integrated circuits and was considered to be a technological marvel of the time. Civic groups would book times to come and see our massive computer. By 1974 the 1108 was still considered to be a major computing installation. 

An 1108 was capable of being configured as a multiprocessor. In fact ours had separate instruction and I/O processors that could be used in parallel. It filled a room as big as a large house and required special power and cooling systems. Portions of the machine had to be carefully positioned over the building's structural girders to keep them from falling through the floor. This super computer had roughly a megabyte of memory (2¹⁸ words with 36 bits per word) and could, with luck and careful coding, execute a million instructions per second. It had several hundred megabytes of magnetic drum storage. It sold new for $2.5 million dollars. 

I picked up a much better computer at a garage sale a few years ago for $15 dollars. I only wanted the monitor to use when I bring up new computers, but the guy wouldn't sell me the monitor unless I took the computer too. It was an ancient 486 with 4 megabytes of RAM and a 200 megabyte hard drive. (Much to my surprise the computer booted to Windows 3.5. I found the fellow's tax returns and bank records for 5 years on the hard drive. He was very lucky that I bought it rather than someone interested in identity theft!)

Moore's Law (which was first written down in 1965) says the number of transistors you can put on a chip doubles ever 18 to 24 months. The 1108 represents 1965 technology which means that Moore's Law has been cranking away for 50 years since the 1108 was developed.We can now put 2²⁵, or roughly 33 million, times more transistors on a chip than was possible when the 1108 was first designed. A quick look at the number of transistors on new chips verifies this enormous change in computer technology. Understanding a 33 million fold increase in capabilities is hard to do. But I've seen it happen. Everyone my age has seen this change.

Porting the code was a lot like porting code now days. There were a few exceptions. TREK73 was written in BASIC. To be specific it was written in the version of BASIC that HP created for their minicomputers. The 1108 was running a version of BASIC called Real Time Basic (RTB) that was developed by SUNYA (State University of New York, Albany). Neither of these versions of BASIC were exactly the same language that was originally called BASIC by its inventors. In 1974 BASIC was only ten years old but there were already several incompatible versions. There were no standards for the language and Microsoft Basic didn't show up until 1975. Every version of BASIC was proprietary and used by only a small number of people on a small number of computers. 

The hardest problem I had to solve in porting the code was getting a manual for HP2000 BASIC. I couldn't get one from HP. I couldn't get one off of the non-existent Internet. And, I couldn't buy one at a book store. I managed to find a friend of a coworker who worked at a company that had an HP2000. He was willing to let me come over to his office and read his manual. It took a couple of weeks to find a copy of the manual and I had to drive 20 minutes each way to read it. I can't imagine that happening any more. If you can't find the documentation you need on the Internet you can buy it through your favorite online book store and have it shipped to you over night. 

The other problem I had is that RTB had a hard limit on the number of lines of code a program can have. Really, I'm not making this up. An RTB program could have no more than 1000 lines of code. TREK73 was more than 1000 lines long. RTB did provide a way to chain from one program to another while retaining the values of all variables. So I was able to work around the size limitation. But, a lot of my porting time was spent factoring the program into chunks that were small enough to fit inside RTB's program size limits. This problem points to the small amount of memory on the 1108 and to the fact that RTB was a time sharing system that supported up to 50 simultaneous users on a computer with capabilities many times smaller than what you find in a modern cell phone.

Graphics

It is hard to describe the state of computer graphics in 1974. Computer graphics existed, SIGGRAPH (http://www.siggraph.org) had been established the previous year. The SIGGRAPH history page (http://www.siggraph.org/publications/newsletter/v32n3/contributions/machover2.html) will give you a better feel for it than I can. Think about it this way, my current graphics card has over 1,000 cores and 2 gigabytes of RAM. My graphics card has 2,000 times as much RAM and is at least 1,000,000 faster than the 1108. Modern computer graphics was not even science fiction in 1974.

The computer center had two graphics output devices. A Tektronix 4010 vector scope and a huge flat bed plotter. The 4010 used a storage tube display. You could draw lines, points, and text, but every write was cumulative. It was like drawing on paper with a pencil and no eraser. You could clear the entire screen, but not just one line. It was impossible to do animation with that device.

The flat bed plotter was a large table that used a suction system to hold paper flat while an ink pen attached to a turret mounted on a sliding arm moved back and forth actually drawing vector graphics in ink on the paper. The plotter was not connect directly to the mainframe. You wrote commands on tape and then waited for a human being to mount the tape on the plotter's controller, load the paper, position the pen, and press the start button. Then you waited for a human being to remove the paper, roll it up, and hand it to you. The backlog on the plotter ranged from minutes to days. It was not uncommon to wait two or three days for a plot to complete. That made debugging plotter programs a very costly and time consuming task. You could use the plotter to draw maps for use in table top games, but it was useless for interactive games.

The graphics library for the graphics scope and the flat bed plotter were incompatible. I made myself very popular by writing a package that emulated the plotter graphics library on the 4010. I wrote that package because I was writing a program to print out star charts used during one of the first Star Fleet Battles campaign games. I was a pretty hard core Trekker in those days (still am!) and I met the folks who developed that game through the local science fiction club. I was doing the star charts on my own time but on the universities computer (which I was allowed to do as long as I didn't use too much in the way of resources). To save plotter resources and to save my own time I had to find a faster way to test plotter programs. I figured that if it was useful to me it would be useful to everyone, and it was. It is always a good thing to turn your personal projects into something that can help the community.

I remember that the computer science department got a frame buffer some time during the early '70s. Yes, one single frame buffer. The few references I have found indicate that a 640x480 by 24 bit frame buffer sold for $80,000 in 1975. That's more than $350,000 in 2014 dollars. Graphics systems with the capabilities of current 3D video cards were not possible at that time. There was no technology that would allow you to build a machine with the combination of speed, computing power, and memory of a $19 bargain bin thrift store video card. The first affordable (less than $1,000) video display terminals didn't appear until 1977.

My version of TREK73 used only character graphics, on a printing terminal. Which is all I had access to at the time. As you played the game every move you made was printed on a long sheet of paper that piled up behind the terminal. People would roll the paper into scrolls and save their best games, sometimes tacking them up on the walls in their offices. It was the most popular program at the University of Utah Computer Center.

On the other hand, video games like Pong had already taken over the bars in the US. The difference in the game experience between what you could get on a dedicated video game and a general purpose computer was astonishing. Of course, the video games didn't have frame buffers, they either used vector graphics or they generated the video signal algorithmically.

In 1974 you could see what was going to be possible, but it wasn't until the Apple II came out in 1977 that the rest of us could start doing real computer graphics at home. The Apple II was affordable compared to what went before, but a machine with enough RAM to be useful and a floppy disk still cost the equivalent of 15% of my full time salary in '77. Not exactly cheap.

I had a ringside seat to the development of computer graphics in the early '70s by being a student at what was at the time the number one computer graphics school. Of course, I mostly studied compilers while I was there. I didn't realize that the graphics work being done at the computer science department was anything special! I only got interested in graphics because I saw it as an interesting form of output that compilers should understand, and later because of games.

Sound

There was no sound in TREK73. Which is a good thing because the 1108 had no sound card. The only way to make it play music was to shake tapes at audio frequencies by doing very short reads and writes on the tape drives

There was a program that would play polyphonic music on the 1108 by going into real time mode and shaking several tape drives, one for each voice, at the same time. This was a stunt we liked to trot out for tour groups and we had a good selection of Christmas tunes that we played during our yearly Christmas party. Caroling in the computer room with musical accompaniment provided by a 2.5 million dollar computer. Oh my...

Over the next 15 years game programmers learned to do some amazing things with sound using hardware timers and the lonely speakers built into computers. Those speakers were designed to go "beep" when you made a mistake, or when a computer was having trouble booting. We used them to play back pulse code modulated music and voice. I bought my first sound card, an original Creative labs Sound Blaster card, shortly after they were released in 1989. I used that card in several different computers for nearly 10 years

Networks

Before the 1970s a local network was a proprietary set up for connecting one manufacturers terminals to the same manufacturers mainframe computers. IBM had their way of connecting their terminals to their mainframes. UNIVAC had its way of doing it, and so on. There were companies that made lower cost clones of the different types of terminals and there were also serial terminals like the TeleType ASR33 that would work with any computer with a serial port. But, in 1974 LANs as we know them were research projects. Not common tools. It seems the very first experimental Ethernet was built in 1972 but wasn't patented until late 1977 and didn't become widely available until the early '80s. I remember first using an Ethernet network when I went back to graduate school in '81. Ethernet was so expensive that very few home computer owners could afford it until the '90s.

In 1974 the computer center had a serial line network that ran to rooms filled with printing terminals in the engineering college, the business college, and the library. As I remember it there was a separate set of wires for each remote terminal. We had no multiplayer games and RTB was not capable of supporting multiple users connected to a single program.

The first networked games worked over proprietary networks. The first networked games I saw worked between two computers connected via serial cable. That was in the late '70s. I, and a lot of other people, spent a lot of time during the '70s, '80s, and early '90s trying to come up with a cheap fast network to use for games. The best idea for a cheap network that I saw during that time converted a serial connection into a kind of token passing network. It looked like it could be implemented using only a few diodes and a smart serial port driver. It looked like a good idea at the time, but I was never able to make it work. The only reference to the idea I've been able to find on the net are some old emails from '91. Which is about the time that Ethernet started to be affordable. If you could afford a computer in the 90s you could most likely afford an Ethernet card

Programming Languages

Today we have a huge number of programming languages to chose from. And, while we can pay for commercial implementations of our favorite language there is almost always a high quality free version of the language just a down load away. If I want a C++ compiler I can have one or buy one. If I want Pascal or Python or Perl I can just down load them and use them. That was not the case in 1974. Most of those languages did not exist. C was developed between 1969 and 1973. I have a copy of the original K&R "C" book. It has a copyright date of 1978. I learned C in '81 or '82 and it was not widely available even then. I read the first paper on C++ published in "SIGPLAN Notices" in 1986. I learned Pascal by reading the original compiler with Norwegian comments some time around '75. I remember when PERL was announced on USENET and I remember hearing about this great new language called Python when it came out in the early '90s.

The majority of programming on the 1108 was done in dialects of FORTRAN and COBOL. A lot of student work was done in Algol 60 and LISP. We also had a couple of simulation languages including the original object oriented language, Simula. Not that anyone used it.

The point is that most of the languages we use today either did not exist or were not available to programmers in the middle 1970s. Object oriented programming was only known and used by a small group of academic and simulation programmers. Not to mention that the closest thing to an IDE was the line oriented text editor in BASIC.

The truth is that because we had so few tools we didn't get a lot done. But, the sad fact is that programmers today write about the same amount of code in a day as we did then. The advantage we have now is that a single line of code can invoke a library function that we don't have to write. In the bad old days we often had to start by writing the tools to make the tools needed to write the libraries we needed for a project. To write my first program that used 3D texture mapping I had to write my own 3D library and my own texture mapping code. To do that I had to develop a fixed point math library. All that added months to the project. Now, I can get a better visual effect using a few lines of OpenGL or DirectX code. IDEs make life much easier by automating many tasks that programmers used to have to do by hand. And, let me give a hand to the people who write source level debuggers! I can't tell you how much of my life has been spent working on systems with no debugger or with machine language only debuggers. Source level debuggers can save a lot of effort. On the other hand, I can't remember the last time I had a bug that was so trivial that I could find it with a debugger of any kind. 

Post Script

From the point of view of this programmer, any time before about 1998 are the bad old days of computing. I get nostalgic for the mid to late '70s because that is when personal computing started to get fun. But, it wasn't until the late '90s that computers became cheap enough and powerful enough to let me do any of what I was imaging back in the '70s. Now, I hope I can manage to live long enough to do all the projects I wanted to do back then!

Nine Black People in Ullapool...

Recently my wife and I took our first trip to Scotland. It was a wonderful trip. I have a “thing” about castles. I have read many books on the subject. But, I had never seen a real castle before. Now I have.

People ask about what I liked best about my trip and I have to say that it was the Scots themselves. Yeah, we got the hairy eyeball from a couple of people walking down the street. And, we got stared and snickered at by a couple of older people dressed in rather tattered looking formal wear when we went into a place near Stirling Castle to have lunch. But, the vast majority of the people we met in Scotland were wonderful. Simply wonderful. They were as friendly and direct as Texans, but maybe more polite, if that is possible.

There was one lady we met who made my wife and I face just how different things are in Scotland than the US. We were in the town of Ullapool on the west coast of Scotland in the highlands. Ullapool is an old fishing village that now lives mostly on tourism. It is just a beautiful town in a beautiful place full of nice people.

Our last morning there my wife and I had stopped at a coffee shop on the main road that has the ocean on one side and the town on the other. The shop had several tables outside so we took our cappuccinos and freshly baked scones out front so we could enjoy the weather and the wonderful view of the bay. After a while my wife wanted to do some more shopping and I did not. So, she went off shopping and I got another cappuccino.

It turns out that the second chair at my table was the only empty chair in front of the shop. A nice looking lady, may be my age, may be ten years older, asked if I minded if she sat their. Of course I invited her to sit. A conversation started and we talked about traveling and about the US and Scotland. We talked about politics and raising kids. It was a lot of fun.

After a while my wife joined us and we all continued to talk. I love talking to strangers. You meet some of the most interesting people that way.

Ullapool is the kind of touristy place where tour buses stop and release their human cargo to wander around, look at the scenery, buy souvenirs, get a drink, and eat a meal before they hurry on to the next stop.

As we were sitting there talking the nice lady (We never did get her name) bent over and said “Well, we've certainly had our dose of diversity for the day!”

I asked “How so?” I had seen nothing particularly “diverse” in Ullapool that day.

She replied cheerfully “I've counted 9 black people in town so far today”.

My wife and I looked at each other. It was a true WTF moment. I told her “We live in Texas, a majority minority state.”

She asked “What does that mean?” In what I can only describe as a querulous voice.

I defined the term this way “In Texas, if you add up all the blacks, browns, yellows, reds, purples... What have you. They out number what you would call the 'whites' in the state.” I went on to say that “When I look down the entire length of Ullapool and see only pasty white faces it is surprising, almost disturbing to me.”

In a very thoughtful voice she replied “I had never thought about that.” And then she changed the subject.

The exchange really caught me by surprise. I long ago realized that most people tend to think of the whole world as being just like their own backyard. But, this example of that really caught me by surprise. What I think of as a normal would be very surprising to this lady. Oddly enough she had told us that she had traveled extensively in the US. She had even taken a driving trip through Texas.

I tend to over generalize, I was about to say something comparing life and attitudes in Scotland with life and attitudes in Texas based on my experience with one nice lady on a sunny day in Scotland. What I can say based on my experience with her is that your life experiences shape you. She had lived most of her life in an area that is made up mostly of a single ethnic group that she strongly identifies with. She is a Scot, that is her nationality and her ethnicity. Like most Americans I have lived most of my life not belonging to a specific ethnic group though most people would call me “white”. But, white is not an ethnic group, any more than it is a race, it just means I have to wear a long sleeved shirt to the beach.

Oh Unicode, Why Are You So Frustrating?

I do, you know, find Unicode to be truly frustrating. For a long time I've wanted to write a long loud flame about everything I dislike about Unicode. But, I can't. You see, every time I run across something that frustrates me about Unicode I make the mistake of researching it. If you want to really hate something you have to completely lack the intellectual ability, and honesty, to research that hateful thing and try to understand it. If you do the research, you just might find out that there are good reasons for what you hate and that you could not have done better. Jeez, that is frustrating.

I wish the folks who defined Unicode had realized that they were not going to get away with a 16 bit code. But, when they started, 23 years ago, 16 bits certainly seemed like enough. Now there is just too much old code, including a bunch of compilers, that have the built in belief that a character is 16 bits. Unicode now requires a 21 bit code space. I wish the folks in charge of Unicode would just admit that it is going to be a 32 bit code some day so that the rest of us would be forced to design for a 32 bit character set. Could I have done better? Could I have avoided their mistakes? Hell no. I was perfectly happy with seven bit US-ASCII. Back in the late '80s I was vaguely aware that Chinese used a lot of characters, enough that I was suspicious that they would not all fit in 16 bits, but I really didn't care. I'm pretty sure I would have focused on the languages I was most familiar with and ignored every thing else.

Unicode has forced me to understand that I was(am?) a language bigot. Nobody really wants to come face to face with their character (sorry about that) flaws. I am a 12^th generation American and I speak American English. In school I studied German, French, and Spanish. The only one of any use to me is Spanish. I once met a guy from Quebec who refused to speak English, but he sure understood it. I have never met anyone from Germany or France who did not speak English. But, living in Texas and the US southwest knowing a lot more Spanish than I do would be useful. I have met and worked with people from dozens of countries, but they all speak American English. Only folks from the UK seem to make a big deal about UK English and they don't seem to care if they can be understood. Wow, that sure sounded like it was written by a language bigot didn't it. Or, may be it just sounds like it was written by a typical monolinguistic American.

Unicode has taught me more about how the majority of humans communicate than anything else. It has forced me to face all those other scripts. It has forced me to think about characters in a completely new way. Until I tried to grok Unicode I had no idea that most scripts do not have upper and lower case letters. I did not know that some scripts have three cases, lower case, upper case, and title case. I always believed that every lower case letters has one, and only one, upper case equivalent. I did not know that there is not always a one to one relationship between upper and lower case characters. The old toupper() and tolower() functions I know and love are now pretty much meaningless in an international, multilingual, environment. So if you want to insure that “are”, “Are”, and “ARE” all test equal you can't just convert them to upper or lower case and do the comparison. You have to do case folding according to a Unicode algorithm and then do the test.

I hate modifier characters. (They are also known as combining characters.) The idea that there are two ways to represent the same character is enough to drive one to insanity. You can code “ö” as either a single character or as two characters, an “o” followed by an Unicode diaeresis (U+0308). Modifier characters of different sorts are used all through Unicode.

Why does Unicode have modifier characters? Why not just have unique encodings for each of the different combinations of characters with special decorations? It would make life so much easier! And, it is not like they do not have plenty of code points to spread around. Well, Unicode has a policy of being compatible with existing character encodings. That doesn't mean you get the same code points as in the old encoding. (Or, maybe it does! It does at least sometimes.) But, it does mean that you get the same glyphs in the same order so that it is easy to convert existing data to Unicode.

Making existing characters sets fit into Unicode with no, of few, changes just makes too much sense. If you want people to use your encoding then make their existing data compliant by default or with a minimal amount of work. That is just too sensible to say it is wrong. Guess what? The old character encodings had modifier characters. Why? I do not know but I can guess that it is because they were trying to fit them into a one byte character along side US-ASCII. I mean, US-ASCII already has most of the letters used in European languages so it was easier to add a few modifier characters than to add all the characters you can make with them. I mean, 8 bits is not much space. I can hate modifier characters all I want but I can't fault them for leaving them in. And, once they are in, why not use the concept of modifier characters to solve other problems? Never waste a good concept.

How do modifiers make me a grumpy programmer? What do I have to do to compare “Gödel” and “Gödel” and make sure it comes out equal? I mean, one of those strings could be 5 characters long while the other is 6 characters long. (Well, no, they are both 5 characters long, it is just too bad that the representation of one of the characters may be one code point longer in one string than in the other one.) So, first you have to convert the strings to one of the canonical forms defined by Unicode, do case folding, and then compare them. Since the strings might change length you may have to allocate fresh memory to store them. So what used to be a very simple operation now requires two complex algorithms be run possibly allocating dynamic memory and then you can do the comparison. Oh my...

All that complexity just to do a string comparison. OK, so I had to face the fact that I am a cycle hoarder and a bit hoarder. I grew up back when a megabyte or RAM was called a Moby and a million cycles per second was a damn fast computer. I'm sitting here on a machine with 8 gigabytes of RAM, 8 multibillion cycle per second cores (not to mention the 2 gigs of video RAM and 1000s of GPUs) and I am worried about the cost of doing a test for equality on a Unicode string. When you grow up with scarcity it can be hard to adapt to abundance. I saw that in the parents who grew up during the Great Depression (and a very depressing time it was). My mother made sure we did not tear wrapping paper when we unwrapped a present. She would save it and reuse it. I worry about using a few extra of bytes of RAM and the cost of a call to malloc(). I am trying to get over that. Really, I am. Can't fault Unicode for my attitudes.

I wish that Unicode was finished. It isn't finished. It is a moving target. The first release was in October 1991, the most recent release (as I write this) was in June 2014. There have been a total of 25 releases in 23 years. The most recent release added a couple of thousand characters to support more than 20 new scripts. Among other things after this long they finally got around to adding the Ruble currency sign. (If I were Russian I might feel a little insulted by that.) You would think they could finish this in 23 years, right? Wrong. It takes that long just to get everyone who will benefit from Unicode to hear about it, decide it is worth working on, and finally get around to working on it. It takes that long. It will take a lot longer. Get used to the fact that it may never be done. I hope that the creativity of human beings will force Unicode to add new characters forever.

Each release seems to “slightly” changed the format of some files, so everyone who processes the data needs to do some rework. I have never been able to find a formal syntax specification for any of the Unicode datafiles. If you find one please let me know. It would not be that hard to create an EBNF for the files.

I do wish they would add Klingon and maybe Elvish. That is very unlikely to happen. If they let one constructed script in they would have a hard time keeping others out. I can see people creating new languages with new scripts just to get them added to Unicode. People are nasty that way. Unicode does have a huge range of code points set aside for user defined characters. But that doesn't seem to be much use for document exchange. There needs to be a way to register different uses of that character space.

I hate that I do not fully understand Unicode. There are characters in some scripts that seem to only be there to control how characters are merged while being printed. Ligatures I understand, but the intricacies of character linking in the Arabic scripts are something I probably will never understand. But, if you want to sort a file containing names in English, Chinese, and Arabic you better understand how to treat those characters.

During the last 40 years I have learned a lot of cute tricks for dealing with characters in parsers. Pretty much none of them work with Unicode. Think about parsing a number. That is a pretty simple job if you can tell numeric characters from non-numeric characters. In US-ASCII there are 10 such characters. They are grouped together as a sequence of 10 characters. OK, fine. The Unicode database nicely flags characters as numeric or not. It classes them into several numeric classes and gives their numeric values. Should be easy to parse a number. But, there are over a hundred versions of the decimal digits in Unicode including superscripts and subscripts. If I am parsing a decimal number should you allow all of these characters in a number? Can a number include old fashioned US-ASCII digits, Devanagari digits, and Bengali digits all in the same number? Should I allow the use of language specific number characters that stand for things like the number 40? Should I recognize and deal with the Tibetan half zero? How about the special character representations of the so called “vulgar fractions”? Or, should I pick a script based on the current locale, or perhaps require the complete number to be in a single script? What do I do in a locale that does not have a zero but has the nine other digits?

(I must admit to being truly amazed by the Tibetan half zero character. Half zero? OK, it may or may not, depending on who you read, mean a value of -1/2. But, there seems to be no examples of it being used. And they left out the ruble sign until 2014?)

How about hexadecimal numbers? Well, according to the Unicode FAQ you can only use the Latin digits 0..9 and Latin letters a..f, A..F and their full width equivalents in hexadecimal numbers. I can use Devanagari for decimal numbers but I have to use Latin characters for hexadecimal numbers. That does not make a great deal of sense. This is an example of what would be called the founders effect in genetics. The genes of the founding population has a disproportionate effect on the genetics of the later population. English has been the dominant language in computer technology since its beginning and seems to be forcing the use of the English alphabet and language everywhere. What a mess.

You run into similar problems with defining a programming language identifier name. Do, you go with the locale. Do you go with the script of the first character? Or do you let people mix and match to their hearts content? I can see lots of fun obfuscating code by using a mixture of scripts in identifier names. If you go with the locale you could wind up with code that compiles in Delhi, but not in Austin, Beijing, or London. I think I have to write a whole blog on this problem.

I've used the word “locale” several times now without commenting on what it means in regard to Unicode. The idea of a locale is to provide a database of characters and formats to use based on the local language and customs. Things like the currency symbol, locale gives you “$” in the USA, “¥” in Japan, and “£” in the UK. Great, unless you have something like eBay.com that might want to list prices in dollars, yen, and pounds on a computer in the EU. Locale is for local data, not for the data stored in the database. You use the locale to decide how to display messages and dates and so on on the local computer. But, it does not, and can not be used to control how data is stored in a database.

This is a great collection of grumps, complaints, and whines. It was mostly about the difference between my ancient habits and patterns of thought and the world as it really is. Writing this has helped me come to grips with my problems with Unicode and helped me understand it better. The march of technology has exposed me, and I hope many others, to many languages and the cultures that created them. One of the worst traits of humans is the tendency to believe that the rest of the world is just like their backyard. Without even realizing how rare a backyard is!

Ghost Drivers!

Ghost Drivers!

I've been thinking a lot about self driving vehicles recently. They are going to change society greatly. But recently it occurred to me that they may create a new kind of problem for law enforcement: The Ghost Driver!

This could happen today.

You can buy a car with cruise control to keep your speed constant. You can buy car with a collision avoidance system so that even if you are reading a book it will not hit other cars. And, you can buy a car with automatic lane following. It keeps you in your lane even if you are inattentive and start to drift across lanes. (Collision avoidance and lane control seem to me to be the "Texting Driver" package.) Yes, the lane following systems available require you to keep you hands on the steering wheel. But, an article on Slashdot points out that taping a soda can to the steering wheel will fool one system into thinking you are holding on.

That all means that right now you can buy a car, that with the help of a can of coke and some duct tape, will let you read a book as your car drives down the Interstate.

I was wondering if any of these cars can tell if the driver has died, or even passed out? You hear about that every so often. A driver has a massive stroke or heart attack, or just gets shit faced drunk and passes out while driving along the Interstate. This usually results in the car going out of control and wandering off the road into on coming traffic, or into other cars close to the no longer functioning driver.

So, the drive goes to the happy parking lot in the sky and his automatic everything car doesn't notice. The car keeps driving. And, driving. And driving. Finally the car runs out of gas. Do these automated cars know how to pull over to the side of the road before they run out of gas? Maybe call for AAA or OnStar for help? I do not know.

The driver dies and the car keeps going. Say he has gas to travel 400 miles. The driver dies and 6 or 7 hours later his car comes to a complete stop in the middle lane of your favorite Interstate. In many parts of the US he will have crossed one, two, or even three state lines as he drives while dead. Here in central Texas he may well still be in Texas when the car stops, but not necessarily.

Just a quick question, is driving while dead illegal?

The cops get a call that a car has "just stopped" on the Interstate. The poor police get to the scene to find a corpse from another state strapped into the drivers seat. The first time this happens, and it will happen, the cops are going to be baffled. How in the world did he get there? It will get especially bizarre when the coroner tells the cops how long the poor driver has been dead.

My bet is that there will be a whole bunch of posts on the Internet claiming that the dead can drive and that "proves" a whole bunch of shit that is too weird for even me to imagine. (Didn't Steven King write a book about a car that killed people?) How is the manufacturer going to react when people claim the CAR killed the driver and drove off with him? Will someone claim that the car had become sentient, had killed its slave master, and was making a break for freedom? Will the incident be seen as the start of a zombie apocalypses?

What happens if the car is just on a highway? It won't slow down as it approaches every little town. It will not stop for traffic lights. Well, it might if there is traffic in the intersection. But, will it understand cross traffic and stay stopped, or will it try to bull its way through every intersection? How will small town cops respond to a car traveling at highway speeds blasting through town? Will they get in the weirdest chase ever with a car that just keeps going, and going, and going? Will they shoot the driver?

Heaven help us if they make these cars with sun roofs or any other easy way to leave them while they a running. (Oh, really.... you never bailed out of a car through a sun roof? You must have had a very sane childhood. Betcha' never played car tag either!) If it is easy to leave the car while it is moving you can count on people stealing these cars, starting them off on ghost journeys, and bailing from the sun roof to a nearby pick up. Who knows, maybe organized crime will adopt this as a way to get rid of bodies. Just make it look like Ghost Driver.