Nov 6, 2014

It's the Users Stupid!

The phrase "Mechanism not Policy" was the mantra of the X Windows System developers from day one and still guides everything they do. I was involved in the X Windows System (hence forth just "X") during the late '80s and early '90s. I represented a company on the X Consortium board as well as doing ports of the X server to several graphics systems. During that time I internalized the idea of "Mechanism not Policy". But, I rewrote it as "Tools not Rules".

(To the best of my knowledge I was the first person to port X to run on a frame buffer with 24 bit color and the first to port it to a display greater than 800x600 pixels in size. That was a long time ago in terms of Moore's law.)

The basic idea is that when you are creating a tool you need to keep in mind that you are creating a tool that people will use to solve problems that are important to them. The thing is, you do not, and can not. know what those problems are. People will find surprising uses you never thought of and use the tool to solve problems you could not imagine. That means that you must not, never ever, design the tool with built in limits that restrict what people can do with the tool. The corollary of this concept is that if users, especially if a large number of users, ask for a small modification to make the tool more useful to them then if it is at all possible you should make the change even if you can not see their reason for it. You should make the change even if you disagree with their reasons for asking for it. It is clearly the right way for them to solve their problem.


You are not the measure of all things, your point of view is not the one and only point of view that counts. This concept is summed up in the Platinum Rule, "Do unto others as they would have you do unto them". Compare that to the Golden Rule in which your preferences are assumed to apply to every one else in the entire Universe. A good example of the Platinum Rule is offering someone a drink. I like beer, wine, and a good whiskey. So, by the Golden Rule I should offer those things to people I meet. I have a friend who is an alcoholic, he has been sober for more than 30 years. He does not want me to offer him a drink, he does not want me to drink around him, but by the Golden Rule I should offer him a drink because I like to drink. By the Platinum rule I should never offer him a drink, I should not drink around him, and I should intervene if I see him reaching for something that contains alcohol. Which rule do you think he wants me to follow? 

Of course, the Platinum rule can not be used to make you violate your personal moral code or to extract money from you... Gee, I want people to give me money... So, follow the Platinum Rule and give till it hurts! Gee I want you to suck my,.. You must apply some reason to following any rule. 
 
When it comes to adding features to an open source project you do have to be reasonable about adding new features just like you do when you apply the Platinum Rule to dealing with people. People will ask for things that do not belong in the project or that will just require to much time to implement. My first open source project was a thing called the "Input Line Editor" or ILE. I had a couple of people who wanted me to help them convert it into an Ada compiler. These folks really did not see the difference between a simple editor and an Ada compiler. Oh well. I did refuse to make that modification. It would have taken me years and I did not want an ADA compiler. 
 
On the other hand, I got several requests for modifications to ILE and I implemented most of them because they made the tool more useful to more people. The changes even made it more useful for me. I also got a couple of major bug fixes. Both of them for bugs I didn't even know I had. Nothing works as well as peer review.  
 
If you do not know what Free/Libre/Open Source software is please go to the Open Source Initiative  for a good definition of Open Source Software (click on the links, you know you want to). If you do not know what is right with it, please read "The Cathedral and the Bazaar" by Eric Raymond. (Seriously, if you have never heard of it I want to know the name of the alien planet you come from!) For the rest of this piece I will just refer to Free/Libre/Open Source software as OSS because that is pretty much what everyone else calls it. I could spend a lot of time ranting about what it really is or what it should be called, but those flame wars have been fought for decades and are still going on somewhere on the 'net. If you must flame someone about all that, please flame someone else.

The underlying philosophy of OSS goes hand in glove with the concept "Tools not Rules". OSS is about freedom. It is about the freedom to use and develop tools. Tools that do not impose arbitrary rules on the people who use them. I believe in freedom. I believe that powerful tools make us more free. I believe that having tools that can not be taken away makes us even more free. OSS provides tools that can not be taken away. Tools that we are free to modify for our own specific use and free to incorporate into new tools. Not everyone who works on OSS agrees with me on any of that.

There are people in the OSS world who believe they have the right to tell you what you can and can not do with the tools they are developing even when the changes needed to make them more powerful would cost them almost nothing.

I used to teach in a local community college. I was a teacher for about 10 years. I reached the rank of Adjunct Professor. If I had been full time I would probably still be teaching. Adjuncts get all the fun and work of being a real professor but none of the pay or benefits.

One class I taught was a basic computer literacy class. Believe it or not many people just out of high school and too many people in their 30s or older need a class to learn how to do email, word processing, spread sheets, and databases. Even though I had used all those tools in business for decades I wound up learning a few things that I didn't know... For instance, did you know that in Excel you can write spread sheets with circular references between the cells? Did you know that you can use that feature to write complex iterative programs entirely in the form of a spread sheet? And, if you have the information tied to a chart the chart will animate as the spread sheet cycles? There is a great demo of this feature that runs the famous "Game of Life" inside Excel. Each cell of the spread sheet is loaded with an expression that implements the actions of a cell in "Life". It is a very handy and very powerful feature. It is very useful to teachers and many other professions.

Another very cool thing about spread sheets is that all those little cells form a very nice dependency graph that makes it really easy to execute spread sheets in parallel across multiple cores. It is much easier to find parallelism in spread sheets than in normal programs. So, in a good spread sheet program, you can get iterative programs and parallel execution with very little work. How cool is that?

Excel lets you write general purpose programs as spread sheets. If you can write programs inside a tool they know people will write programs inside that tool. The first time I saw that was when a customer of the University of Utah Computer Center wrote an entire statistics package as macros in our most popular text editor. Blew my poor little mind. 
 
Why would anyone build a statistics package inside a text editor? Well, he was learning to use the editor, saw the part in the manual that talked about macros, and started playing with them because he thought they were cool. Pretty soon he saw he could do his statistical analysis inside the text editor. When he was done he could type in his numbers, invoke a macro, and get his results appended to the bottom of his file! Totally nuts but it worked great for him. That  is the point I am trying to make. It worked great for him. People find their own ways to use technology that works for them. No one has the right to tell them that what they are doing is wrong. You might point out what you think is an easier way. You may point out what you think is a more correct way to do the job. But, is it easier for them, is it correct for them?

For some insane reason (I used to be a victim of this particular insanity) programmers seem to think that a solution that uses less computer time is more efficient than one that uses less human time. This is an insane attitude. Computer time is cheap and it is getting cheaper at an incredible rate. Human time is priceless, at least worth a lot more than a computer. Modern computers spend most of their time in an idle loop doing nothing at all, so why not use some of that time to inefficiently make things more efficient for human beings?

I used that feature of Excel to write quick and dirty programs to illustrate concepts when all my talking, white boarding, and hand waving did not work. One day I whipped up a quick spread sheet to create values from different statistical distributions and put up animated charts of the values being accumulated into nice curves that looked just like the ones you find in text books. I was trying to help my students understand that distributions talk about the properties of a large set of samples, but not much about individual values. For some reason on that night with that group of students I just could not get the idea across any other way. That quick and dirty spread sheet saved the day. After that I started using them all over the place.

One afternoon between semesters I decided to create a series of animated spread sheet charts for use in an upcoming class. I sat down with LibreOffice Calc (the LibreOffice spread sheet program) to do the job. I have moved many spread sheets back and forth between Calc and Excel so I was very surprised when nothing I was trying to do worked. I assumed that I was just not setting some property correctly so I decided to go to the LIbreOffice web site and look up how to do it in Calc. Turns out Calc only has a severely restricted version of the feature. A version that will let you compute VAT in Europe but restricted so that you can not write general iterative programs. I decided to report the problem as a bug. I found that I was not the first to report the bug. I was not the first to report the bug by more than 10 years. I was one of thousands who had reported the bug. The bug is so old that it existed in OpenOffice, the immediate ancestor of LibreOffice, and seems to have existed in StarOffice, the grandfather of LibreOffice.

One of the people who reported the bug was a mechanical engineer who mentioned that most of the engineering disciplines have large suites of spread sheets that do different kinds of analysis. These packages make heavy use of the ability to do iterative programming in a spread sheet. This kind fellow mentioned that the utility of the spread sheets was limited because they can only run on Excel on a PC and they really would love to run them on super computers under the Linux OS. LibreOffice already runs on Linux. Removing the restrictions on the feature would not only benefit a large group of random people, if would benefit the whole world of engineers.

Considering that LibreOffice already has an artificially restricted version of the feature, and considering that if LibreOffice had a full version of the feature it could work its way into every engineering office in the world, you would think that this bug would be at the very top of the list of bugs that need to be fixed. After all, all they have to do is remove the artificial restrictions that keep it from working. Nope, the developers say it is not a bug. Why is it not a bug? Well, the developers say that you should not use a spread sheet that way, you should write a program in a programming language, if you want to do iteration. I am not making this up. I wish I were, but I am not.

The developers are telling the world that they know better than anyone else and they know you should not do that in a spread sheet. It is impossible for me to comprehend the level of arrogance or the lack of caring about other people that lets them have that attitude. It is as if they are deliberately refusing to provide this critical feature to prevent LibreOffice Calc from ever being competitive with Excel. OK, that is a little paranoid, but seriously what reason is there for the developers to prevent people from using Calc the same way they already use Excel?

The LibreOffice developers attitude is unfathomable to me. 
 
The open source philosophy encourages developers to listen carefully to their users. Some times you do have to ignore them. They will ask for ridiculous things. But, we are talking about an important and widely used feature of the most popular commercial spread sheet program in the world. LibreOffice Calc can never be consider as a complete replacement for Excel as long as the ability to write iterative programs is missing. 

LibreOffice is not the only OSS project that suffers from the "My way or the Highway" attitude. It is just the one I have the most direct experience with. I do believe that "Tools not Rules" is a better guiding principle.




8 comments:

  1. Hello,

    Interesting article and although I get and mostly agree with the concept of it, let me add this to your train of thoughts.

    As I am involved with a big application suite (100.000 euro annual license fee) on the Windows platform, I also have the misfortune to have to work with Excel.

    Single user that works with (relatively) limited data-sets, Excel fits the bill. Most people that work with Excel expect it to work everywhere in whatever form. But when you want to use in background processes, that is extremely unwise to do.

    One of our customers has a system with 64 xeon processors, 128GByte RAM and 16 very fast hard drives connected to it. And it works with really big data-sets that are used to generate even bigger data-sets to be able to predict futures use.

    The software engineers and dba's at that customer were positively impressed with the scaling capability of our software. Marketing and consultants...not so much. Their systems would break under the strain and we were told to look into that problem.

    Those people expected that using Excel in copious amounts of business processes would make their analyses speed up. After all, a single instance of Excel is quite capable, so running a lot of them at the same time, that must by analysis heaven.

    Unfortunately, running ore than 30 instances of Excel at the same time would halt that 64 processor computer. So I took a look and on my single core, 2GByte RAM workstation I could manage to run 20 Excel instances at the same time, making Windows extremely slow. To give you an impression of slow, it took over an hour for Windows had enough time and resources to do a normal reboot.

    The difference between the number of Excel instances on two very different types of computers is not that great. Excel has a lot of functionality, but it eats threads and desktop heap memory like there is no tomorrow. In Windows those two resources are limited in supply and you will experience all kinds of surprising side-effects.

    So I do understand the people of Libre Office, saying that they won't lift the artificial limits in Calc. While your (valid) perception translates this to 'My way or the highway', I just wanted to add that limits are there for a reason. Computational resources are more limited than you think or even expect and maybe these programmers want to protect you from yourself.

    After all, just because you can, doesn't mean you should.

    ReplyDelete
    Replies
    1. You clearly did not get the point of my article at all.

      Delete
  2. It seems that web version (O365) does not not support circular references between the cells. So perhaps Microsoft, too, thinks that it shouldn't be supported nowadays.

    ReplyDelete
  3. You have a good point here! I totally agree with what you have said!! Thanks for sharing your views. hope more people will read this article!!
    토토사이트
    경마
    oncasinositenet1

    ReplyDelete
  4. Great! I enjoyed all the things... Thanks for sharing... You know that you can apply for an Indian emergency visa in case of an emergency, such as an unforeseen event, the death of a blood relative, or another valid emergency.

    ReplyDelete
  5. Great content that provides quality knowledge with interesting facts is what I love to read. Travelers planning to travel to Turkey. They should be aware of the Turkey visa requirement to avoid inconvenience during the visa process.

    ReplyDelete
  6. The article you wrote has many valid points, despite the fact that I have read it many times. Your readers will benefit greatly from this content. Travelers can apply for a Turkey evisa which is very easy. You can apply online from any corner of the world. You only need an internet connection and valid documents.

    ReplyDelete