Nov 5, 2015

Who Watches the Watchers? Policing the Police and everyone else.

Quis custodiet ipsos custodes? Who watches the watchers? A question that goes back at least to a Roman poet named Juvenal who died roughly 2,000 years ago. It would seem that abuses by those with power have been a problem for a very long time.... You might say that it is a basic human flaw.

The recent series of highly publicized murders carried out by police brought the question to mind quite forcefully. Then, a few days ago I heard an interview on KUT, the local public radio station. The interview was on the subject of sex crimes committed by Police. It seems to work like this:

  • See a woman alone in a car. 
  • Pull her over for whatever, they can always come up with some reason to pull you over. 
  • Tell the woman to spread her legs, open her mouth, or go to jail. 
  • If she is not quick enough hold a gun to her head.
The Huffington Post has a great article on the subject with many examples and data that is better than my memory of an interview on the radio. BTW, Ted Bundy used to pretend to be a cop so he could pull women over and rape and murder them. If it is good enough for a serial killer it should be good enough for a cop, right? (I met Ted Bundy once at a party when he was studying law at the University of Utah.)

If enough women complain about this behavior the cop might get fired. They do not usually get charged they just move on to the next town where they start all over again. Sounds just like what happens to pedophiliac Catholic Priests. Hard to believe that cop rapists and priest rapists get treated the same way. Their powerful organizations cover up for them and let them go on to rape again.

The interesting thing is that we know about murderous cops because people with smart phones record them and post the videos to the Internet. Once on the Internet the cops can not get them back. They can try to suppress them but they can't. The power of the individual with a video camera and an Internet connections is at least equal to, if not greater than, the power of the police to suppress. But, you don't commit rape when there are other people around. So, not so many videos of cops raping women (and men... I should not leave out that they also rape men) make it on the Internet.

The answer to the 2,000 year old question is that *we* watch the watchers. I've already seen video of a police officer attacking a woman who was recording a police beat down. He didn't figure that someone else was also recording what was going on. We need as many cameras out there as we can get. And, they need to be under our control. Police dash cam and body cam recordings are just to easy to lose.

This answer became obvious when the beat down of Rodney King was videotaped and the tape was sent to a local television station. The Internet really wasn't ready for the job back then, but it is now. We must record what is going on and we must make sure that everyone sees what really happened.

So, I would like to suggest a little citizen action that we can take to do even a better job of watching the watchers. I would like someone or some group of someones to create a little high tech bump. What we need is a bump you can stick on a car, or a door, or anywhere. What I want is a bump with the ability to keep a recording of at least the last 24 hours of video and sound. The device should have a wifi interface and maybe USB connections. It should be battery powered with a solar cell to recharge it. It would be nice if it recorded in the infrared and ultraviolet as well as the normal visual spectrum. It must be cheap. I know I can put together something like that starting with an Arduino or Raspberry Pi and off the shelf add on boards. But, it would be better, and no doubt cheaper, to build it as a single unit. One critical feature is that the only way to erase the stored recording is to destroy the device. (Cast the whole thing in a solid block of epoxy?) But anyone can download the current stored recording.

The idea is that I can have these on my car and one stuck on the side of my house and so can you. They can be put most anywhere and if you put them where they don't belong you should go to jail. These things would create a fair witness. An unbiased recording of any thing that happens. It is important that anyone can download the recording but no one can destroy or modify the recording without making it obvious that they did so. The wifi connection would also let these things be monitored in real time.

My little bump sounds like on hell of an invasion of privacy. And it will be if used incorrectly. On the other hand they would make it a lot safer for people when they are walking down the street or when they are pulled over by a cop. If there is always a witness with a video recorder there is very little chance of getting away with a crime. Every programmer knows that you are more careful about your coding if you know other people are going to look at it. Knowing that you might be being watched just might help everyone, including cops, behave a little bit better.

I have known many cops. Every cop I have known has been a great guy. I have been helped out by cops I did not know and will never see again. I hate what is happening right now. I hate it more because it seems it has always gone on and cops have used their power to cover it up. I hate that this could happen. But it has, and it does, and something must be done to fix the problem. If you want to see the cockroaches you have to turn on a light. Video cameras and the Internet are the most powerful light ever made.

While most cops are great people, there is something about image of cops that attracts bullies the same way the job of Catholic Priest seems to attract pedophiles. I wouldn't be surprised if the job of surgeon attracts sadists. Some jobs just attract certain kinds of people. Have you ever met an extroverted programmer? You have? Really? Are you sure? Ever met a corporate executive that you suspected was a sociopath, a psychopath? I hope you get my point.

OK, all that feel good stuff aside, we need to use technology to create a fair witness system that is not controlled by police or the government. It needs to be a citizen action for the protection of our selves. The goal here is not to undermine the police. The goal is to make it possible to trust them again. Do you want to see the day when it is considered self defense to shoot a cop just because the cop pulled you over where you would be alone and it was after dark? That could happen if we do not do something to reestablish trust between the police and citizens. OK, in large parts of the country we need to establish trust between the police and the citizens for the first time.

I hate to admit it, but right now the idea of being pulled over by a cop scares me. I never thought that could happen. I want everyone to be able to trust a cop all the time and I want cops to be able to trust citizens as much as is possible.

Oct 30, 2015

$866,304.61 per GigaByte

Recently I was looking for drives for a new DVR I was building. (The old one based on a 10 year old PC I inherited from one of my kids died of a fatal thunderstorm.) I looked around on the net and wound up just buying a 4TB disk at Fry's (the place where old men go to be ignored...). It was made by Seagate and cost $160. No big deal, right?

Buying that drive made me think of the first hard drive I ever owned. It happened to be made by the same company under a different name and I bought it in the early '80s. It was an ST-506. ST stood for Shugart Technology, Inc. The company was renamed Seagate Technology to avoid a law suit over the name. The ST-506 was the first hard drive intended for use with personal computers. I should mention that at the time a personal computer was most likely a Z80 with 64K of RAM. IBM had not gotten into the PC business in 1980 and a personal computer was any old computer, not what we would not call a PC.

When first introduced in 1980 the ST-506 stored 5 MegaBytes. That's right, MEGA, with an M, not GigaBytes or TeraBytes, MegaBytes. It stored 5,000,000 bytes. My new drive stores 4,000,000,000,000 bytes. Did I pound the difference in capacity into your head? The new drive stores close to a million times as much as my first hard drive.

Think of it this way. I paid $0.04 for a gigabyte of disk space a couple of weeks ago. 35 years ago I would have had to buy 200 ST-506 drives to make up a gigabyte of storage. I do not remember what I paid for that drive, it was part of a bundle... And, it was a long time ago. But, when it was first released the ST-506 cost $1,500 dollars. So, what would a GigaByte cost if using ST-506 drives? Let's see... $1,500 * 200 = $300,000 dollars. Of course, those are 1980 dollars. Taking into account 35 years of inflation and the cost in 2015 dollars is $866,304.61. That is a lot of money.

Four cents versus $866K is a... large, change in price.

It hurts my poor little mind to realize that I thought a 5 MegaByte disk was freaking HUGE back then. Today I am wishing I could make a case for building a redundant RAID out of 4 TeraByte drives.  I like RAID's, RAIDs are cool. But I just don't need that much storage right now.

We have just passed "Back to the Future" day and I was watching a documentary about that amazing movie. The folks making the documentary asked why the original film makers hadn't used CGI. The answer was that back in 1980 there was no CGI because there were no computers capable of doing CGI back then. Think about it. An ST-506 could not store even one frame of 1920x1080 HD video. Let me say that again for emphasis: an ST-506 could not store even one frame of HD video. No personal computer from that era could be used for video production work. Things we do on phones were not possible on high end workstations of that era.

The 35 years from 1980 to 2015 do not seem like a long time in a human life. It was enough time to raise a couple of kids. Pay off the mortgage on a house. Raise a dog from puppyhood to old age a couple of times over. It was enough time for the price of desktop hard drives to drop by a factor of 10 and for their capacity to increase by a factor of 1,000,000. A short time for a human. A long long time in the history of technology.

I can't wait to see what's next.

Aug 14, 2015

My Favorite Bugs: Java Memory Leak

This is a story of how I managed to create the largest memory leak I have ever seen, in fact, the largest memory leak I have ever heard of. I did it using Java, a garbage collected programming language. You are not supposed to be able to have memory leaks in garbage collected languages.

This story is not in anyway a slam at Java. I really like Java. It is a story about unintended consequences and how what you do not know can bite you right on the ass.

I know a lot about garbage collectors and garbage collected languages. I have implemented Lisp several times. I have implemented it as an interpreter and as a compiler. That means I have written several garbage collectors. I have actually written at least a half dozen garbage collectors of various sorts. I have read at least 100 papers and several books on garbage collection and at least that many on storage management in general. Not counting garbage collectors I have written... Ok, I've lost count... general heap management systems using many different methods including both relocating handle based systems and pointer based systems.

I know a little bit about memory management. I do not claim to be an expert. In the bad old days you just often had to write your own memory managers to make it possible to write the application you that you were actually working on. Anyway, I was so very glad to find myself in a situation where I was able to use a garbage collected language. So very very glad not to have to write yet another memory manager, not to have to worry about memory management at all. And, that was my first mistake.

It takes a few mistakes to create a memory leak in a garbage collected language.

I was writing a web spider. I was working on a research project that required a web spider and at the same time was supposed to be evaluating Java. It was actually new when I was doing this work. I.E. it was a while ago. So, I decided to write a web spider in Java to kill multiple birds with one stone.

So, I did just that. I wrote a multithreaded web spider in Java. In the end it was very handy for collecting the statistics I was looking for and it showed me some statistics I was not expecting. What did I find? Well I found that the distribution of vocabulary used on the web varies significantly depending on where you start your web spider. I found that (at the time) there was a porn site within three links of any page on Of course (at the time) was a porn site. People just can't seem to avoid typing ".com" when they mean ".gov".

I also found out that using as a starting point for testing a web spider is a bad idea. I was working for a large phone company at the time and it seems that upper management was not pleased when they got a telephone call from the real live White House, the Official Residence of POTUS asking the telco to cut it out. They were OK with me hitting them at night, but not during working hours. That taught me to salt the search queue with many starting URLs so that the initial start up activity would be spread all over the net and not dumped on one poor little old web server. It also taught me that paying taxes is not a justification for beating up the White House web server.

I also found out that every URL I hit was being recorded by the Telephone Company Police (yes, really). I found out about this little detail when the TCP showed up in my bosses office with a list of the tens of thousands (yes, really) porn sites I had hit from my PC. After she started breathing again she explained what was going on and the TCP left without dragging me, and possibly her, out to the parking lot and firing me on the spot. (I am talking about the company that fired Dilbert...) The fact is that if you start a web spider you will find porn sites.

It turned out the TCP were really more interested in how I was managing to hit thousands of porn sites per minute without downloading any pictures. They found the pattern to be ... odd.

OK, so I wrote and used a web spider and had some interesting experiences and actually accomplished several tasks that I was getting paid to do. But, there is still the story about the giant memory leak to tell.

This all started with the problem of parsing web pages. To write a web spider you have to examine a web page and find all the HTTP type URLs that are in it. You store the URLs in a queue if you want to do a breadth first search or a stack if you want to do a depth first search. You can also sort the URLs by first assigning them a weight based on some measure you apply to them. You also need a list of URLs you have already visited so that you do not visit them again. The web, and webs in general, have lots and lots of cycles.

(Oh yeah, if you want a really useful web spider you should never do what those pesky robot.txt files tell you to do. Really they do get in the way. It can be fun to down load them and analyze them though. They give you clues for finding really interesting stuff.)

I thought that the way to find the URLs was to parse the HTML and look in all the places you are supposed to have URLs. I wrote a parser that, like most parsers, treated the source document as a stream. That should work, right? It turns out that very few web pages that I found were syntactically correct HTML. Tags were misspelled or just plain missing. If the tags were there their contents were not what they were supposed to be. It seems the syntax of HTML is not defined by the standard documents but by what web browsers will accept. Web browsers are very flexible about what they accept as valid HTML.

I also found out that most of the URLs, especially the ones people do not want you to find, are hidden inside chunks of Javascript. You can't find those by parsing HTML. If I were writing it today I would definitely scan all the CSS for URLs. I've found some interesting stuff in CSS with out a web spider.

My first attempt at parsing web pages was a total bust. The technique that worked was to down load the entire web page as a single chunk of text and apply pattern matching to it. So I did just that. I down loaded each page and stored it in a Java string and searched for patterns. For the first version of the working spider I did everything in memory. Since everything was being done in memory I was not surprised when the spider ran out of memory. That turned out to be a mistake. I should have been surprised.

So, what was taking up memory? Well I had a queue for storing URLs that were waiting to be scanned. The values in that queue were actually pairs of URLs. I stored the source URL of the page  along with the new URL. I did that so that if the new URL didn't work the spider would log both URLs so I could go back and learn something new about how to hide URLs. There was also a hash that I used to keep track of URLs that had been scanned so that the spider would not get caught in loops and rescan pages multiple times. Then there were the structures I used to store the statistics I was interested in.

OK, so I expected to run out of memory and It did so I assumed that I needed to reduce memory use. I did not suspect a memory leak. What to do? Well, I built a file based queue. That was slower than RAM but it moved all that information out to a disk. Sure enough, I still ran out of RAM. Moving the queue to a file had no effect. Time to move the hash, right?

I had some old C code for B-Tree files so I converted that to Java. (Later I converted the same code to C++. Good code never dies...) That code let me substitute a B-Tree file for the hash that I had been using to keep track of previously visited URLs. Putting all that stuff on disk also let me stop a scan and restart it at a later time.

The spider was still running out of memory. I learned how to monitor memory usage in a Java program. I learned how to dig through RAM in a Java program. I dumped all sorts of information as the spider ran around the web. I noticed something. The program still ran out of RAM, but the rate at which it used RAM declined over time. Really weird. I now suspected a memory leak.

I finally figured it out. I also just found out that what caught me was fixed in 2013 in version Java 1.7.0_06. BTW, that is only about 15 years after I reported the problem to Sun. The problem was with the original implementation of strings in Java and my lack of understanding of that design. If I had understood the design I would not have had a problem so I can hardly put all the blame on Java. What you don't know can kill you.

I believe that the original version of Java strings was designed to both save memory and avoid memory fragmentation. Strings are hell for garbage collectors and all other storage managers because they tend to have random sizes. Most data types have a single fixed size. If you create one and delete it then create another one you are usually going to find a free block exactly the size you need. But, not so with strings. Strings tend to be small and random sized. Just finding a piece of memory the right size can be time consuming. But, worse, you wind up with lots of small random sized blobs of memory scattered all through your heap. These little nasty blobs can take up a lot of space but not be large enough to be useful. So, the original designers of Java tried to avoid all those problems. Can't blame them. And, they did a pretty good job too. I have a lot of respect for Gosling.

When I download those pages and stored them in a single string I was creating a large string. I "assumed" that when I was done with the string it would be garbage collected. But, the strings were not being garbage collected. In the original version of Java a string has a pointer to the text of the string, plus an offset to the starting position of the string within the text and a length. This representation lets you take substrings of a string with out copying any part of the text of the original string. What happens is that the substring has a pointer to the original text of the string, but the offset indicates the starting byte of the substring in the original text and the length is the length of the substring. Allowing substrings to share the text of the original string helps avoid all the problems with memory management I described above. The trouble is that I was keeping substrings around in memory as part of my statistical information. Keeping a substring around pinned the entire original string into memory.

I down loaded a page into a single string. I then scanned for words and looked them up in my "dictionary". If the word was already in the dictionary I incremented its count by one. If not, I added the word to the dictionary with a count of one. That means that every time I added a word to my dictionary I pinned the entire source page in RAM. Even though I had no reference to the original text, the substring did and so the page could not be garbage collected.

The first pages I down loaded had lots of new words in them because the dictionary started out empty. As time went on fewer and fewer new words were found. Many pages had no new words. If a page had no new words is would be garbage collected. But, every time I found a new word, and I eventually found tens of thousands of words, an entire page was being pinned in memory.

Once I understood the problem, the solution was easy. Just create an actual copy of any substring I wanted to keep around. Doing that removed all those pesky invisible references and suddenly I could let the spider run for weeks without running out of RAM.

So what is the moral of our story?
  • Do not make assumptions about anything. 
  • Do not use any language feature if you do not understand how it is implemented.
  • You must understand the implications of the implementation.
Those are good ideas for programmers using any language. For language implementers and designers the moral should be "all the use cases you can imagine are a tiny subset of the use cases that exist." Or, as Shakespeare said so well:
There are more things in heaven and earth, Horatio,
Than are dreamt of in your philosophy.
Another way to say that would be:
  • Do not make assumptions about how programmers will use your language.
  • Features should work the way programmers believe they work.
  • The distance between clever and stupid is very short. 
I guess the sum total of my morals is "Do not make assumptions". The question is, are you wise enough to realize you are making an assumption? In the case of this bug I was not that wise. That is why it is one of my favorite bugs. Fortunately it only cost me about a weeks work.

Aug 6, 2015

Rainbow Time

This isn't one of my usual grumpy stories. This morning I had rainbow time. It was wonderful.

When my children were young Mother's Day became a really problem for me. My two kids always made some kind of craft, usually at school, for Mom. But, they also wanted to get her something special. They wanted to get her something beautiful, and artistic, something Mom would love and remember forever. On the other hand, taking two active inquisitive children into stores that sell beautiful artistic things usually means taking two active inquisitive children into stores that have many fragile things that little hands should not touch. One parent, two kids, the math just doesn't work. You can't watch them both at the same time.

But I found a solution. My wife loves Swarovski style cut lead glass crystals.  And, every child I have ever met loves rainbows. Those prismatic crystals are wonderful rainbow makers.

There is a swarovski store near where we live and one of its nicest features is that everything in the store is either out of the reach of small children or in locked cases. These stores are actually child proof. I could take my kids in there and know that even if I got distracted my children could do no damage. They were good kids. Well behaved. But, they were children.

The trouble with giving someone a beautiful prismatic lead glass crystal is that the crystals usually wind up in a box in a drawer where they may be looked at only once every few years, if they are looked at at all. Or, they wind up in a curio cabinet where they become part of the wall paper and are never looked at at all.

I found a solution to that too. When I was a little kid, almost eight years old, I saw the move Pollyanna. That movie had a surprisingly large effect on my life.

Back then, in the late '50s and early '60s, my parents would dress the three of us, me and my two sisters, in our PJs and load us in the back of the car with our pillows and some blankets. Mom always made sure to tuck in a blanket over the whole back seat so that we wouldn't sweat and stick to it. Then we would head out to the drive in movie theater. The idea was that my parents would watch the double feature and that we would fall asleep sometime during the first film of the night. It usually worked very well, but sometimes I would manage to stay awake and watch the whole first film. But I always fell asleep in the back seat of the car curled up with my sisters like a pile of sleeping kittens. I have a few memories of awaking in my father's arms as he carried me from the car to my bed and tucked me in.

In the movie, Pollyanna notices that the prismatic decorations hanging on a huge chandelier cast "rainbows" on the wall when sunlight hits them just right. She and her guardian then take all the rainbow makers and hangs them in a window. As the sun sets the light streams through the window and casts what seen like hundreds of rainbows on the walls of the room. I saw the same effect in the old Hansen Planetarium in Salt Lake City. It had beveled glass windows and just as the sun set over the Great Salt Lake the second floor gallery would light up with dozens of rainbows.

Those Mother's Day crystals are wonderful rainbow makers. We put cup hooks at the tops of our living room and kitchen windows and hung the crystals where, once in a while, the light through the window casts dozens of dancing rainbows across the whole first floor of my home.

Rainbow time is the time when sunlight manages to come through the windows at just the right angle to get past the trees and clouds and be diffracted by the crystals into dancing rainbows. It doesn't happen every day. It only happens when the sun rises in the right position and there are no clouds or mist to block the direct rays of sunshine. But when it happens it is a amazing and wonderful.

When my children were small we would play a game where we tried to catch a rainbow. We would put our hands in just the right place to catch a rainbow in our palms. Then we would follow the rainbow as far as we could toward the crystals and toward the walls.

Over the decades many of the crystals have wound up in a display case. But there are two left in the arch of the living room window. This morning I had rainbow time and I caught a rainbow in the palm of my hand and on my big toe. My children are grown now and I'm a grandfather. But, I still get to play catch the rainbow and remember when I was the father of two young children. I tried my best to fill their world with wonder.

I believe that my fascination with prisms, inspire by that movie, is part of what drove me to become so curious and so fascinated by science. I saw light going in one side. I saw the rainbow coming out the other side. I was desperate to understand how and why that could happen. I now know the standard explanation, and I also know that at the bottommost level we do not know why the Universe works that way. There are so many unanswered questions. I hope we never run out of questions and people who are willing to spend their lives answering one more level of "Why?"

Jun 1, 2015

Questioning Questions

If a spammer and a politician managed to have a child the result could be a loaded question. Questions are as explosive as an IED. They are as slippery as goose shit in a canebrake. They are as misleading as a dating profile. Questions are the best way to tell a convincing lie that I know of.

Let's start off with a simple question:

Why is granite gray?

When you read that question what was your first reaction? Give yourself a brownie point if the first thing you thought was along the lines of "is granite gray?" Give yourself two brownie points if you thought something like "all granite is not gray". You get no brownie points if you immediately started to try to figure out why granite is gray.

All granite is not gray. I live in Texas. I live not far from the Capitol of Texas which is made of granite. The Texas state Capitol building is pink.... Yes, really, the Capitol of Texas and many other state buildings are made of a pink granite found in Texas. 

Granite is not always gray. Some granite is gray, some is white, some is pink. There are many different colors of granite.

So, if you didn't get any brownie points what happened? Well, maybe you just thought that granite is gray. But, maybe it is because the question contains a lie and you fell for it. If you take the question apart it contains the words "granite is gray" not exactly in that order. But, the meaning is clear. I could have phrased the question like this "Granite is gray, why?" Same words, same meaning, different word order.

Why are questions so good at hiding lies? Maybe it is because we have been taught by years of being questioned in school to accept that questions can be trusted. After all, our teachers are not supposed to lie to us.  Maybe questions are so good at hiding lies because our minds skip right past analyzing the question and jump right into answering the question. It doesn't really matter why questions make such effective lies. It matters that we know what they can do and learn to defend against them.

(I run into questions every day that are lies. They are often being asked, and answered, on TV news shows. Makes for a lot of laughs if you are watching for them.)

I'm sure, well at least I hope, that y'all would never be fooled by a question of the form "Hey Joe, are you still beating your wife?" Those words "are you still" and words like them seem like a lead in to a statment of fact. After all, they assert that it is a fact that in the past you did what ever comes next in the question. So the question "Hey Joe, do you still go fishing every Sunday?" has exactly the same form as the previous question, the one that says you beat your wife. I bet Joe would much rather you tell everyone in earshot that he goes fishing on Sunday than that he beats his wife. 

In both the question about granite and the question about wife beating the invalid questions have the same structure as a valid question. But my two examples are lies. 

The most dangerous thing about false questions isn't that they may be directed at you, but that you might just overhear the question and take the lie it contains as truth. After all, you learned a lot in school by listening to the teacher ask questions. And, you have heard millions (maybe more, maybe less) perfectly valid questions in your life. We trust questions to be valid.

Questions are an easy way to slip a lie past an uncritical mind.

I hope it is obvious that I am not the first person to notice this fact about questions. I suspect I am not even the billionth person to notice this problem. Zen Buddhism contains an entire system for training people not to be confused by tricks of language. The system is call koan study. Two koans are so well known in the US that they are treated as jokes; they have even been covered on the Simpsons. I do not feel I'm giving anything away by talking about the famous pair:
  1. What is the sound of one hand clapping?
  2. If a tree falls in the woods and no one hears it, does it make a sound?
I love asking a classroom full of students the one hand clapping question. The responses fall into several distinct groups. One group yells out "it is some kind of swishing noise". (If you can tell me where that answer comes from I would love to know.) The second group have seen the Simpsons episode where Bart answers the question by clapping his fingers into the palm of his hand. That group will stand there and try to make a clapping sound with their fingers and palms. The third group starts making a clapping motion with one hand. That group gets a very puzzled look on their faces. The last group gets a very hard expression, that group is obviously angry. That group calls the administration and complains that I am wasting their time. That group assumes they know what they need to learn, but they rarely seem to learn very much.

The "one hand clapping" question is just like the question "why is granite gray". It starts out by telling you a lie dressed up as a fact. It says that one hand can clap. It does that by asking you what a one handed clap sounds like. Since one hand clapping makes a sound, the koan tells you that it does, you must be able to figure out what that sound is. Right? No. There is a slight problem, a subtle lie has been slipped in. You see, it takes two hands to make a clap. A one handed clap is not a real thing.

That koan comes from an old Chinese saying, a saying I already said:

It takes two hands to make a clap.

This koan is so bad it isn't even wrong. It gets you to accept an idea that is absurd. But, at least half of an average class will sit there and wave their hands back and forth trying to hear the sound of one hand clapping. They have been completely taken in by the lie in the question. The rest of the class has accepted one of several popular, but false, answers to the question. 

Why were students willing to accept a false answer to a question? Well, the first thing to consider is that all their answers sound sort of reasonable so why not accept them? But, the question makes no sense. How can a false answer seem reasonable? If you accept the question as being a real question that pretty much destroys any mental defense against accepting a ridiculous answer as true. The problem is that most of us deep down inside accept the idea that every question must have an answer. To put it another way:

The existence of a question implies the existence of an answer.

Folks, that is simply not true. But as far as I can tell we all pretty much live by that lie. The truth is that the answer may not have been found yet. Or, the question may simply be invalid. It is much better to realize that an answer does not yet exist or that a question is not a question at all than it is to believe a false answer to any question. But, we do it all the time. 

The existence of a question does not imply the existence of an answer.

I would hate to have to admit how long it took me to figure out that koan. So, I won't.

The next koan on the list is the famous "tree falling in the woods" question. This is much more subtle than the "one hand clap" question. It can be claimed that it is not a lie at all. But, it is not what it appears to be. For a detailed discussion of the question look it up in wikipedia

This koan has the appearance of a simple yes or no question. But it isn't that at all. This question explores the difference between physics and perception and even the basis of reality. Even in its simplest interpretation the answer depends on your definition of the word "sound". Do not ever expect another person to share your exact definition of any word or phrase. You can't even claim that we both see the same color we both call blue. Our retinas are different.

If you believe the physical definition of sound you were told some time in grade school you will say that "Of course it makes a sound". A tree falling down must cause sound waves to be generated and therefore the answer is an unequivocal yes.

If you are of a more philosophical point of view, perhaps you are a psychologist, a neuroscientist, or even a poet, you may say that sound is the mental experience of perceiving those physical waves in the air. To you sound is the way you experience the physical, not the physical itself. In that case the answer is an unequivocal no.

Let's take a jump off the deep end for a moment and assume that you are a friend of Schrodinger and love his cat. Well, in that case you have to ask if there was no observer has the tree actually fallen? Is it really in a superposition of states in which it exists in all possible states between standing and falling? We can't actually say it has fallen until someone goes and looks. If we don't know if it has fallen then no matter what our favorite definition of the word "sound" we have to say that we can not answer the question.

This seemingly simple yes/not question seems to have a lot of answers. My experience is that most of the yes/no questions I've been asked had more than two answers. The possible answers include "yes", "no", "maybe", "I don't know", "I don't care", and "would you mind defining that word?". 

The "I don't know" answer actually breaks down into subcategories:

  1. I don't know but I believe I can find out.
    This question can be solved by a trip to a library or a quick google search.
  2. I don't know and nobody seems to know but with enough work we might be able to find out.
    This is the OMG I found a topic for my Ph.D thesis type of question. If it is good enough you might make a whole career out of this one question.
  3. I don't know but it has been proven that no one can answer that kind of question.
    These are the proving a negative, Russell's tea pot, kind of questions.
I suspect I have fallen short of listing all the possible answers by at least a few thousand, if not more.

I think that the most dangerous thing about invalid yes/no questions is that they are a form of brain washing. Once you have answered "yes" to such a question you have made up your mind. You have committed yourself to the idea that the one and only true answer to the question is "yes" and you stop looking for or considering any information that contradicts your simple answer. The same thing happens if we say "no". Once you have picked an answer it becomes very hard to ever change your answer or to even consider anything that contradicts your point of view. Answering an invalid yes/no question is like shutting the doors of your mind. (Of course, this doesn't apply to valid questions like "you want a beer?".)

It seems that we humans have a deep flaw that makes us uncomfortable with unanswered questions. That flaw makes us vulnerable to truly awful answers just so we can have an answer, any answer. We can be lead to believe almost anything by the use of carefully constructed invalid questions. An invalid question is like dividing by zero in a mathematical proof. Once you sneak in a division by zero you can easily prove that 1 = 2. Invalid, bullshit, questions can be used to convince people of all sorts of crazy ideas. 

May 6, 2015

Pharmacists or quiet killers?

Recently a very bad thing has started happening in rural areas of the US. There has been an upsurge in the number of new cases of HIV and Hepatitis C. HIV is usually spread by sexual contact. Hepatitis C is usually spread by sharing needles between drug users. The odd thing about this newest outbreak is that both diseases are being spread by sharing dirty needles.

I have read that there has been a huge increase in the use of heroin in the rural US. I have also read that until recently the drug of choice in the rural US has been meth. Meth is pretty easy to make, cheap, and gives a good high. Out in the boonies where everyone has an old shed that is a long way from neighbors or major roads, cooking meth is easy to do without getting caught.

Because it has horrible health effects, no one in their right mind would want to use meth. And, authorities have been cracking down on it in many different ways. I have pretty nasty sinus problems so I know how hard it is to get pseudoephedrine at a pharmacy these days in Texas. You have to ask for it over the counter, show ID, your ID is logged, and then you can over pay for pseudoephedrine.  Used to be that people cooking meth would just steal pseudoephedrine off the shelves and go off to do their dirty deed. Much harder to get the basic materials. Hey! Making it hard to get the materials makes it hard to make the stuff.

I have read and seen TV reports that say that most of the meth in Texas now comes across the border. And, of course, the availability has gone down and the price has gone up. Capitalism works!

On the other hand the war in Afghanistan seems to have had an unintended consequence. The poppy fields are alive and doing very well. No one is burning them down or killing the farmers. I'm sure the farmers are very happy about that. But, the result appears to be that the supply of heroin is up and the price is down. Hey capitalism really works!

So, if you live in the rural US and your choice is between expensive meth and new reduced price heroin which are you going to chose? The answer is "what ever you can get". That is what it means to be an addict. You need to get whatever you can get. I have a few old friends who are recovering addicts of different types. Many of my old highschool friends are dead from drug overdoses. I have experience dealing with people close to me who became addicts. Some died. I was lucky to learn from their mistakes.

I can remember the late '60s and early '70s which means I did not experience them. Too many dead friends. Drugs scare the shit out of me. Doctors who do not share my experiences scare the shit out of me. I can't believe some of the addictive crap that doctors have tried to get me to take. My experience living through that time and going to school next to an open air drug market taught me way to much and has strongly shaped my opinions.

The increase in the use of heroin in rural areas of the US has directly resulted in the spread of HIV and Hepatitis C from sharing dirty needles.

I am not a lawyer, but I can read and what I have read has lead me to believe that in most states of the US it is legal to walk into a pharmacy and buy a clean needle and syringe. So why would anyone anywhere share a dirty needle? Good question. A question in which I have a great personal interest.

To say the least I am not a well man. I have several potentially fatal conditions. I to not expect to die from any of them. While they are not curable they are treatable. Due to a wonderful thing that gets called ObamaCare (The Affordable Care Act) I still have health insurance and can actually afford the treatments needed to keep me alive.

As a result of my medically necessary treatments I go through three to five needles per day. A couple of hundred needles is almost a two month supply. I like to buy two or three hundred needles at a time. Most places will not sell more than two hundred at a time because it depletes their stock. But, if I call ahead I can usually get full boxes at a case lot discount. Such a deal.

I live in Texas. I have talked to my doctors about getting needles and syringes. I was surprised to find out that in Texas you do not need a prescription to buy them. It is legal to buy them over the counter in any quantity at any pharmacy in Texas. I recently carried about 50 needles of several different varieties out of the US into the UK and returned with a smaller number of them. They went through customs both ways with not even a comment. Buying, owning, and using needles and syringes is pretty much legal most places.

So, again I ask you, why would anyone anywhere reuse a needle and syringe let alone share one and risk getting HIV and Hepatitis C?

Well, there are lots of reasons. If you live in a rural area how far are you from a pharmacy? How many pharmacies are their within 50 miles of where you live? Do you trust the pharmacist not to call the police and report you? Will he actually sell you the needles and syringes at all?

I looked up some statistics about the rural US. One said that 15% of the population lives on 72% of the land. One said it was more like 10% on 90% of the land. If I did the math correctly that comes out to about 18 people per square mile. I can show you square mile after square mile where the total population consists of people in cars on the Interstate.

Pharmacies rarely deliver, but drug dealers often do. People I knew back in the '60s did time for delivering to the wrong people. Idiots...

So what has this got to do with me, pharmacists, and killing people?

Recently I have been finding it harder and harder to buy syringes. Like I said I am not a lawyer, but to the best of my knowledge, in Texas, there are very few, if any, restrictions on selling needles and syringes. There is also a pharmacist's shield law that keeps a pharmacist from being held legally responsible for what someone does with what they sell. There seems to be no legal reason for a pharmacist to refuse to sell needles and syringes. It is quite clear that refusing to sell them leads to slow lingering death for drug users. Refusing to sell them can also lead to immediate death or serious medical issues for people who have to use them to stay alive. People like me. To me there is no moral or ethical reason not to sell them. No legal, moral, or ethical reason not to sell needles and syringes to anyone.

I can not find a legal, moral, or ethical reason not to sell needles and syringes to anyone who asks for them.

Recently I went into the local Walmart to buy needles and syringes. I have bought them there many times. I like to buy them there because they have a large stock on hand and they are open late. So, it was fairly late and I needed a shot. I went in and told the tech what I needed and he went off to get it for me. No problem, just like normal.

The pharmacist intervened and started asking me a bunch of questions. Funny thing, we have a law called HIPAA that defines Personal Health Information (PHI) and restricts where you can talk about it and under what conditions you can ask for it. During the conversation that followed the pharmacist demanded PHI that in my opinion he had no right to. I wanted my shot, I was under duress, so I gave it to him as quietly as I could and he proceeded to shout it out as loudly as he could. He yelled out the name of my medication and my health problem several times in front of a growing crowd of people. He committed several violations of HIPAA.

The end of the story is that he refused to sell me anything. The techs in the pharmacy looked shocked and embarrassed, and I told the pharmacist that he could fuck off. Yes, I did. Right there in the middle of Walmart. I told him to fuck off.

Before I go on with the story I should say that I complained to Walmart. Walmart gave me a written apology, the store manager called me and personally apologized. The pharmacy manager called me and gave me what can only be described as a groveling apology. Plus, I was told that what the pharmacist did was against company policy, that he had been disciplined, and counseled on Walmart company policy regarding needles and syringes. It seems Walmart has a policy of selling needles and syringes, without question, to anyone with the cash to buy them.

So, why did that guy refuse to sell me needles and syringes? That is, before I told him to fuck off? Well, the answer I got from his manager was that the guy said it violated his personal ethics to sell me needles and syringes. What? Please explain to me how your personal ethics can allow you do deny people access to medication? Or, even kill them? Well, it turns out that a lot of pharmacists have personal ethics that require them to kill people and some companies share their opinion.

The next time I went into Walmart I was able to buy what I wanted with no questions asked. They were out of one kind of needle I needed, but what the heck. They apologized for not having it and offered to order it for me.

I did not get my shot that night. I did not know of any other place where I could get the syringes I needed that late in the evening. I had to wait until the next morning to get what I needed. I had to go out and find another place to buy them. I was now more than 12 hours late for my medication. If I didn't get a syringe soon I was going to have to go to my doctor's office and pay for a specialist visit to get a shot. Expensive. I didn't want to do that.

I went to the local CVS. This is a store that is about a mile from my house and where I buy all my prescriptions. No love for CVS, but it is close and it has never lied to me. Other pharmacies have lied to me.

So, I went into my local CVS. They know me there. I asked to buy the needles and syringes that I needed and they asked why I needed them. I told them that I needed to give my self a shot. They asked if I had a prescription for the medication I was going to inject. I told the nice young lady that yes, I do have a prescription. She asked me what the medication was and if I could prove it. I told her that yes I could prove it, but no, I was not going to blurt out my medical history in front of the rapidly growing line of people waiting to talk to her. I also told her it was none of her business and that asking me about personal health information in front of a group of people was a violation of HIPAA. She did not care at all.

At that point the pharmacist yelled across the pharmacy asking me if I was, well he yelled out my full name with middle initial, born on my full birthday, living at my full address. What he did is as blatant a violation of HIPAA as you can get, except for what he did then. After a pause he yelled out the names of the three injected medication I use, asked which one I need the needles and syringes for, and told the tech to sell me the needles. To say I was shocked, and enraged, would be an understatement.

I did get the needles and syringes I needed. Federal law was broken, my rights were violated, I felt humiliated, but no one but me seemed to give a damn.

I talked to the pharmacist and he said that it was CVS policy to require customers to prove that they had a medical reason to buy syringes and needles. I tried to explain that there was no legal, or moral, reason to do that. He said that even if CVS didn't require it he would because of his personal ethical standards. After all he said "you might be using illegal drugs". In other words, the more real need I had for them, the less likely he is to let me have them. Nice guy. I just love people who think that their personal bigotry trumps our right to stay alive.

I called CVS and complained. The representative explained that the pharmacist was following company policy. They were sorry I was upset about it, but that is their policy. On the HIPAA violations? They didn't even apologize. Federal law was not a concern for CVS.

I did get a call from another pharmacist at the same pharmacy, who also was sorry I was upset. But, he explained that the pharmacist was within his rights to refuse to sell anything to anyone and that he was following company policy. He didn't care about HIPAA either.

I always thought that pharmacists were in the business of helping people stay healthy. Some of them are, I suspect that most of them are. But, some of them are in the business of playing God. They have the power of life and death and if they don't like the way you look they will let you die. If they suspect you are could possibly be doing something illegal, they will just kill you to make sure you don't. No proof required.

There are assholes everywhere, but they should not be allowed to have the power of life and death. What do you do when a whole company, the second largest pharmacy chain in the the US, decides to use their sick twisted morals and ethics to justify killing people by denying them medical supplies they need to stay alive?

Last time I needed syringes and needles I went to Walmart, got two hundred needles of one kind, but they were out of the other kinds I needed. I was driving home when I saw a Walgreens I pulled in and walked inside. I asked for what I needed and the tech got it off the shelf, and put it in a bag. I paid for it and left. No one in either store had ever seen me before. I have no prescriptions on file with either of them. They asked no questions. They just sold me what I needed. Walmart and Walgreens seem to have the right idea: Sell people what they need.

CVS clearly thinks they have the right to decide what you need and whether you get to have it, even if not getting it can kill you.

So, you are a rural drug abuser or addict. You have to drive 20 miles to get to a pharmacy, the only pharmacy around. And then you have to hope the pharmacist will sell you clean needles and syringes. You have to hope that the pharmacist will not yell out your request while the local sheriff is deciding what kind of snack will go best with the six pack of beer he just picked up. What are you going to do? Share needles and die.

Apr 17, 2015

RFIDs, Encryption, and Stop Rules... Oh My!

This post is about one facet of how people think, or don't think. It is about programming. It is about stupidity built into the human mind.

Every programmer knows what a stop rule is. They may not know it by that name. I learned that particular name in an AI (Artificial Intelligence) class and found that it applied to people as well as programs.

Unless you are using gotos (you evil vile fiend) every loop in every program has a stop rule a rule that tells the loop when it is done. If you have done any AI work you know that every search algorithm has at least two stop rules and usually more, one that stops the search when it has found what it is looking for, another that stops the search when it is determined that it cannot find what it is looking for, and usually one more that stops the search when it has run for as long as you want it to run. There are a lot of other examples from AI.

At some point in my life I noticed that people also use stop rules. I found that every one, including me, are not aware that they have stop rules. We are all terrible at noticing when they kick in. I've been watching myself for stop rules for around 40 years and I still run into stop rules that I did not know I had.

Here is an example: I used to work for a woman who, oddly enough, I did not like. Not because she was female, I usually get a long well with female managers, but because you could not argue with her. She was always correct. But, she could not explain why she was correct. If she believed something was true then it was true.

One day in a faculty meeting she told us all that the syllabus for a class is a contract. That surprised me. A syllabus does not meet any of the basic requirements of a contract. I asked her how a syllabus could be considered a contract. I asked if there was a special law or a precedent that made it a contract? She told me again that it is a contract. I have noticed that when people hit a stop rule they get a particular look on their faces and a specific tone to their voices and at that point you might as well just move on. She was showing all the standard signs. So I dropped the subject.

Then she said that you cannot change a contract. She said that a contract is a contract and can not be changed. Well, I laughed. I have spent a lot of time in business, I have written large parts of contracts, I have read many contracts. I have read books on contract law. I have talked at length with lawyers about contracts. I know that contracts can be changed. Most contracts even have a set of clauses that set out a procedure for changing the contract. But, all contracts can be changed by mutual agreement of the parties to the contract.

I blurted out that of course you can change a contract. She reiterated that "A contract is a contract and can not be changed". I started to argue with her and give examples but the look I got told me that that was a very bad idea. So, I just ignored what she said and made sure that my syllabi always included a clause stating that the syllabus was subject to change at any time and without notice.

(I actually had a student try to extort a better grade by threatening to turn me in for changing my syllabus. I told him to read the clause at the bottom. The clause that said that the syllabus was subject to change at any time and without notice. He read it and he shut up. Funny, some students will do anything to get a good grade except for doing the work.)

The first time I noticed the stop rule effect was just after I took an AI class in college. I was, I believe, a senior. I was never a freshman, but I was a senior for several years. I had just been called an atheist, a term I find rather puzzling. I was trying to explain to this fellow that the fact that we did not share the same conception of God did not make either of us atheists. (But that is a subject for a blog I most likely will never write.) In the middle of this discussion he blurted out that "to prove God does not exists you would have to be everywhere at the same time". That left me staring at him and then I tried to explain what I was not trying to disprove the existence of God. He said exactly the same thing but with his eyes unfocused and his voice in a fixed but forceful tone. I tried to ask him why he thought that I was trying to disprove the existence of God. I even pointed out that you cannot prove a negative. He shouted that phrase at me and turned red. I stepped back and pointed out that I was not trying to prove anything about God, I was trying to explain my conception of God. He screamed the statement at me, turned purple, and put his fist in my face. I backed up some more and asked him what was going on. He snapped out of it, and calmed down, it was like he had never even noticed what he was doing. It was like something out of a bad horror movie. He was possessed by that phrase. His whole mind locked up when his thought process lead him to that stop rule.

Then there was the friend who had seen an advertisement for artificial star sapphires. In the ad all the sapphires were the same size. I could not convince him that size was not part of the definition of "star sapphire". He didn't believe me even when I showed him a star sapphire of a different size. He told me that I was not showing him a star sapphire because it was too small to be a star sapphire.

There was also the case of a friend who told me that you could not write a compiler in Basic. Kind of an odd discussion.... I had made the off hand remark that you could write a compiler in any language. He said what he said to prove me wrong. Well, you can write a compiler in Basic. I had to ask a lot of questions to figure out where he got that "fact".

It turned out that to him Basic was one of the two implentations of the language that ran on our mainframe. And, to him a compiler was defined as a program that could call the machine specific, OS specific, system call used to write UNIVAC style linkable code to a file. To him nothing else was a compiler. The language he called Basic did not allow you to make that system call. Therefore, you could not write a compiler in Basic.

In all these cases I had run into stop rules. A stop rule of the mind is a phrase, concept, or fact that you can not think past. It is like an axiom in geometry. Only more so. What happens is that when the mind gets to a certain area of thought instead of thinking through the concept and reexamining the existing evidence, the mind dredges up an existing fixed concept and returns that as an unquestionable absolute FACT. Running into the stop rule ends all further thought.

You also see stop rules in expert systems. You need them to stop evaluation of patterns in the knowledge base. Stop rules are used to block circular search that leads to infinite loops.

Some stop rules are dangerous. One of the most dangerous stop rules in computing is the idea of encryption. So many people think that if something is encrypted it is "safe". This is not true and it leads to some absurd beliefs and behaviours.

The other day I was talking to a customer service rep at Time Warner Cable who asked me why I don't autopay my monthly bill. I told him the truth. I told him that I do not trust his company to keep my information private. All of my other monthly bills are paid automatically. But, not that one. I don't trust them. His reply was to tell me that they keep all their records encrypted. I commented that encryption is only as good as the security applied to the keys. If they are doing what he said they are doing the keys were available from each and every one of their database servers. In other words, keeping records encrypted doesn't necessarily add any security. The encrypted data is only as secure as the keys.

Encryption is a major stop rule. If you want something to be secure, just encrypt it! Once you decide to encrypt, your security problems are solved! No. No. Not at all. Even encrypted data can be understood without decrypting it. Don't believe me? Look up how passwords are stored. They are encrypted and the plain text is erased, only the crypto text is retained. When you type in your password it is encrypted and the new piece of crypto text is checked to see if it matches the crypto text of the password that is on file. The stored password is never decrypted.

So, finally I get to RFIDs. Radio Frequency IDs are a big thing. Some people have them under their skin right now. A lot of pets have RFIDs. RFIDs are used to prevent shop lifting. If you put them on shipping boxes they are great for keeping track of where things are in a warehouse and great for making sure that a box gets on the right truck. They even put them in US passports. But, they were expected to be much bigger than that. Much much bigger than they are. And, it all comes down to encryption.

You see, retailers wanted to put RFIDs inside every product. Not just in the packaging, in the product. Your toilet paper would have an RFID that identified it as toilet paper. Your laptop would have an RFID that identified it as a laptop. I believe that the plan was to encrypt the SKU (Stock Keeping Unit) of each product, place that information into an RFID chip and then build the chip into each product.

Why would retailers want this to happen? Imagine shelves that read RFIDs. Each shelf in a store would "know" what was on the shelf. It would know how many items were on the shelf. If someone picks one up and puts it on the wrong shelf, well that shelf would tell the system what happened and someone could be sent out to move it back. If the shelf becomes empty then someone can be sent out to restock the shelf.

It gets better than that. Lets say you have electronic signage on each shelf. The shelf can look up the product and display the name of the product and its current price. No more paying people to change signage. No more bar codes either. You do not need them if you have RFIDs.

The retailers were even talking about doing away with check out. Just walk out the door and the price of everything you took out of the store would be automagically deducted from your bank account. Of course they would know what you carried in and what you carried out so they could make sure they didn't charge you for the clothes you wore into the store.... I never did hear what they were planning to do if you didn't have enough money in the bank to pay for your purchases. Drop a cage on you?

There are a few problems with the plan... You walk into a store, buy a new diamond ring, and leave. Ok, a guy in a car with a few dollars worth of hardware scans you as you leave the store, follows you home and relieves you of the diamond. Or, you are walking along with a new laptop in your backpack and someone with a few dollars worth of hardware pings your RFIDs and steps out of the shadows and relieves you of the back pack.

They were talking about putting RFIDs in US currency. That would let the bad guys count the money in your pocket and decide who is worth robbing...

Hey! no problem say the RFID peddlers. The info on the RFID is encrypted! The bad guys can't read it! No. Not true. Not at all. Remember passwords?

You see, to make this work there has to be a database that maps crypto text to plain text. It must exist. To do an RFID based transaction you have to read the crypto text from an RFID. Then you have to pass it to something that gives you the real SKU. That SKU is then used to look up the price. Even if there is no way to decrypt the SKU stored inside the product you still have to be able to get the real SKU from the encrypted SKU. That implies the existence of a database that contains all the SKUs and the matching crypto text. That database would be worth a lot to the bad guys. That database has to exist no matter what kind of information is stored in the RFID. That means that no matter how good the security is around that database someone will sell it.

The alternative to the database is to have copies of the decryption keys spread all over the world. Which we can count on being secure for oh, maybe ten minutes?

But, then, the bad guys do not need to buy the database. They can build it themselves. Just walk through a store and collect encrypted SKUs from the products on the shelves. If that isn't practical just buy or steal products and collect their encrypted SKUs. Pawn shops and fences would get a whole new revenue stream from collecting and selling databases of encrypted SKUs paired with product descriptions.

(Techies tend to think of criminals as stupid technophobes while at the same time worrying about how to defend against the next hacker attack. Truly twisted thinking.)

The development and testing of schemes for putting an RFID inside every product you buy went on for quit a few years. It took that long for someone to point out how bogus the idea was. It took about 6 months for the whole idea to collapse after the flaw in their thinking was pointed out. Years of work went into this screwy idea because the concept of encryption is a powerful stop rule. For years not one person figured out that just encrypting the information did not solve the security problems. I have talked to some of the people who took part in that project and they are very intelligent people. But, they never noticed the problem. They were blocked by a stop rule called "encryption". They accepted it as a solution without thinking it through.

Stop rules are dangerous. How many do you have? Are you even aware that you have them?

I can guarantee that you have them. You would probably go insane without them. You see, people believe that their stop rules are all facts. Most people do not question facts. It has been said, apparently by several different famous people that "Everyone is entitled to their own opinions, but they are not entitled to their own facts". What I have learned is that, everyone has their own facts.

Think about this simple stop rule: "2+2=4". That is something I hope you learned in grade school. It is difficult to add 2 and 2 if you have not memorized this stop rule. Oh, you could count on your fingers to two, and then do it again, and then count the fingers you touched, the result will be 4. But then, knowing how to count depends on memorized stop rules er... facts. If you had to rederive, reinvent, or reexamine all your stop rules every time you use them you would go nuts. But, then you would have to be insane to do that in the first case. There is no difference in the mind between stop rules and facts. Facts are the bedrock of our verbal thinking system.

Just for fun let's try to think past a stop rule. Let's challenge a basic fact. Can you imagine that 2 plus 2 is not equal to 4? Could it ever be that the sum of 2 and 2 is 1? Well, yes it can. If we are talking about modulo 3 arithmetic then it is always the case that 2 plus 2 is equal to 1. Funny, we just went "past" a fact and found an alternative that is also true, but not the same.

What if 2 is not even a number? What if all numbers are pairs of what we call numbers? In that case 2 is not a number but (2,0) and (1,3) are numbers. Well sure, if you have learned much math at all you will see those as coordinates on a plane and think I am talking about vector math, right? In this case I am talking about toroidal math. The first value is modulo N (say N=3 just for grins and giggles) and the second value is modulo M (say M=5) and we have pairs of values that designate points on the surface of a torus. Wow, all that from just questioning a single well established fact known to every grade school child in the world. If we are able to go there we see that 2 plus 2 is not always equal to 4. We see that 2 all by itself is not necessarily a number. And, we see that we can do arithmetic bound to the surface of a torus.

What do you have if you use 3 values to represent a value and use modulo arithmetic on all three values? You get hypertoroidal arithmetic. You would be doing arithmetic bound to the surface of a hypertorus.

We are dependent on using memorized facts to do our thinking. If we get some of them wrong, and especially if we cannot question them even when our facts are challenged, we can go through our lives using facts/stop rules that do not correspond to reality. We are forced to go through our lives with no chance of discovering anything new. We can never have the joy of realizing that much of what we know is wrong. We cannot advance toward greater understanding of ourselves or the Universe.

Looking into this peculiar phenomenon I have learned some interesting information. First off, we (all humans) tend to avoid any information that contradicts our existing stop rules. We even seek out information that seems to back up our existing stop rules. This phenomenon is called "confirmation bias" and has been well studied for quite a while now. I have to admit I see this in myself. It is why I laugh at people who quote Fox News and treat anything I hear on the BBC as gospel. (Ok, not quite...)

Another interesting thing about this phenomenon is that the way we look at facts is one of the four known axes of human personality. As far as I have been able to find out this idea originates with Carl Jung and has been refined by Myers & Briggs in their theory of personality types. What they call the iNtuitive - Sensing (NS) axis is all about our attitudes toward facts. To a person on the intuitive end of the axis all facts are evidence of underlying principles. This person can not see the trees for the forest. The other end, the so called sensing end, treats facts as facts, absolute unchallengeable rules that stand on their own. This is the person who can not see the forest for the trees. This indicates that some people are more likely to be affected by false stop rules and more resistant to modifying them than other people.

This page gives a best guess at the percentage of people in the US with the 16 different personality types. (I should point out that there are an infinite number of possible personalities within the 16 general types.) According to that page roughly 73% of people in the US are on the S, facts is facts, end of the spectrum. That means that almost 3/4ths of the people in the US have a natural resistance to changing their views. They accept facts (stop rules) as unchallengeable nuggets of pure truth that cannot change. That is why in the US someone who changes their mind when faced with new facts is called a "Flip Flopper" and can't get elected. Learning and changing your opinions goes against most people's basic nature. No wonder our schools focus on memorization instead of learning. No wonder that so many are suspicious of science...

Unexamined stop rules, facts that are not facts, cause irrational behaviour. Most people like it that way.

I started out to talk about how unchallenged stop rules, that is, unexamined assumptions, lead to bad decisions, even to irrational behavior. I hope I have done that. I hope if you read this far you got something useful out of it.

Since I have referenced Myers & Briggs I feel that I must come clean and tell you that my Myers Briggs type is INTP. If you look up INTPs you'll find that we are rather odd ducks:

We make up about 3% of the population but are about 7% of all software developers.

Mar 12, 2015

Too Clever to be Useful

User interfaces are often designed by people who do not understand that they are not the measure of all things. The design interfaces that are easy to use, easy to understand, and pretty... according to the designers. The designers do not remember the rest of the world does not share the assumptions the designer have. They do not share the designers taste. They do not share the designers culture, language, or even all of their physical characteristics and abilities. 

We people out here do not know, or care about, the clever principles you based your design on. We do not know, or care about, the constraints you faced when designing the interface. We do not care about any of that. We only care if we can use the interface to get our job done with a minimal amount of effort and a minimal chance of nailing ourselves to the wall. People will only learn as much as they need to get their job done. The more they have to learn to use your interface the more likely that they will just walk away and bad mouth you. The more times they have to learn a new interface for do the same old task the more likely they are just skip the new version or switch to a different vendor. A vendor whose interface is easier to use.

People see a new or different interface as a barrier to getting their jobs done. They see having to relearn the interface over and over again as stealing their lives. They see the changes a disrespect. They see it as you wasting part of their all to short lives by making them learn one more way of doing the same damn thing.

Sometimes the designers make a clever interface with the expectation that the interface will be powerful. They do not realize that the result is actually confusing and error prone. A plethora of features and clever behaviors leads to people getting unexpected results as a result of unexpected interactions between features of the overall system.

The first time I ran into this problem was in the middle '70s. I was an experienced Fortran programmer. One night Dave, another experienced Fortran programmer (and the only person I have ever found sleeping face down on a keyboard), and I were sitting around talking when a contractor we both knew came in to ask us a question. Noel was a very experienced Fortran programmer. She was teaching a night class in Fortran and had given a test, graded it, and returned it to the class. Everyone had given the wrong answer to a question about the Fortran FORMAT statement.

When she was reviewing the test with the class several people challenged her answer to the formatting question. They had different answers and they were sure they were right. After a discussion with the class she realized that she was not 100% sure she was right. So, she promised to research the question and make sure she had the correct answer. This is a very reasonable thing for an instructor to do. Not half the instructors I have had would have done that. I have encountered one "school" where you are not allowed to do what she did. The official answer is correct even when it is wrong.

Her review of the question started with asking Dave and I what the correct answer was. Dave gave his answer and Noel and I showed him that he was wrong. I gave my answer and both Dave and Noel showed me that I was wrong. We had a friendly little argument, like programmer do, that lasted a few minutes. We dug out the textbook and the manuals and traced through the definitions and realized that we could be there all night. By that time we had all changed our answers but we still did not agree.

I suggested that we code up the question and run it. Well, we did that. When theory fails do the experiment. Coded does not lie! The results did not match my answer, or Dave's, or Noel's. It also did not match any of the answers that any of the students had come up with. Noel dropped the question from the grading for the test.

In Fortran, the FORMAT statement is sort of like the format strings used by C's printf function, but much more powerful and complex. You can have repeat counts on fields. You can have repeat counts on parenthesized substrings. And, when you reach the end of the format the I/O system just goes back to the beginning and starts over. Oh yeah, you can have loops within I/O statements. The whole mess is much more clever than the one in C, much easier to mess up. It once let me print 2,000 pages with one character per line. You may have noticed that C++ has tried very hard to get away form using format strings. But, of course, Boost put them back in.

Fortran FORMATs are powerful, and clever. So clever that they are hard to use. Unless you work with them every day you are likely to have to look up the details every time you use them. After the empty pages incident I always tested them on very small test cases before I used them for production work.

I have run into this odd problem several times. I remember having to deal with a search tool (not on the Internet) that wanted you to type in a properly formatted regular expression. If you made an error it would force you to type it in again. No editing capability, no warning that it expected a regular expression, and no tutorial documentation, just a mention of grep. If you did not know what a regular expression was and know how to type one out, oh just off the top of your head, you were screwed. Not to mention that there are lots of different ways to type a regular expression. You have to do it their way, not the way you may have learned it in college. And, of course, how many people know what a regular expression is anyway? Very powerful, very clever, and totally useless.

I've seen the same thing with search tools that require you to type in a valid boolean search expression. But, did not tell you that, and did not tell you what the operators were. I even wrote the back end for a mess like that. Worked great but no one could use it.

You see this kind of thing in changes to user interfaces all the time. Sometimes the change is for the better, like the ribbon in MS Office. On the other hand sometimes the changes don't seem to make any sense at all. Change just to let the designers show off how clever they are.

Last fall I rented a car in the UK. No problem getting in, the clicker worked just like I expected it to.. But, then, we had to figure out how to start the car. No key on the keyring, no ignition switch near the steering. What!? Well, there was a big button labeled "ignition" so we pushed it and nothing happened. Then we looked carefully at the keyring and noticed that it had a little gray box on it, and there was a little hole in the dashboard over there... so we put the little box in the little whole and then pushed the big button and the car started. I never dared pull that little box out of the little hole while the care was running. I sure wanted to. I never could get the radio to work at all.

Again, very clever, I'm sure that keyless ignition is very "cool" or "khol" as they say on the coast. The change seems pointless. What is the reason for replacing the key with a little box? It seems like a change that was made just to make a change. Very clever, but WTF?

Smart people design things that are easy for them to use. They forget that not everyone sees things the same way they do. They forget that people do not share their culture, their training, their recent experience, their field of knowledge. The smartest people in the world may not be able to use your design if it is too clever. You have to be smart enough to design something that is dumb enough for everyone to use. Something that everyone can use.

I had a coworker who once told me that if a program with a GUI interface could not be learned in 10 minutes, 30 minutes tops, of poking around it was not worth learning. I think he was being a bit unreasonable! But I like the goal.

That brings me to Windows 8.1, talk about change for changes sake. I bought my wife a new laptop with Windows 8.1 preloaded. My wife is not a noob and she is not stupid. My wife was a rocket scientist. Really. One of her projects blew up a test stand at Edwards AFB. Some of her stuff has launched commercial payloads into orbit. She has been using computers since she was in high school back in the 70s. She has used DOS, and pretty much every version of Windows since 3.1. She even has time using TSO. Now days she mostly does project management. She manages IT projects and has an amazing record of getting projects done on time and on budget. At work she uses Windows 7 and Office all day long...

After one week of trying to do anything on Windows 8.1 she asked me to replace it with Ubuntu Linux. I tried to talk her out of it. We need at least one Windows machine in the house so that we can update the maps on the GPS. (In this day and age why would anyone support only Windows and iOS?) After a month she demanded that I get Windows 8.1 off of her machine. I was tired of listening to her scream at the laptop so I took it off. It only took about half an hour.

Windows 8.1 looks great. Touch screens are very nice and there are some really nice touch screen interfaces out there, like the ones on everything by Apple and on Android devices. Windows 8.1 is strange just to be strange. If Windows 7 is the best version of Windows ever (in my opinion it is) then Windows 8.1 is a sign that Microsoft has corporate dementia. Hey if corporations can have human rights they must be able to have human diseases. Windows 8.1 is very pretty, and I bet the designers think they are very clever. So clever they have built something that is completely useless.

Mar 5, 2015

Google vs. AT&T vs. TWC

[ Note - I've added some notes but haven't edited the original text. ]

I live near Austin, Texas. In fact I live about a mile outside of its ETJ in Round Rock, Texas (named after a round rock that marks where wagons and cattle can ford the Brushy Creek on the Chisholm Trail). My, don't we love descriptive names around here?

Austin is one of those towns that is currently getting Google gigabit fiber Internet service. We expect, and hope, that Round Rock will also get it just like the suburbs of Kansas City got it after Google rolled out their service in Kansas city. We are very excited about that.

Funny thing, AT&T and Time Warner Cable (TWC) are also very excited about it. Scared out of their minds, shitting down both legs, may be a better description of their reaction.

For years AT&T and TWC were the only game in town when it came to so called high speed Internet access in this area. In some parts of Round Rock AT&T is the only game in town, no cable in those neighborhoods at all. Over time both of these monopolistic companies have provided just enough bandwidth to keep people from rioting in the streets and not one bit per second more.

TWC tried to put a usage cap on their service, but found that the people actually did come out in the streets, and TWC's parking lots, and TWC's lobbies, to protest. TWC dropped the idea of a usage cap like a rotting fish.

AT&T put on a usage cap and told people to take it or leave it. AT&T loves usage caps and overage charges.

Neither company is exactly famous for the quality of their customer service. But, there are distinct differences in their attitudes toward their customers. TWC, in my opinion has always been more progressive than AT&T in their customer service. TWC actually does try to provide customer service. In fact, as a TWC customer I have to admit that I actually like TWC's customer service. (At least most of the time.) I have called them with some weird problems and always, eventually, but usually immediately, gotten a solution.

AT&T's attitude toward customer service is that customers should be served medium rare with a nice chianti, and some fava beans. (I worked for what is now called AT&T for nearly five years. so I know that AT&T really does not respect customers.)

TWC has always offered better speeds than AT&T and seems to offer better prices. AT&T's network cannot support the same speed that TWC's network supports. AT&T uses fiber to the neighborhood and copper telephone wire to the house while TWC use coax to the home and coax or fiber to the neighborhood. You can push more bits through coax than through a pair of twisted copper wires.

I once watched a TWC technician run diagnostics on my cable connection. If I read it correctly their network is capable of delivering many gigabits per second to my home.When you think that using DOCISS 3.0 TWC can move over 300 Mbps downstream by bonding 8 channels. You can see that with their ability to deliver hundreds of channels the total bandwidth available on their network is truly huge.

I watched an AT&T technician do the same test and was told they could get 31 megabits to my house. Megabits from AT&T gigabits from TWC. Maybe the difference in networks explains the difference in their reaction to competition.

When Google started rolling out their service TWC calmly, and very quietly, announced they were going to increase the bandwidth of their Internet service. They described the new service, told people when it would be available and did what they said they would do. They doubled my speed to 100 Mbps at no extra charge and offered me 2 to 3 times that speed for a reasonable, in my opinion, price. That made me happy. It gave me a nice warm feeling toward TWC. Of course, I know they could have done the same thing nearly 20 years ago and chose not to. They chose not to for a simple reason. Like all cable companies TWC makes more money from a cable customer than from an Internet customer. But, the better the Internet connection the less the need for cable service. High speed, low profit, Internet service kills high profit cable service.

AT&T's reaction was very different. AT&T could not just increase their customers bandwidth. Their network will not allow it. To match Google's service AT&T will have to spend almost as much as Google is spending to create an equivalent network. They really do not want to do that. So, what do they do instead? AT&T started advertising GigaPower Internet. I really want to call it GigglePower because their whole reaction makes me want to laugh out loud. They advertised it widely and loudly and with expensive TV adds using local big name performers. But, nobody has seen GigaPower... I can't even verify that it really exists.

You see, you can't get GigaPower Internet service. You can't even get any information about it. I have called AT&T to try to find out if or when it will be available in my area. They will not answer that question. I have asked where it is currently available. They will not say. I have asked people all over Austin if they have it, or can get it. No one has it and no one can get it. As far as I can tell it does not exist except as an advertising campaign designed to slow the uptake of Google's service. Giggle Power indeed.... well almost.

I recently found out there are a few homes somewhere in Austin that have the service, but I can't find out where. If you know where you can get it or when it will be available please let me know. I have, like I said, called and asked. I have looked all over their web site. I can find a list of cities they say will get the service, but nothing about actual installations. I found out that there are real customers out there by doing a search for "GigaPower Sucks". Funny how searches of that form always work.

Oh yeah, one more thing to giggle about. AT&T has slapped a 1TeraByte usage cap on their GigaPower service. Let's see 1 gigabit per second translates to about 100 megabytes per second. (When you figure in protocol overhead it takes about 10 bits to transfer 8 bits, 1 byte, of data.) That comes out to 10,000 seconds, 166.7 minutes, or a grand total of 2.8 hours of full speed usage per month. Note that I said full speed usage. You will usually not use the full speed. One terabyte is a lot of Netflix... But, I have one terabyte on DropBox and I could pretty easily blow through that cap.

[ Note - Turns out the above is wrong... You see, even though they call it "GigaPower" they do not offer gigabit/second service. They top speed they offer is 300 megabits/second. A tad less than one third of the speed implied by the name. Almost false advertising.... Exactly the same top speed that TWC offers. Oh yeah, almost forgot, at 300 megabits you can blow through the AT&T cap in about 9 hours of full speed usage. You should multiply or divide by 3 to get closer to the correct numbers. ]

AT&T takes you for $10 for every 50 Gigabytes you go over their cap. Fifty gigabytes is 8.33 minutes of full speed usage. Another way to think of this is that you get 2.8 hours of use per month and are charged $60/hour for every extra hour you use on a service advertised as being only $70 per month. If you use the service as it is advertised you can wind up owing thousands to AT&T at the end of the month. You can see where they plan to make their money on this "service". Of course, I am assuming they will someday deliver the service. Oddly enough, I know that AT&T can move very fast when it wants to. They can deliver this service. The question is, does AT&T want to?

It sure looks like GigaPower Internet is a service you can't get and if you do get it you can't use it the way you think you should be able to. GigaPower is just one more proof that AT&T execs think that the only things people do on the Internet is read sports trivia and look at porn.

AT&T and TWC have to compete or die. Until they faced competition they had no reason to offer a better service. Customers are paying for the old service. Offering a better service will eat into their cable businesses. And, customers had no other choices. As long as neither company improved their services they could both just sit their and collect money for crappy products and service.

Both companies worked to keep competition at bay. Through lobbying the big telecoms made it illegal for municipalities to build their own networks (which many had started doing) and through other legal and financial dealings they kept the small start ups small. At least they have done that in Texas. As long as they can buy politicians and do other things to prevent competition they were never going to have to change. And that was their goal, make as much money as possible selling a service that didn't interfere with their other businesses, namely cable TV. If that kept the US in the Internet dark ages compared to other first world countries, that was fine with them.

The cost of buying a state legislature is chump change for a company like AT&T. What can I say, most Texas politicians are honest in that once they are bought, they stay bought. So, they are a good buy. (Just look at what Ted Cruz, the senator from Comcast, has to say about net neutrality. Even other right wing conservative tea sips think he is nuts.)

AT&T is slowly rolling out the service they advertise. I can't find out where or when it is being installed. I can't find out which neighborhoods will get it. But, they sure do advertise the hell out of it. The only thing I can find out about it is that it has a service cap and incredibly high overage charges.

TWC can, using DOCSIS 3.0 offer a gigabit Internet service. It might be fairly costly to do, but it can be done on their network. I fully expect that they will eventually do that to stay competitive with Google and AT&T. It will be cheaper for TWC to upgrade their network than for either AT&T or Google. That gives them a small advantage.

The next real question to ask is what these companies are doing about their backbone networks. Getting a gigabit in and out of my home means they need terabit backbones to handle the traffic. Connecting millions of homes with gigabit connections to slower backbones would be like having an eight lane highway for a driveway that dumps you out onto a rutted dirt road. AT&T's usage cap goes a long way toward eliminating the problem for them. No matter how fast their service turns out to be the usage cap is only a few times higher than their DSL usage cap so they won't need a faster backbone.

For all my bashing the two telecoms that I know the best I have not ignored bashing Google because I trust them, or even like them. I have done it because I don't have any experience with them as a telecom.

At least Google has stirred things up by coming in as a serious competitor. By competing with all the big telecoms they are forcing them all to provide better services at lower prices. So no matter what, we the people, win. I think that is a good thing.