The Software Developer

For a moment try to imagine how much code is in the average application running on Windows.  The EZGUI 5.0 Visual Designer, plus its runtime DLL’s come to about 65,000 to 70,000 lines of code. That may sound like a lot, but its actually quite small compared to many applications today.

Now add to this scenario many hundreds of thousands of lines of code in the operating system, possibly millions. Now do you really think that all that code is 100% bug free ?

Probability alone would dictate that a number of bugs get through unchecked. Now some may say thats just something we have to live with, so why make a big deal about it, but it is a big deal and it should be of concern to software developers. Many software bugs can be quite serious to the end users who depend upon the software.  So can something be done about it ?

It all starts with ones attitude towards software development. Code needs to be tested by the programmers writing it, rather than leave that to beta testers. It is so easy to pawn off a beta version of an application and wait for end users to find bugs, rather than take the time to do some inhouse testing of your own. To illustrate, look at how car manufacturers test their cars for safety. They run them through a battery of tests “inhouse” (crash the cars on purpose) to find the flaws and then fix them. Imagine if they simply built cars and let the consumers “beta test” their cars and then when they see plenty of flaws which could hurt someone, they decide to fix them. No one would put up with that! But yet when it comes to software, we often believe this is normal. It shouldn’t be.

I will leave discussing the details on how to debug software for future articles, but suffice it to say that there are keys to debugging software. The first starts with our attitude as programmers. Second, how we write code is critical, from the language we use (is it easy to read and maintain?) to the style of coding we use. If we write code which takes a rocket scientist to read, maybe we need to find a better style of coding.  Modular code which is well written need not be overly complex. The tools we use to write applications with need to be reliable too. It is also important to take preventive steps when writing code. For example, unless the code is time critical, it doesn’t hurt to write code which tests parameters to see if they are valid, before using them. Also make sure one always cleans up after they do something which could leave behind a problem (ie. in Windows you have to make sure when you change the state of a DC you restore the DC back to its original state when done). Modern computers are very fast today, so the few extra CPU cycles used to do some house cleaning won’t hurt the overall speed of an application.

As a programmer I visit the online programming forums, particularly the PowerBasic forums, since I write all my software in Powerbasic. I often see errors in other peoples code which demonstrates a simple lack of attention to detail. True code posted on a forum is not expected to be “mission ready” so to speak, but is often just test code, but none the less the errors in such code do often demonstrate the writers lack of attention to details and lack of testing (debugging) skills. I have even seen some who will start questioning the reliability of the compiler, when something doesn’t work right, instead of first considering whether they did something wrong.

Let me say that learning to test and debug ones own software should be a priority for programmers and it requires experience and training. I might be a bit of a perfectionist (some times thats bad and some times its good), but when I find a bug in my software I will do whatever it takes to figure out the cause of the problem, even if it takes hours or even days. Some problems are difficult because it isn’t so much that you wrote bad code, but some times you are dealing with the operating system and it has its own quirks at times.

Now I take this subject very seriously, because not only do I write software, but I write tools for programmers who will use my software to write their software.  In essence, I have two levels of end users. First the programmer who uses my tools and then their end users who use their software, created using my tools. I have double duty so to speak. I can’t prevent a programmer from writing bad code, but I can help a programmer to write better code and can try to protect a programmer from easy mistakes. EZGUI’s runtime does a lot of parameter checking to make sure the user (programmer) does not do the wrong things. It even checks the window classes to make sure you don’t a listview command with say a treeview control.  Sure that is extra overhead, but it makes the software my customers write more reliable. I actually had one customer years ago who purposely tried to crash the EZGUI runtime by passing a bunch of bad parameters to EZGUI commands. He was trying to make the runtime GPF, on purpose. He told me he couldn’t do it. The point is, he tried to break my software on purpose, but found it difficult to do, which was all because I wrote the software to expect bad parameters and to take measures to deal with them. In some cases EZGUI would simply use a default, instead of the parameter passed. In other cases EZGUI would simply do nothing (ignore the programmers request by the command call). In my case I am trying to protect the programmer (users of my tools) from himself. Its enough I have to try to prevent bugs from getting into my own code, but I also have to protect users of my software from the bugs they may introduce into their applications. Now that is a challenge!

For example, a number of times I have seen Powerbasic programmers on the forums post code where they confuse different API terms, functions, messages with similiar ones. I have seen where an API message is used for the totally wrong control class. A message may be meant for a button control and someone tries to use it with a label (static) control. I have also noticed that some may attempt to write code to do something of which they lack the proper knowledge about. I find it is better to do some research first to fully understand the Windows API’s you are using, before writing any code.  One thing I try to avoid, is writing code in areas where I lack a basic understanding of the principles involved. I do a lot of research learning new things long before I even attempt to start writing code in such new areas.  As my knowledge base grows, I continue to add new subjects to that knowledge base. I would not have attempted to start writing OpenGL code even a year or two ago, but now I am digging deeper and deeper into OpenGL and have written a very powerful GLCanvas control with its own 3D Graphics script Language. For awhile I simply “browsed” the internet and “dabbled” with OpenGL. I read articles which were simple in nature so I could get the basics. I finally picked up a great book on OpenGL and started digging deeper. I played with examples of OpenGL code on the Powerbasic forums to experiment with. Now I never use anyone elses code in my software but that does not mean I don’t learn from others. I just like to experiment with others code to see what happens when I start modifying it and see how the API’s used respond. Once I reach the point where I begin to grasp a new subject, I then start completely from scratch. No ones code is used as a basis for my own. I simply start building an understanding of each API used, one step at a time. It is slow at first, but I build the project one step at a time, test it, add more, test it, add more, test it and so on.

My GLCanvas control was a perfect example of this. Actually this new control, which is in EZGUI 5.0 (not released yet) is two projects in one. The new control uses EZGUI’s new superclassing engine, which I built and fine tuned while writing this control. The GLCanvas control is a hybrid control based on my Canvas control. I then began to add OpenGL features one step at a time, piece by piece. I write code to test my code! Yes, I build the control in steps, and then write application code to use the control to see how it performs. I don’t have to have beta testers “tear into the control” looking for bugs, because my goal is to make it flawless right from the start if possible. Sure those who are currently working with the PreRelease version of EZGUI 5.0 may find a bug or two, but I don’t expect them to be looking for bugs. I consider that my job, not theirs. I guess thats the key! As a programmer, I consider testing and debugging my responsibility, not the end users.  Unless a programmer has the right attitude, no training in debugging will help them.

So are you ready to learn how to be a better debugger ?

Thats a subject for another article !

To be continued …

Debugging is one of those areas in programming which in my opinion doesn’t get enough emphasis when teaching programming. In this article I will discussion why it is important and also some suggestions to help in debugging. Since Powerbasic is my primary programming language, all suggestions will be geared towards debugging in PowerBasic for Windows.

First the “why” it is so important.

No matter how fancy and polished your software may look, if it has serious bugs in it, it is useless. Also it is very easy for a programmer to blame the end user when problems occur when using the software, rather than the programmer taking a hard look at his software to see if it really does have a bug. Lastly it is too easy for programmers to feel that “all software has bugs in it”, so “end users will just have to live with it”.

Personally I feel such feelings are absolutely wrong and may indicate the programmer is not as qualified as they make think they are. While it is true, that 100% bug free software may not ever be possible, it is also true that when a programmer becomes aware of known bugs and simply does nothing about them (at minimum warn the user and provide a workaround if possible until it can be fixed) something is definitely wrong.

Maybe the problem with buggy software has something to do with how programmers write software, so lets consider this. First, it is important to use good programming habits to decrease the risk of bugs. One of my favorites is the simple #DIM ALL compiler directive in PowerBasic. You would be surprised how many PB’ers (thats short for PowerBasic programmers) don’t ever use this directive. This simple directive forces the compiler to expect that every variable used must be declared at some point (ie. LOCAL, GLOBAL statements). This prevents so many simple typing errors from getting through, because if you misspell a variable name, the compiler won’t think it is just another variable, but it will actually generate an error message when compiling. This compiler directive is an absolute must in my opinion.

Another area has to do with variable names. While one does not have to follow any rules on this, one should seriously consider how their name conventions will effect debugging. For example I like to always define Global variables with a common prefix. One I use a lot is App_ which indicates this variable is global and available to the enitre app. This makes code so much more readable and it prevents a lot of problems.

Next, while it may not be the popular and so called “standard” today when writing code, I prefer to write variables with type identifiers rather than the syntax of LOCAL MyVar AS LONG. Now I can see if a variable may need to change because of changes in the Windows API, so it may be better for some variables to be defined with a type indentifier, so in that case definitely do it. But in code which will likely never experience a change in data types, use the type identifiers. It makes code more readable at a glance, because you can tell what type a variable is simply by its name. Now if you must avoid using type identifiers, then maybe name the variables with a prefix for each type. For example an asciiZ string variable could start with a small z (ie. zText) and a dword variable could start with dw (ie. dwSomeVar). Also it is a good idea to use some prefixes in general to indicate what is stored in a variable. For example I like to always add a small h to variable names which store some kind of handle (ie. hWnd, hDlg, hCtrl, hPen, hBrush).

Put some serious thought into your style of coding with the idea of making code more readable. If a bug is found later on, you may find yourself having to go back and read code you wrote last year, so you need to be able to make sense of it quickly.

Another important step in debugging software, is to write modular code. It need not be object oriented to be modular. I write all my code using good old fashioned procedural style coding (ie. sub, function), yet I tend to write very modular code. If code is not time critical (speed) it doesn’t hurt to break it up into smaller blocks of code. You will be amazed how much easier it is to debug modular code. Once a subroutine or function is debugged, then every other routine which calls it, is guaranteed to work better. Reuse code as much as possible. This make applications smaller, but also makes software more reliable because it is easier to debug.

Now the next area is where I think many programmers may have a difficulty. I strongly feel that debugging is the responsibility of the programmer and not of the users. It is easy for a programmer to fall into the trap of thinking, they can write all the code they want and if there are bugs, the Beta testers will catch them. Beta testing should not be ones sole means of debugging. A programmer should test everything he writes, even writing test applications to test the new code. If you write a database engine, then create a test app to create a database with it and make sure it works properly. The point is, be willing to put your code through the paces to see if it works. Also be willing to put your code to the test by pushing it to its max. If you write a database engine, don’t test it with a small tiny database, but test it with a huge, larger than normal database. What works well when not under load, may simply choke when put under a much larger load. For example, in autoparts stores when they bench test an alternator or a battery, they have to put it under some heavy load, because it may pass under a light load, but then fail under a heavier load. The same is true for sofware.

For example, I was bench testing so to speak my EZSprite software (a 2D Sprite engine). It ran well with say 12 or 24 sprites. One day I was comparing my sprite engine to another graphic tool which could animate objects like sprites and while the other tool worked well with a small load, when I put it under a much larger load, say 100 objects, it demonstrated that it was not as fast as you would have thought. I figured I needed to put EZSprite under a much larger load to see if it would do the same. I created a test app where EZSprite used the exact same sprite animation as the other tool and then had it create and animate (move and frames) 500 sprites. Thats a lot of sprites for a software based 2D sprite engine. Amazingly it did quite well and actually was faster than the tool I was comparing it to. Only when the other tool resorted to using hardware support, did it run faster than EZSprite.

The idea is to bench test ones applications and to put it through the paces, before it even gets to the end user. You will find that bugs (or poor speed) will show up more quickly when you purposely are looking for them. With EZGUI, I find most bugs even before my customers ever notice they are there, because I do a lot of testing.

Now one will say that testing takes valuable time I just don’t have. My answer is “make the time”, because if you don’t want to test your software you shouldn’t be writing it. Look at car manufacturers where they run their cars through torture tests and crash them, just to see how they perform when pushed to the limit.  You have to be willing to do the same with software.

Debugging and testing is a art form. You have to learn how to do it and how to do it right. It is not enough to learn how to write code. You have to learn the fine art of debugging. Now a warning! Do not think the solution to debugging is simply making the compiler or other third party tools do all the work for you. Some programmers feel they are debuggers when the use software tools such as third party debuggers to “catch” bugs. The best debugger is you the programmer. Software is stupid compared to a human being. Thats the truth. The human mind, when trained to look for bugs, is better than all the debugger tools combined. Now this does not mean you can’t or shouldn’t use such tools. It just means don’t think that a software tool will relieve you of the responsibility of learning the art of debugging and the need to write quality, modular code. Just for your information, personally I don’t use any of the debug tools built into the compiler and I don’t use any third party debuggers available and my software has a reputation of being “solid as a rock”. This is because I have taken the time to learn the art of debugging. I learn from my mistakes and over time I change my coding habits to add new ways to produce better code. I test my code, specifically new routines I write before I use it in production code. I experiment a lot to make sure I know what I am doing with some new API before I introduce it to my actual software products.

Lastly, I spend a lot of time testing and experimenting to make sure what I write works as intended.

If you want to be “great programmer”, then take the time to learn the art of debugging. Strive for 100% bug free software. Learn from your mistakes and be willing to change your style of coding when it becomes obvious you need to. Lastly, don’t expect the end users (the customers) to be your debuggers and testers. Consider it your responsibility!

Budding (new) programmers may wonder, “what makes a good programmer” and what advice would you offer that would help make me a better programmer ?

My answer would be:

Stop being a programmer for a moment (maybe longer) !

Maybe an illustration would help here.  The engineers who design automobiles are likely very knowledgable about their craft, some even being experts in their field. But one failing I see in such “experts” is that they fail to put themselves into the shoes of those who eventually must repair those automobiles (all cars break down some day). Out of shear necessity I have had to learn how to work on my own cars. I recently had to rebuild the engine on my car. Guys (and gals) who work on cars often find themselves saying, “what was the guy who designed this thinking !”. As an example, my trusty old Geo Prism (which finally died after over 335,000 miles on it) had the alternator right up front near the top of the engine. It was so easy to take off, that I once went to the autoparts store, purchased a new alternator, went out into the parking lot and replaced the old one with the new one in a very short time and then walked right back into the store to get my refund for the core price on the old one.  Now my current car,  has the alternator hidden underneath the engine on the far back side of the engine in a place which makes it very hard to remove. Sure there is a plastic panel you can take off to get better access to it, but it was still a real pain to remove and replace. The point is, those who design stuff often fail to grasp what those who will ultimately use (and/or repair) it think.

When designing software, “stop being a programmer for a moment” and consider those who will use your software. One important aspect is whether the software has bugs in it. When software “breaks”, the end user can’t go to a mechanic and say “fix this”. Often they can’t even grasp what is actually happening and then end up suffering just trying to figure out what has happened to their favorite software. Programmers may feel that “bugs are inevitable and must be lived with” and that “I don’t have time to track down every bug”. If you feel that way, then IMO you are not ready to be called a programmer yet.  Now you may say that “time is money” and I can’t afford to spend that much time on debugging my software. Well, this experienced programmer can tell you that it can be done, but it requires two things.

(1) Debugging is an art!

Maybe more time should be spent on training programmers on how to debug their software. IMO, it is an art, and a rare one at that. I can assure you of one thing, that the compiler (or programming environment) can’t be your only means of debugging. The debugging skills of a compiler can’t compare to the debugging skills of the human mind. You can’t program a debugger that thinks like a human being. Despite all the science fiction you may watch and how they portray computers, simply put “computers are dumb”.  Computers just are faster than us when it comes to counting and stuff like that, but they are as “dumb as a door nail” (as the saying goes). Yes, computers are stupid! A young child only a few years old can master two languages at one time (ie. both Spanish and English spoken in their family) and yet the greatest super computer can’t even master one language (so called artificial intelligence is a overrated).  Debugging requires the powers of the human mind, requires the desire to “make it right” for the sake of the end user and requires experience.

Actually debugging is similiar to being a auto mechanic. Good mechanics are the ones who “care about the customer” and who will do whatever it takes to get it right. Good mechanics are those who “think it through”, rather than simply just start replacing parts. Good mechanics are those who have developed the “art” of diagnosis. The same holds true for programmers. Debugging is an art, which comes from experience, as well as the desire to “do it right” for the “sake of the end user”.

(2) Be a learner, rather than acting like a master.

What I mean is this. No matter how experienced you are as a programmer, often your end user is far more experienced than you when it comes to the task at hand, that your software must handle. For example, if you write software for a business (ie. a custom application for handling some important task they must do), then you need to be a “learner” which means take time to learn everything you can (in a reasonable amount of time of course) about the task at hand and the business who will use your software. I have written software for all sorts of businesses, from mom and pop video stores, quality control for manufacturing, lumber supply store accounting, machine shop estimating, etc. Each time I had to “learn” as much as I could about the end users and what they required for the task at hand. You may know more about programming than your end user, but you likely don’t know more than your end user about the task at hand. Also the needs of end users tend to differ from person to person, so one may need to ask lots of questions and to get answers from multiple users.

Beta testing is a good example of where sometimes this goes wrong. Most programmers think of beta testing as a means to get end users to find all the bugs in their software. You give a bunch of beta testers some poorly tested software and expect them to figure out where all the problems are. I think that is all wrong. You are the programmer and its your job to find most of the bugs before your end users get the software. Today, I don’t Beta test my software to find bugs. I have a program called a “PreRelease version” where a select group of users can purchase and use the software and provide feedback on what features are still needed for their specific situations. I add new features in steps during this program, but the discussions with users are not about all the bugs they find (because they find very few and I often find most bugs before they are even aware they are there), but they are about what features would help them solve problems. I want them to use the software in real world situations, so I can see what features I failed to consider would be required for their individual needs. I then add new features based on the feedback, but the debugging is my responsibility. As far as writing the software and being a debugger, I have to be the expert. But when it comes to improving the feature set of the software (new ideas), I have to be the “learner”. I have to listen to my customers problems. I have to see their needs. I have to try to put myself into their “shoes” (as the saying goes). Its kind of like an old time cobbler (shoe maker). He is the expert in making shoes, but he still has to measure the feet of his customers, he still has to put himself into their “shoes” so to speak (meaning he has to consider how they will use his shoes and the unique needs of each user).

Being a good programmer require some modesty and humility. Be willing to learn. Recognize that you don’t know it all. Recognize that others are effected by your work (for good or for bad). Don’t be lazy! Debug your own software, thats your job! Put yourself into the shoes of those who will use your software. Stop being a programmer for a moment. Be objective about your own work and consider the needs of those who depend upon your software.