Lessons learned from building a GUI framework
I have been programming for nearly three decades now. Actually my first exposure to programming was back in 1975 in high school and I did take a college course in computer science in 1976, coding in Fortran using punch cards to input my code. In the 1980’s I learned how to code in BASIC (using a real compiler too) and 6502 machine language on a Commodore 64 computer. I even wrote my own compiler. I wrote software for local businesses, first for CPM computers and then IBM compatible computers. Some of that software I wrote is still in use today.
About a decade and half ago I started developing tools for programmers. At a time when WIN32 programming had gone out of fashion and most programmers were writing software for Windows using managed (aka. dot.net) languages, I ventured into the world of low level WIN32 programming. Rather than code in C or C++, I used a BASIC language compiler which had many of the features found in a C compiler, including things like data pointers, code pointers and the capability to access the WIN32 API directly. It was difficult to even find books which taught WIN32 programming (without things like MFC, ATL or OOP), so I had to search for the best used books I could find which taught the subject well. All the books were for C (not C++) and not for BASIC either. Learning how to write custom controls using WIN32 techniques was a challenge, since the only book I could find dated back to 16 bit Windows. Obviously it was not a popular subject. Even learning how to work with threads at a low level was a challenge, but I fortunately found one of the best books I could find on the subject which went into detail about how threads work and the pros and cons to using threads.
To make things even more challenging, the compiler I was using was being marketed as an addon to Visual Basic (classic) for writing small and fast DLL’s and while it was capable of being used to write an application (EXE) for Windows, it had no GUI command set, no visual designer, but simply a code editor and a compiler which could access the WIN32 API. I wasn’t about to hand code applications using only the WIN32 API. So what was a programmer to do ?
Building a GUI engine and the lessons learned
If there was one thing I had learned over the years, it was the importance of building code libraries. Modular design was (and still is) vital for fast software development. The WIN32 API is actually very low level and not well suited for fast software development. RAD (Rapid Application Development) was also something I appreciated. Not only did I need to write my own higher level GUI framework, but I also had to build my own RAD tools as well. It was a slow process and it took years to accomplish. I am still adding to my tools and there is still much to learn (I haven’t mastered low level OLE and COM yet, but I am working on that). Also while some programmers today would rather fast-track it and take shortcuts and simply use libraries written by others, particularly open source, I have always prefered to use only my own code and so I wrote everything myself. No copy and paste for me ! While it would take longer to accomplish things, I decided to first learn the necessary API techniques and then once I was confident I had the skills needed, I would proceed to write each part of my library code from scratch. No open source, no borrowed code. But what lessons did I learn through all of this?
First, the Windows WIN32 API is an extremely powerful framework. While it is a bit terse at times and the documentation could benefit from a little clearer explanations, it is still a well documented API with a huge amount of resources available for those who know how to find them. In this world obsessed with object oriented programming, it is amazing how efficient the WIN32 API is and it is basicly a flat API. Most work done using the WIN32 API is done using procedural coding styles, rather than object oriented styles. This means that applications written using the WIN32 API are more likely to be smaller and faster than their object oriented counterparts. Actually, at a time when many are looking for ways to build software which is faster and smaller, well suited to todays mobile computers (Windows 8 Desktop), using the WIN32 API is an amazingly powerful alternative. Just consider, the WIN32 API dates back to Windows 95 when a typical PC was less powerful than some of the cheapest cell phones of today. CPU’s were rated in megahertz speeds, rather than gigahertz. RAM memory was only a few megabytes in size compared to todays typical PC with multiple gigabytes of RAM. Hard-drives were only a few megabytes in size. Are you looking for a way to build smaller, faster software which requires minimal resources ? Maybe WIN32 programming is worth a look.
Of course, coding using the WIN32 API directly is likely too slow for developing large applications, but no one said you can’t build modular, reusable code using the WIN32 API, to speed up development. That is exactly what I did. Windows has one of the most efficient library formats around which is the dynamic link library or DLL. DLL’s are simple, yet powerful. No system registration required. No COM required. Completely transportable (meaning just copy and run). While an application can be written where DLL’s are automatically loaded by the EXE when it is first run, DLL’s can also be loaded via code ( LoadLibrary API function ) on demand. Applications can actually poll any DLL ( GetProcAddress API function ) to see if a specific API exists first and if so, then call it via a pointer. This is how I overcame problems with different versions of Windows. While most API’s I used were available on all versions of Windows, from Windows 95 to Windows 8, some were specific to certain versions of Windows. No problem ! You can load a system DLL and test whether an API exists or not. If it does not, then you can switch to an alternative or simply not make a feature available to your application. One interesting feature of the LoadLibrary API function, is that the file extenstion of a DLL does not need to be .dll . You can define your own file extensions and use them for DLL’s. I used this for writing a plugin format for my visual designers. I compile the plugins to DLL’s and then change the file extension to one I use only for plugins. This way my visual designer can simply look for plugins which match the file extension I use. Dynamic Link Libraries are an amazingly simple yet powerful way to build libraries and they are sharable by multiple applications, even ones written in different programming languages. For example, I can write a DLL in BASIC which can be used by an application written in C.
Lessons from classic Visual Basic
One of the best (and easiest to use) development environments for designing Windows applications was classic Visual Basic. The key to its ease of use was its drag and drop front end. Rather than simply code everything by hand, one could literally draw the user interface of an application. But Visual Basic had it weaknesses too. I found this out when I was a beta tester for a company who built a powerful multimedia development system. Their development system was written in C. It had the ability to generate a multimedia presentation which could be run on a web page via an ActiveX plugin (for Internet Explorer), but they had no utility for generating the HTML web page one needed for running the presentation. One had to hand code the HTML web page. I decided to write a nice utility using Visual Basic, which they could use with their development system. The C programmer who built the multimedia development system, liked what I created, but he was bothered by the number of OCX controls my Visual Basic application required. He asked me if I could write the application so it didn’t need the OCX controls. It appears that it was more trouble to build the installation program for their software when OCX’s were used. DLL’s were simple, since they can just be copied anywhere (system folder or app folder). OCX components were a bit more trouble to install and they wanted a trouble free installation program. I was not versed with the WIN32 API yet at the time, so it was a challenge to rewrite my app to get rid of its dependencies which were OCX controls. The core DLL runtime was not a problem though.
I learned a few lessons from Visual Basic which I carried with me when I started to learn the Windows WIN32 API. First, rather than use OCX components I chose to rely on standard custom window classes compiled into DLL’s instead. This made the runtimes transportable (just copy and run) and didn’t require any special installation. DLL’s (Dynamic Link Libraries) have served me well and are very versatile. They also tend to be much smaller than their OCX (object oriented) counterparts. DLL’s loaded on demand also allow more flexibility when trying to build reusable libraries which will run on as many different versions of Windows as possible. I actually designed my GUI framework so it would be able to run even on Windows 95/98, even those versions of Windows did not support all the features in the framework. By building libraries as DLL’s which directly access the WIN32 API, it also allowed me to build a framework which was surprisingly small, requiring minimal resources.
One interesting facit of classic Visual Basic was that while you had to distribute a runtime DLL with your applications, the applications EXE tended to be quite small in comparison to an application written in C using only the WIN32 API. because much of the functionality of the GUI framework was in that DLL, the executable part didn’t really have to have a lot of code in it. The runtime did most of the work. In a similar fashion, I chose to do the same thing. By building as much functionality into my GUI framework as possible, the executable part of an application would be significantly small. Now while a runtime may initially make the entire application a bit larger than a purely WIN32 counterpart with everything in an EXE, as the application gets larger and more complex one should reach a tipping point where the EXE and runtime DLL together actually may be smaller than if one had built it as a single EXE using the WIN32 API alone.
So a runtime made sense to me. But like classic Visual Basic, the runtime needed to be small in size. By todays standards the 1.4 megabyte runtime of classic Visual Basic is tiny. Knowing that the extra OCX controls (and common dialogs) would add significantly to the size of a classic Visual Basic application and defeat the small runtime potential, I decided that I needed to build a core runtime with as much functionality as I could so I would not have to have a number of extra dependencies, just for basic functionality most applications would require. A single runtime would have shared code in it, which would have to be duplicated if it was broken up into many separate parts (like Visual Basic OCXs), so by squeezing as much functionality into a single DLL made sense. My goal was to keep below the size of the code Visual Basic runtime library. What surprised me was how much one could put into a small runtime library, when using native WIN32 coding. If there is a lesson from this discussion, it is how powerful the WIN32 native API is and how small and fast software becomes when using it. So did I meet my goal ?
One of the key decisions I made was to follow the 80/20 rule. In essence the principle is that most people (80 percent) only use about 20 percent of the features in a product (in this case software). Now the actual percentages may differ of course, but the principle is what matters. When building a GUI framework for Windows, I decided that I should code features based on what is most commonly used, rather than try to encapsulate everything (basically too much). The WIN32 API is huge, but most programmers only use a small subset of that API. So building what is needed and useful, one could build a framework which is very small. If a feature is needed, which is not in the framework, one could simply resort to the Windows API directly. This meant that I had to build the framework so it would be easy to integrate WIN32 code into it. This meant making sure things like actual window, font or bitmap handles were available, even if not normally used by the framework command set. This way if WIN32 API code was inserted, one had access t everything needed to make the call properly. I could not use the typical “black box” approach where everything is hidden from the programmer. This meant supporting things like subclassing, superclassing, window procedure hooks, etc. If the GUI framework couldn’t do something, then one shouldn’t have to “jump through hoops” to do it using the WIN32 API.
By using the 80/20 rule it allowed me to build a decent GUI framework, while keeping it small. But there was another thing I learned from Visual Basic which was important. While Visual Basic was powerful, it wasn’t always easy to customize things going beyond what it provided. For example, if a programmer wanted a slightly different Listview or Treeview control, there were few options available for customizing. Programmers would be more apt to purchase a third party custom control (OCX) with the functionality they wanted. This was fine and good, but is brought more problems. Now you have another dependency. Now you are dependent upon another party for a significant part of your software (meaning you now get the bugs by that third party developer). Now programmers are use to dealing with the issues of programming addons or components, but wouldn’t it be nice if you could simply use the existing Windows controls and then customize them yourself. The less dependencies the better. So when I built my own GUI framework, I decided to delve into all the customization features the WIN32 made available and surprisingly there were many. Actually, the WIN32 API was extremely well designed for endless options for customization. OwnerDraw and CustomDraw are two features in the WIN32 API which allow you all sorts of ways to customize controls, so I began to tap into them and this opened up all sorts of possibilities. Now a single window class (control) now could serve many different purpose via customization. I learned about superclassing, which allows you to build new window classes based on existing ones and with little overhead (unlike inheritance with objects). The WIN32 API provides all sorts of little tricks which can be used to customize window classes.
Even things like the Common Dialogs are customizable. You can create hooks into their core dialog procedures and then modify how they work and look. The WIN32 API is very low level, but it is its low level nature which made it possible for nearly endless ways of customization. In time though I found that there are a number of things which simply put, are missing in the WIN32 API. Functionality which would have made sense, but just didn’t exist directly in the API’s. Much of this kind of functionality is more high level, but still needed. But the raw power of the WIN32 API allows a programmer to build such functionality, what ever it is. The core tools are there. You just have to know how to use them. So what were some of those higher level features which were missing in the WIN32 API ?
Building higher level features
I will mention a few and how I dealt with them. The first was something I needed for building RAD tools. It is something which is obviously not easy to do with the raw WIN32 API. Now you may think, “I can do that using dot.net” or some other GUI framework. But I am talking about the raw WIN32 API here. There is a lot of higher level functionality which could have been embeded into it, rather than have to add it with one more layer on top. But fortunately the tools are there in the WIN32 API, so I had little choice, since I was building my own GUI framework. What I needed was a way to impliment true drag and drop so I could build a Visual Designer front end, which most programmers today can’t live without. Think it is easy ? If you browse the internet looking for indie (independent) programming languages, whether BASIC, C or some new different language, one thing you may find in common. At least in the BASIC world this is true. While some very nice compilers have been developed by these developers, the area which most are weak in is the RAD area. Why ? Because it is not easy to build a WYSIWYG style RAD visual designer. But this kind of functionality has so many useful purposes in software development. Shouldn’t programming languages make it easy to do ? They should, but often they don’t and the WIN32 API is not a big help here. If you think this is a simple task, then why not take a look at Visual Basic and how it improved over the years when it comes to this. Do you have an old version of Visual Basic, say 1.0 or 2.0 lying around ? Pull it out and compare how the visual designer works compared to say Visual Basic 5.0 or 6.0. The old versions are very rudimentary in comparison. Why ? Because it is a lot of work figuring out how to make Windows do what you want when it comes to WYSIWYG drag and drop. I found this out the hard way. It took me years to figure it out. How to do rubberbanding properly with a snap to grid effect ! How to have objects which are the real controls and not some kind of placeholders ! For this I had to build a subclassing engine which took over control of different window classes (controls), so it could prevent the controls from responding to mouse clicks when in drag/size mode, but regain functionality when in test mode. One big challenge was in figuring out how to drag hundreds of controls at one time with speed and minimal flicker. The WIN32 API has the core functionality there to build such things, but few programmers have the time to build it from scratch.
Another high level featured I wanted was what us old timer programmers would refer to as “sprites”. Not DirectX based sprites or animation, but simple GDI based non-rectangular animated images. Sprites should have been inherent to Windows from day one, but sadly they were not. But the designers of Windows did though provide one low level feature which many graphic programmers are grateful for. That is DIB’s or Device Independent bitmaps. Unlike DDB’s (Device Dependent Bitmaps or normal bitmaps), DIB’s are so versatile and they can be the basis of so many different things. DIB’s were my choice when I built my own custom Canvas control. DIB’s were my choice when I build a 2D Sprite engine into my Canvas control. DIB’s allow you direct access to the pixel data, without having to move the data to a separate buffer first. DIB’s can also be huge, while DDB’s are limited in size (and memory). Oh how DIB’s make a difference. You can even write code with high performance and it only be GDI based using DIB’s.
But the one thing I did different than Visual Basic, was rather than depend upon components for extra functionality, I tapped into the WIN32 API in ways to provide all sorts of extra functionality which would be customizable. Since Graphic oriented controls are so useful, it made sense to build a single graphic control which had the ability to be turned into almost anything. The Sprite engine turned a simple graphic Canvas control into something would could do almost anything. I later expanded upon that and built upon that a 3D layer using OpenGL which could handle 3D objects over the top of a normal Canvas and in a real window class (control) so I cna have multiple instances of the 3D Canvas on a single parent window.
So how big is this GUI framework ?
Well consider this. The framework handles most of the common GUI tasks one would require, such as all the standard controls and key features, most of the common controls and key features, the common dialogs , but also with all the following:
- Graphic engine for printing and drawing and ownerdraw
- Subclassing, Superclassing engines
- Drag and Drop engine for build Visual Designers
- MultiMonitor support
- Theme Drawing support
- HLP and HTML Help display commands
- OwnerDraw and CustomDraw engine
- Drag Handle custom control
- Masked Edit custom control
- Shape/Hotspot/SplitterBar control
- 2D and 3D Colored Button drawing engine
- 22 Image Filters for drawing Bitmaps
- Canvas custom control with 2D Sprite engine
- glCanvas custom control with 3D scripting engine
- MCI multimedia custom control with EZ scripting language
- Turtle Graphic (vector) custom control
- Files Listbox custom control
- Property Listbox custom control
- Thread engine
- nearly 900 GUI commands in command set
Did I reach my goal of a runtime as small as the classic Visual Basic core runtime ?
YES ! The entire GUI framework is only about 1 megabyte in size. That is about 70 percent of the size of just the core Visual Basic runtime and it does not need any extra OCX controls for things like the common controls or common dialogs. Yes, by tapping into the power of the WIN32 API, I was able to build a GUI framework which has many features not found in Visual Basic, but still be 30% smaller in size. That GUI framework is called:
EZGUI 5.0 Professional was written using PowerBasic and the native WIN32 API. EZGUI requires only a minimal footprint (memory and disk space) and yet it packs a very large feature set. EZGUI is an excellent framework for developing Windows (x86 desktop) applications for todays mobile computers and even embeded Windows (x86) devices.
Want to see EZGUI in action ?
Download this test application I wrote to test different features of the framework to see how they worked on Windows 8:
Windows 8 backward compatiblity test app
Download this 3D model viewer demo application I created to test the 3D engine with STL 3D models:
3D Model Viewer for STL models