The importance of a low level native flat API in Windows.

Note:  This article has been updated (8/19/2019)!

Innovation in software development is always important.  New ideas for better software often requires “thinking outside the box” as they say. In the Windows world, software development can easily become stagnated if developers push too much for more higher level functionality at the expense of low level functionality.

WIN32 API

Having been a WIN32 API programmer for the last 15 years, while most programmers are using the much higher level dot.net and WINRT runtimes, I see more and more why low level flat APIs are so important. Actually, the term low level is deceptive. Low level does mean more work compared to high level, but actually low level also means simplicity.  The concepts of low level, once learned, are actually simpler in many ways, not being hindered by overly complex higher level concepts. It is the extra work that low level requires which often turns off programmers, since it is easy to desire a quick fix for a solution (meaning little or no work). But low level coding also provides more choices for customization. It encourages new ways of doing things, since it is only limited by ones imagination and skill.

Windows, while still based on its core WIN32 API, has lost much of its simplicity. The use of object oriented programming promotes the building of more and more complex code bases, which may seemingly help programmers by providing more higher level functionality, but it actually strays away from the beautiful simplicity of a low level flat API like the WIN32 API. So while one sees the benefits of the higher level code base, why should one even care about the low level native API ?

The high level must be built from the low level.

What we may fail to appreciate is that the high level functionality must be built using the low level.  The more one improves the low level functionality, the more options one has to develop the higher level engines software is built upon. I see this more and more when new hardware devices come to market. For example, why does a hardware manufacturer first provide access to its devices via a high level class library, rather than first via a low level simple flat API say for C, rather than C++ ? Build the low level API’s first. Then one can more easily build better higher level libraries. If low level libraries are available first, some software developers may find ways to exceed the functionality first intended by the hardware manufacturer.  What few may appreciate is that some times a low level, simple, flat API may provide better performance than a high level class library might. Even things like COM , namespace classes, etc are built upon more simpler low level API’s. For example, if one were to see how a class object used in C++ could be emulated via just using C (with things like vector arrays) one would appreciate that the higher level is actually much more complex than one might think. True, modern compilers hide such complexity from us, but that does not mean it is no longer complex. There was a simple rule in programming. Less code means less work for the CPU. The problem though is that if the code is very high level, then the less code may hide the reality of what a compiler is generating and programmers lose control of performance. Less code no longer means less work for the CPU.

High level libraries are important for faster software development. But if an operating system like Windows is only improved at the higher level, then what happens is that innovation decreases. True, the average programmer may seem more productive, but the real innovators, the ones who tinker with the low level, have less and less to work with. Where are the new compiler builders ? While innovation increases when it comes to scripted languages, where are the new native code compilers coming from ? In the “indie” (independent) world of programming languages, who is building the next generation of native code compilers ?

A lesson learned from building a GUI framework.

Having spent the last 15 years developing a GUI framework for use with a native code compiler, working only with the native WIN32 API, I have learned a thing or two about Windows, programming languages and native code programming. First, I have a deep appreciation for the WIN32 API and how much it offers. Sadly though, with the advent of dot.net little has been done to improve the low level WIN32 API. The API docs often make reference now to some API’s being labeled as legacy. Yet likely the functionality of such API’s is still maintained in some form or fashion in the Windows core DLL’s since the higher level functionality may be based on it. In the old days, when there was a need to update an API, Microsoft at times would create a similar API call, but simply append an EX at the end of the function name. For example, CreateWindow became CreateWindowEx. This made sense when significant functionality needed to be added. But at least the core API continued to be improved. Today, the core APIs in Windows seem to have little or no improvement, but instead have been replaced by COM object calls instead. This programmer suggests that it may be better to provide both, low level API improvements and higher level companions. Sadly though native API’s are an afterthought rather than the primary concentration today.

Watch the video of Herb Sutters talks “Why C++?” on Channel 9.

If high level functionality becomes more important than low level, then we may lose the real software innovators who create totally new ways to do things.

The richness of the WIN32 is, simply put, amazing! Microsoft put so many different ways of customizing things into the core WIN32 API, the choices are almost endless.  Manufacturers of new hardware could learn a thing or two from this, by building low level libraries for their hardware which would appeal to C (not simply C++, but real C) programmers first and then build higher level wrappers over them for other languages. I had the opportunity to beta test a unique user input device which is now becoming popular and was shocked to find the interface libraries were only provided in C++ class libraries. The idea of providing the library in a simple DLL for use with C appeared to not have been considered and workarounds for languages like C had to be done via a C++ wrapper converting the library to a simple flat API usable by C. Personally, I would have thought that C would be the first choice, a simple flat API and then build a C++ class on top of that.  It is the old “cart before the horse” syndrome.

In building my own GUI framework (on top of the WIN32), I found that building the framework in levels made a lot of sense. Often I would start adding functionality as a low level feature first and then build upon it. For example, I wanted to tap into OwnerDraw so I could customize controls. Rather than build a high level engine on top of OwnerDraw, I simply built a simpler low level ownerdraw engine which allowed me to have access to the core functionality of ownerdraw. It would take a lot more work for a user to draw something, but it provided more choices at a lower level. Then I built upon that engine a higher level (I called it Simplified OwnerDraw) where common tasks were built into higher level functions, so many common tasks could be done with just one line of code. But users of the framework though had a choice. They could use the higher level functions (ie. draw a 3D Button) if it sufficed, but if they needed more they could fall back to lower level drawing and draw things exactly as they desired. But I learned that the low level had to come first.

It would be nice!

As a Windows programmer I think it would be nice to see a resurgence in interest in improving and building upon the low level WIN32 API. By going back and improving the core APIs in Windows, even higher level layers like dot.net could benefit. It could even be possible to see new innovations in the Windows world beyond what we think possible currently. Imagine a full blown version of Windows capable of running on Intel’s Edison device ! By fine tuning the WIN32 API and building upon it, it is possible. This is not simply wishful thinking or theory. When I was designing my own GUI framework (on top of the WIN32 API), I purposedly didn’t use the latest version of Windows or the latest computers. Unlike most programmers who write their software on the lastest, fastest computers and then test them on a legacy PC to see if it will work, I did the opposite. For example, when Windows XP was the current version of Windows I developed my GUI framework on a Windows 95 computer with a very slow 233 mhz CPU. While most XP computers were running CPUs in the gigahertz ranges (2 ghz average), I was developing on a 233 mhz CPU based computer and less RAM than typical for an XP computer.  If my GUI framework ran on a less powerful machine, then imagine how well it would run on the latest PC. That is the power of the raw WIN32 API ! Minimal hardware and better performance.

It is not inconceivable to be able to write powerful Windows software which could run on an Intel Edison device (aka. Quark chip). Windows currently can’t, but it could if the operating system was slimmed down and the WIN32 API was tapped into.

UPDATE:

Windows raw power overlooked !

It is now nearly 20 years since I started development on my GUI framework for Windows (EZGUI 5.0 Professional) and I am even more convinced of the importance of the original WIN32 flat API. Why ? Because of its raw power.  Just look at the state of Windows software today. It tends to be bloated and slow and requires the latest hardware. Yet, apps written using the core WIN32 API using a procedural style of coding are fast, small in size and require minimal hardware.  As proof of this, consider my GUI framework (EZGUI 5.0 Professional).  Even compared to the “classic” Visual Basic 5.0/6.0 runtimes , EZGUI’s runtimes (DLL’s) are tiny in comparison. Visual Basic required a number of additional OCX controls just to provide access to core features of Windows, such as the Common Controls, RichEdit and Common Dialogs. Classic VB 6.0 has a core runtime which is about 1.3 megabytes in size. To use many of the other normal features in Windows, such as Common Dialogs, Common Controls, RichEdit and MCI requires maybe another megabyte of runtimes. This is small compared to dot.net runtimes, but it is actually large compared to a pure WIN32 procedural (flat API) GUI framework such as EZGUI 5.0 Professional.

The EZGUI core runtime is only about 760 Kilobytes (less than 1 megabyte).  Yet, this runtime contains more features than the VB 6.0 runtime does, when it comes to GUI features. Consider what I squeezed into the EZGUI runtime, despite its tiny size:

  • All the standard controls
  • Most of the Common controls (except header and property sheet)
  • RichEdit control (supports 1.0 through 4.0 versions)
  • Forms (standard, MDI, container)
  • Thread engine
  • Ownerdraw engine
  • subclassing engine
  • superclassing engine
  • Theme support (for drawing)
  • multi-monitor support
  • print engine
  • GDI graphics engine

Besides core features in Windows, EZGUI provides custom engines not found in Windows (and not in classic Visual Basic) and a number of custom controls I built from scratch, such as:

  • Property Listbox custom control
  • Files Listbox custom control
  • 3D Button custom controls (via ownerdraw)
  • Masked Edit custom control
  • Turtle Graphics custom control with scripting language
  • Canvas custom control with 2D Sprite engine, multiple buffers, DIB sections (direct access to pixels) and more
  • advanced Graphics engine for Canvas including 360 degree rotation, transparent and alphablending, etc.
  • Shape/Hotspot/Splitterbar custom control
  • MCI custom control with its own scripting language

Now add to this something I have never seen in any other GUI framework, which Windows should have had but never did:

  • Visual Designer drag and drop engine
  • Drag Handle custom control

This Visual Designer engine is for building Visual Designer, like we use in programming IDE’s. It does things like a snap-to grid, drag handles, drawing rectangle/ellipse or line to create controls, etc. and it can drag hundreds of controls at one time smoothly with little flicker (uses a backbuffer for drawing and bitblts each move).

The API for the EZGUI framework in nearly a programming language of itself with a nearly a 1000 GUI commands, not counting script languages and it has its own event engine as well.

If you add just one more runtime DLL (at 225 kilobytes) to this, it also adds an OpenGL based Canvas control with its own 3D scripting language and it supports the STL 3D model format (used in 3D printing). It can load (and display) a STL 3D model with almost 2 million polygons in it in just a couple seconds on a low end mass market PC (like you get at Walmart).

To better appreciate what this framework can do, just download a test app I wrote with it which demonstrates many of its features as see for yourself:

http://cwsof.com/download/testwin8.zip

Here are some 3D examples using the framework:

http://cwsof.com/download/ezmodvw11.zip

http://cwsof.com/download/ez5demo3d.zip

So with a runtime less than 1 megabyte in size (smaller than an old fashioned floppy disk), one can do an amazing amount of stuff. That is the power of the WIN32 API used in a procedural style.

If Microsoft could produce a “lite” version of Windows with just the core WIN32 API in it which could run on minimal hardware, such as a single board computer (aka. like Raspberry PI, but x86), I could write powerful software which would “fly” on the device.

My GUI framework was written using the Powerbasic native Windows compiler. It has the raw power of C, but using BASIC.  Yes, using the core WIN32 flat API, one can push the limits of Windows, even on Windows 10.

tagged with , , , , , ,